Code generation method, device, computer equipment and storage medium

Abstract

The present application relates to a code generation method, device, computer equipment and storage medium. The method comprises: extracting at least two fusible network layers from a neural network; obtaining multiple operations contained in each of the at least two fusible network layers; combining the multiple operations contained in each of the at least two fusible network layers to obtain a fusion node; obtaining the quantized implementation code of the fusion node according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusible network layer and the type of backend hardware. The method uses automatic generation instead of manual addition, which can reduce labor costs and has a low error rate.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a code generation method, apparatus, computer device and storage medium. Background Art

[0002] At present, after the fusion process, the neural network will generate many new nodes. Common fusionable network layers are: Eltwise layer and activation layer. If the multiple operations contained in the Eltwise layer and the multiple operations contained in the activation layer are combined, many fusion nodes can be obtained. These nodes need to add quantization implementation code to deploy the neural network to the backend hardware.

[0003] In the prior art, quantization implementation code can be added to each fusion node by handwriting the kernel. However, with the increase of new neural network layers, the number of fusion nodes increases exponentially. The method of manually adding code undoubtedly requires increased labor costs and has a high error rate. Summary of the invention

[0004] Based on this, it is necessary to provide a code generation method, device, computer equipment and storage medium that can reduce labor costs in response to the above technical problems.

[0005] In a first aspect, the present application provides a code generation method, the method comprising:

[0006] Extract at least two fusion-capable network layers from the neural network;

[0007] Acquire a plurality of operations included in each of the at least two fusible network layers;

[0008] Combining multiple operations included in each of the at least two fusion-capable network layers to obtain a fusion node;

[0009] According to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer and the type of backend hardware, the quantization implementation code of the fusion node is obtained.

[0010] In one of the embodiments, before obtaining the quantized implementation code of the fusion node according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer, and the type of backend hardware, the method further includes:

[0011] For each of the at least two convergible network layers, obtain the code of each operation of the multiple operations included in the convergible network layer, and disassemble the code to obtain an abstract syntax tree corresponding to the operation. The abstract syntax trees corresponding to the multiple operations included in the convergible network layer constitute the abstract syntax tree of the convergible network layer.

[0012] In one of the embodiments, obtaining the quantized implementation code of the fusion node according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer, and the type of backend hardware includes:

[0013] According to the operation combination corresponding to the fusion node, an abstract syntax tree of the operations included in the operation combination is obtained from the abstract syntax tree of each fusionable network layer;

[0014] Parsing the abstract syntax tree of the operations included in the operation combination to obtain a code corresponding to the operation combination;

[0015] According to the code corresponding to the operation combination and the type of the backend hardware, the quantization implementation code of the fusion node is obtained.

[0016] In one embodiment, the acquiring the quantization implementation code of the fusion node according to the code corresponding to the operation combination and the type of the backend hardware includes:

[0017] The code corresponding to the operation combination is converted into a form adapted to the type of the backend hardware, and the conversion result is used as the quantization implementation code.

[0018] In one embodiment, the at least two fusible network layers include: an Eltwise layer and an activation layer; the multiple operations included in the Eltwise layer are: dot product, addition and subtraction of sum, and taking the maximum value max; the multiple operations included in the activation layer are: tanh, sigmoid, relu and prelu; the operation combination corresponding to the fusion node is one of the following combinations: a combination of dot product and tanh, a combination of dot product and sigmoid, a combination of dot product and relu, a combination of dot product and prelu, a combination of addition and subtraction of sum and tanh, a combination of addition and subtraction of sum and sigmoid, a combination of addition and subtraction of sum and relu, a combination of addition and subtraction of sum and prelu, a combination of taking the maximum value max and tanh, a combination of taking the maximum value max and sigmoid, a combination of taking the maximum value max and relu, or a combination of taking the maximum value max and prelu.

[0019] In one of the embodiments, the type of the backend hardware is one of the following types: a central processing unit CPU, a graphics processing unit GPU, or a data processing unit DPU.

[0020] In a second aspect, the present application also provides a code generation device. The device comprises:

[0021] An extraction module, used for extracting at least two fusion-capable network layers from a neural network;

[0022] An acquisition module, used for acquiring a plurality of operations included in each of the at least two fused network layers;

[0023] A fusion module, used for combining a plurality of operations included in each of the at least two fusion-capable network layers to obtain a fusion node;

[0024] A generation module is used to obtain the quantized implementation code of the fusion node according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer, and the type of backend hardware.

[0025] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0026] Extract at least two fusion-capable network layers from the neural network;

[0027] Acquire a plurality of operations included in each of the at least two fusible network layers;

[0028] Combining multiple operations included in each of the at least two fusion-capable network layers to obtain a fusion node;

[0029] According to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer and the type of backend hardware, the quantization implementation code of the fusion node is obtained.

[0030] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0031] Extract at least two fusion-capable network layers from the neural network;

[0032] Acquire a plurality of operations included in each of the at least two fusible network layers;

[0033] Combining multiple operations included in each of the at least two fusion-capable network layers to obtain a fusion node;

[0034] According to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer and the type of backend hardware, the quantization implementation code of the fusion node is obtained.

[0035] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0036] Extract at least two fusion-capable network layers from the neural network;

[0037] Acquire a plurality of operations included in each of the at least two fusible network layers;

[0038] Combining multiple operations included in each of the at least two fusion-capable network layers to obtain a fusion node;

[0039] According to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer and the type of backend hardware, the quantization implementation code of the fusion node is obtained.

[0040] The above code generation method, device, computer equipment and storage medium extract at least two fusible network layers from a neural network; obtain multiple operations contained in each of the at least two fusible network layers; combine the multiple operations contained in each of the at least two fusible network layers to obtain a fusion node; and obtain the quantized implementation code of the fusion node according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusible network layer and the type of back-end hardware. This method uses automatic generation instead of manual addition, which can reduce labor costs and has a low error rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a fusion node in an embodiment;

[0042] Figure 2 A schematic diagram of a flow chart of a code generation method in an embodiment;

[0043] Figure 3 A schematic diagram of a neural network in one embodiment;

[0044] Figure 4 is a flowchart of a code generation method in another embodiment;

[0045] Figure 5 is a flowchart of a code generation method in another embodiment;

[0046] Figure 6 is a structural block diagram of a code generating device in one embodiment;

[0047] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] In the process of deploying neural networks on backend hardware, combining two operations and using one computing unit to process the combination can save computing resources compared to using two computing units to process the two operations separately. A fusible network layer in a neural network means that if multiple operations contained in one network layer can be combined with multiple operations contained in another network layer, then both network layers are fusible network layers.

[0050] Common network layers that can be fused are: Eltwise layer and activation layer. Figure 1 As shown in the figure, the Eltwise layer contains multiple operations: dot product, addition and subtraction of sum, and maximum value max; the activation layer contains multiple operations: tanh, sigmoid, relu, and prelu. If the multiple operations contained in the Eltwise layer and the multiple operations contained in the activation layer are combined, 12 fusion nodes can be obtained. The operation combinations corresponding to these 12 fusion nodes are: combination of dot product and tanh, combination of dot product and sigmoid, combination of dot product and relu, combination of dot product and prelu, combination of addition and subtraction of sum and tanh, combination of addition and subtraction of sum and sigmoid, combination of addition and subtraction of sum and relu, combination of addition and subtraction of sum and prelu, combination of maximum value max and tanh, combination of maximum value max and sigmoid, combination of maximum value max and relu, and combination of maximum value max and prelu. These nodes need to add quantization implementation code to deploy the neural network to the backend hardware.

[0051] It should be noted that the fusion network layer can also be other network layers. The above-mentioned Eltwise layer and activation layer are only examples and do not constitute a limitation on the present application. In addition to tanh, sigmoid, relu and prelu, the multiple operations contained in the Eltwise layer may also include Leakly Relu and RRelu. The embodiments of the present application only use tanh, sigmoid, relu and prelu as examples for illustration, which does not constitute a limitation on the present application.

[0052] In some embodiments, quantization implementation code can be added to each fusion node by handwriting the kernel. However, with the increase of new neural network layers, the number of fusion nodes increases exponentially. The method of manually adding code undoubtedly requires increased labor costs and has a higher error rate.

[0053] To this end, an embodiment of the present application provides a code generation method, which pre-establishes an abstract syntax tree of a fusible network layer. When the neural network needs to be deployed to a back-end hardware device, the fusible network layer is extracted from the neural network, and multiple operations contained in each fusible network layer are combined to obtain a fusion node. Then, based on the pre-established abstract syntax tree and the type of back-end hardware, a quantization implementation code is automatically generated. This method uses automatic generation instead of manual addition, which can reduce labor costs and has a lower error rate.

[0054] The code generation method provided in the embodiment of the present application can be applied to any device with corresponding processing capabilities, such as: a personal computer, a laptop computer, a smart phone, a tablet computer, an Internet of Things device or a server, etc., and the embodiment of the present application does not limit this. The above code generation method is described below in conjunction with specific embodiments.

[0055] In one embodiment, a code generation method is provided. Figure 2 As shown, the method includes:

[0056] S102. Extract at least two fusion-capable network layers from the neural network.

[0057] Optionally, the neural network in the embodiment of the present application may be a deep neural network (DNN), a recurrent neural network (RNN), or a convolutional neural network (CNN). The embodiment of the present application does not limit the type of neural network.

[0058] Optionally, a fusible network layer in the embodiment of the present application means that if multiple operations included in one network layer can be combined with multiple operations included in another network layer, then both network layers are fusible network layers.

[0059] Optional, common fused network layers include: Eltwise layer and activation layer, which can traverse the network layers included in the neural network. If it is determined that the neural network contains Eltwise layer and activation layer, the Eltwise layer and activation layer can be used as fused network layers.

[0060] S104: Obtain multiple operations included in each of the at least two convergible network layers.

[0061] Optionally, the functions of each network layer in the neural network are different, and the operations contained in each network layer are also different. After extracting the fusible network layer, the fusible network layer is analyzed to determine the multiple operations contained in the fusible network layer. For example, if the fusible network layer is an Eltwise layer, the multiple operations contained in the Eltwise layer can be determined as: dot product, addition and subtraction, and maximum value max; if the fusible network layer is an activation layer, the multiple operations contained in the activation layer can be determined as: tanh, sigmoid, relu, and prelu.

[0062] S106: Combine multiple operations included in each of the at least two fusion-capable network layers to obtain a fusion node.

[0063] Optionally, after determining the operations included in each fusionable network layer, the operations included in each fusionable network layer are combined to obtain multiple fusion nodes. Exemplarily, if the multiple operations included in the Eltwise layer and the multiple operations included in the activation layer are combined, 12 fusion nodes can be obtained, and the operation combinations corresponding to the 12 fusion nodes are: a combination of dot product and tanh, a combination of dot product and sigmoid, a combination of dot product and relu, a combination of dot product and prelu, a combination of addition and subtraction of sum and tanh, a combination of addition and subtraction of sum and sigmoid, a combination of addition and subtraction of sum and relu, a combination of addition and subtraction of sum and prelu, a combination of taking the maximum value of max and tanh, a combination of taking the maximum value of max and sigmoid, a combination of taking the maximum value of max and relu, and a combination of taking the maximum value of max and prelu.

[0064] S108. Obtain the quantized implementation code of the fusion node according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer, and the type of backend hardware.

[0065] Optionally, during the offline development phase, for each fusible network layer, an abstract syntax of the fusible network layer may be established. When the neural network needs to be deployed to the back-end hardware, for each generated fusion node, based on the operation combination corresponding to the node, the abstract syntax tree of the relevant operation is extracted from the abstract syntax tree of the fusible network layer, the extracted abstract syntax tree is parsed to obtain the code of the fusion node, and the code is converted into a form supported by the back-end hardware to obtain the quantized implementation code.

[0066] Here are some examples:

[0067] Neural network Figure 3 As shown, the neural network includes an Eltwise layer and an activation layer. If the multiple operations contained in the Eltwise layer and the multiple operations contained in the activation layer are combined, 12 fusion nodes can be obtained. Figure 3 One of the fusion nodes is shown, and the operation combination corresponding to the fusion node is: dot product and relu and combination. The fusion node is recorded as Plugin, and the quantization implementation code of Plugin can be generated by the method of the embodiment of the present application.

[0068] The code generation method provided in the embodiment of the present application extracts at least two fusible network layers from a neural network; obtains multiple operations contained in each of the at least two fusible network layers; combines the multiple operations contained in each of the at least two fusible network layers to obtain a fusion node; and obtains the quantized implementation code of the fusion node according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusible network layer, and the type of back-end hardware. The method uses automatic generation instead of manual addition, which can reduce labor costs and has a low error rate.

[0069] As mentioned above, the quantization implementation code is generated by using the abstract syntax tree that can fuse the network layer. In one embodiment, see Figure 4 As shown, before using the abstract syntax tree of the fused network layer, the abstract syntax tree of the fused network layer can be obtained in the following manner:

[0070] S107. For each of the at least two convergible network layers, obtain the code of each operation of the multiple operations included in the convergible network layer, and disassemble the code to obtain an abstract syntax tree corresponding to the operation. The abstract syntax trees corresponding to the multiple operations included in the convergible network layer constitute the abstract syntax tree of the convergible network layer.

[0071] Here are some examples:

[0072] Assume that the fusible network layer is an Eltwise layer. The Eltwise layer contains multiple operations: dot product, addition and subtraction of sum, and maximum value max. Taking addition and subtraction of sum as an example, the pseudo code of the addition and subtraction of sum is:

[0073] for(i=0;i <size;i++){

[0074] vector_c[i]=vector_a[i]+vector_b[i];

[0075] }.

[0076] The above code can be broken down into sub-expressions such as ForExprAST (for loop expression), AssignExprAST (assignment operation), SubscriptExprAST (variable index, such as vector_a[i]), BinaryExprAST (variable addition). The above ForExprAST, AssignExprAST, SubscriptExprAST and BinaryExprAST are called the abstract syntax trees corresponding to addition and subtraction of sum. Using the same method, we can get the abstract syntax trees corresponding to the dot product and the maximum value max. The abstract syntax tree corresponding to the dot product, the abstract syntax tree corresponding to addition and subtraction of sum, and the abstract syntax tree corresponding to the maximum value max constitute the abstract syntax tree of the Eltwise layer. Similarly, the abstract syntax tree of the activation layer can be obtained.

[0077] After obtaining the abstract syntax trees of each fused network layer in the offline development phase, the quantization implementation code can be automatically generated based on the abstract syntax tree replacement method. Figure 5 As shown, the quantization implementation code can be generated in the following ways, including:

[0078] S501. According to the operation combination corresponding to the fusion node, an abstract syntax tree of the operations included in the operation combination is obtained from the abstract syntax tree of each fusionable network layer.

[0079] Optionally, as described above, the abstract syntax tree of the fusible network layer is composed of the abstract syntax trees corresponding to the operations contained in the fusible network layer. Therefore, after obtaining the fusion node, it is possible to first determine which combination of operations the fusion node is, and then search for the abstract syntax tree corresponding to the relevant operation in the abstract syntax tree of each fusible network layer.

[0080] Here are some examples:

[0081] As described above, if the multiple operations included in the Eltwise layer and the multiple operations included in the activation layer are combined, 12 fusion nodes can be obtained. The operation combinations corresponding to these 12 fusion nodes are: the combination of dot product and tanh, the combination of dot product and sigmoid, the combination of dot product and relu, the combination of dot product and prelu, the combination of addition and subtraction of sum and tanh, the combination of addition and subtraction of sum and sigmoid, the combination of addition and subtraction of sum and relu, the combination of addition and subtraction of sum and prelu, the combination of taking the maximum value of max and tanh, the combination of taking the maximum value of max and sigmoid, the combination of taking the maximum value of max and relu, and the combination of taking the maximum value of max and prelu. Taking the combination of dot product and relu as an example, the abstract syntax tree of the dot product operation can be found in the abstract syntax tree of the Eltwise layer, and the abstract syntax tree of the relu operation can be found in the abstract syntax tree of the activation layer.

[0082] S502: Parse the abstract syntax tree of the operations included in the operation combination to obtain a code corresponding to the operation combination.

[0083] Optionally, the parsing process here can be the reverse process of the above-mentioned disassembly process. For example, assuming that the operations included in the operation combination are: dot product and relu, the abstract syntax trees corresponding to the dot product and relu can be parsed respectively to obtain the code corresponding to the operation combination.

[0084] S503: Obtain the quantization implementation code of the fusion node according to the code corresponding to the operation combination and the type of the backend hardware.

[0085] Optionally, the type of the backend hardware is one of the following types: a central processing unit CPU, a graphics processing unit GPU, or a data processing unit DPU.

[0086] Optionally, the code corresponding to the operation combination may be converted into a form adapted to the type of backend hardware, and the conversion result may be used as the quantized implementation code of the fusion node.

[0087] The code generation method provided in the embodiment of the present application provides a detailed process of how to automatically generate quantization implementation code based on an abstract syntax tree. Compared with the method of manually adding code in the prior art, it improves the adding efficiency, reduces labor costs, and reduces the error rate.

[0088] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0089] Based on the same inventive concept, the embodiment of the present application also provides a code generation device for implementing the code generation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more code generation device embodiments provided below can refer to the limitations on the code generation method above, and will not be repeated here.

[0090] In one embodiment, Figure 6 As shown, a code generation device is provided, comprising:

[0091] An extraction module 601 is used to extract at least two fusible network layers from a neural network;

[0092] An acquisition module 602 is used to acquire a plurality of operations included in each of the at least two convergible network layers;

[0093] A fusion module 603, configured to combine a plurality of operations included in each of the at least two fusion-capable network layers to obtain a fusion node;

[0094] The generating module 604 is used to obtain the quantized implementation code of the fusion node according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer and the type of backend hardware.

[0095] Optionally, the acquisition module 602 is further used for:

[0096] For each of the at least two convergible network layers, obtain the code of each operation of the multiple operations included in the convergible network layer, and disassemble the code to obtain an abstract syntax tree corresponding to the operation. The abstract syntax trees corresponding to the multiple operations included in the convergible network layer constitute the abstract syntax tree of the convergible network layer.

[0097] Optionally, the generating module 604 is specifically used for:

[0098] According to the operation combination corresponding to the fusion node, acquiring the abstract syntax tree of the operation included in the operation combination from the abstract syntax tree of each fusionable network layer;

[0099] Parsing the abstract syntax tree of the operations included in the operation combination to obtain a code corresponding to the operation combination;

[0100] According to the code corresponding to the operation combination and the type of the backend hardware, the quantization implementation code of the fusion node is obtained.

[0101] Optionally, the generating module 604 is specifically used for:

[0102] The code corresponding to the operation combination is converted into a form adapted to the type of the backend hardware, and the conversion result is used as the quantization implementation code.

[0103] Optionally, the at least two fusible network layers include: an Eltwise layer and an activation layer; the multiple operations contained in the Eltwise layer are: dot product, addition and subtraction of sum, and taking the maximum value max; the multiple operations contained in the activation layer are: tanh, sigmoid, relu and prelu; the operation combination corresponding to the fusion node is one of the following combinations: a combination of dot product and tanh, a combination of dot product and sigmoid, a combination of dot product and relu, a combination of dot product and prelu, a combination of addition and subtraction of sum and tanh, a combination of addition and subtraction of sum and sigmoid, a combination of addition and subtraction of sum and relu, a combination of addition and subtraction of sum and prelu, a combination of taking the maximum value max and tanh, a combination of taking the maximum value max and sigmoid, a combination of taking the maximum value max and relu, or a combination of taking the maximum value max and prelu.

[0104] Optionally, the type of the backend hardware is one of the following types: a central processing unit CPU, a graphics processing unit GPU, or a data processing unit DPU.

[0105] Each module in the above code generation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0106] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as an abstract syntax tree. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a code generation method is implemented.

[0107] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0108] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0109] Extract at least two fusion-capable network layers from the neural network;

[0110] Acquire a plurality of operations included in each of the at least two fusible network layers;

[0111] Combining multiple operations included in each of the at least two fusion-capable network layers to obtain a fusion node;

[0112] According to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer and the type of backend hardware, the quantization implementation code of the fusion node is obtained.

[0113] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0114] For each of the at least two convergible network layers, obtain the code of each operation of the multiple operations included in the convergible network layer, and disassemble the code to obtain an abstract syntax tree corresponding to the operation. The abstract syntax trees corresponding to the multiple operations included in the convergible network layer constitute the abstract syntax tree of the convergible network layer.

[0115] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0116] According to the operation combination corresponding to the fusion node, an abstract syntax tree of the operations included in the operation combination is obtained from the abstract syntax tree of each fusionable network layer;

[0117] Parsing the abstract syntax tree of the operations included in the operation combination to obtain a code corresponding to the operation combination;

[0118] According to the code corresponding to the operation combination and the type of the backend hardware, the quantization implementation code of the fusion node is obtained.

[0119] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0120] The code corresponding to the operation combination is converted into a form adapted to the type of the backend hardware, and the conversion result is used as the quantization implementation code.

[0121] In one embodiment, the at least two fusible network layers include: an Eltwise layer and an activation layer; the multiple operations included in the Eltwise layer are: dot product, addition and subtraction of sum, and taking the maximum value max; the multiple operations included in the activation layer are: tanh, sigmoid, relu and prelu; the operation combination corresponding to the fusion node is one of the following combinations: a combination of dot product and tanh, a combination of dot product and sigmoid, a combination of dot product and relu, a combination of dot product and prelu, a combination of addition and subtraction of sum and tanh, a combination of addition and subtraction of sum and sigmoid, a combination of addition and subtraction of sum and relu, a combination of addition and subtraction of sum and prelu, a combination of taking the maximum value max and tanh, a combination of taking the maximum value max and sigmoid, a combination of taking the maximum value max and relu, or a combination of taking the maximum value max and prelu.

[0122] In one embodiment, the type of the backend hardware is one of the following types: a central processing unit CPU, a graphics processing unit GPU, or a data processing unit DPU.

[0123] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0124] Extract at least two fusion-capable network layers from the neural network;

[0125] Acquire a plurality of operations included in each of the at least two fusible network layers;

[0126] Combining multiple operations included in each of the at least two fusion-capable network layers to obtain a fusion node;

[0127] According to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer and the type of backend hardware, the quantization implementation code of the fusion node is obtained.

[0128] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0129] For each of the at least two convergible network layers, obtain the code of each operation of the multiple operations included in the convergible network layer, and disassemble the code to obtain an abstract syntax tree corresponding to the operation. The abstract syntax trees corresponding to the multiple operations included in the convergible network layer constitute the abstract syntax tree of the convergible network layer.

[0130] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0131] According to the operation combination corresponding to the fusion node, acquiring the abstract syntax tree of the operation included in the operation combination from the abstract syntax tree of each fusionable network layer;

[0132] Parsing the abstract syntax tree of the operations included in the operation combination to obtain a code corresponding to the operation combination;

[0133] According to the code corresponding to the operation combination and the type of the backend hardware, the quantization implementation code of the fusion node is obtained.

[0134] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0135] The code corresponding to the operation combination is converted into a form adapted to the type of the backend hardware, and the conversion result is used as the quantization implementation code.

[0136] In one embodiment, the at least two fusible network layers include: an Eltwise layer and an activation layer; the multiple operations included in the Eltwise layer are: dot product, addition and subtraction of sum, and taking the maximum value max; the multiple operations included in the activation layer are: tanh, sigmoid, relu and prelu; the operation combination corresponding to the fusion node is one of the following combinations: a combination of dot product and tanh, a combination of dot product and sigmoid, a combination of dot product and relu, a combination of dot product and prelu, a combination of addition and subtraction of sum and tanh, a combination of addition and subtraction of sum and sigmoid, a combination of addition and subtraction of sum and relu, a combination of addition and subtraction of sum and prelu, a combination of taking the maximum value max and tanh, a combination of taking the maximum value max and sigmoid, a combination of taking the maximum value max and relu, or a combination of taking the maximum value max and prelu.

[0137] In one embodiment, the type of the backend hardware is one of the following types: a central processing unit CPU, a graphics processing unit GPU, or a data processing unit DPU.

[0138] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0139] Extract at least two fusion-capable network layers from the neural network;

[0140] Acquire a plurality of operations included in each of the at least two fusible network layers;

[0141] Combining multiple operations included in each of the at least two fusion-capable network layers to obtain a fusion node;

[0142] According to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer and the type of backend hardware, the quantization implementation code of the fusion node is obtained.

[0143] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0144] For each of the at least two convergible network layers, obtain the code of each operation of the multiple operations included in the convergible network layer, and disassemble the code to obtain an abstract syntax tree corresponding to the operation. The abstract syntax trees corresponding to the multiple operations included in the convergible network layer constitute the abstract syntax tree of the convergible network layer.

[0145] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0146] According to the operation combination corresponding to the fusion node, an abstract syntax tree of the operations included in the operation combination is obtained from the abstract syntax tree of each fusionable network layer;

[0147] Parsing the abstract syntax tree of the operations included in the operation combination to obtain a code corresponding to the operation combination;

[0148] According to the code corresponding to the operation combination and the type of the backend hardware, the quantization implementation code of the fusion node is obtained.

[0149] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0150] The code corresponding to the operation combination is converted into a form adapted to the type of the backend hardware, and the conversion result is used as the quantization implementation code.

[0151] In one embodiment, the at least two fusible network layers include: an Eltwise layer and an activation layer; the multiple operations included in the Eltwise layer are: dot product, addition and subtraction of sum, and taking the maximum value max; the multiple operations included in the activation layer are: tanh, sigmoid, relu and prelu; the operation combination corresponding to the fusion node is one of the following combinations: a combination of dot product and tanh, a combination of dot product and sigmoid, a combination of dot product and relu, a combination of dot product and prelu, a combination of addition and subtraction of sum and tanh, a combination of addition and subtraction of sum and sigmoid, a combination of addition and subtraction of sum and relu, a combination of addition and subtraction of sum and prelu, a combination of taking the maximum value max and tanh, a combination of taking the maximum value max and sigmoid, a combination of taking the maximum value max and relu, or a combination of taking the maximum value max and prelu.

[0152] In one embodiment, the type of the backend hardware is one of the following types: a central processing unit CPU, a graphics processing unit GPU, or a data processing unit DPU.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0154] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A code generation method, characterized in that: The method comprises: Extract at least two fusion-capable network layers from the neural network; Acquire a plurality of operations included in each of the at least two fusible network layers; Combining multiple operations included in each of the at least two fusion-capable network layers to obtain a fusion node; According to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer, and the type of backend hardware, the quantized implementation code of the fusion node is obtained; Before obtaining the quantization implementation code of the fusion node according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer, and the type of backend hardware, the method further includes: For each of the at least two convergible network layers, obtain the code of each operation of the multiple operations included in the convergible network layer, and disassemble the code to obtain an abstract syntax tree corresponding to the operation, and the abstract syntax trees corresponding to the multiple operations included in the convergible network layer constitute the abstract syntax tree of the convergible network layer; The acquiring, according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer, and the type of backend hardware, the quantization implementation code of the fusion node comprises: According to the operation combination corresponding to the fusion node, an abstract syntax tree of the operations included in the operation combination is obtained from the abstract syntax tree of each fusionable network layer; Parsing the abstract syntax tree of the operations included in the operation combination to obtain a code corresponding to the operation combination; According to the code corresponding to the operation combination and the type of the backend hardware, obtaining the quantization implementation code of the fusion node; The at least two fusible network layers include: an Eltwise layer and an activation layer; the multiple operations contained in the Eltwise layer are: dot product, addition and subtraction of sum, and taking the maximum value max; the multiple operations contained in the activation layer are: tanh, sigmoid, relu and prelu; the operation combination corresponding to the fusion node is one of the following combinations: a combination of dot product and tanh, a combination of dot product and sigmoid, a combination of dot product and relu, a combination of dot product and prelu, a combination of addition and subtraction of sum and tanh, a combination of addition and subtraction of sum and sigmoid, a combination of addition and subtraction of sum and relu, a combination of addition and subtraction of sum and prelu, a combination of taking the maximum value max and tanh, a combination of taking the maximum value max and sigmoid, a combination of taking the maximum value max and relu, or a combination of taking the maximum value max and prelu.

2. The method according to claim 1, characterized in that The acquiring the quantization implementation code of the fusion node according to the code corresponding to the operation combination and the type of the backend hardware includes: The code corresponding to the operation combination is converted into a form adapted to the type of the backend hardware, and the conversion result is used as the quantization implementation code.

3. The method according to claim 1, characterized in that The type of the backend hardware is one of the following types: a central processing unit CPU, a graphics processing unit GPU or a data processing unit DPU.

4. A code generating device, characterized in that: The device comprises: An extraction module, used for extracting at least two fusion-capable network layers from a neural network; An acquisition module, used for acquiring a plurality of operations included in each of the at least two fused network layers; A fusion module, used for combining a plurality of operations included in each of the at least two fusion-capable network layers to obtain a fusion node; A generation module, used for acquiring the quantized implementation code of the fusion node according to the operation combination corresponding to the fusion node, the abstract syntax tree of each fusionable network layer and the type of backend hardware; The acquisition module is further used for: For each of the at least two convergible network layers, obtain the code of each operation of the multiple operations included in the convergible network layer, and disassemble the code to obtain an abstract syntax tree corresponding to the operation, and the abstract syntax trees corresponding to the multiple operations included in the convergible network layer constitute the abstract syntax tree of the convergible network layer; The generation module is specifically used for: According to the operation combination corresponding to the fusion node, an abstract syntax tree of the operations included in the operation combination is obtained from the abstract syntax tree of each fusionable network layer; Parsing the abstract syntax tree of the operations included in the operation combination to obtain a code corresponding to the operation combination; According to the code corresponding to the operation combination and the type of the backend hardware, obtaining the quantization implementation code of the fusion node; The at least two fusible network layers include: an Eltwise layer and an activation layer; the multiple operations contained in the Eltwise layer are: dot product, addition and subtraction of sum, and taking the maximum value max; the multiple operations contained in the activation layer are: tanh, sigmoid, relu and prelu; the operation combination corresponding to the fusion node is one of the following combinations: a combination of dot product and tanh, a combination of dot product and sigmoid, a combination of dot product and relu, a combination of dot product and prelu, a combination of addition and subtraction of sum and tanh, a combination of addition and subtraction of sum and sigmoid, a combination of addition and subtraction of sum and relu, a combination of addition and subtraction of sum and prelu, a combination of taking the maximum value max and tanh, a combination of taking the maximum value max and sigmoid, a combination of taking the maximum value max and relu, or a combination of taking the maximum value max and prelu.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images