Compilation method, device, electronic device and storage medium of neural network model

Abstract

The disclosed embodiments disclose a method, device, electronic device and storage medium for compiling a neural network model, wherein the method comprises: determining an intermediate representation corresponding to a preset network layer of the neural network model to be compiled; determining a first target layer based on the intermediate representation; processing a first input tensor and a second input tensor of the first target layer based on a preset operation to obtain a first processing layer corresponding to the first target layer; the preset operation at least comprises a convolution operation; obtaining an optimization model corresponding to the intermediate representation based on the first processing layer corresponding to the first target layer; and compiling the optimization model into a binary target model corresponding to the neural network model to be compiled. The disclosed embodiments implement element-wise maximum and element-wise minimum operations based on convolution operations, so that the element-wise maximum and element-wise minimum operations of the neural network model can be implemented through accelerator hardware, effectively improving the model reasoning speed, thereby improving data processing efficiency.

Description

Technical Field

[0001] The present disclosure relates to artificial intelligence technology, and in particular to a compilation method, device, electronic device and storage medium for a neural network model. Background Art

[0002] With the rapid development of artificial intelligence, a series of lightweight neural network models are gradually being used on mobile terminals such as mobile phones, PCs (personal computers), and in-vehicle devices to achieve machine learning tasks such as object detection and classification. Among these neural network models, some models contain element-wise maximum layers, such as element-wise maximum layers (i.e., elementwise min layers) and element-wise minimum layers (i.e., elementwise max layers). It is necessary to perform element-wise minimum or element-wise maximum on the two input tensors of the layer. In order to improve the reasoning speed of the neural network, a special acceleration chip (or accelerator) is usually used to implement the operation of the neural network. However, the existing acceleration chips usually do not support element-wise maximum operations, resulting in low model data processing efficiency. Therefore, how to implement element-wise maximum operations based on chips to further improve the reasoning speed of the neural network has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] In order to solve the technical problem that the above acceleration chip does not support the element-by-element maximum value operation, the present disclosure is proposed. The embodiments of the present disclosure provide a method and device for compiling a neural network model, an electronic device and a storage medium.

[0004] According to one aspect of an embodiment of the present disclosure, a method for compiling a neural network model is provided, including: determining an intermediate representation corresponding to a preset network layer of a neural network model to be compiled; determining a first target layer based on the intermediate representation; processing a first input tensor and a second input tensor of the first target layer based on a preset operation to obtain a first processing layer corresponding to the first target layer; the preset operation includes at least a convolution operation; based on the first processing layer corresponding to the first target layer, obtaining an optimization model corresponding to the intermediate representation; and compiling the optimization model into a binary target model corresponding to the neural network model to be compiled.

[0005] According to another aspect of an embodiment of the present disclosure, a compilation device for a neural network model is provided, including: a first determination module, used to determine an intermediate representation corresponding to a preset network layer of a neural network model to be compiled; a second determination module, used to determine a first target layer based on the intermediate representation; a first processing module, used to process a first input tensor and a second input tensor of the first target layer based on a preset operation to obtain a first processing layer corresponding to the first target layer; the preset operation at least includes a convolution operation; a second processing module, used to obtain an optimization model corresponding to the intermediate representation based on the first processing layer corresponding to the first target layer; and a compilation module, used to compile the optimization model into a binary target model corresponding to the neural network model to be compiled.

[0006] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the compilation method of the neural network model described in any of the above embodiments of the present disclosure.

[0007] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the compilation method of the neural network model described in any of the above embodiments of the present disclosure.

[0008] Based on the compilation method and device, electronic device and storage medium of the neural network model provided by the above embodiments of the present disclosure, by converting at least one layer of the network layers such as taking the maximum value and the minimum value of the element in the neural network model into a convolution operation, the model's operations such as taking the maximum value and the minimum value of the element can be implemented through accelerator hardware, effectively improving the model reasoning speed, thereby improving data processing efficiency.

[0009] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the compilation method of the neural network model provided by the present disclosure;

[0012] Figure 2 It is a flowchart of a method for compiling a neural network model provided by an exemplary embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of step 203 provided by an exemplary embodiment of the present disclosure;

[0014] Figure 4 is a schematic diagram of the principle of element-wise addition operation provided by an exemplary embodiment of the present disclosure;

[0015] Figure 5 is a flowchart of step 203 provided by another exemplary embodiment of the present disclosure;

[0016] Figure 6 is a schematic diagram of a first preset convolution kernel provided by an exemplary embodiment of the present disclosure;

[0017] Figure 7 It is a schematic diagram of the optimization principle of element-by-element maximum value operation provided by an exemplary embodiment of the present disclosure;

[0018] Figure 8 is a flowchart of step 203 provided by another exemplary embodiment of the present disclosure;

[0019] Fig. 9 It is a schematic diagram of the optimization principle of element-by-element minimum operation provided by an exemplary embodiment of the present disclosure;

[0020] Fig.10 is a flowchart of step 202 provided by an exemplary embodiment of the present disclosure;

[0021] Fig.11 is a structural diagram of a compiling device for a neural network model provided by an exemplary embodiment of the present disclosure;

[0022] Fig.12 is a structural schematic diagram of a first processing module 503 provided by an exemplary embodiment of the present disclosure;

[0023] Fig.13 is a structural schematic diagram of a first processing module 503 provided by another exemplary embodiment of the present disclosure;

[0024] Fig.14 is a structural schematic diagram of a first processing module 503 provided by another exemplary embodiment of the present disclosure;

[0025] Fig.15 It is a schematic diagram of the principle of implementing the element-by-element maximum value operation based on a convolution calculation unit including sumin provided by an exemplary embodiment of the present disclosure;

[0026] Fig.16It is a schematic diagram of the principle of implementing element-by-element minimum operation based on a convolution calculation unit including sumin provided by an exemplary embodiment of the present disclosure;

[0027] Fig.17 is a flow chart of the second determination module 502 provided by an exemplary embodiment of the present disclosure;

[0028] Fig.18 It is a structural schematic diagram of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0029] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0030] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0031] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0032] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0033] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0034] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0035] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0036] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0037] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0038] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0039] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0040] The disclosed embodiments can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0041] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0042] SUMMARY OF THE DISCLOSURE

[0043] In the process of implementing the present disclosure, the inventors found that in some neural network models, at least one of an elementwise maximum layer (elementwise min layer) and an elementwise minimum layer (elementwise max layer) is included. The elementwise maximum layer requires performing elementwise maximum operations on the two input tensors of the layer, and the elementwise minimum layer requires performing elementwise minimum operations on the two input tensors of the layer. In order to improve the reasoning speed of the neural network, a special acceleration chip is usually used to implement the operation of the neural network. However, existing acceleration chips usually do not support elementwise maximum operations and elementwise minimum operations, resulting in low model data processing efficiency.

[0044] Exemplary Overview

[0045] Figure 1 This is an exemplary application scenario of the compilation method of the neural network model provided in the present invention.

[0046] For a neural network model having an element-by-element maximum layer and / or an element-by-element minimum layer, the compilation method of the neural network model provided by the present disclosure can be used to optimize the intermediate representation of the neural network model in the compilation stage to obtain an optimized model, and the element-by-element maximum operation and / or element-by-element minimum operation of the intermediate representation can be implemented based on the convolution operation, and the convolution operation can be implemented on the accelerator, thereby effectively improving the reasoning speed of the neural network model, and solving the problem that the prior art acceleration chip does not support the element-by-element maximum operation and / or element-by-element minimum operation. The compilation of the neural network model can be implemented by a compiler, and the compiler can include a front end, a middle end, and a back end, and the front end, the middle end, and the back end respectively include different modular compiler components and tool chains, wherein the front end converts the neural network model to be compiled from a programming language into an intermediate representation, the middle end optimizes the intermediate representation to obtain an optimized model, and the back end converts the optimized model of the intermediate representation into a machine-recognizable binary target model, the target model includes a binary instruction sequence recognizable by the accelerator, and the accelerator completes the real-time reasoning of the neural network by executing these instructions. The method disclosed in the present invention is to optimize the intermediate representation in the mid-end optimization process, and then convert the optimized model into a binary target model through the back-end. The specific working principle of the front-end is not disclosed and will not be elaborated.

[0047] Exemplary Methods

[0048] Figure 2 is a flowchart of a method for compiling a neural network model provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as an in-vehicle computing platform, such as Figure 2 As shown, the following steps are included:

[0049] Step 201, determine the intermediate representation corresponding to the preset network layer of the neural network model to be compiled.

[0050] Among them, the neural network model to be compiled is a neural network model with at least one element-by-element maximum layer and / or at least one element-by-element minimum layer. The preset network layer can be any network layer in the neural network model to be compiled, such as a convolutional layer, a pooling layer, an element-by-element maximum layer, an element-by-element minimum layer, etc. The preset network layer may also only refer to the element-by-element maximum layer and the element-by-element minimum layer in the neural network model to be compiled. It can be specifically set according to actual needs, and the present disclosure does not limit it. For example, if the neural network model to be compiled includes N element-by-element maximum layers, the preset network layer may refer to the N element-by-element maximum layers. For another example, if the neural network model to be compiled includes N element-by-element maximum layers and M element-by-element minimum layers, the preset network layer may refer to the N element-by-element maximum layers and the M element-by-element minimum layers. Determining the intermediate representation corresponding to the neural network model to be compiled at the preset network layer refers to converting the preset network layer from the programming language into another intermediate representation (IR) with a complexity between the programming language and the binary code (machine code), which can also be called intermediate code. The intermediate code is an internal representation of the source program, and its function is to make the structure of the compiler program logically simpler and clearer, and easier to optimize. Different compilers have different intermediate codes. For example, the TVM (Tensor Virtual Machine) compiler compiles the neural network model to be deployed into Relay intermediate code. Relay is a versatile programming language that can be used for the intermediate representation of machine learning system expression. In this way, model files under different front-end frameworks can be compiled into the intermediate Relay code of the TVM compiler to achieve the unification of different front-end frameworks, thereby avoiding the complex process of cross-compiling codes of different front-end network frameworks, reducing workload, and improving model deployment efficiency. For example, the language of the intermediate Relay code of the TVM compiler can be regarded as the "intermediate representation" described in the present disclosure. The present disclosure does not limit the specific compiler, so the specific language of the intermediate representation is not limited to the Relay language. The present disclosure does not limit the specific implementable computer-recognizable language.

[0051] Step 202: determine a first target layer based on the intermediate representation.

[0052] Among them, the first target layer can be an intermediate representation corresponding to at least one of the element-by-element maximum layer and the element-by-element minimum layer that need to be subsequently optimized. For example, the intermediate representation includes the intermediate representation corresponding to any network layer in the neural network model to be compiled, that is, the preset network layer includes any network layer in the neural network model to be compiled, then it is necessary to determine at least one element-by-element maximum layer and / or at least one element-by-element minimum layer that meets the preset rules from the layers of the intermediate representation as the first target layer. For another example, the preset network layer refers to N element-by-element maximum layers, then at least one of the N element-by-element maximum layers can be used as the first target layer according to actual needs, and the specific setting can be based on actual needs.

[0053] Step 203: Based on a preset operation, the first input tensor and the second input tensor of the first target layer are processed to obtain a first processing layer corresponding to the first target layer; the preset operation at least includes a convolution operation.

[0054] Among them, the preset operation at least includes a convolution operation, that is, the element-by-element maximum operation or the element-by-element minimum operation of the first target layer is implemented based on the convolution operation, so that the operation of the first target layer can be implemented through the accelerator hardware, effectively improving the model reasoning speed. The first input tensor and the second input tensor are two input tensors of the first target layer, and the first target layer obtains the corresponding first output tensor by taking the element-by-element maximum value or the element-by-element minimum value between the first input tensor and the second input tensor. The obtained first processing layer achieves the purpose of the first target layer based on the preset operation that at least includes the convolution operation, that is, the two input tensors of the first processing layer are still the first input tensor and the second input tensor, and the second output tensor obtained by the first processing layer is the same as the first output tensor of the first target layer. It’s just that the specific operation process of the first processing layer is different from that of the first target layer.

[0055] Step 204: Based on the first processing layer corresponding to the first target layer, an optimization model corresponding to the intermediate representation is obtained.

[0056] Specifically, obtaining the first processing layer corresponding to the first target layer realizes the optimization of the first target layer. In practical applications, other layers can also be optimized according to actual needs. The present disclosure does not limit this. The optimization model corresponding to the intermediate representation may include the optimization layer corresponding to the corresponding layer in the intermediate representation, and may also include the optimization layer corresponding to other layers. It can be set according to actual needs.

[0057] Step 205, compile the optimized model into a binary target model corresponding to the neural network model to be compiled.

[0058] Specifically, after obtaining the optimized model, the optimized model can be compiled into a binary target model corresponding to the neural network model to be compiled through the compiler backend. The specific transformation method from the optimized model to the target model can be implemented in any feasible way according to actual needs, and this disclosure will not go into details.

[0059] The compilation method of the neural network model provided in this embodiment, by taking at least one layer of the network layers such as taking the maximum value and the minimum value of the element in the neural network model as the first target layer, converting the first target layer into a first processing layer based on convolution operation, so that the model's operations such as taking the maximum value and the minimum value of the element can be implemented through accelerator hardware, effectively improving the model reasoning speed, thereby improving data processing efficiency.

[0060] In an alternative example, Figure 3 203 is a flowchart of step 203 provided by an exemplary embodiment of the present disclosure. In this example, the preset operation includes a first convolution operation, a first element-wise addition operation, a second convolution operation, and a second element-wise addition operation. Step 203 may specifically include the following steps:

[0061] Step 2031: perform a first convolution operation on the second input tensor to obtain a first result.

[0062] Among them, the first convolution operation can be implemented based on a preset convolution kernel and a preset bias parameter, wherein the preset convolution kernel can be set according to actual needs, for example, the preset convolution kernel can be a 1*1 convolution kernel with a weight value of -1, and the preset bias parameter can be determined according to the value range of the numerical type to which the characteristic element of the second input tensor belongs. Specifically, for example, the preset bias parameter can be -HALF, HALF = (L_MAX-L_MIN+1) / 2, wherein L represents the numerical type, L_MAX and L_MIN respectively represent the maximum and minimum values ​​of the value range of the numerical type, such as the numerical type of the characteristic element is signed int8, then the value range is -128 to 127, and there can be a total of 256 values, L_MAX = 127, L_MIN = -128, then the determined preset bias parameter is -256 / 2 = -128, and different numerical types, if the value range is different, the preset bias parameter is different. The first result obtained by the first convolution operation is a tensor of the same size as the second input tensor.

[0063] Optionally, the preset convolution kernel can be any m*m convolution kernel, where m is an integer greater than or equal to 1, that is, the size of the preset convolution kernel is not limited, as long as there is a weight value of -1 and the other weight values ​​are 0, the specific padding can be set according to actual needs, as long as it can ensure the same function as the 1*1 convolution kernel with a weight value of -1, the specific preset convolution kernel can be set according to actual needs. For example, if m=3, the center weight value is -1 and the other weight values ​​are 0, then the padding is 1, ensuring that each element of the second input tensor is multiplied by -1. For another example, if m=2 and the upper left corner weight value is -1, then the bottom and right sides of the second input tensor need to be padded to ensure that the last row and last column of the second input tensor are multiplied by -1 during convolution. The specific padding principle will not be repeated here.

[0064] For the same numerical type, the preset convolution kernel and preset bias parameters corresponding to the element-by-element maximum operation and element-by-element minimum operation optimization are the same.

[0065] Step 2032: Perform a first element-wise addition operation on the first result and the first input tensor to obtain a second result.

[0066] The first element-by-element addition operation refers to adding the characteristic element of the first result to the characteristic element of the corresponding position of the first input tensor as the characteristic element of the position in the second result.

[0067] For example, Figure 4 This is a schematic diagram of the principle of element-wise addition operation provided by an exemplary embodiment of the present disclosure. Tensors A and B are added element by element, and the characteristic element at any position in tensor C (for example, the first characteristic element 21) is the sum of the characteristic element at that position in tensor A (i.e., 12) and the characteristic element at that position in tensor B (i.e., 9). This example only shows the principle of one channel, and the principle of any channel is the same as this example, which will not be repeated here.

[0068] Step 2033: perform a second convolution operation on the second result to obtain a third result.

[0069] Among them, the second convolution operation can set the corresponding convolution kernel and bias parameters according to actual needs. The specific principle is similar to step 2031 and will not be repeated here. For the optimization of element-by-element maximum operation and element-by-element minimum operation, the convolution kernel weight and bias parameters in this step are different, which are specifically set according to the actual operation.

[0070] For example, for the element-by-element maximum operation, during optimization, the convolution kernel of the second convolution operation is 1*1, the weight value is 1, and the bias parameter is HALF, where HALF represents half of the number of values ​​in the value range of the feature element value type. For the element-by-element minimum operation, during optimization, the convolution kernel of the second convolution operation is 1*1, the weight value is -1, and the bias parameter is -HALF. The specific settings can be made according to actual needs.

[0071] Optionally, the convolution kernel of the second convolution operation can also be an m*m convolution kernel, where m is an integer greater than 1, any one of the weight values ​​is 1 (when the element is taken as the maximum value) or -1 (when the element is taken as the minimum value), and the other weight values ​​are 0. The padding is set according to actual needs. As long as it can ensure that each characteristic element of the second result is multiplied by 1 or -1, it can be set according to actual needs, and the specific principle will not be repeated here.

[0072] Step 2034: perform a second element-wise addition operation on the third result and the second input tensor or the first input tensor to obtain a fourth result.

[0073] The principle of element-wise addition operation is as mentioned above and will not be described again here.

[0074] In this step, for the element-by-element maximum operation and the element-by-element minimum operation, the element-by-element maximum operation is different from the object of element-by-element addition of the third result. The element-by-element minimum operation is the second element-by-element addition operation of the third result and the second input tensor, and the element-by-element minimum operation is the second element-by-element addition operation of the third result and the first input tensor. The specific settings can be based on actual needs.

[0075] Step 2035: Use the fourth result as the output result corresponding to the first target layer to obtain the first processing layer corresponding to the first target layer.

[0076] The operation of the above-mentioned steps 2031-2034 makes the fourth result obtained the same as the original output result of the first target layer, so that the fourth result can be used as the output result corresponding to the first target layer, and then the processing process including the above-mentioned first convolution operation, first element-by-element addition operation, second convolution operation and second element-by-element addition operation is used as the first processing layer obtained by optimizing the first target layer. The first processing layer realizes the element-by-element maximum operation or element-by-element minimum operation of the first target layer based on the convolution operation combined with the element-by-element addition operation.

[0077] The present invention combines convolution operation with element-by-element addition operation to realize element-by-element maximum operation and / or element-by-element minimum operation, so that the element-by-element maximum operation and / or element-by-element minimum operation in the model can be realized by hardware on the acceleration chip, effectively improving the model reasoning speed.

[0078] In an alternative example, Figure 5 is a flowchart of step 203 provided by another exemplary embodiment of the present disclosure. In this example, the first target layer is an element-by-element maximum value layer; based on a preset operation, step 203 processes the first input tensor and the second input tensor of the first target layer to obtain a first processing layer corresponding to the first target layer, including:

[0079] Step 2031a, performing a convolution operation on the second input tensor based on the first preset convolution kernel and the first bias parameter to obtain a first tensor corresponding to the second input tensor; the first bias parameter is determined according to the value range of the numerical type to which the characteristic element of the second input tensor belongs and the first rule.

[0080] Among them, the first preset convolution kernel can be a 1*1 convolution kernel with a weight value of -1, or it can be an m*m convolution kernel, any of which has a weight value of -1 and the other weight values ​​are 0. Padding is set according to actual needs, as long as it can ensure that each characteristic element of the second input tensor is -1. For specific principles, please refer to the above content. The first bias parameter is determined based on the number of values ​​included in the value range of the numerical type to which the characteristic element of the second input tensor belongs. Please refer to the above content for details. For example, the number of values ​​included in the value range of the numerical type is n, such as the number of values ​​included in the value range of signed int8 is 256, then the first bias parameter can be set to -HALF(n), HALF(n) means half of n, which can be referred to as HALF, such as n=256, HALF=128.

[0081] For example, Figure 6 is a schematic diagram of a first preset convolution kernel provided by an exemplary embodiment of the present disclosure. The first preset convolution kernel is a 3*3 convolution kernel with a padding of 1, a center weight value of -1, and other weight values ​​of 0.

[0082] Step 2032a, performing element-wise addition operation on the first tensor and the first input tensor to obtain a second tensor.

[0083] The principle of element-wise addition operation is as mentioned above and will not be described again here.

[0084] Step 2033a, performing a convolution operation on the second tensor based on a second preset convolution kernel and a second bias parameter to obtain a third tensor corresponding to the second tensor, wherein the second bias parameter is determined according to a value range of a numerical type to which a characteristic element of the second input tensor belongs and a second rule.

[0085] Among them, the second preset convolution kernel can be an m*m convolution kernel, m is a positive integer, any weight value of the second preset convolution kernel is 1, and the other weight values ​​are 0, and padding is set according to actual needs, as long as it can ensure that each characteristic element of the second tensor is multiplied by 1. The specific principle can be found in the above content and will not be repeated here. The second rule can be set according to actual needs, for example, the second bias parameter is HALF, HALF is half of the number of values ​​included in the value range corresponding to the numerical type, such as HALF = 128.

[0086] Step 2034a, perform element-wise addition operation on the third tensor and the second input tensor to obtain a fourth tensor.

[0087] The principle of element-wise addition operation is as mentioned above and will not be described again here.

[0088] Step 2035a, using the fourth tensor as the result of taking the element-wise maximum value of the first input tensor and the second input tensor, to obtain the first processing layer corresponding to the first target layer.

[0089] The specific operation of this step refers to the aforementioned step 2035, which will not be described in detail.

[0090] Among them, the result of the element-by-element addition in the above steps 2032a and 2034a, when the characteristic element overflows the value range of the numerical type to which the characteristic element belongs, the limit value (or saturation value) of the value range at the overflow end is used as the result of the operation. That is, the values ​​of the characteristic elements in the second tensor and the fourth tensor are all within the value range corresponding to the numerical type. Overflow means that the operation result exceeds the value range corresponding to the numerical type, such as the value range of signed int8 is -128 to 127. When the operation result is greater than 127 or less than -128, it means that the operation result overflows the value range. Saturation means that when overflow occurs, the operation result needs to be adjusted to the limit value of the value range. For example, when it is greater than 127, the operation result is determined to be 127, and when it is less than -128, the operation result is determined to be -128. In view of the above situation, in actual application, the bit width supported in the calculation process of steps 2031a-2032a is greater than the bit width of the second tensor value type to ensure that there is no overflow and saturation during the calculation process, and only the calculation result of step 2032a is overflowed and saturated.

[0091] The specific operation principle of the above steps 2031a-2035a can be expressed as:

[0092] F3=F1-F2-HALF

[0093] F0=F2+F3+HALF

[0094] Among them, F1 represents the first input tensor, F2 represents the second input tensor, F3 represents the second tensor, and F0 represents the fourth tensor.

[0095] For example, Figure 7 This is a schematic diagram of the optimization principle of the element-by-element maximum operation provided by an exemplary embodiment of the present disclosure. This example takes the numerical type of signed int8 as an example. The first output result obtained by taking the maximum value of tensor D (first input tensor) and tensor E (second input tensor) by element is tensor F (first output tensor). The optimized operation process is to perform a convolution operation on tensor E to obtain tensor G (first tensor), add tensor G and tensor D by elements to obtain tensor H (second tensor), and the obtained tensor H is the result after overflow saturation. For example, the actual result of adding element -143 in tensor G and element 6 in tensor D is -137, which is -128 after overflow saturation. The tensor H is convoluted to obtain tensor I (third tensor), and tensor I and tensor E are added by elements to obtain tensor J (fourth tensor). It can be seen that tensor J is the same as tensor F, so the element-by-element maximum operation is realized based on convolution operation, element-by-element addition operation and overflow saturation.

[0096] The present invention implements element-by-element maximum value operation through a first preset convolution kernel and a corresponding first bias parameter, a second preset convolution kernel and a corresponding second bias parameter, combined with element-by-element addition operation and saturation value, so that the element-by-element maximum value operation of the model can be implemented through the hardware of the acceleration chip, effectively improving the reasoning speed of the model.

[0097] In an alternative example, Figure 8 203 is a flowchart of step 203 provided by another exemplary embodiment of the present disclosure. In this example, the first target layer is an element-by-element minimum layer. Step 203 processes the first input tensor and the second input tensor of the first target layer based on a preset operation to obtain a first processing layer corresponding to the first target layer, including:

[0098] Step 2031b, performing a convolution operation on the second input tensor based on the third preset convolution kernel and the third bias parameter to obtain a fifth tensor corresponding to the second input tensor; the third bias parameter is determined according to the value range of the numerical type to which the characteristic element of the second input tensor belongs and the third rule.

[0099] Among them, the third preset convolution kernel can be an m*m convolution kernel, m is a positive integer, any weight value of the third preset convolution kernel is -1, and the other weight values ​​are 0, and padding is set according to actual needs, as long as it can ensure that each characteristic element of the second input tensor is multiplied by -1. The specific principle can be found in the above content, which will not be repeated here. The third rule for determining the third bias parameter can be set according to actual needs. For example, the third bias parameter is -HALF, and HALF is half of the number of values ​​included in the value range corresponding to the numerical type. No more details are given.

[0100] Step 2032b, perform element-wise addition operation on the fifth tensor and the first input tensor to obtain a sixth tensor.

[0101] The principle of element-wise addition operation is as mentioned above and will not be described here. When the characteristic element of the result of element-wise addition of the fifth tensor and the first input tensor overflows, the limit value of the value range of the overflow end is used as the characteristic element of the sixth tensor.

[0102] Step 2033b, performing a convolution operation on the sixth tensor based on the fourth preset convolution kernel and the fourth bias parameter to obtain a seventh tensor corresponding to the sixth tensor; the fourth bias parameter is determined according to the value range of the numerical type to which the characteristic element of the second input tensor belongs and the fourth rule.

[0103] Among them, the fourth preset convolution kernel can be an m*m convolution kernel, m is a positive integer, any weight value of the fourth preset convolution kernel is -1, and the other weight values ​​are 0, and padding is set according to actual needs, as long as it can ensure that each characteristic element of the sixth tensor is multiplied by -1. The specific principle can be found in the above content, which will not be repeated here. The fourth rule for determining the fourth bias parameter can be set according to actual needs. For example, the fourth bias parameter can be -HALF, and HALF is half of the number of values ​​included in the value range corresponding to the numerical type. No more details are given.

[0104] Step 2034b, perform element-wise addition operation on the seventh tensor and the first input tensor to obtain the eighth tensor.

[0105] The principle of element-wise addition operation is as mentioned above and will not be described again here.

[0106] Step 2035b, using the eighth tensor as the result of taking the element-wise minimum of the first input tensor and the second input tensor, to obtain the first processing layer corresponding to the first target layer.

[0107] For specific operations, please refer to the aforementioned step 2035, which will not be repeated here.

[0108] Among them, in the calculation result of step 2034b, when the characteristic element overflows the value range of the numerical type to which the characteristic element belongs, the boundary value of the value range at the overflow end is used as the calculation result.

[0109] Regarding overflow, please refer to the above content and will not be repeated here.

[0110] Exemplarily, the specific operation principle of the above steps 2031b-2035b can be expressed as:

[0111] F3=F1-F2-HALF

[0112] F0=F1-F3-HALF

[0113] Among them, F1 represents the first input tensor, F2 represents the second input tensor, F3 represents the sixth tensor, and F0 represents the eighth tensor.

[0114] For example, Fig. 9 This is a schematic diagram of the optimization principle of the element-by-element minimum operation provided by an exemplary embodiment of the present disclosure. This example takes the numerical type of signed int8 as an example. The first output result obtained by taking the minimum value of tensor D (the first input tensor) and tensor E (the second input tensor) by element is tensor F (the first output tensor). The optimized operation process is to perform a convolution operation on tensor E to obtain tensor G (the fifth tensor), add tensor G and tensor D by elements, and obtain tensor H (the sixth tensor). The obtained tensor H is the result after overflow saturation. For example, the actual result of adding element -122 in tensor G and element -15 in tensor D is -137, which is -128 after overflow saturation. The tensor H is convoluted to obtain tensor I (the seventh tensor), and tensor I and tensor D are added by elements to obtain tensor J (the eighth tensor). It can be seen that tensor J is the same as tensor F, so the element-by-element minimum operation is realized based on convolution operation, element-by-element addition operation and overflow saturation.

[0115] In an optional example, for a computing unit having a convolution operation with an element-by-element addition function, a first convolution operation and a first element-by-element addition operation are fused into a first comprehensive operation; and a second convolution operation and a second element-by-element addition operation are fused into a second comprehensive operation.

[0116] Specifically, some neural network accelerators have a convolution calculation unit containing sumin, which can realize the convolution and element-wise addition functions. In these accelerators, it is equivalent to integrating the element-wise addition operation into the convolution. For such accelerators, when accelerating the reasoning of the model, the first convolution operation and the first element-wise addition operation can be merged into the first comprehensive operation, that is, the tensor to be convolved and the tensor to be element-wise added with the convolution result are input into the convolution calculation unit containing sumin together, and the output result is the result after the convolution operation and the element-wise addition operation. For example, the second input tensor that needs convolution operation in the above-mentioned step 2031a and the first input tensor that needs to be element-wise added with the convolution result (first tensor) of the second input tensor in step 2032a can be input into the convolution calculation unit containing sumin, wherein the first input tensor is input through sumin, and the second tensor can be obtained by correspondingly configuring the calculation unit, and the specific principle will not be repeated. For the convolution calculation unit containing sumin, since the element-by-element addition operation is included in the convolution, the bit width supported by its internal operation is larger than the bit width of the numerical type of the first input tensor, and the output bit width is the same as the numerical type, so as to realize the aforementioned element-by-element maximum value and element-by-element minimum value functions.

[0117] In an alternative example, Fig.10 2 is a flow chart of step 202 provided by an exemplary embodiment of the present disclosure. In this example, determining the first target layer based on the intermediate representation in step 202 includes:

[0118] Step 2021, traverse each layer of the intermediate representation, take each layer as the second target layer, and obtain the operation parameters of the second target layer.

[0119] Among them, the operation parameters can be relevant parameters corresponding to the second target layer, such as max, min, or elementwise max, elementwise min, etc., which are used to represent the parameters of element-by-element maximum or minimum operations or relevant parameters of other layers, etc. They can be set according to actual needs. As long as they can be used to determine whether it is an element-by-element maximum value layer or an element-by-element minimum value layer, the present disclosure does not limit them.

[0120] Step 2022: The second target layer whose operating parameters satisfy the preset rules is used as the first target layer.

[0121] The preset rule is a rule for determining whether the operation parameters satisfy the characteristics of the element-by-element maximum layer or the element-by-element minimum layer to be optimized. It can be set according to actual needs, for example, it can be a matching rule, and a keyword that can determine the element-by-element maximum layer or the element-by-element minimum layer is preset, and the operation parameters of the second target layer are matched with the keywords to determine whether the second target layer satisfies the preset rule, which is not specifically limited.

[0122] Any method for compiling a neural network model provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any method for compiling a neural network model provided in the embodiments of the present disclosure may be executed by a processor, such as a processor executing any method for compiling a neural network model mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.

[0123] Exemplary Devices

[0124] Fig.11 is a schematic diagram of the structure of a neural network model compilation device provided by an exemplary embodiment of the present disclosure. The device of this embodiment can be used to implement the corresponding method embodiments of the present disclosure, such as Fig.11 The device shown includes: a first determining module 501 , a second determining module 502 , a first processing module 503 , a second processing module 504 and a compiling module 505 .

[0125] The first determination module 501 is used to determine the intermediate representation corresponding to the preset network layer of the neural network model to be compiled; the second determination module 502 is used to determine the first target layer based on the intermediate representation determined by the first determination module 501; the first processing module 503 is used to process the first input tensor and the second input tensor of the first target layer determined by the second determination module 502 based on a preset operation to obtain a first processing layer corresponding to the first target layer; the preset operation at least includes a convolution operation; the second processing module 504 is used to obtain an optimization model corresponding to the intermediate representation based on the first processing layer corresponding to the first target layer obtained by the first processing module 503; the compilation module 505 is used to compile the optimization model obtained by the second processing module 504 into a binary target model corresponding to the neural network model to be compiled.

[0126] In an alternative example, Fig.12 1 is a schematic diagram of the structure of the first processing module 503 provided by an exemplary embodiment of the present disclosure. In this example, the preset operation includes a first convolution operation, a first element-wise addition operation, a second convolution operation, and a second element-wise addition operation; the first processing module 503 includes: a first operation unit 5031, a second operation unit 5032, a third operation unit 5033, a fourth operation unit 5034, and a first processing unit 5035.

[0127] The first operation unit 5031 is used to perform a first convolution operation on the second input tensor to obtain a first result; the second operation unit 5032 is used to perform a first element-wise addition operation on the first result obtained by the first operation unit 5031 and the first input tensor to obtain a second result; the third operation unit 5033 is used to perform a second convolution operation on the second result obtained by the second operation unit 5032 to obtain a third result; the fourth operation unit 5034 is used to perform a second element-wise addition operation on the third result obtained by the third operation unit 5033 and the second input tensor to obtain a fourth result; the first processing unit 5035 is used to use the fourth result obtained by the fourth operation unit 5034 as the output result corresponding to the first target layer, and obtain the first processing layer corresponding to the first target layer.

[0128] Fig.13 1 is a schematic diagram of the structure of the first processing module 503 provided by another exemplary embodiment of the present disclosure. In an optional example, the first target layer is an element-by-element maximum layer; the first processing module 503 includes: a first convolution operation unit 5031a, a first addition operation unit 5032a, a second convolution operation unit 5033a, a second addition operation unit 5034a and a first processing unit 5035a.

[0129] The first convolution operation unit 5031a is used to perform a convolution operation on the second input tensor based on the first preset convolution kernel and the first bias parameter to obtain a first tensor corresponding to the second input tensor; the first bias parameter is determined according to a value range of a numerical type to which a characteristic element of the second input tensor belongs and a first rule;

[0130] A first addition operation unit 5032a, configured to perform an element-wise addition operation on the first tensor obtained by the first convolution operation unit 5031a and the first input tensor to obtain a second tensor;

[0131] A second convolution operation unit 5033a is used to perform a convolution operation on the second tensor obtained by the first addition operation unit 5032a based on a second preset convolution kernel and a second bias parameter to obtain a third tensor corresponding to the second tensor, wherein the second bias parameter is determined according to a value range of a numerical type to which a characteristic element of the second input tensor belongs and a second rule;

[0132] A second addition operation unit 5034a is used to perform an element-wise addition operation on the third tensor obtained by the second convolution operation unit 5033a and the second input tensor to obtain a fourth tensor;

[0133] The first processing unit 5035a is used to use the fourth tensor obtained by the second addition operation unit 5034a as the result of taking the element-wise maximum value of the first input tensor and the second input tensor to obtain the first processing layer corresponding to the first target layer.

[0134] In an alternative example, Fig.14 1 is a schematic diagram of the structure of the first processing module 503 provided by another exemplary embodiment of the present disclosure. In this example, the first target layer is an element-by-element minimum layer; the first processing module 503 includes: a third convolution operation unit 5031b, a third addition operation unit 5032b, a fourth convolution operation unit 5033b, a fourth addition operation unit 5034b and a second processing unit 5035b.

[0135] The third convolution operation unit 5031b is used to perform a convolution operation on the second input tensor based on a third preset convolution kernel and a third bias parameter to obtain a fifth tensor corresponding to the second input tensor; the third bias parameter is determined according to the value range of the numerical type to which the characteristic element of the second input tensor belongs and the third rule; the third addition operation unit 5032b is used to perform an element-by-element addition operation on the fifth tensor obtained by the third convolution operation unit 5031b and the first input tensor to obtain a sixth tensor; the fourth convolution operation unit 5033b is used to perform an element-by-element addition operation on the third addition operation unit 5032 based on a fourth preset convolution kernel and a fourth bias parameter. b performs a convolution operation on the sixth tensor obtained to obtain a seventh tensor corresponding to the sixth tensor; the fourth bias parameter is determined according to the value range of the numerical type to which the characteristic elements of the second input tensor belong and the fourth rule; the fourth addition operation unit 5034b is used to perform an element-by-element addition operation on the seventh tensor obtained by the fourth convolution operation unit 5033b and the first input tensor to obtain an eighth tensor; the second processing unit 5035b is used to use the eighth tensor obtained by the fourth addition operation unit 5034b as the result of taking the element-by-element minimum of the first input tensor and the second input tensor to obtain a first processing layer corresponding to the first target layer.

[0136] In an optional example, for a computing unit having a convolution operation with an element-by-element addition function, a first convolution operation and a first element-by-element addition operation are fused into a first comprehensive operation; and a second convolution operation and a second element-by-element addition operation are fused into a second comprehensive operation.

[0137] For example, Fig.15This is a schematic diagram of the principle of implementing the element-by-element maximum value operation based on a convolution calculation unit containing sumin provided by an exemplary embodiment of the present disclosure. In which, kernel(-1) represents the first preset convolution kernel, the weight value is -1, bias1 represents the first bias parameter, conv1 and conv2 represent the convolution calculation unit containing sumin, kernel(+1) represents the second preset convolution kernel, bias2 represents the second bias parameter, F1 represents the first input tensor, F2 represents the second input tensor, F3 represents the second tensor, and F0 represents the fourth tensor.

[0138] For example, Fig.16 This is a schematic diagram of the principle of implementing the element-by-element minimum operation based on a convolution calculation unit including sumin provided by an exemplary embodiment of the present disclosure. Among them, the kernel(-1) above represents the third preset convolution kernel, the weight value is -1, bias3 represents the third bias parameter, the kernel(-1) below represents the fourth preset convolution kernel, the weight value is -1, bias4 represents the fourth bias parameter, conv3 and conv4 represent convolution calculation units including sumin, F1 represents the first input tensor, F2 represents the second input tensor, F3 represents the sixth tensor, and F0 represents the eighth tensor.

[0139] In an alternative example, Fig.17 : is a flow chart of the second determination module 502 provided by an exemplary embodiment of the present disclosure. In this example, the second determination module 502 includes:

[0140] The first acquisition unit 5021 is used to traverse each layer of the intermediate representation, take each layer as the second target layer, and obtain the operation parameters of the second target layer; the third processing unit 5022 is used to take the second target layer whose operation parameters obtained by the first acquisition unit 5021 meet the preset rules as the first target layer.

[0141] Exemplary Electronic Devices

[0142] The embodiment of the present disclosure also provides an electronic device, including: a memory for storing a computer program;

[0143] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method described in any of the above embodiments of the present disclosure is implemented.

[0144] Fig.18 FIG. 1 is a schematic diagram of a structure of an application embodiment of the electronic device disclosed in the present invention. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0145] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0146] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the methods of the various embodiments of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0147] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0148] For example, the input device 13 may be the microphone or microphone array mentioned above, which is used to capture the input signal of the sound source.

[0149] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0150] The output device 14 can output various information to the outside, including the determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0151] Of course, to simplify, Fig.18 Only some of the components related to the present disclosure in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0152] Exemplary computer program products and computer-readable storage media

[0153] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0154] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0155] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0156] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0157] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0158] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0159] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0160] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0161] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0162] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0163] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for compiling a neural network model, comprising: Determine the intermediate representation corresponding to the preset network layer of the neural network model to be compiled; Based on the intermediate representation, determining a first target layer; Based on a preset operation, a first input tensor and a second input tensor of the first target layer are processed to obtain a first processing layer corresponding to the first target layer; The preset operation at least includes a convolution operation; Based on the first processing layer corresponding to the first target layer, obtaining an optimization model corresponding to the intermediate representation; Compiling the optimized model into a binary target model corresponding to the neural network model to be compiled; Wherein, determining the first target layer based on the intermediate representation includes: Traversing each layer of the intermediate representation, taking each layer as a second target layer, and obtaining an operation parameter of the second target layer; The second target layer whose operation parameters satisfy preset rules is used as the first target layer.

2. The method according to claim 1, wherein: The preset operation includes a first convolution operation, a first element-wise addition operation, a second convolution operation, and a second element-wise addition operation; The step of processing the first input tensor and the second input tensor of the first target layer based on the preset operation to obtain a first processing layer corresponding to the first target layer includes: Performing a first convolution operation on the second input tensor to obtain a first result; Performing a first element-wise addition operation on the first result and the first input tensor to obtain a second result; Performing a second convolution operation on the second result to obtain a third result; Performing a second element-wise addition operation on the third result and the second input tensor or the first input tensor to obtain a fourth result; The fourth result is used as the output result corresponding to the first target layer to obtain the first processing layer corresponding to the first target layer.

3. The method according to claim 1, wherein: The first target layer is an element-wise maximum value layer; The step of processing the first input tensor and the second input tensor of the first target layer based on the preset operation to obtain a first processing layer corresponding to the first target layer includes: Performing a convolution operation on the second input tensor based on a first preset convolution kernel and a first bias parameter to obtain a first tensor corresponding to the second input tensor; the first bias parameter is determined according to a value range of a numerical type to which a characteristic element of the second input tensor belongs and a first rule; Performing an element-wise addition operation on the first tensor and the first input tensor to obtain a second tensor; Performing a convolution operation on the second tensor based on a second preset convolution kernel and a second bias parameter to obtain a third tensor corresponding to the second tensor, wherein the second bias parameter is determined according to a value range of a numerical type to which a characteristic element of the second input tensor belongs and a second rule; Performing an element-wise addition operation on the third tensor and the second input tensor to obtain a fourth tensor; The fourth tensor is used as the result of taking the element-wise maximum value of the first input tensor and the second input tensor to obtain the first processing layer corresponding to the first target layer.

4. The method according to claim 1, wherein: The first target layer is an element-by-element minimum layer; and based on a preset operation, the first input tensor and the second input tensor of the first target layer are processed to obtain a first processing layer corresponding to the first target layer, including: Performing a convolution operation on the second input tensor based on a third preset convolution kernel and a third bias parameter to obtain a fifth tensor corresponding to the second input tensor; the third bias parameter is determined according to a value range of a numerical type to which a characteristic element of the second input tensor belongs and a third rule; Performing an element-wise addition operation on the fifth tensor and the first input tensor to obtain a sixth tensor; Performing a convolution operation on the sixth tensor based on a fourth preset convolution kernel and a fourth bias parameter to obtain a seventh tensor corresponding to the sixth tensor; the fourth bias parameter is determined according to a value range of a numerical type to which a characteristic element of the second input tensor belongs and a fourth rule; Performing an element-wise addition operation on the seventh tensor and the first input tensor to obtain an eighth tensor; The eighth tensor is used as the result of taking the element-wise minimum of the first input tensor and the second input tensor to obtain the first processing layer corresponding to the first target layer.

5. The method according to claim 2, wherein: For a computing unit having a convolution operation with an element-wise addition function, the first convolution operation and the first element-wise addition operation are fused into a first comprehensive operation; and the second convolution operation and the second element-wise addition operation are fused into a second comprehensive operation.

6. A compiling device for a neural network model, comprising: A first determination module is used to determine the intermediate representation corresponding to the preset network layer of the neural network model to be compiled; A second determination module, configured to determine a first target layer based on the intermediate representation; A first processing module, configured to process the first input tensor and the second input tensor of the first target layer based on a preset operation to obtain a first processing layer corresponding to the first target layer; The preset operation at least includes a convolution operation; A second processing module, configured to obtain an optimization model corresponding to the intermediate representation based on the first processing layer corresponding to the first target layer; A compiling module, used to compile the optimization model into a binary target model corresponding to the neural network model to be compiled; The second determining module comprises: A first acquisition unit, configured to traverse each layer of the intermediate representation, take each layer as a second target layer, and acquire an operation parameter of the second target layer; The third processing unit is used to use the second target layer whose operating parameters meet the preset rules as the first target layer.

7. The device according to claim 6, wherein: The preset operation includes a first convolution operation, a first element-wise addition operation, a second convolution operation, and a second element-wise addition operation; The first processing module comprises: A first operation unit, configured to perform a first convolution operation on the second input tensor to obtain a first result; A second operation unit, configured to perform a first element-wise addition operation on the first result and the first input tensor to obtain a second result; A third operation unit, configured to perform a second convolution operation on the second result to obtain a third result; a fourth operation unit, configured to perform a second element-wise addition operation on the third result and the second input tensor to obtain a fourth result; The first processing unit is used to use the fourth result as the output result corresponding to the first target layer to obtain the first processing layer corresponding to the first target layer.

8. A computer-readable storage medium storing a computer program for executing the method for compiling a neural network model according to any one of claims 1 to 5.

9. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the compilation method of the neural network model described in any one of claims 1-5 above.

Images