Compilation method and device of neural network model, electronic device and storage medium

Abstract

This disclosure provides a method, apparatus, electronic device, and storage medium for compiling a neural network model. The method includes: for a first pointwise convolutional layer in at least one pointwise convolutional layer of a first model corresponding to the neural network model to be compiled, performing a data structure transformation on the original input tensor of the first pointwise convolutional layer based on a preset tensor folding rule to obtain a transformed first input tensor, such that a first distance between the size of the first input tensor in at least one dimension and the size required for alignment with the multiplier is less than a second distance corresponding to the original input tensor; obtaining an optimized second pointwise convolutional layer corresponding to the first pointwise convolutional layer based on the first input tensor; obtaining an optimized model corresponding to the first model based on the second pointwise convolutional layer; and generating a binary target model corresponding to the neural network model to be compiled based on the optimized model. This disclosure can effectively reduce alignment padding, thereby improving the effective utilization rate of the multiplier.

Description

Technical Field

[0001] The present disclosure relates to artificial intelligence technology, and in particular to a method and device for compiling a neural network model, an electronic device, and a storage medium. Background Art

[0002] With the rapid development of artificial intelligence, a series of lightweight deep neural networks, mainly MobileNet, are gradually being used in mobile terminals such as mobile phones, PCs (personal computers), and in-vehicle devices to achieve machine learning tasks such as object detection and classification. In these deep neural networks, there are usually point-wise convolution layers that require point-wise convolution operations. In order to improve the reasoning speed of deep neural networks, chips that are specially designed to accelerate deep neural networks are usually used to implement deep neural network operations. The chip usually has a large multiplier array. When used for convolution calculations, the multiplier array usually has alignment requirements, and the input data needs to be filled according to the alignment requirements, which brings additional overhead to the operation process, resulting in low effective utilization of the multipliers. Summary of the invention

[0003] In order to solve the above technical problems, the present disclosure is proposed. The embodiments of the present disclosure provide a method and device for compiling a neural network model, an electronic device and a storage medium.

[0004] According to one aspect of an embodiment of the present disclosure, a method for compiling a neural network model is provided, comprising: obtaining a first model corresponding to a neural network model to be compiled, the first model being an intermediate representation corresponding to the neural network model to be compiled; for a first point-by-point convolution layer in at least one point-by-point convolution layer in the first model, performing a data structure transformation on an original input tensor of the first point-by-point convolution layer based on a preset tensor folding rule to obtain a transformed first input tensor, so that a first distance between a size of the first input tensor in at least one dimension and a size required for alignment of a multiplier is less than a second distance corresponding to the original input tensor, the second distance being a distance between a size of the original input tensor in at least one dimension and a size required for alignment of a multiplier; the at least one dimension is a dimension other than a channel dimension; obtaining an optimized second point-by-point convolution layer corresponding to the first point-by-point convolution layer based on the first input tensor; obtaining an optimized model corresponding to the first model based on the optimized second point-by-point convolution layer; and generating a binary target model corresponding to the neural network model to be compiled according to the optimized model.

[0005] According to another aspect of an embodiment of the present disclosure, a compilation device for a neural network model is provided, comprising: a first acquisition module, used to acquire a first model corresponding to a neural network model to be compiled, wherein the first model is an intermediate representation corresponding to the neural network model to be compiled; a first processing module, used to perform data structure transformation on an original input tensor of at least one point-by-point convolution layer in the first model based on a preset tensor folding rule to obtain a transformed first input tensor, so that a first distance between a size of the first input tensor in at least one dimension and a size required for alignment of a multiplier is smaller than a second distance corresponding to the original input tensor, wherein the second distance is a distance between a size of the original input tensor in at least one dimension and a size required for alignment of the multiplier; the at least one dimension is a dimension other than a channel dimension; a second processing module, used to obtain an optimized second point-by-point convolution layer corresponding to the first point-by-point convolution layer based on the first input tensor; a third processing module, used to obtain an optimized model corresponding to the first model based on the optimized second point-by-point convolution layer; and a fourth processing module, used to generate a binary target model corresponding to the neural network model to be compiled according to the optimized model.

[0006] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the compilation method of the neural network model described in any of the above embodiments of the present disclosure.

[0007] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the compilation method of the neural network model described in any of the above embodiments of the present disclosure.

[0008] Based on the compilation method and device of the neural network model, the electronic device and the storage medium provided by the above-mentioned embodiments of the present disclosure, by optimizing at least one point-by-point convolution layer of the intermediate representation of the neural network model to be compiled based on the tensor folding rule, the data structure of the input tensor of the point-by-point convolution layer can be transformed according to the alignment requirement, so that the size of the transformed input tensor in at least one dimension is closer to or equal to the size required by the alignment of the multiplier, thereby reducing the amount of alignment padding, and further improving the effective utilization of the multiplier, thereby solving the problem of low effective utilization of the multiplier due to the alignment requirement in the prior art.

[0009] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the compilation method of the neural network model provided by the present disclosure;

[0012] Figure 2 It is a flowchart of a method for compiling a neural network model provided by an exemplary embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of a method for compiling a neural network model provided by another exemplary embodiment of the present disclosure;

[0014] Figure 4 is a schematic diagram of the principle of data structure transformation provided by an exemplary embodiment of the present disclosure;

[0015] Figure 5 It is a schematic diagram of the principle of point-by-point convolution layer optimization provided by an exemplary embodiment of the present disclosure;

[0016] Figure 6 is a flow chart of step 202 provided by an exemplary embodiment of the present disclosure;

[0017] Figure 7 is another flowchart diagram of step 202 provided by an exemplary embodiment of the present disclosure;

[0018] Figure 8 is another flow chart of step 202 provided by an exemplary embodiment of the present disclosure;

[0019] Fig. 9 is another flow chart of step 202 provided by an exemplary embodiment of the present disclosure;

[0020] Fig.10 is a flowchart of a method for compiling a neural network model provided by another exemplary embodiment of the present disclosure;

[0021] Fig.11 is a structural diagram of a compiling device for a neural network model provided by an exemplary embodiment of the present disclosure;

[0022] Fig.12 is a structural schematic diagram of a second processing module provided by an exemplary embodiment of the present disclosure;

[0023] Fig.13 is a structural schematic diagram of a first processing module provided by an exemplary embodiment of the present disclosure;

[0024] Fig.14 is a structural schematic diagram of a first processing module provided by another exemplary embodiment of the present disclosure;

[0025] Fig.15 is a structural schematic diagram of a first processing module provided by yet another exemplary embodiment of the present disclosure;

[0026] Fig.16 It is a structural schematic diagram of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0027] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0028] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0029] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0030] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0031] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0032] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0033] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0034] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0035] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0036] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0037] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0038] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0039] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0040] SUMMARY OF THE DISCLOSURE

[0041] In the process of implementing the present disclosure, the inventors found that in acceleration chips that implement deep neural network operations, convolution calculations are usually performed through a large-scale multiplier array. However, during the operation, the multiplier array usually has alignment requirements. For the point-by-point convolution layer in the deep neural network, if the input data does not meet the alignment requirements, the input data needs to be padded so that the input data meets the alignment requirements. Since the padding data is invalid for the actual calculation, it still occupies a certain amount of multiplier resources, which brings additional overhead to the operation process, resulting in low effective utilization of the multiplier.

[0042] Exemplary Overview

[0043] Figure 1 This is an exemplary application scenario of the compilation method of the neural network model provided in the present invention.

[0044] For a neural network model with a point-by-point convolution layer, the technical solution provided by the present disclosure can be used to obtain an optimized model by optimizing the first model of the intermediate representation of the neural network model in the compilation stage, and the input tensor of the point-by-point convolution layer of the first model is transformed in the dimension other than the channel dimension, so that the transformed first input tensor meets or is closer to the alignment requirement of the acceleration chip, so that when performing point-by-point convolution, the invalid data that needs to be filled is reduced, thereby effectively improving the effective utilization rate of the acceleration chip multiplier. The compilation of the neural network model can be implemented by a compiler, and the compiler may include a front end, a middle end and a back end, and the front end, the middle end and the back end respectively include different modular compiler components and tool chains, wherein the front end converts the neural network model to be compiled from a programming language into a first model of the intermediate representation, the middle end optimizes the first model to obtain the optimized model, and the back end converts the optimized model of the intermediate representation into a machine-recognizable binary target model, the target model includes a binary instruction sequence recognizable by the acceleration chip, and the acceleration chip completes the real-time reasoning of the neural network by executing these instructions. The method disclosed in the present invention is to optimize the first model in the mid-end optimization process, and then convert the optimized model into a binary target model through the back end. The specific working principle of the front end is not disclosed and will not be described in detail. In the embodiment disclosed in the present invention, the dimensions of the original input tensor include at least 3 dimensions, that is, in addition to the number of channels, it also includes at least two dimensions, not limited to the two dimensions of height and width, and there can be more other dimensions.

[0045] Exemplary Methods

[0046] Figure 2 1 is a flowchart of a method for compiling a neural network model provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device, such as a terminal device or a server, such as Figure 2As shown, the following steps are included:

[0047] Step 201, obtain a first model corresponding to the neural network model to be compiled, where the first model is an intermediate representation corresponding to the neural network model to be compiled.

[0048] Among them, the neural network model to be compiled can be a trained neural network model with at least one point-by-point convolution layer, such as a neural network model from a model training tool such as pytorch, tensorflow, mxnet, etc. The neural network model to be compiled is converted from a programming language to an intermediate representation through a neural network compiler to obtain a first model of the intermediate representation corresponding to the neural network model to be compiled. The specific method of converting from the programming language to the intermediate representation can be any feasible method according to actual needs, which will not be elaborated in this disclosure.

[0049] Step 202, for the first point-by-point convolution layer in the at least one point-by-point convolution layer in the first model, perform data structure transformation on the original input tensor of the first point-by-point convolution layer based on a preset tensor folding rule to obtain the transformed first input tensor, so that a first distance between a size of the first input tensor in at least one dimension and a size required for alignment of the multiplier is smaller than a second distance corresponding to the original input tensor, and the second distance is a distance between a size of the original input tensor in at least one dimension and a size required for alignment of the multiplier; and at least one dimension is a dimension other than the channel dimension.

[0050] The first point-by-point convolution layer can be any point-by-point convolution layer in the first model. The first point-by-point convolution layer has an input tensor (called the original input tensor). The point-by-point convolution layer is to perform point-by-point convolution on the original input tensor to obtain an output tensor (called the original output tensor). The convolution kernel required for point-by-point convolution is a 1*1 convolution kernel.

[0051] The preset tensor folding rules can be set according to actual needs. Different tensor folding rules can be set for different input tensors, and there is no specific limitation. The data structure transformation of the original input tensor is performed between the dimensions of the original input tensor except the channel dimension, and the data of one dimension is folded and moved to another dimension. For example, the original input tensor with a data structure of H*W*C is transformed into a first input tensor of (H / 2)*(2W)*C, where H, W, and C represent the height, width, and number of channels of the tensor, respectively. The transformed first input tensor meets or is closer to the alignment requirement of the multiplier in at least one direction of the height direction and the width direction, that is, the first distance between the size of the first input tensor in at least one dimension and the alignment requirement size is less than the second distance between the size of the original input tensor and the alignment requirement size. The first distance refers to the difference between the alignment requirement size and the first input tensor in at least one dimension. The second distance refers to the absolute value of the difference in size on one dimension. The second distance refers to the absolute value of the difference between the alignment requirement size and the size of the original input tensor in at least one dimension. For example, if the alignment requirement size is 2*64*32 and the original input tensor is 8*8*512, the second distance in the height dimension is 6, and the second distance in the width dimension is 56. The preset tensor folding rule is to fold half of the size in the height direction and move it to the width direction. Then, the first input tensor obtained by the data structure transformation is 4*16*512. Then, the sizes of the first input tensor in the height and width directions are closer to the alignment requirements than the original input tensor. When filling, the amount of invalid data to be filled is reduced, thereby improving the effective utilization of the multiplier.

[0052] Step 203: Based on the first input tensor, obtain an optimized second point-by-point convolutional layer corresponding to the first point-by-point convolutional layer.

[0053] Among them, since the first input tensor is a tensor after the data structure of the original input tensor is transformed, the point-by-point convolution layer with the first input tensor as the input tensor is used as the optimized second point-by-point convolution layer corresponding to the first point-by-point convolution layer, and the output tensor of the second point-by-point convolution layer is the first output tensor obtained by point-by-point convolution of the first input tensor. In order to ensure that the original output of the first point-by-point convolution layer is obtained, the first output tensor needs to be restored, which can be set according to actual needs.

[0054] Step 204: Based on the optimized second point-by-point convolutional layer, an optimized model corresponding to the first model is obtained.

[0055] In practical applications, at least one point-by-point convolution layer in the first model can be optimized. In order to further improve the effective utilization rate of the multiplier and the operation efficiency, all point-by-point convolution layers in the first model can be optimized to finally obtain an optimized model. Specifically, the optimization model can be initialized, and the layers in the first model can be traversed. For non-point-by-point convolution layers, they can be added directly to the optimization model without processing. For the point-by-point convolution layer, it can be optimized as the first point-by-point convolution layer in the above manner, and the intermediate representation of the original input tensor data structure transformation, the optimized second point-by-point convolution layer, and the restored intermediate representation of the first output tensor of the second point-by-point convolution layer are added to the optimization model, and so on until all the layers of the first model are traversed, that is, the optimization model corresponding to the first model is obtained.

[0056] Step 205: Generate a binary target model corresponding to the neural network model to be compiled based on the optimization model.

[0057] Specifically, after obtaining the optimized model, the optimized model can be compiled into a binary target model corresponding to the neural network model to be compiled through the compiler backend. The specific transformation method from the optimized model to the target model can be implemented in any feasible way according to actual needs, and this disclosure will not go into details.

[0058] The compilation method of the neural network model provided in this embodiment optimizes the neural network model during the compilation process of the neural network model, and transforms the size of the original input tensor of the point-by-point convolution layer of the neural network model into a size that meets or is closer to the multiplier alignment requirement, so that when the first input tensor of the optimized point-by-point convolution layer is operated, the invalid data that needs to be filled is reduced, thereby effectively improving the effective utilization of the multiplier, and further improving the reasoning speed.

[0059] In an alternative example, Figure 3 FIG. 2 is a flow chart of a method for compiling a neural network model provided by another exemplary embodiment of the present disclosure. In this example, step 203 may specifically include the following steps:

[0060] Step 2031: Use the first input tensor as the input tensor of the second point-by-point convolutional layer, and use the first output tensor obtained by performing point-by-point convolution on the first input tensor as the output tensor of the second point-by-point convolutional layer to obtain the second point-by-point convolutional layer.

[0061] Specifically, the first input tensor is a tensor obtained by performing data structure transformation on the original input tensor, the point-by-point convolution layer using the first input tensor as the input tensor serves as the optimized second point-by-point convolution layer corresponding to the first point-by-point convolution layer, and the output tensor of the second point-by-point convolution layer is the first output tensor obtained by performing point-by-point convolution on the first input tensor.

[0062] Accordingly, after step 203, the method of the present disclosure further includes:

[0063] Step 301, restore the first output tensor to the original output tensor corresponding to the original input tensor.

[0064] Specifically, since the first output tensor is obtained by performing point-by-point convolution on the first input tensor, and the first input tensor is obtained by performing data structure transformation on the original input tensor, point-by-point convolution only affects the number of channels and does not affect the dimensions in the height and width directions. Therefore, the data structure of the first output tensor is consistent with the first input tensor in the height and width directions. In order to ensure that the original output tensor of the first point-by-point convolution layer is obtained, the first output tensor needs to be restored, and the data structure of the first output tensor is transformed into the same structure as the original output tensor to obtain the original output tensor.

[0065] The restoration rule for restoring the first output tensor to the original output tensor needs to be set according to the preset tensor folding rule. The restoration rule is the inverse transformation of the preset tensor folding rule to ensure the correspondence of the tensor feature elements during the transformation process, that is, to ensure that the position of the feature elements in the restored output tensor is consistent with the original output tensor obtained by point-by-point convolution based on the original input tensor.

[0066] Specifically, the data structure transformation from the original input tensor to the first input tensor and from the first output tensor to the original output tensor can be achieved through cyclic data movement. For example, taking the first input tensor as an example, the first tensor is initialized, and a loop mode can be set according to a preset format to perform cyclic movement, traverse each characteristic element of the first input tensor, and put it into the first tensor, thereby realizing the transformation of the data structure. Since the present disclosure involves the folding and movement of dimensions other than the channel dimension, the order of the characteristic elements in the channel direction is kept unchanged, and the dimensions in the height direction and the width direction are transformed by movement. The specific loop mode can be set according to actual needs.

[0067] For example, Figure 4 This is a schematic diagram of the principle of data structure transformation provided by an exemplary embodiment of the present disclosure. In this example, the original 4*4*4 input tensor is transformed into a first 2*8*4 input tensor. If the point-by-point convolution operation on the original input tensor includes 6 convolution kernels, that is, 6 1*1*4 convolution kernels, then the point-by-point convolution operation on the first input tensor also uses these 6 1*1*4 convolution kernels, which are represented as Kernel1, Kernel2, ..., Kernel6, respectively. Figure 56 convolution kernels perform point-by-point convolution on the first input tensor, generate 2*8 feature maps of 6 channels, form a 2*8*6 first output tensor, and restore the first output tensor to the original 4*4*6 output tensor. For a convolution kernel, such as Kernel1, it includes 4 weights in the depth direction, expressed as Among them, the subscript 11 represents the position of the weight in the height and width direction of Kernel1, and the superscripts 1 to 4 represent the position of the weight in the depth direction. The original input tensor is represented as a 4*4 feature map of 4 channels. Each feature element in the feature map can be represented as Where i, j, and t represent the position of the feature element in the length direction, width direction, and channel direction of the first input tensor, respectively. i=1,2,…,H1,j=1,2,…,W1,t=1,2,…,C1. In this example, H1=4, W1=4, C1=4, and the 4*4 feature map of the tth channel is expressed as:

[0068]

[0069] The first input tensor obtained by transforming the original input tensor data structure is represented as a 2*8 feature map of 4 channels, and each 2*8 feature map is represented as:

[0070]

[0071] That is, the third and fourth rows of the feature map of each channel of the original input tensor are folded and moved, and placed behind the first and second rows respectively.

[0072] The first output tensor obtained by performing point-by-point convolution on the first input tensor is represented as a 2*8 feature map of 6 channels, and the feature map of the sth (s=1, 2, ..., 6) channel is represented as:

[0073]

[0074] Among them, the feature map of the first channel is obtained through the convolution operation of Kernel1, which is expressed as:

[0075]

[0076] Similarly, the feature map of the sth channel is obtained through Kernels convolution, which will not be repeated here.

[0077] The original output tensor obtained by restoring the first output tensor is represented as a 4*4 feature map of 6 channels. Specifically, the last 4 columns of the first output tensor are moved below the first 4 columns. Corresponding to the previous folding operation, the feature elements at the corresponding positions are restored to the original output tensor positions. The specific expression is as follows:

[0078]

[0079] in, Corresponding The weighted sum based on Kernel1's weights, that is, corresponding to Based on the weighted sum of Kernel1's weights, after transforming into the original output tensor, Still at position 31. Perform point-by-point convolution on the original input tensor, and the characteristic element at position 31 of channel 1 in the obtained result is also Weighted sum based on Kernel1's weights.

[0080] It can be seen that since the point-by-point convolution only affects the number of channels and has no effect on the order of the feature elements in the height and width directions, the feature elements in the original output tensor finally restored are the same as the output tensor obtained by directly performing point-by-point convolution on the original input tensor.

[0081] In actual applications, when folding, when the characteristic elements of the moved part are spliced ​​with the unmoved part, the order of the characteristic elements is not limited to the order in the above example, and can be set according to actual needs.

[0082] The present disclosure can obtain the original output tensor of the original point-by-point convolution layer by restoring the output tensor of the optimized point-by-point convolution layer, thereby improving the effective utilization rate of the multiplier while ensuring the function of the original point-by-point convolution layer.

[0083] In an alternative example, Figure 6 202 is a flowchart of step 202 provided by an exemplary embodiment of the present disclosure. In this example, step 202 may specifically include the following steps:

[0084] Step 2021, for a first point-by-point convolution layer in at least one point-by-point convolution layer in the first model, based on the size of the original input tensor of the first point-by-point convolution layer and preset multiplier parameters, determine at least one folding method of the original input tensor.

[0085] Specifically, the preset multiplier parameters are parameters determined according to the actual situation of the multiplier of the acceleration chip. The preset multiplier parameters may include the alignment requirement size. In practical applications, they may also include the multiplier size, multiplier organization, clock cycle, etc., without specific limitation. The folding method refers to the method of transforming the data structure of the original input tensor. Since one dimension needs to be folded and moved to another dimension, such as H=8, W=8, and the H dimension is folded and moved to the W dimension, there will be multiple folding methods, such as one folding method of H1=4, W1=16, and another folding method of H2=2, W2=32. The principle of folding and moving is that the product of the height dimension and the width dimension remains unchanged, that is, the number of characteristic elements in the tensor is guaranteed to remain unchanged.

[0086] A feasible folding method of the original input tensor is determined based on preset multiplier parameters, that is, it is ensured that the size of the first input tensor obtained by performing data structure transformation on the original input tensor using these folding methods can meet or be closer to the size required by the alignment.

[0087] Step 2022: for each folding method in at least one folding method, perform data structure transformation on the original input tensor according to each folding method to obtain a second input tensor under each folding method.

[0088] After determining at least one feasible folding method of the original input tensor, the second input tensor of the original input tensor under each folding method can be obtained according to each folding method. The specific data structure transformation process can be achieved through cyclic data movement.

[0089] Step 2023, based on the second input tensor under each folding method, determine the performance evaluation results corresponding to each folding method.

[0090] Specifically, after determining the second input tensor under each folding method, the performance of each folding method can be evaluated based on each second input tensor to obtain the performance evaluation results corresponding to each folding method. Among them, the performance evaluation can be implemented in any feasible way, such as preset performance evaluation rules to calculate the score of each folding method under the preset performance evaluation rules. The score represents the performance. The specific preset performance evaluation rules can be set according to actual needs. For example, the preset performance evaluation rules can be determined based on the calculation amount of the point-by-point convolution layer and the calculation amount that the acceleration chip can process per second. The rule is to calculate the computing time required for the point-by-point convolution layer. The shorter the computing time, the better the performance.

[0091] Optionally, for each folding method, the compiler can also generate folding instructions from the original input tensor to the first input tensor, point-by-point convolution instructions for processing the folded first input tensor, and restoration operation instructions for restoring the first output tensor. These instructions are another intermediate representation close to machine instructions between the above-mentioned intermediate representation and binary machine instructions. The performance of each folding method is evaluated based on a preset performance model and these instructions of another intermediate representation. Here, the preset performance model can be set according to actual needs. For example, it can be a simple or complex calculation formula to calculate the computing time required for these instructions under each folding method.

[0092] Optionally, for each folding method, binary instructions may be directly generated by the compiler backend and sent directly to an actual chip for actual verification to obtain performance evaluation results corresponding to each folding method.

[0093] Step 2024: According to the performance evaluation results corresponding to each folding method, a folding method with the best performance is determined as a target folding method.

[0094] Specifically, after obtaining the performance evaluation results corresponding to each folding method, the folding method with the best performance can be selected as the target folding method to perform data structure transformation on the original input tensor.

[0095] Step 2025: Use the second input tensor in the target folding mode as the first input tensor.

[0096] Specifically, the second input tensor under the target folding method with the best performance is used as the first input tensor, which improves the effective utilization of the multiplier while ensuring the overall performance.

[0097] The present invention evaluates the performance of different folding methods and selects the folding method with the best performance to transform the data structure of the original input tensor, so as to balance the impact on performance caused by adding the data structure transformation of the original input tensor and the processing process of restoring the first output tensor in the process of improving the effective utilization of the multiplier through the input tensor data structure transformation, thereby ensuring high overall performance while improving the effective utilization of the multiplier.

[0098] In an alternative example, Figure 7 is another flowchart of step 202 provided by an exemplary embodiment of the present disclosure. In this example, after step 2024, it also includes:

[0099] Step 2026, obtain the performance evaluation result corresponding to the original input tensor.

[0100] Step 2027, when it is determined that the performance evaluation result of the target folding method is better than the performance evaluation result corresponding to the original input tensor, the second input tensor under the target folding method is used as the first input tensor.

[0101] Specifically, since the optimization of the convolutional layer will introduce additional data structure transformation processes, it will have a certain impact on the overall performance. Therefore, in practical applications, the overall performance after optimization can be compared with that before optimization. When the performance evaluation result after optimization is better than that before optimization, the first convolutional layer can be optimized. If the performance after optimization is worse than that before optimization, the first convolutional layer can be not optimized according to actual needs. Of course, if only the effective utilization rate of the multiplier is considered, the first convolutional layer can also be optimized. The specific settings can be based on actual needs.

[0102] The present invention compares the performance evaluation result of the folding method with the best performance with the performance evaluation result before optimization. When the performance evaluation result after folding is better than that before optimization, the first convolutional layer is optimized based on the folding method with the best performance, thereby avoiding the additional overhead caused by the folding and restoration process exceeding the time saved by the effective utilization of the multiplier, thereby ensuring that the model performance will not be reduced while improving the effective utilization of the multiplier.

[0103] In an optional example, the preset multiplier parameters at least include dimensions that require alignment and preset thresholds for alignment of each dimension; accordingly, determining at least one folding method of the original input tensor based on the size of the original input tensor and the preset multiplier parameters in step 2021 includes:

[0104] If, based on the size of the original input tensor and the preset multiplier parameters, it is determined that the size of the original input tensor in the first dimension is greater than the preset threshold value that needs to be aligned in the first dimension, then at least one folding method is determined based on the size in the first dimension and the preset threshold values ​​that need to be aligned in each dimension.

[0105] Among them, the first dimension can be any dimension in the original input tensor except the channel dimension. When the size of the original input tensor in a certain dimension exceeds the alignment requirement, folding and moving the dimension can bring better benefits. Therefore, in order to further improve the optimization effect, when it is determined that the size of the original input tensor in the first dimension is greater than the preset threshold that needs to be aligned with the first dimension, it is determined that the first dimension needs to be folded, and then at least one folding method of the first dimension is determined based on the size in the first dimension and the preset threshold that each dimension needs to be aligned in the alignment requirement.

[0106] For example, if the height H direction exceeds the alignment requirement, the height of the original input tensor is aligned according to the hardware alignment requirement of the acceleration chip. Assume where p i is a prime number, H inis the height dimension of the original input tensor, H chip To accelerate the chip alignment requirements in the height direction, all possible height dimensions after folding are H chip *∏ j p j , that is, H in In addition to H chip All factors other than can be folded into other directions. For example, the height H of the original input tensor in The size of the 8*8 pixel is 8. After removing the acceleration chip alignment requirement 2, the remaining 4 can be folded into 2 or 4 in the width W direction, that is, the height size after folding is 4 (8 / 2) or 2 (8 / 4), and the size in the width direction is doubled (2 times or 4 times), that is, 8*8 is transformed into 4*16 or 8*8 is transformed into 2*32. For example, 9*9 is transformed into 3*27.

[0107] The present disclosure folds the first dimension whose size is larger than the alignment requirement into other dimensions so that while the first dimension meets or approaches the alignment requirement, other dimensions can also meet or approach the alignment requirement, thereby further improving the optimization effect.

[0108] In an alternative example, Figure 8 FIG. 2 is another flow chart of step 202 provided by an exemplary embodiment of the present disclosure. In this example, step 202 specifically includes:

[0109] 4011. For a first point-by-point convolution layer in at least one point-by-point convolution layer in the first model, determine an effective utilization rate of the first point-by-point convolution layer based on preset multiplier parameters.

[0110] Among them, the effective utilization rate refers to the proportion of the operation of the valid data in the input data by the multiplier during the operation to the operation that the multiplier itself can complete. The effective utilization rate of the first point-by-point convolution layer in the present disclosure is the effective utilization rate of the point-by-point convolution operation based on the multiplier on the input tensor of the first point-by-point convolution layer. The size of the effective utilization rate is affected by the degree of matching between the input tensor data size and the multiplier alignment requirement size. For example, when the multiplier alignment requirement size is certain, for some small-sized tensor data, since the size does not meet the alignment requirement, invalid data needs to be filled for alignment, which seriously affects the effective utilization rate of the multiplier. For example, in the neural network model, point-by-point convolution is needed to process a data of H=8, W=8, C=512. Since the multiplier array requires W to be aligned to 64, the proportion of effective calculation in actual operation is only 8 / 64=12.5%.

[0111] Step 4012: Based on the effective utilization rate of the first point-by-point convolutional layer, determine whether the first point-by-point convolutional layer needs to be optimized.

[0112] In practical applications, whether the point-by-point convolution layer needs to be optimized can be determined based on the effective utilization rate of the point-by-point convolution layer in different neural network models. For example, an effective utilization rate threshold can be set. When the effective utilization rate of the first point-by-point convolution layer of the first model of the intermediate representation is lower than the effective utilization rate threshold, the first point-by-point convolution layer is optimized. Otherwise, it is not optimized. The specific setting can be based on actual needs.

[0113] In an optional example, whether optimization is needed can also be determined based on the running time of the first point-by-point convolution layer. For example, a running time threshold can be set. When the running time of the first point-by-point convolution layer is higher than the running time threshold, it is determined that the first point-by-point convolution layer needs to be optimized. The specific setting can be based on actual needs. The running time can be obtained by calculating the effective utilization rate and the clock cycle, and the details will not be repeated here.

[0114] Step 4013: If the first point-by-point convolution layer needs to be optimized, the data structure of the original input tensor of the first point-by-point convolution layer is transformed based on a preset tensor folding rule to obtain a first input tensor.

[0115] The present disclosure can effectively avoid unnecessary optimization processing by judging whether optimization is needed based on the effective utilization rate of the point-by-point convolutional layer. For the point-by-point convolutional layer whose original input tensor already meets the alignment requirements, there is no need to perform optimization processing, thereby reducing unnecessary waste of resources and further improving data processing efficiency.

[0116] In an alternative example, Fig. 9 FIG. 2 is another flow chart of step 202 provided by an exemplary embodiment of the present disclosure. In this example, step 202 specifically includes:

[0117] Step 4021, traverse each layer of the first model, take each layer as the target layer, and obtain the operation parameters of the target layer.

[0118] Specifically, after obtaining the first model, the layers of the first model can be traversed, and each layer can be used as the target layer to obtain the operation parameters of the target layer. The operation parameters are parameters related to the operation of the target layer. The operation parameters of different layers may be different. For example, for the convolution layer, the operation parameters may include the convolution kernel size, the convolution kernel sliding step, etc., and the point-by-point convolution layer has its own unique operation parameters, such as the convolution kernel size of 1*1, the convolution kernel sliding step of 1*1, the filling size of 0*0, and no special properties such as holes and grouping.

[0119] Step 4022: determine whether the target layer is a point-by-point convolution layer based on the operating parameters and preset judgment rules.

[0120] Among them, the preset judgment rules can be set according to the actual parameter characteristics of the point-by-point convolution layer. For example, when the convolution kernel size of the target layer is 1*1, the convolution kernel sliding step is 1*1, the filling size is 0*0, and there are no special attributes such as holes and grouping, the target layer is judged to be a point-by-point convolution layer.

[0121] Step 4023: If the target layer is a point-by-point convolution layer, the target layer is used as the first point-by-point convolution layer, and the original input tensor of the first point-by-point convolution layer is transformed in data structure based on a preset tensor folding rule to obtain a first input tensor.

[0122] Specifically, for non-point-by-point convolutional layers, no optimization is required and they are directly added to the optimization model. For point-by-point convolutional layers, they are optimized and then added to the optimization model. After traversing all layers, the complete optimization model corresponding to the first model is obtained. In the optimization model, the first point-by-point convolutional layer in the first model is transformed into three parts. The first part is the intermediate representation of the original input tensor transformed into the first input tensor, the second part is the second point-by-point convolutional layer with the first input tensor as the input tensor (the output is the first output tensor), and the third part is the intermediate representation of the first output tensor transformed into the original output tensor, thereby achieving the function of the original point-by-point convolutional layer while improving the effective utilization rate.

[0123] In an alternative example, Fig.10 FIG. 1 is a flowchart of a method for compiling a neural network model provided by another exemplary embodiment of the present disclosure. In this example, the method of the present disclosure specifically includes:

[0124] 1. Get the first model corresponding to the neural network model to be compiled.

[0125] 2. Traverse each layer of the first model, take each layer as the target layer, and obtain the operation parameters of the target layer.

[0126] 3. According to the operating parameters and preset judgment rules, determine whether the target layer is a point-by-point convolution layer.

[0127] 4. If the target layer is a point-by-point convolution layer, the target layer is used as the first point-by-point convolution layer, and the effective utilization rate of the first point-by-point convolution layer is determined based on the preset multiplier parameters.

[0128] 5. Based on the effective utilization rate of the first point-by-point convolutional layer, determine whether the first point-by-point convolutional layer needs to be optimized.

[0129] 6. If the first point-by-point convolution layer needs to be optimized, at least one folding method of the original input tensor is determined based on the size of the original input tensor of the first point-by-point convolution layer and the preset multiplier parameters.

[0130] 7. For each folding method in at least one folding method, perform data structure transformation on the original input tensor according to each folding method to obtain a second input tensor under each folding method.

[0131] 8. Based on the second input tensor under each folding method, determine the performance evaluation results corresponding to each folding method.

[0132] 9. According to the performance evaluation results corresponding to each folding method, determine the folding method with the best performance as the target folding method.

[0133] 10. Get the performance evaluation results corresponding to the original input tensor.

[0134] 11. When it is determined that the performance evaluation result of the target folding method is better than the performance evaluation result corresponding to the original input tensor, the second input tensor under the target folding method is used as the first input tensor.

[0135] 12. Use the first input tensor as the input tensor of the optimized second point-by-point convolution layer corresponding to the first point-by-point convolution layer, and use the first output tensor obtained by performing point-by-point convolution on the first input tensor as the output tensor of the second point-by-point convolution layer to obtain the second point-by-point convolution layer.

[0136] 13. Restore the first output tensor to the original output tensor corresponding to the original input tensor.

[0137] 14. Based on the optimized second point-by-point convolutional layer, an optimized model corresponding to the first model is obtained.

[0138] 15. Based on the optimization model, generate a binary target model corresponding to the neural network model to be compiled.

[0139] Any method for compiling a neural network model provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any method for compiling a neural network model provided in the embodiments of the present disclosure may be executed by a processor, such as a processor executing any method for compiling a neural network model mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.

[0140] Exemplary Devices

[0141] Fig.11 is a schematic diagram of the structure of a neural network model compilation device provided by an exemplary embodiment of the present disclosure. The device of this embodiment can be used to implement the corresponding method embodiments of the present disclosure, such as Fig.11 The device shown includes: a first acquisition module 501 , a first processing module 502 , a second processing module 503 , a third processing module 504 and a fourth processing module 505 .

[0142] The first acquisition module 501 is used to acquire a first model corresponding to the neural network model to be compiled, where the first model is an intermediate representation corresponding to the neural network model to be compiled.

[0143] The first processing module 502 is used to perform data structure transformation on the original input tensor of the first point-by-point convolution layer in at least one point-by-point convolution layer in the first model obtained by the first acquisition module 501 based on a preset tensor folding rule to obtain the transformed first input tensor, so that a first distance between the size of the first input tensor in at least one dimension and the size required for alignment of the multiplier is smaller than a second distance corresponding to the original input tensor, and the second distance is the distance between the size of the original input tensor in at least one dimension and the size required for alignment of the multiplier; and at least one dimension is a dimension other than the channel dimension.

[0144] The second processing module 503 is used to obtain an optimized second point-by-point convolution layer corresponding to the first point-by-point convolution layer based on the first input tensor obtained by the first processing module 502.

[0145] The third processing module 504 is used to obtain an optimized model corresponding to the first model based on the optimized second point-by-point convolutional layer obtained by the second processing module 503.

[0146] The fourth processing module 505 is used to generate a binary target model corresponding to the neural network model to be compiled according to the optimization model obtained by the third processing module 504.

[0147] In an alternative example, Fig.12 It is a structural diagram of the second processing module provided by an exemplary embodiment of the present disclosure. In this example, the second processing module 503 includes: a first processing unit 5031 and a second processing unit 5032. The first processing unit 5031 is used to use the first input tensor obtained by the first processing module 502 as the input tensor of the second point-by-point convolution layer, and use the first output tensor obtained by performing point-by-point convolution on the first input tensor as the output tensor of the second point-by-point convolution layer. The second processing unit 5032 is used to restore the first output tensor to the original output tensor corresponding to the original input tensor.

[0148] Fig.13 It is a structural diagram of a first processing module provided by an exemplary embodiment of the present disclosure.

[0149] In an optional example, the first processing module 502 includes: a first unit 5021, a second unit 5022, a third unit 5023, a fourth unit 5024 and a fifth unit 5025. The first unit 5021 is used to determine at least one folding method of the original input tensor based on the size of the original input tensor and the preset multiplier parameter; the second unit 5022 is used to transform the data structure of the original input tensor according to each folding method in the at least one folding method obtained by the first unit 5021, and obtain the second input tensor under each folding method; the third unit 5023 is used to determine the performance evaluation results corresponding to each folding method based on the second input tensor under each folding method obtained by the second unit 5022; the fourth unit 5024 is used to determine the folding method with the best performance as the target folding method according to the performance evaluation results corresponding to each folding method obtained by the third unit 5023; the fifth unit 5025 is used to use the second input tensor under the target folding method obtained by the fourth unit 5024 as the first input tensor.

[0150] In an optional example, the first processing module 502 also includes: a sixth unit 5026, used to obtain a performance evaluation result corresponding to the original input tensor; accordingly, the fifth unit 5025 is specifically used to use the second input tensor under the target folding method as the first input tensor when it is judged that the performance evaluation result of the target folding method obtained by the fourth unit 5024 is better than the performance evaluation result corresponding to the original input tensor obtained by the sixth unit 5026.

[0151] In an optional example, the preset multiplier parameters include at least the dimensions that require alignment and the preset thresholds for alignment of each dimension. The first unit 5021 is specifically used for: if it is determined based on the size of the original input tensor and the preset multiplier parameters that the size of the original input tensor in the first dimension is greater than the preset threshold for alignment of the first dimension, then based on the size in the first dimension and the preset thresholds for alignment of each dimension, determine at least one folding method.

[0152] In an alternative example, Fig.14 It is a structural diagram of a first processing module provided by another exemplary embodiment of the present disclosure. In this example, the first processing module 502 includes: a seventh unit 5027, an eighth unit 5028 and a ninth unit 5029. The seventh unit 5027 is used to determine the effective utilization rate of the first point-by-point convolution layer based on preset multiplier parameters; the eighth unit 5028 is used to determine whether the first point-by-point convolution layer needs to be optimized based on the effective utilization rate of the first point-by-point convolution layer obtained by the seventh unit 5027; the ninth unit 5029 is used to transform the data structure of the original input tensor of the first point-by-point convolution layer based on a preset tensor folding rule to obtain the first input tensor if the eighth unit 5028 determines that the first point-by-point convolution layer needs to be optimized.

[0153] In an alternative example, Fig.15 It is a structural diagram of the first processing module provided by another exemplary embodiment of the present disclosure. In this example, the first processing module 502 includes: a third processing unit 601, a fourth processing unit 602 and a fifth processing unit 603. The third processing unit 601 is used to traverse the layers of the first model obtained by the first acquisition module 501, take each layer as the target layer, and obtain the operation parameters of the target layer; the fourth processing unit 602 is used to determine whether the target layer is a point-by-point convolution layer according to the operation parameters obtained by the third processing unit 601 and the preset judgment rules; the fifth processing unit 603 is used to take the target layer as the first point-by-point convolution layer if the fourth processing unit 602 determines that the target layer is a point-by-point convolution layer, and perform data structure transformation on the original input tensor of the first point-by-point convolution layer based on the preset tensor folding rule to obtain the first input tensor.

[0154] Exemplary Electronic Devices

[0155] The embodiment of the present disclosure also provides an electronic device, including: a memory for storing a computer program;

[0156] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the compilation method of the neural network model described in any of the above embodiments of the present disclosure is implemented.

[0157] Fig.16 FIG. 1 is a schematic diagram of a structure of an application embodiment of the electronic device disclosed in the present invention. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0158] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0159] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the compilation method of the neural network model of each embodiment of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0160] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0161] For example, the input device 13 may be the microphone or microphone array mentioned above, which is used to capture the input signal of the sound source.

[0162] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0163] The output device 14 can output various information to the outside, including the determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0164] Of course, to simplify, Fig.16 Only some of the components related to the present disclosure in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0165] Exemplary computer program products and computer-readable storage media

[0166] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps in the method of compiling a neural network model according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0167] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0168] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps of the method for compiling a neural network model according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0169] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0170] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0171] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0172] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0173] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0174] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0175] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0176] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for compiling a neural network model, comprising: Obtaining a first model corresponding to the neural network model to be compiled, where the first model is an intermediate representation corresponding to the neural network model to be compiled; For a first point-by-point convolution layer in at least one point-by-point convolution layer in the first model, a data structure transformation is performed on an original input tensor of the first point-by-point convolution layer based on a preset tensor folding rule to obtain a transformed first input tensor, so that a first distance between a size of the first input tensor in at least one dimension and a size required for alignment of a multiplier is less than a second distance corresponding to the original input tensor, and the second distance is a distance between a size of the original input tensor in at least one dimension and a size required for alignment of a multiplier; the at least one dimension is a dimension other than a channel dimension; Based on the first input tensor, obtaining an optimized second point-by-point convolutional layer corresponding to the first point-by-point convolutional layer; Based on the optimized second point-by-point convolutional layer, obtaining an optimized model corresponding to the first model; According to the optimization model, a binary target model corresponding to the neural network model to be compiled is generated.

2. The method according to claim 1, wherein: The step of obtaining an optimized second point-by-point convolutional layer corresponding to the first point-by-point convolutional layer based on the first input tensor includes: Using the first input tensor as the input tensor of the second point-by-point convolution layer, and using a first output tensor obtained by performing point-by-point convolution on the first input tensor as the output tensor of the second point-by-point convolution layer; After obtaining the optimized second point-by-point convolution layer corresponding to the first point-by-point convolution layer based on the first input tensor, the method further includes: The first output tensor is restored to the original output tensor corresponding to the original input tensor.

3. The method according to claim 2, wherein: The performing data structure transformation on the original input tensor of the first point-by-point convolutional layer based on a preset tensor folding rule to obtain a transformed first input tensor includes: Determining at least one folding mode of the original input tensor based on the size of the original input tensor and the preset multiplier parameters; For each folding method in the at least one folding method, respectively transform the data structure of the original input tensor according to each folding method to obtain a second input tensor under each folding method; Based on the second input tensor under each of the folding modes, determining the performance evaluation results corresponding to each of the folding modes; According to the performance evaluation results corresponding to the folding methods, a folding method with the best performance is determined as a target folding method; The second input tensor in the target folding mode is used as the first input tensor.

4. The method according to claim 3, wherein: After determining the folding method with the best performance as the target folding method according to the performance evaluation results corresponding to each of the folding methods, the method further includes: Obtaining a performance evaluation result corresponding to the original input tensor; When it is determined that the performance evaluation result of the target folding method is better than the performance evaluation result corresponding to the original input tensor, the second input tensor under the target folding method is used as the first input tensor.

5. The method according to claim 3, wherein: The preset multiplier parameters at least include dimensions that require alignment and preset thresholds for alignment of each dimension, and the determining, based on the size of the original input tensor and the preset multiplier parameters, at least one folding method of the original input tensor includes: If, based on the size of the original input tensor and the preset multiplier parameter, it is determined that the size of the original input tensor in the first dimension is greater than the preset threshold value that needs to be aligned in the first dimension, then based on the size in the first dimension and the preset threshold value that needs to be aligned in each dimension, determine the at least one folding method.

6. The method according to claim 1, wherein: Before performing data structure transformation on the original input tensor of the first point-by-point convolutional layer based on the preset tensor folding rule to obtain the transformed first input tensor, the method further includes: Determining an effective utilization rate of the first point-by-point convolutional layer based on preset multiplier parameters; Based on the effective utilization rate of the first point-by-point convolutional layer, determining whether the first point-by-point convolutional layer needs to be optimized; The performing data structure transformation on the original input tensor of the first point-by-point convolutional layer based on a preset tensor folding rule to obtain a transformed first input tensor includes: If the first point-by-point convolution layer needs to be optimized, a data structure transformation is performed on an original input tensor of the first point-by-point convolution layer based on a preset tensor folding rule to obtain the first input tensor.

7. The method according to claim 1, wherein: The step of performing data structure transformation on an original input tensor of the first point-by-point convolution layer in the at least one point-by-point convolution layer in the first model based on a preset tensor folding rule to obtain a transformed first input tensor includes: Traversing each layer of the first model, taking each layer as a target layer, and obtaining operation parameters of the target layer; According to the operating parameters and preset judgment rules, determining whether the target layer is a point-by-point convolution layer; If the target layer is a point-by-point convolution layer, the target layer is used as the first point-by-point convolution layer, and the original input tensor of the first point-by-point convolution layer is transformed in data structure based on a preset tensor folding rule to obtain the first input tensor.

8. A compiling device for a neural network model, comprising: A first acquisition module, used to acquire a first model corresponding to the neural network model to be compiled, where the first model is an intermediate representation corresponding to the neural network model to be compiled; A first processing module, for performing data structure transformation on an original input tensor of the first point-by-point convolution layer in at least one point-by-point convolution layer in the first model based on a preset tensor folding rule to obtain a transformed first input tensor, so that a first distance between a size of the first input tensor in at least one dimension and a size required for alignment of a multiplier is smaller than a second distance corresponding to the original input tensor, and the second distance is a distance between a size of the original input tensor in at least one dimension and a size required for alignment of a multiplier; the at least one dimension is a dimension other than a channel dimension; A second processing module, configured to obtain an optimized second point-by-point convolutional layer corresponding to the first point-by-point convolutional layer based on the first input tensor; A third processing module, configured to obtain an optimized model corresponding to the first model based on the optimized second point-by-point convolutional layer; The fourth processing module is used to generate a binary target model corresponding to the neural network model to be compiled according to the optimization model.

9. A computer-readable storage medium storing a computer program for executing the method for compiling a neural network model according to any one of claims 1 to 7.

10. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the compilation method of the neural network model described in any one of claims 1-7 above.

Images