Compiling method and device of neural network model, storage medium and electronic equipment

Abstract

A method and apparatus for compiling a neural network model, a storage medium, and an electronic device are disclosed. The method includes obtaining a neural network model to be compiled, the neural network model to be compiled including a first network layer corresponding to a first operation type, the first operation being a lookup table operation and using a first table; based on a specified constant range and the first table, transforming the first operation of a first input feature map of the first network layer into a second operation to obtain a second network layer, the second operation being an operation supported by a neural network accelerator; and based on network layers other than the first network layer in the neural network model to be compiled and the second network layer, generating a first target neural network model through a compilation process. The disclosed embodiments can implement a lookup table operation without data migration, thereby effectively saving bandwidth and time, and also avoiding the occupation of additional resources.

Description

Technical Field

[0001] This disclosure relates to artificial intelligence technology, and in particular to a method, apparatus, storage medium, and electronic device for compiling a neural network model. Background Technology

[0002] In some cases, on-chip neural network accelerators do not support table lookup operations, but the actual operation of the neural network model requires table lookup operations. The current solution is to move the data that needs to be looked up to the central processing unit (CPU) to perform the lookup operation, and then move the result obtained by the CPU back to the neural network accelerator. However, this solution wastes bandwidth and time because it requires data movement. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure is proposed. Embodiments of this disclosure provide a method, apparatus, storage medium, and electronic device for compiling a neural network model.

[0004] According to one aspect of the present disclosure, a method for compiling a neural network model is provided, comprising:

[0005] Obtain the neural network model to be compiled, wherein the neural network model to be compiled includes a first network layer whose corresponding operation type is a first operation, and the first operation is a lookup operation and the table used is a first table;

[0006] Based on a specified constant range and the first table, the first operation of the first input feature map of the first network layer is transformed into a second operation to obtain a second network layer. The second operation is an operation supported by a neural network accelerator.

[0007] Based on the network layers other than the first network layer in the neural network model to be compiled, and the second network layer, a first target neural network model is generated through compilation processing.

[0008] According to one aspect of the present disclosure, a neural network model compilation apparatus is provided, comprising:

[0009] The first acquisition module is used to acquire the neural network model to be compiled, wherein the neural network model to be compiled includes a first network layer whose corresponding operation type is a first operation, and the first operation is a lookup table operation and the table used is a first table;

[0010] The second acquisition module is used to transform the first operation of the first input feature map of the first network layer in the neural network model to be compiled, which is obtained by the first acquisition module, into a second operation based on a specified constant range and the first table, so as to obtain a second network layer. The second operation is an operation supported by the neural network accelerator.

[0011] The first generation module is used to generate a first target neural network model by compiling the network layers other than the first network layer in the neural network model to be compiled, obtained by the first acquisition module, and the second network layer obtained by the second acquisition module.

[0012] According to another aspect of this disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for executing the compilation method of the neural network model described above.

[0013] According to another aspect of this disclosure, an electronic device is provided, the electronic device comprising:

[0014] processor;

[0015] Memory used to store the processor's executable instructions;

[0016] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the compilation method of the neural network model described above.

[0017] Based on the neural network model compilation method, apparatus, storage medium, and electronic device provided in the above embodiments of this disclosure, the table lookup operation can be transformed into an operation supported by the neural network accelerator during the compilation of the neural network model. In this way, even if the neural network accelerator does not support the table lookup operation, the table lookup operation can be indirectly implemented through an operation equivalent to the table lookup operation. Therefore, the embodiments of this disclosure can implement the table lookup operation without data migration, thereby effectively saving bandwidth and time, and avoiding the occupation of additional resources.

[0018] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1This is a schematic diagram illustrating the principle of table lookup operations in an embodiment of this disclosure.

[0021] Figure 2 This is a flowchart illustrating a method for compiling a neural network model provided in an exemplary embodiment of this disclosure.

[0022] Figure 3 This is a flowchart illustrating a method for compiling a neural network model provided in another exemplary embodiment of this disclosure.

[0023] Figure 4 This is a schematic diagram of the first set of convolution kernels in an exemplary embodiment of this disclosure.

[0024] Figure 5 This is a schematic diagram of the third convolution kernel set in an exemplary embodiment of this disclosure.

[0025] Figure 6 This is a schematic diagram of the operational logic of the second operation in an embodiment of this disclosure.

[0026] Figure 7 This is a flowchart illustrating a method for compiling a neural network model provided in another exemplary embodiment of this disclosure.

[0027] Figure 8 This is a schematic diagram of the structure of a compilation device for a neural network model provided in an exemplary embodiment of this disclosure.

[0028] Figure 9 This is a schematic diagram of the structure of a compilation device for a neural network model provided in another exemplary embodiment of this disclosure.

[0029] Figure 10 This is a schematic diagram of the structure of a compilation apparatus for a neural network model provided in another exemplary embodiment of the present disclosure.

[0030] Figure 11 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0031] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0032] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0033] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0034] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0035] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0036] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0037] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0038] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0039] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0040] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0041] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0042] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0043] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0044] Application Overview

[0045] Some chips can have neural network accelerators deployed on them; for example, artificial intelligence (AI) chips can have brain processing units (BPUs) deployed on them. It should be noted that neural network accelerators can be used to implement neural network models, such as neural network models for object detection.

[0046] In the process of developing this disclosure, the inventors discovered that in some cases, on-chip neural network accelerators do not support lookup table operations, while neural network models require lookup table operations during actual operation. Lookup table operations can be used to simulate arbitrary fixed-point functions, such as trigonometric function operations, exponential operations, and logarithmic operations. For example, if feature map A requires a lookup table operation, and the lookup table operation simulates f(x) = sinx, then the sine value needs to be calculated for each element in feature map A, and feature map B, as the lookup result, needs to be generated. Each element in feature map B is the sine value corresponding to the corresponding element in feature map A.

[0047] For situations where on-chip neural network accelerators do not support table lookup operations, but the actual computation of the neural network model requires them, the current solution is to move the data requiring table lookup to the CPU for the lookup operation, and then move the result obtained by the CPU back to the neural network accelerator. However, this solution wastes bandwidth and time due to the data movement, has high implementation costs, and consumes CPU resources. Therefore, a more reasonable solution is needed to address this situation.

[0048] Exemplary System

[0049] It should be noted that even if on-chip neural network accelerators do not support table lookup operations, they often support some common operation types, such as convolution, bias, ReLU, and elementwise operations.

[0050] In view of this, in the embodiments of this disclosure, such as Figure 1 As shown, the neural network model involves two stages: compilation and execution. During compilation, table lookup operations are transformed into operations supported by the neural network accelerator, but these operations are not actually executed during compilation. During execution, table lookup operations are indirectly implemented through the actual execution of the operations supported by the neural network accelerator. Thus, even if the neural network accelerator does not support table lookup operations, these operations can still be performed without data migration or additional resource consumption.

[0051] Exemplary methods

[0052] Figure 2 This is a flowchart illustrating a method for compiling a neural network model provided in an exemplary embodiment of this disclosure. Figure 2 The method shown can be applied to compilers. Figure 2 The method shown includes steps 210, 220 and 230, which are explained below.

[0053] Step 210: Obtain the neural network model to be compiled. The neural network model to be compiled includes a first network layer with the corresponding operation type being the first operation. The first operation is a lookup operation and the table used is the first table.

[0054] It should be noted that the neural network model to be compiled refers to the neural network model that needs to be compiled. The neural network model to be compiled may include multiple network layers. Among the multiple network layers, there may be at least one network layer whose corresponding operation type belongs to table lookup operation. Different network layers in the at least one network layer may use different tables. Any network layer in the at least one network layer can be used as the first network layer in step 210. The table used by the first network layer is the first table.

[0055] Step 220: Based on the specified constant range and the first table, the first operation of the first input feature map of the first network layer is transformed into a second operation to obtain the second network layer. The second operation is an operation supported by the neural network accelerator.

[0056] Optionally, the specified constant range can be an 8-bit constant range, i.e., [-128, 127]. Of course, the specified constant range can also be a 4-bit, 16-bit, or other constant ranges. For ease of understanding, the embodiments of this disclosure all use the case of specifying an 8-bit constant range as an example. Thus, assuming that the fixed-point function simulated by the operation type corresponding to the first network layer is f(x), the first table includes f(-128), f(-127), ..., f(126), f(127) in sequence. That is, the first table includes 256 values, and the number of values ​​included in the first table is the same as the number of constants in the specified constant range.

[0057] Optionally, the second operation includes, but is not limited to, common operation types such as convolution, bias, modified linear unit operation, and element-wise operation.

[0058] It should be noted that in step 220, transforming the first operation into the second operation essentially means transforming the first operation into an equivalent operation. In other words, the result obtained by performing the equivalent operation is the same as the result obtained by performing the table lookup operation.

[0059] Step 230: Based on the network layers other than the first network layer and the second network layer in the neural network model to be compiled, a first target neural network model is generated through compilation processing.

[0060] Optionally, the network layers in the neural network model to be compiled, excluding the first network layer, include, but are not limited to, network layers whose corresponding operation type is convolution (i.e., convolutional layer), network layers whose corresponding operation type is pooling (i.e., pooling layer), and network layers whose corresponding operation type is modified linear unit operation (i.e., ReLU layer).

[0061] In step 230, the compiler backend can perform compilation processing based on the network layers in the neural network model to be compiled, excluding the first network layer, and the second network layer obtained through operation and transformation, thereby generating the binary first target neural network model. The specific compilation processing method can be any feasible method according to actual needs, and this disclosure will not elaborate on it.

[0062] Based on the neural network model compilation method provided in the above embodiments of this disclosure, the table lookup operation can be transformed into an operation supported by the neural network accelerator during the compilation process of the neural network model. In this way, even if the neural network accelerator does not support the table lookup operation, the table lookup operation can be indirectly implemented through an operation equivalent to the table lookup operation. Therefore, the embodiments of this disclosure can implement the table lookup operation without data migration, thereby effectively saving bandwidth and time, and avoiding the occupation of additional resources.

[0063] exist Figure 2 Based on the illustrated embodiments, as Figure 3 As shown, step 220 includes steps 2201, 2203 and 2205.

[0064] Step 2201: Based on the number of channels C of the first input feature map and the number of constants L in the specified constant range, determine the first convolution kernel set and the second convolution kernel set.

[0065] Here, the number of channels C may be 3, 4, 6, 8, 10 or other values, which will not be listed here.

[0066] Here, since the specified constant range is the constant range corresponding to 8 bits, the number of constants L can specifically be 256.

[0067] Optionally, the first set of convolutional kernels includes L*C convolutional kernels with a width of 1, a height of 1, and a number of channels of C. Furthermore, the i-th element of the L*(i-1)+1 to L*i-th convolutional kernels in the first set of convolutional kernels is -1, and the remaining elements in the first set of convolutional kernels are all 0.

[0068] Since L is 256, the first set of convolutional kernels can include 256*C convolutional kernels, each with a width of 1, a height of 1, and a channel count of C. Furthermore, the 256*C convolutional kernels in the first set satisfy the following conditions: the first element of the 1st to 256th convolutional kernels is -1; the second element of the 257th to 512th convolutional kernels is -1; the third element of the 513th to 768th convolutional kernels is -1; and so on for subsequent convolutional kernels. All other elements are 0. For details on the first set of convolutional kernels, please refer to [link to documentation]. Figure 4 , Figure 4In the first set of convolutional kernels, kernel0 to kernel256*C-1 represent the 256*C convolutional kernels. Figure 4 In the equation, x0, x1, ..., x[C-1] indicate the position of -1.

[0069] In this way, by determining the shape of the convolution kernel set based on the number of channels C and the number of constants L, and by placing -1 at a specific position in the convolution kernel set and placing 0 at the remaining positions in the convolution kernel set, the first convolution kernel set can be constructed efficiently and reliably.

[0070] Optionally, the second convolutional kernel set includes L*C convolutional kernels with a width of 1, a height of 1, and a number of channels of C. Furthermore, the j-th element of the L*(j-1)+1 to L*j-th convolutional kernels in the second convolutional kernel set is -2, and the remaining elements in the second convolutional kernel set are all 0.

[0071] It should be noted that the construction method of the second convolutional kernel set is similar to that of the first convolutional kernel set. The construction method of the second convolutional kernel set can be found in the explanation of the construction method of the first convolutional kernel set; the only difference is... Figure 4 x0, x1, ..., x[C-1] in the text indicate the location of -2.

[0072] In this way, by determining the shape of the convolution kernel set based on the number of channels C and the number of constants L, and by placing -2 at specific positions in the convolution kernel set and 0 at the remaining positions in the convolution kernel set, the second convolution kernel set can be constructed efficiently and reliably.

[0073] Step 2203: Determine the third convolution kernel set based on the first table, the number of channels C, and the number of constants L.

[0074] Optionally, the third convolutional kernel set includes C convolutional kernels with a width of 1, a height of 1, and a channel number of L*C. In the s-th convolutional kernel of the third convolutional kernel set, the L consecutive elements starting from the L*(s-1)+1-th element are the L values ​​in the first table, and the remaining elements in the third convolutional kernel set are all 0.

[0075] Since L is 256, the L values ​​in the first table are f(-128), f(-127), ..., f(126), f(127). The third convolutional kernel set can include 256 convolutional kernels with a width of 1, a height of 1, and 256*C channels. Furthermore, the C convolutional kernels in the third convolutional kernel set satisfy the following: the first to the 256th elements in the first convolutional kernel set are f(-128), f(-127), ..., f(127). (126), f(127), the 257th to 512th elements in the second convolutional kernel set are f(-128), f(-127), ..., f(126), f(127) respectively, the 513th to 768th elements in the third convolutional kernel set are f(-128), f(-127), ..., f(126), f(127) respectively, and so on. Except for these elements, all other elements are 0. For details on the third convolutional kernel set, please refer to [link to documentation]. Figure 5 , Figure 5 In the set of kernels, kernel0 to kernelC-1 represent the C convolutional kernels included in the third convolutional kernel set. Figure 5 The curly braces indicate the positions of f(-128), f(-127), ..., f(126), and f(127).

[0076] In this way, by determining the shape of the convolution kernel set based on the number of channels C and the number of constants L, and by placing the values ​​in the first table at specific positions in the convolution kernel set and placing the element 0 at the remaining positions in the convolution kernel set, the third convolution kernel set can be constructed efficiently and reliably.

[0077] Step 2205: Based on the first set of convolutional kernels, the second set of convolutional kernels, the third set of convolutional kernels, and L constants within a specified constant range, transform the first operation of the first input feature map of the first network layer into the second operation.

[0078] Optionally, the second operation includes:

[0079] The first feature map is obtained by performing a convolution operation between the first set of convolution kernels and the first input feature map;

[0080] Perform a first bias operation on the first feature map to obtain the second feature map;

[0081] The second feature map is subjected to a modified linear unit operation to obtain the third feature map;

[0082] The second set of convolution kernels is convolved with the third feature map to obtain the fourth feature map, which has the same width, height and number of channels as the second feature map.

[0083] The fifth feature map is obtained by adding each element in the fourth feature map to each element in the second feature map.

[0084] Perform the second bias operation on the fifth feature map to obtain the sixth feature map;

[0085] The third set of convolutional kernels is convolved with the sixth feature map to obtain the output feature map of the second network layer.

[0086] In the first bias operation, the L bias values ​​used for the L channels corresponding to any original channel of the first input feature map are L constants in a specified constant range, and the bias value used for the second bias operation is the constant 1.

[0087] It should be noted that the output feature map of the second network layer obtained after the operation transformation is the same as the output feature map of the first network layer before the operation transformation.

[0088] Assuming the first set of convolutional kernels is represented as -1W, the second set of convolutional kernels as -2W, the third set of convolutional kernels as Wt, the first input feature map as F1, and the output feature map of the second network layer as F0, then we have:

[0089] F2=ReLU(-1w*F1+Bi) (1)

[0090] F3=-1w*F1+Bi (2)

[0091] F4=ReLU(-2w*F2+F3+1) (3)

[0092] F0=Wt*F4 (4)

[0093] In this diagram, "*" represents a convolution operation, and "+" represents an element-wise add operation. Thus, -1w*F1 can be used as the first feature map, Bi can be used as the bias parameter for the first bias operation (i.e., the bias parameter can include 256 constants in [-128, 127]), F3 can be used as the second feature map, F2 can be used as the third feature map, -2w*F2 can be used as the fourth feature map, -2w*F2+F3 can be used as the fifth feature map, the 1 after -2w*F2+F3 can be used as the bias parameter for the second bias operation (i.e., the bias parameter can include the constant 1), and F4 can be used as the sixth feature map.

[0094] It should be noted that the above formula (1) is used to obtain F2 from F1, and F2 is 256 times larger than F1 in the channel direction; the above formula (2) is used to obtain F3 from F1, and F3 is 256 times larger than F1 in the channel direction.

[0095] In a specific example, if an element at a certain position in the first input feature map (hereinafter referred to as the target element) is 1, then convolving -1w with F1 will result in a first element group corresponding to the target element in -1w*F1. This first element group contains 256 elements, all of which are -1, and these 256 elements are located in different channels. Performing a first bias operation on -1w*F1 will result in a second element group corresponding to the target element in F3. This second element group contains 256 elements, and these 256 elements are sequentially -129, -128, ..., 0, 1, ..., 125, 126. Performing a ReLU operation on F3 will result in a third element group corresponding to the target element in F2. This third element group contains 256 elements, and these 256 elements are sequentially 0, 0, ..., 0, 1, 2, 3, 4, ..., 125, 126. By convolving -2w with F2, the resulting -2w*F2 contains a fourth element group corresponding to the target element. This fourth element group contains 256 elements, which are 0, 0, ..., 0, -2, -4, -6, -8, ..., -250, -252. By adding -2w*F2 and F3 element-wise, the resulting -2w*F2+F3 contains a fifth element group corresponding to the target element. This fifth element group contains 256 elements, which are -129, -128, ..., -1, 0, -1, -2, -3, ..., -125, -126. By performing a second bias operation on -2w*F2+F3, the resulting -2w*F2+F3+1 contains a sixth element group corresponding to the target element. This sixth element group contains 256 elements, which are -128, -127, ..., 0, 1, 0, -1, ..., -124, -125 in sequence. By performing a ReLU operation on -2w*F2+F3+1, the resulting F4 contains a seventh element group corresponding to the target element. This seventh element group contains 256 elements, which are 0, 0, ..., 0, 1, 0, 0, ..., 0, 0 in sequence. That is, only one specific element among these 256 elements is 1, and the rest are 0. Thus, when performing the convolution operation between Wt and F4, the convolution result corresponding to the target element is exactly f(1), thereby realizing the second operation. Furthermore, the second operation is equivalent to the first operation, and thus the same operation result can be obtained. Optionally, the specific operational logic of the second operation can be found in [reference needed]. Figure 6 .

[0096] It is easy to see that the second operation specifically includes convolution operation, bias operation, corrected linear unit operation, and element-wise addition operation. Since neural network accelerators usually support convolution operation, bias operation, corrected linear unit operation, and element-wise addition operation, this can ensure the normal and fast implementation of the second operation, which is conducive to ensuring the normal implementation of table lookup operation.

[0097] In the embodiments of this disclosure, by referring to the number of channels C of the first input feature map and the number of constants L in the specified constant range, the first convolution kernel set and the second convolution kernel set can be determined efficiently and reliably. By referring to the first table, the number of channels C and the number of constants L, the third convolution kernel set can be determined efficiently. In this way, based on the first convolution kernel set, the second convolution kernel set, the third convolution kernel set, and the L constants in the specified constant range, the first operation can be transformed into an equivalent operation efficiently and reliably, thereby indirectly realizing the table lookup operation.

[0098] In one optional example, step 2201 includes:

[0099] In response to the absence of a set of convolutional kernels corresponding to the number of channels C in the preset correspondence, a first set of convolutional kernels is constructed based on the number of channels C and the number of constants L, and a correspondence between the number of channels C and the first set of convolutional kernels is added to the preset correspondence.

[0100] In response to the existence of a set of convolutional kernels corresponding to the number of channels C in the preset correspondence, the set of convolutional kernels corresponding to the number of channels C in the preset correspondence is determined as the first set of convolutional kernels.

[0101] As can be seen from the above introduction to the first convolutional kernel set, among the data on which the first convolutional kernel set is constructed, only the number of channels C can be regarded as a variable (because the number of channels in the input feature maps of different network layers may be different), and the rest of the data (such as the number of constants L) are constants. Conversely, if the number of channels in the input feature maps of two network layers is the same, then when the first convolutional kernel set is constructed for these two network layers respectively, the two first convolutional kernel sets constructed will be the same.

[0102] In view of this, in the embodiments of this disclosure, after obtaining the number of channels C and the number of constants L, it can be determined whether there is a set of convolutional kernels corresponding to the number of channels C in the preset correspondence.

[0103] If the judgment result is that there is no set of convolutional kernels corresponding to the number of channels C in the preset correspondence, it can be assumed that the first set of convolutional kernels has not been constructed for the network layer with the number of channels C before. Therefore, the first set of convolutional kernels can be constructed based on the number of channels C and the number of constants L as described above, and the correspondence between the number of channels C and the constructed first set of convolutional kernels can be added to the preset correspondence.

[0104] If the judgment result is that there is a set of convolutional kernels corresponding to the number of channels C in the preset correspondence, it can be assumed that the first set of convolutional kernels has been constructed for the network layer with the number of channels C. Therefore, the set of convolutional kernels corresponding to the number of channels C can be directly obtained from the preset correspondence as the first set of convolutional kernels.

[0105] In the embodiments of this disclosure, by setting a preset correspondence, the constructed first set of convolutional kernels can be reused by network layers with the same number of corresponding channels, thereby effectively saving system resources. Similarly, the constructed second set of convolutional kernels can be reused by network layers with the same number of corresponding channels, and the constructed third set of convolutional kernels can be reused by network layers with the same number of corresponding channels and the same table size.

[0106] exist Figure 2 Based on the illustrated embodiments, as Figure 7 As shown, the method further includes steps 212, 214, 216, 218 and 219.

[0107] Step 212: Determine whether the neural network model to be compiled also includes a third network layer that meets preset conditions. The preset conditions include: the operation type corresponding to the third network layer is the third operation, the third operation belongs to the lookup table operation, one of the first network layer and the third network layer uses the output result of the other, and the second table is the table used by the third operation. If it includes the third network layer, proceed to step 214; if it does not include the third network layer, proceed to step 220.

[0108] Optionally, the neural network model to be compiled may contain information on the usage of the output results of each network layer. In this way, the network layer that uses the output results of the first network layer can be determined based on the information on the usage of the output results of the neural network model to be compiled, and it can be determined which network layer's output results are used by the first network layer.

[0109] In one example, the first network layer in the neural network model to be compiled is conv1, and the output of conv1 is represented as follows: conv2_output = conv(conv1_output). Therefore, it can be determined that the network layer conv2 uses the output of conv1. Since the operation type corresponding to conv2 is convolution, which is not a lookup operation, conv2 cannot be used as the third network layer that meets the preset conditions. If the network layer using the output of conv1 is another network layer, and the operation type corresponding to that network layer is a lookup operation, then that network layer can be used as the third network layer that meets the preset conditions, and the table used by the third network layer becomes the second table.

[0110] Step 214: Nest the first function associated with the first table and the second function associated with the second table to generate the third function.

[0111] It should be noted that the first function associated with the first table refers to the fixed-point function simulated by the lookup operation corresponding to the first network layer, and the second function associated with the second table refers to the fixed-point function simulated by the lookup operation corresponding to the third network layer.

[0112] Function nesting typically refers to using some functions as parameters of another function. Suppose the first function associated with the first table is f(x) = sinx, the second function associated with the second table is f(x) = cosx, and the third network layer uses the output of the first network layer; then the third function obtained through function nesting can be f(x) = cos(sinx). Similarly, suppose the first function associated with the first table is f(x) = x^2, the second function associated with the second table is f(x) = cosx, and the first network layer uses the output of the third network layer; then the third function obtained through function nesting can be f(x) = (cosx)^2.

[0113] Step 216: Determine the third table based on the third function and the specified constant range.

[0114] In step 216, each constant within the specified constant range can be sequentially used as the value of the independent variable in the third function. The operation represented by the third function is then performed on each constant to obtain the result value corresponding to each constant, and a third table is generated accordingly. For example, if the specified constant range is [-128, 127], and the result values ​​of -128, -127, ..., 126, 127 are z0, z1, z2, ..., z255 respectively, then the third table can be constructed by sequentially arranging z0, z1, z2, ..., z255.

[0115] Step 218: Based on the specified constant range and the third table, the combined operation of the first operation of the first input feature map and the third operation of the second input feature map of the third network layer is transformed into a fourth operation to obtain the fourth network layer. The fourth operation is an operation supported by the neural network accelerator.

[0116] It should be noted that, based on the specified constant range and the third table, the specific implementation of transforming the combined operation (which is the operation represented by the third function) of the first operation of the first input feature map and the third operation of the second input feature map of the third network layer into the fourth operation can be referred to the above description of the implementation of transforming the first operation into the second operation based on the specified constant range and the first table. This embodiment of the present disclosure will not elaborate further on this. Thus, the fourth operation can be obtained. Similar to the second operation, the fourth operation may also include convolution operation, bias operation, modified linear unit operation, element-wise addition operation, etc.

[0117] Step 219: Based on the network layers other than the first and third network layers in the neural network model to be compiled, as well as the fourth network layer, a second target neural network model is generated through compilation processing.

[0118] In step 219, the compiler backend can perform compilation processing based on the network layers other than the first and third network layers in the neural network model to be compiled, as well as the fourth network layer obtained through operation and transformation, thereby generating a binary second target neural network model. The specific compilation processing method can be any feasible method according to actual needs, and this disclosure will not elaborate on it.

[0119] In the embodiments of this disclosure, when a third network layer satisfying preset conditions exists in the neural network model to be compiled, a third function can be obtained by nesting the first function corresponding to the first network layer and the second function corresponding to the third network layer. The third function and a specified constant range are then used to generate a third table. Based on the specified constant range and the third table, the first operation of the first input feature map and the third operation of the second input feature map of the third network layer can be transformed together. Thus, during the compilation stage of the neural network model, the table lookup operations involving two related network layers (i.e., the first network layer and the third network layer satisfying preset conditions) can be transformed into operations supported by the neural network accelerator (specifically, the fourth operation). Subsequently, through the execution of the fourth operation, the table lookup operations involving these two network layers can be implemented simultaneously, thereby improving computational efficiency.

[0120] The compilation method for any neural network model provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to terminal devices and servers. Alternatively, the compilation method for any neural network model provided in this disclosure can be executed by a processor, such as by a processor executing the compilation method for any neural network model mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0121] Exemplary device

[0122] Figure 8 This is a schematic diagram of the structure of a compilation device for a neural network model provided in an exemplary embodiment of this disclosure. Figure 8 The apparatus shown includes a first acquisition module 810, a second acquisition module 820, and a first generation module 830.

[0123] The first acquisition module 810 is used to acquire the neural network model to be compiled. The neural network model to be compiled includes a first network layer whose corresponding operation type is the first operation. The first operation is a lookup table operation and the table used is the first table.

[0124] The second acquisition module 820 is used to transform the first operation of the first input feature map of the first network layer in the neural network model to be compiled acquired by the first acquisition module 810 into a second operation based on a specified constant range and a first table, so as to obtain the second network layer. The second operation is an operation supported by the neural network accelerator.

[0125] The first generation module 830 is used to generate a first target neural network model by compiling the network layers other than the first network layer in the neural network model to be compiled obtained by the first acquisition module 810 and the second network layer obtained by the second acquisition module 820.

[0126] In an optional example, such as Figure 9 As shown, the second acquisition module 820 includes:

[0127] The first construction submodule 8201 is used to determine the first convolutional kernel set and the second convolutional kernel set based on the number of channels C of the first input feature map and the number of constants L in the specified constant range;

[0128] The second construction submodule 8203 is used to determine the third set of convolution kernels based on the first table, the number of channels C, and the number of constants L;

[0129] Transformation submodule 8205 is used to transform the first operation of the first input feature map of the first network layer into the second operation based on the first convolutional kernel set constructed by the first construction submodule 8201, the second convolutional kernel set constructed by the first construction submodule 8201, the third convolutional kernel set constructed by the second construction submodule 8203, and L constants in a specified constant range.

[0130] In one optional example,

[0131] The first set of convolutional kernels includes L*C convolutional kernels with a width of 1, a height of 1, and a number of channels of C. Furthermore, the i-th element of the L*(i-1)+1 to L*i-th convolutional kernels in the first set of convolutional kernels is -1, and the remaining elements in the first set of convolutional kernels are all 0.

[0132] And / or,

[0133] The second convolutional kernel set includes L*C convolutional kernels with a width of 1, a height of 1, and a number of channels of C. Furthermore, the j-th element of the L*(j-1)+1 to L*j-th convolutional kernels in the second convolutional kernel set is -2, and the remaining elements in the second convolutional kernel set are all 0.

[0134] And / or,

[0135] The third convolutional kernel set includes C convolutional kernels with a width of 1, a height of 1, and L*C channels. In the s-th convolutional kernel of the third convolutional kernel set, the L consecutive elements starting from the L*(s-1)+1-th element are the L values ​​in the first table. The remaining elements in the third convolutional kernel set are all 0.

[0136] In one optional example, the second operation includes:

[0137] The first feature map is obtained by performing a convolution operation between the first set of convolution kernels and the first input feature map;

[0138] Perform a first bias operation on the first feature map to obtain the second feature map;

[0139] The second feature map is subjected to a modified linear unit operation to obtain the third feature map;

[0140] The second set of convolution kernels is convolved with the third feature map to obtain the fourth feature map, which has the same width, height and number of channels as the second feature map.

[0141] The fifth feature map is obtained by adding each element in the fourth feature map to each element in the second feature map.

[0142] Perform the second bias operation on the fifth feature map to obtain the sixth feature map;

[0143] The third set of convolutional kernels is convolved with the sixth feature map to obtain the output feature map of the second network layer.

[0144] In the first bias operation, the L bias values ​​used for the L channels corresponding to any original channel of the first input feature map are L constants in a specified constant range, and the bias value used for the second bias operation is the constant 1.

[0145] In one optional example, the first build submodule 8201 includes:

[0146] The first processing unit is used to respond to the absence of a set of convolutional kernels corresponding to the number of channels C in the preset correspondence, construct a first set of convolutional kernels based on the number of channels C and the number of constants L, and add a correspondence between the number of channels C and the first set of convolutional kernels to the preset correspondence;

[0147] The second processing unit is used to determine the set of convolutional kernels corresponding to the number of channels C in the preset correspondence as the first set of convolutional kernels in response to the existence of a set of convolutional kernels corresponding to the number of channels C in the preset correspondence.

[0148] In an optional example, such as Figure 10 As shown, the device also includes:

[0149] The judgment module 812 is used to determine whether the neural network model to be compiled acquired by the first acquisition module 810 also includes a third network layer that meets preset conditions. The preset conditions include: the operation type corresponding to the third network layer is a third operation, the third operation belongs to a table lookup operation, one of the first network layer and the third network layer uses the output result of the other, and the second table is the table used by the third operation. If it includes the third network layer, the second generation module 814 is triggered; if it does not include the third network layer, the second acquisition module 820 is triggered.

[0150] The second generation module 814 is used to nest the first function associated with the first table and the second function associated with the second table to generate a third function;

[0151] Module 816 is used to determine the third table based on the third function generated by the second generation module 814 and the specified constant range;

[0152] The third acquisition module 818, based on the specified constant range and the third table determined by the determination module 816, transforms the combined operation of the first operation of the first input feature map and the third operation of the second input feature map of the third network layer into a fourth operation to obtain the fourth network layer. The fourth operation is an operation supported by the neural network accelerator.

[0153] The third generation module 819 is used to generate a second target neural network model by compiling the network layers other than the first and third network layers in the neural network model to be compiled obtained by the first acquisition module 810 and the fourth network layer obtained by the third acquisition module 818.

[0154] Exemplary electronic devices

[0155] Below, for reference Figure 11 This describes an electronic device according to embodiments of the present disclosure. The electronic device may be either or both of a first device and a second device, or a standalone device independent of them, which may communicate with the first device and the second device to receive acquired input signals from them.

[0156] Figure 11 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0157] like Figure 11 As shown, the electronic device 1100 includes one or more processors 1101 and memory 1102.

[0158] The processor 1101 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1100 to perform desired functions.

[0159] The memory 1102 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1101 may execute the program instructions to implement the compilation methods of the neural network models of the various embodiments of this disclosure described above, and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0160] In one example, the electronic device 1100 may also include an input device 1103 and an output device 1104, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0161] For example, when the electronic device is a first device or a second device, the input device 1103 can be the microphone or microphone array described above, used to capture the input signal from the sound source. When the electronic device is a standalone device, the input device 1103 can be a communication network connector, used to receive the acquired input signal from the first device and the second device.

[0162] In addition, the input device 1103 may also include, for example, a keyboard, a mouse, etc.

[0163] The output device 1104 can output various information to the outside, including determined distance information, direction information, etc. The output device 1104 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0164] Of course, for the sake of simplicity, Figure 11 Only some of the components of the electronic device 1100 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1100 may include any other suitable components depending on the specific application.

[0165] Exemplary computer program products and computer-readable storage media

[0166] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods for compiling neural network models according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0167] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0168] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the compilation methods of neural network models according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0169] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0170] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0171] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0172] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0173] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0174] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for compiling a neural network model, comprising: Obtain the neural network model to be compiled, wherein the neural network model to be compiled includes a first network layer whose corresponding operation type is a first operation, and the first operation is a lookup operation and the table used is a first table; Based on a specified constant range and the first table, the first operation of the first input feature map of the first network layer is transformed into a second operation to obtain a second network layer. The second operation is an operation supported by a neural network accelerator. The number of constants in the specified constant range is the same as the number of values ​​included in the first table. The second operation is an operation equivalent to the first operation. Based on the network layers other than the first network layer in the neural network model to be compiled, and the second network layer, a first target neural network model is generated through compilation processing; The step of transforming the first operation of the first input feature map of the first network layer into a second operation based on a specified constant range and the first table includes: Based on the number of channels C of the first input feature map and the number of constants L in the specified constant range, determine the first convolution kernel set and the second convolution kernel set; Based on the first table, the number of channels C, and the number of constants L, determine the third set of convolution kernels; Based on the first set of convolutional kernels, the second set of convolutional kernels, the third set of convolutional kernels, and L constants within the specified constant range, the first operation of the first input feature map of the first network layer is transformed into a second operation.

2. The method according to claim 1, wherein, The first set of convolutional kernels includes L*C convolutional kernels with a width of 1, a height of 1, and a number of channels of C. Furthermore, the i-th element of the L*(i-1)+1 to L*i-th convolutional kernels in the first set of convolutional kernels is -1, and the remaining elements in the first set of convolutional kernels are all 0. And / or, The second convolutional kernel set includes L*C convolutional kernels with a width of 1, a height of 1, and a number of channels of C. Furthermore, the j-th element of the L*(j-1)+1 to L*j-th convolutional kernels in the second convolutional kernel set is -2, and the remaining elements in the second convolutional kernel set are all 0. And / or, The third convolutional kernel set includes C convolutional kernels with a width of 1, a height of 1, and a channel count of L*C. In the s-th convolutional kernel of the third convolutional kernel set, the L consecutive elements starting from the L*(s-1)+1-th element are the L values ​​in the first table. The remaining elements in the third convolutional kernel set are all 0.

3. The method according to claim 2, wherein, The second operation includes: The first feature map is obtained by performing a convolution operation between the first set of convolution kernels and the first input feature map. Perform a first bias operation on the first feature map to obtain a second feature map; The second feature map is subjected to a modified linear unit operation to obtain the third feature map; The second set of convolutional kernels is convolved with the third feature map to obtain a fourth feature map, which has the same width, the same height and the same number of channels as the second feature map. The fifth feature map is obtained by adding each element in the fourth feature map to each element in the second feature map. Perform a second bias operation on the fifth feature map to obtain a sixth feature map; The third set of convolutional kernels is convolved with the sixth feature map to obtain the output feature map of the second network layer; Specifically, when performing the first bias operation, the L bias values ​​used for the L channels corresponding to any original channel of the first input feature map are L constants in the specified constant range, and the bias value used for the second bias operation is constant 1.

4. The method according to claim 1, wherein, The determination of the first convolutional kernel set based on the number of channels C of the first input feature map and the number of constants L in the specified constant range includes: In response to the absence of a set of convolutional kernels corresponding to the number of channels C in the preset correspondence, a first set of convolutional kernels is constructed based on the number of channels C and the number of constants L, and a correspondence between the number of channels C and the first set of convolutional kernels is added to the preset correspondence. In response to the existence of a set of convolutional kernels corresponding to the number of channels C in the preset correspondence, the set of convolutional kernels corresponding to the number of channels C in the preset correspondence is determined as the first set of convolutional kernels.

5. The method according to claim 1, further comprising: In response to the neural network model to be compiled further comprising a third network layer that meets preset conditions, the first function associated with the first table and the second function associated with the second table are nested to generate the third function; The preset conditions include: the operation type corresponding to the third network layer is a third operation, the third operation is a table lookup operation, and one of the first network layer and the third network layer uses the output result of the other; the second table is the table used by the third operation. Based on the third function and the specified constant range, determine the third table; Based on the specified constant range and the third table, the combined operation of the first operation of the first input feature map and the third operation of the second input feature map of the third network layer is transformed into a fourth operation to obtain a fourth network layer. The fourth operation is an operation supported by the neural network accelerator. Based on the network layers other than the first and third network layers in the neural network model to be compiled, and the fourth network layer, a second target neural network model is generated through compilation processing; In response to the fact that the neural network model to be compiled does not include the third network layer, the step of transforming the first operation of the first input feature map of the first network layer into the second operation based on the specified constant range and the first table is performed.

6. A compilation device for a neural network model, comprising: The first acquisition module is used to acquire the neural network model to be compiled, wherein the neural network model to be compiled includes a first network layer whose corresponding operation type is a first operation, and the first operation is a lookup table operation and the table used is a first table; The second acquisition module is used to transform the first operation of the first input feature map of the first network layer in the neural network model to be compiled, which is obtained by the first acquisition module, into a second operation based on a specified constant range and the first table, to obtain a second network layer. The second operation is an operation supported by the neural network accelerator. The number of constants in the specified constant range is the same as the number of values ​​included in the first table. The second operation is an operation equivalent to the table lookup operation. The first generation module is used to generate a first target neural network model by compiling the network layers other than the first network layer in the neural network model to be compiled, obtained by the first acquisition module, and the second network layer obtained by the second acquisition module. The second acquisition module includes: The first construction submodule is used to determine the first convolutional kernel set and the second convolutional kernel set based on the number of channels C of the first input feature map and the number of constants L in the specified constant range; The second construction submodule is used to determine the third set of convolutional kernels based on the first table, the number of channels C, and the number of constants L; The transformation submodule is used to transform the first operation of the first input feature map of the first network layer into a second operation based on the first set of convolutional kernels constructed by the first construction submodule, the second set of convolutional kernels constructed by the first construction submodule, the third set of convolutional kernels constructed by the second construction submodule, and L constants in the specified constant range.

7. A computer-readable storage medium storing a computer program for executing a method for compiling a neural network model according to any one of claims 1-5.

8. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the compilation method of the neural network model according to any one of claims 1-5.

Images