Compilation method, compiler and related product for neural network

Abstract

The present disclosure discloses a compilation method for a neural network model and related products, including a compiler, a device and a computer readable storage medium. The compilation method can be applied to a computing processing device included in a combined processing device, which can include one or more data processing devices. The aforementioned combined processing device can also include an interface device and other processing devices. The computing processing device interacts with other processing devices to jointly complete a user-specified computing operation. The combined processing device can also include a storage device connected to the device and other processing devices, respectively, for storing data of the device and other processing devices. By utilizing the scheme of the present disclosure, a neural network with variable tensor shape can be compiled, thereby improving the compilation performance.

Description

Technical Field

[0001] This disclosure generally relates to the field of software. More specifically, this disclosure relates to a compilation method for neural networks, a compiler, an electronic device, a computer program product, and a system for real-time compilation of variable neural network models. Background Technology

[0002] Traditional deep learning frameworks (such as TensorFlow, MXNet, Caffe, and PyTorch) rely on manual optimization of operators and the creation of runtime graph interpreters to address issues like memory allocation and scheduling. Furthermore, deep learning compilers within these frameworks typically perform automatic or semi-automatic code generation during the optimization process to replace manual optimization.

[0003] While deep learning compilers offer some advantages over existing technologies in compilation, their performance optimization for neural networks with variable input tensor shapes is generally inferior to that for neural networks with invariant input tensor shapes. For example, in scenarios where neural networks are used for object detection with varying bounding box sizes, the application of deep learning compilers suffers from certain limitations. Specifically, for neural networks with different bounding box sizes, the deep learning compiler needs to perform multiple compilation operations to ensure optimization performance, resulting in significant time costs. Furthermore, after each compilation operation, the intelligent processor can only run the compiled file once, which causes significant inconvenience and a poor user experience. Summary of the Invention

[0004] In view of the technical problems mentioned in the background section above, this disclosure proposes a compilation scheme for neural networks in several aspects. By assigning values ​​to the shape of the input tensor, it can generate a single code for neural networks with different input tensor shapes, thereby reducing the amount of code and shortening the compilation time. The neural networks of the embodiments of this disclosure can be applied to various fields, such as image processing, speech processing, text processing, etc., and these processes may include, but are not limited to, recognition and classification.

[0005] In a first aspect, this disclosure provides a compilation method for a neural network, comprising: acquiring a computation graph of the neural network, wherein the computation graph includes computation nodes and data nodes and their respective label data, wherein the label data includes shape parameters for describing the shape of a tensor; assigning values ​​to the shape parameters in the label data of the data nodes; and generating operator code for the corresponding operator in the computation nodes based on the assigned shape parameters and an operator output shape formula library.

[0006] In a second aspect, this disclosure provides a compiler for a neural network, comprising: an acquisition unit for acquiring a computation graph of the neural network, wherein the computation graph includes computation nodes and data nodes and their respective label data, wherein the label data includes shape parameters for describing the shape of a tensor; an assignment unit for assigning values ​​to the shape parameters in the label data of the data nodes; and an operator generation unit for generating operator code for corresponding operators in the computation nodes based on the assigned shape parameters and an operator output shape formula library.

[0007] In a third aspect, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any of the first aspects of this disclosure.

[0008] In a fourth aspect, this disclosure provides a computer program product including a computer program that, when executed by a processor, implements the method of any one of the first aspects of this disclosure.

[0009] In a fifth aspect, this disclosure provides a system for real-time compilation of a variable neural network model, comprising: an interconnected host and a device, wherein the host includes a compiler according to a second aspect of this disclosure; and wherein the device is configured to execute code compiled by the compiler.

[0010] By utilizing the compilation methods, compilers, electronic devices, computer program products, and systems for real-time compilation of variable neural network models provided above, the solution disclosed herein can generate a single code for neural networks with different input tensor shapes by assigning values ​​to the input tensor shape, thereby reducing code size and shortening compilation time. When running the compiled code generated according to the solution disclosed herein, the user simply fills in a new input tensor shape to compile neural networks with different input tensor shapes, making it user-friendly. Attached Figure Description

[0011] The above and other objects, features, and advantages of exemplary embodiments of the present disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the present disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0012] Figure 1 A flowchart illustrating a compilation method for a neural network according to an embodiment of the present disclosure is shown;

[0013] Figure 2 A calculation diagram according to an embodiment of the present disclosure is shown;

[0014] Figure 3 A schematic diagram of the tensor shape of an embodiment of the present disclosure is shown;

[0015] Figure 4 Another computational diagram of an embodiment of this disclosure is shown;

[0016] Figure 5 A flowchart illustrating a compilation method for a neural network according to another embodiment of this disclosure is shown;

[0017] Figure 6 A computational graph is shown that includes category data for indicating neural network data categories according to embodiments of this disclosure;

[0018] Figure 7 Another computational diagram of an embodiment of this disclosure is shown;

[0019] Figure 8 A functional block diagram of a compiler for neural networks according to an embodiment of this disclosure is shown;

[0020] Figure 9 A structural diagram of a combined processing apparatus according to an embodiment of this disclosure is shown; and

[0021] Figure 10 A schematic diagram of the structure of a board according to an embodiment of this disclosure is shown. Detailed Implementation

[0022] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0023] The specific embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0024] Figure 1 A flowchart illustrating a compilation method for a neural network according to an embodiment of this disclosure is shown. Figure 1 As shown, in step S101, the computation graph of the neural network is obtained. In one embodiment, the computation graph may include computation nodes and data nodes and their respective label data, wherein the label data may include shape parameters for describing the shape of the tensor. Depending on the application scenario, the shape parameters may be one or more, and may include constant data and / or variable data. In the context of this disclosure, the shape parameters may also be used to describe the length of each specific order of the tensor (to be discussed later). Figure 3(Description). To facilitate understanding of the scheme disclosed herein, the following will first combine... Figure 2 Let's introduce the computational graph of neural networks.

[0025] A neural network (i.e., a neural network model in the context of this invention) typically includes an input layer, convolutional layers, activation function layers, pooling layers, and / or fully connected layers. Depending on the application, the number of layers in a neural network can range from a few to hundreds. In computational operations, each layer of the neural network can execute a corresponding operator; for example, a convolutional layer executes the convolution operator to perform a convolution operation. Typically, a computational graph can be used to describe the computation process of operators in a neural network model, thereby demonstrating the computations and states involved in the neural network model.

[0026] Figure 2 A computation graph 200 according to an embodiment of this disclosure is shown. In the field of machine learning, a computation graph is a directed graph describing a computation process, which typically contains a set of nodes and edges. Specifically, nodes can represent data input nodes, output nodes, model parameters, etc. Additionally, nodes can also represent various operation processes, including mathematical operations, variable reading and writing, data filling, etc. Therefore, the nodes in the computation graph can be broadly divided into data nodes 202 and computation nodes 204, shown as circles and rectangles respectively in the figure. Edges in the computation graph (indicated by 206 in the figure) can generally be divided into two categories: edges that transmit specific data and edges that represent control dependencies between nodes. All data in a neural network model can be modeled as tensors, which can represent all data in the computation graph, and tensors are passed between nodes in the computation graph. Stateful parameters in the computation graph, such as model parameters, can be stored using variables. Corresponding to variables are constants (i.e., Constant or Const), used to represent immutable parameters.

[0027] The shape of a tensor corresponds to its shape attribute, representing the length of each order. A tensor can be viewed as an N-dimensional array, and the order of a tensor can be understood as the dimension of the array, while the length of each order in a tensor is represented by its shape. To facilitate understanding of the scheme disclosed herein, the following will continue to combine... Figure 3 The shape of tensors is introduced.

[0028] Figure 3 A schematic diagram of the tensor shape of an embodiment of this disclosure is shown. (As follows) Figure 3As shown, a 0th-order tensor corresponds to a scalar data, and its shape is empty; a 1st-order tensor corresponds to a one-dimensional vector, and its shape contains one element, the value of which is the length of the vector. In the example shown, the shape of the 1st-order tensor is (3); a 2nd-order tensor corresponds to a matrix, and in the figure, a 3×3 matrix is ​​used as an example, so the shape of the 2nd-order tensor is (3, 3); a 3rd-order tensor corresponds to a three-dimensional array, and its shape should contain 3 elements, each element value corresponding to the length of each order. Therefore, the shape of the 3rd-order tensor in the example is (2, 3, 3). It can be seen that the shape parameters used to describe the length of each order of the tensor can be variable data or constant data. Among them, the variable data in the shape parameters of the tensor shape can refer to the data whose parameter values ​​can only be obtained at runtime. The constant data in the shape parameters of the tensor shape can refer to the data whose parameter values ​​can be obtained at compile time.

[0029] In addition to shape, tensors also have attributes such as data type, name, operation that computes the tensor, and a computation graph containing the tensor. Tensor data types include signed integers, unsigned integers, floating-point numbers, complex numbers, Boolean numbers, strings, and so on.

[0030] In the context of this disclosure, all data nodes and computation nodes in the computation graph have associated tensors as described above. Specifically, the label data of computation nodes and data nodes in the computation graph is the tensor associated with that computation node or data node. To facilitate understanding of the scheme of this disclosure, the following will be combined with... Figure 4 The computational graph with labeled data is explained.

[0031] Figure 4 Another computational diagram 400 of an embodiment of this disclosure is shown. (As...) Figure 4As shown, computation graph 400 includes five data nodes and two computation nodes. The five circular or elliptical boxes representing "Input," "Convolution Kernel," "Bias," "Hidden," and "Output" are data nodes, while the two rectangular boxes representing "Convolution" and "Activation" are computation nodes. In computation graph 400, the rectangular boxes at each node represent the tensor data associated with that node, i.e., the node's label data. Taking the label data "Shape=(in=?,ic=256,ih=?,iw=?),Type=int" at the "Input" node as an example, 'shape' represents the shape of the tensor at the "Input" node, and 'Type' represents the data type of the tensor at the "Input" node. The tensor shape at the "Input" node is a 4th-order tensor, i.e., the number of input tensors (in), the number of input tensor channels (ic), the height of the input tensor (ih), and the width of the input tensor (iw). The data type of the tensor at the "Input" node is int (integer). The input tensor data, input tensor height, and input tensor width are variables, while the number of input tensor channels is a constant (256).

[0032] Similarly, the tensor shape at the "kernel" node is a 4th-order tensor, with the following parameters: kernel tensor count (kn), kernel tensor channels (kc), kernel tensor height (kh), and kernel tensor width (kw). The tensor data type at the "kernel" node is int (integer). The kernel tensor count, kernel tensor channels, kernel tensor height, and kernel tensor width are all constants. The tensor shape at the "hidden" node is also a 4th-order tensor, with the following parameters: hidden tensor count (tn), hidden tensor channels (tc), hidden tensor height (th), and hidden tensor width (tw). The tensor data type at the "hidden" node is float. The hidden tensor count, hidden tensor channels, hidden tensor height, and hidden tensor width are all variables. The tensor shape at the "output" node is a 4th-order tensor, with the following parameters: output tensor count (on), output tensor channels (oc), output tensor height (oh), and output tensor width (ow). The tensor data type at the "output" node is float. Among them, the number of output tensors (on), the number of output tensor channels (oc), the height of output tensor (oh), and the width of output tensor (ow) are variables.

[0033] The tensor shape at the "bias" node is a first-order tensor, i.e., the bias amount (bn), and the tensor data type at the "bias" node is floating-point (float). Among them, the bias amount (bn) is constant (512). The tensor shape at the "convolution" node is a second-order tensor, i.e., stride height (sh) and stride width (sw), and both stride height (sh) and stride width (sw) are constant.

[0034] Continue to refer to Figure 1After obtaining the computational graph of the neural network as described above, in step S102, the shape parameters in the label data of the data nodes are assigned values. As previously mentioned, the shape parameters disclosed herein may include variable data and / or constant data.

[0035] In this embodiment, the aforementioned variable data / constant data can be variable data / constant data at input nodes in the computation graph or variable data / constant data at hidden nodes in the computation graph. Furthermore, this embodiment does not limit the position of the variable data / constant data in the computation graph. Taking variable data at input nodes as an example, if the tensor shape at the input node is a 4th-order tensor (in, ic, ih, iw), where each order tensor is a variable, then the tensor shape at the input node can be assigned values, generating the following assignment code:

[0036] shapes[0] = in; / / Assign values ​​to the input tensor shapes

[0037] shapes[1] = ic;

[0038] shapes[2] = ih;

[0039] shapes[3] = iw;

[0040] Among them, shapes[0], shapes[1], shapes[2], and shapes[3] are the addresses of the tensor shape storage area.

[0041] In this embodiment, the shape of the tensor, which is an N-order tensor, can be a variable for each order or a variable for some orders. Taking the label data at the input node as an example, if the tensor shape at the input node is a 4-order tensor (in, ic, ih, iw), where in, ih, and iw are variables and ic is a constant IC, then the tensor shape (i.e., the shape parameter) at the input node is assigned a value, generating the following assignment code:

[0042] shapes[0] = in; / / Assign values ​​to the input tensor shapes

[0043] shapes[1] = IC;

[0044] shapes[2] = ih;

[0045] shapes[3] = iw;

[0046] Among them, shapes[0], shapes[1], shapes[2], and shapes[3] are the addresses of the tensor shape storage area.

[0047] Next, in step S103, the operator code of the corresponding operator in the calculation node is generated based on the assigned shape parameters and the operator output shape formula library.

[0048] In one implementation, the computation nodes in the computation graph contain operations for the neural network, and each computation node has an operator corresponding to the neural network operation. Here, the neural network operation operators may include, but are not limited to, convolution, max pooling, biasing, softmax, sigmoid, ReLU, etc. Based on the properties of each operator, there is a fixed correspondence between the input tensor shape and the output tensor shape of each operator. Taking the ReLU operator as an example, after performing a ReLU activation operation on a 4th-order tensor with an input tensor shape of (in, ic, ih, iw), the resulting output tensor shape is (in, ic, ih, iw). That is, for the ReLU operator, the input tensor shape (in, ic, ih, iw) and the output tensor shape (on, oc, oh, ow) have the following formula: on = in, oc = ic, oh = ih, ow = iw.

[0049] The above only shows the output shape formula of the ReLU operator. As those skilled in the art will know, each operator in a neural network operation has a corresponding output shape formula. Therefore, the output shape formulas of various operators can be collected to form an operator output shape formula library.

[0050] In this embodiment, after assigning values ​​to the shape parameters at the data node, the computation nodes in the computation graph can be traversed to find the computation nodes connected to the data node. Then, the operator code corresponding to the operator in the computation node can be generated based on the assigned shape parameters and the operator output shape formula library. The following uses a convolution computation node in the computation graph as an example to illustrate the code generation process of the convolution operator corresponding to this computation node.

[0051] Taking the variable data at the input node as an example, if the tensor shape at the input node is a 4th order tensor (in, ic, ih, iw), after assigning values ​​to the variable data at the input node, the variable data (in = shapes[0], ic = shapes[1], ih = shapes[2], iw = shapes[3]) is obtained. By traversing the computation nodes in the computation graph, the computation node connected to the input node is found to be a convolution. According to the output shape formula library of the convolution operator: on = in, oc = kn, oh = (ih-kh) / sh+1, ow = (iw-kw) / sw+1, the output shape of the convolution operator corresponding to the computation node can be obtained as follows:

[0052] shapes[4] = shapes[0];

[0053] shapes[5] = KN;

[0054] shapes[6]=(shapes[2]-KH) / SH+1;

[0055] shapes[7]=(shapes[3]-KW) / SW+1;

[0056] Wherein, KN is the number of kernel tensors (constant type) at the "convolution kernel" data node connected to the computing node, KH is the height of the kernel tensor (constant type) at the "convolution kernel" data node connected to the computing node, KW is the width of the kernel tensor (constant type) at the "convolution kernel" data node connected to the computing node, SH is the stride height (constant type) at the computing node, and SW is the stride width (constant type) at the computing node. Wherein, shapes[4], shapes[5], shapes[6], shapes[7] are the storage addresses of the tensor shape of the output tensor of the convolution operator. The method for determining the storage address of the tensor shape can be found in the description below. Thus, the shape parameters of the output tensor of the "hidden" convolution computing node are assigned values. Next, the tensor shape parameters at the output node can be assigned using the shapes[4] to shapes[7] obtained above, so as to obtain the assigned tensor shape data (on=shapes[4],oc=shapes[5],oh=shapes[6],ow=shapes[7]).

[0057] Furthermore, the operator code of the convolution computation node can be obtained based on the assigned shape parameters and the attributes of the convolution computation node. For example, the operator code of this convolution computation node can be represented as follows:

[0058] int temp[shape[4]*shape[5]*shape[6]*shape[7]];

[0059] conv(input,kernel,bias,temp,shape[0],shape[1],shape[2],shape[3],KN,KH,KW,SH,SW).

[0060] Similarly, corresponding operator code can be generated for each computation node in the computation graph.

[0061] As can be seen, the compilation method for neural networks disclosed herein represents the shape parameters (such as variable data) in the input data nodes using a unified assignment, and derives the corresponding output data node through an operator output shape formula library. Then, a similar assignment operation can be performed on the shape of the output data node, thus eliminating the need to concern oneself with the specific shape of the input data during the neural network compilation process. Based on this assignment operation, the scheme disclosed herein can compile computation graphs with different input tensor shapes into a single code, thereby reducing the amount of compiled code and significantly shortening compilation time. Furthermore, when running the compiled code generated according to the scheme disclosed herein, the user only needs to input a new input tensor shape to compile neural networks with different input tensor shapes, thus obtaining a relatively user-friendly experience.

[0062] Figure 5 A flowchart illustrating a compilation method for a neural network according to another embodiment of this disclosure is shown.

[0063] like Figure 5 As shown, in step S501, the computational graph of the neural network is obtained. It can be understood that step S501 and... Figure 1 The middle step S101 performs essentially the same operation, therefore regarding Figure 1 The description of step S101 also applies to step S501 here, and therefore will not be repeated.

[0064] In step S502, a pre-code is generated based on the input data node, the output data node, and the label data.

[0065] In this embodiment, the input data node refers to the data node connected to the first computation node in the computation graph in the input direction. Figure 4 Taking the computation graph 400 shown as an example, the input data nodes of this computation graph 400 include the "input," "convolution kernel," and "bias" data nodes, indicated by circles or ellipses in the graph. The output data nodes refer to the data nodes corresponding to the output layer of the neural network in the computation graph, still denoted as... Figure 4 Taking the computation graph 400 shown as an example, the output data nodes of the computation graph 400 include the "output" data nodes shown as ellipses in the figure. If we take... Figure 2 Taking the computation graph 200 shown as an example, the output data node of the computation graph 200 is 210.

[0066] In this embodiment, label data refers to the label data of all nodes in the computation graph, including the label data of computation nodes and the label data of data nodes. In one implementation, the label data here may include variable data and / or constant data.

[0067] In this embodiment, the pre-assignment code refers to the code used to assist in generating the assignment code according to step S503 and the operator code according to step S504. Taking the assignment code generated according to step S503 as "shapes[0] = in; shapes[1] = IC; shapes[2] = ih; shapes[3] = iw" as an example, since IC is a constant, that is, data that the user cannot change, in order to make the assignment code executable, the constant IC needs to be defined before generating the assignment code. Here, the code that defines the constant IC is the pre-assignment code. For example, the pre-assignment code is "#define IC 256".

[0068] After generating the aforementioned pre-processing code, in step S503, the shape parameters in the label data corresponding to the data node are assigned values. It can be understood that step S503 is related to... Figure 1 Step S102 performs essentially the same operation, so it will not be described again here. Next, in step S504, the operator code for the corresponding operator in the computation node is generated based on the assigned shape parameters and the operator output shape formula library. Similarly, step S504 is... Figure 1 The intermediate step S103 is basically the same, so it will not be described again.

[0069] In some optional implementations of this embodiment, the pre-code includes constant definition code and function header code. Constant definition code refers to code that defines constant data in the label data at the computation graph node, such as "#define IC 256" (where IC is a constant). In some embodiments, constant data in the label data at at least one input data node in the computation graph and the corresponding computation node can be defined. Figure 4 Taking the computation graph 400 shown as an example, one input data node in the computation graph 400 is: an "Input" data node, a "Convolution Kernel" data node, and a "Bias" data node. The computation node connected to this input node is the "Convolution" computation node. First, the first constant data in the label data of the "Input" data node, the "Convolution Kernel" data node, and the "Bias" data node are read: ic = 256, bn = 512, kn = 512, kc = 256, kh = 2, kw = 2, and the constant data in the label data of the "Convolution" computation node are read: sh = 2, sw = 2. Then, the first and second constants are named: ic is named IC, bn is named BN, kc is named KC, kn is named KN, kh is named KH, kw is named KW, sh is named SH, and sw is named SW. Finally, the code for defining the named first and second constants is generated to obtain the constant definition code. The following "constant definition" code is generated:

[0070] #define IC 256;

[0071] #define KC 256

[0072] #define KN 512;

[0073] #define BN 512

[0074] #define KH 2;

[0075] #define KW 2;

[0076] #define SH 2;

[0077] #define SW 2.

[0078] In other embodiments, constant data at all nodes in the computation graph can be defined to generate constant definition code. The specific method for generating constant definition code is basically the same as the method for generating constant definition code at the input data nodes and corresponding computation nodes, and therefore will not be described again.

[0079] In some other embodiments, naming the first constant and the second constant further includes: in response to detecting that there are duplicate constant elements in the first constant and / or the second constant, merging and naming the duplicate constant elements. Figure 4 Taking the computation graph 400 shown as an example, the constants "ic" in the label data of the "input" data node and "kc" in the label data of the "convolution kernel" data node are the same (both are 256), and the constants "kn" in the label data of the "convolution kernel" data node and "bn" in the label data of the "bias" data node are the same (both are 512). Therefore, "ic" and "kc" can be merged and named "IC", and "kn" and "bn" can be merged and named "KN". After merging and naming, the defined constant codes for the input data nodes and corresponding computation nodes of the computation graph 400 are as follows:

[0080] #define IC 256;

[0081] #define KN 512;

[0082] #define KH 2;

[0083] #define KW 2;

[0084] #define SH 2;

[0085] #define SW 2.

[0086] The function header code refers to the code preceding the function body, providing information about the function's prototype, such as its return type, the order and type of each parameter. In this disclosed embodiment, the function body can be the operator code of the operator corresponding to the computation node in the computation graph generated according to step S103 of this disclosed embodiment. In some embodiments, the function header code can be generated based on variable data in the input data node, output data node, and label data. The function header code may include a function name and parameter codes. Generating the function header code based on variable data in the input data node, output data node, and label data includes: generating the function name and parameter codes based on data in the label data of at least one input data node and at least one output data node.

[0087] by Figure 4 Taking the computation graph 400 shown as an example, the input data nodes in this computation graph 400 are: "Input" data node, "Convolution Kernel" data node, and "Bias" data node, and the output data node is: "Output" data node. The label data of the input data nodes and output data nodes in the computation graph 400 can be traversed to generate the following function name and parameter code:

[0088] void kernel(int*input,int*kernel,float*bias,float*output,int in,intih, int iw).

[0089] In this code, "int*input" indicates that the data type of the "input" data node is integer, "int* kernel" indicates that the data type of the "kernel" data node is integer, "float*bias" indicates that the data type of the "bias" data node is integer, "float*output" indicates that the return data type of the function body is floating-point, and "int in, in ih, intiw" indicates that the data types of the input tensor count (in), input tensor height (ih), and input tensor width (iw) are integer. "void kernel" indicates that the function name is "kernel".

[0090] In other embodiments, the function header code may include requesting a shape area code. Generating the function header code based on variable data in the input data nodes, output data nodes, and label data includes: generating the requesting shape area code based on the number of data nodes containing variable data in the computation graph and the size of the tensor shape storage area determined by the data corresponding to each data node. Figure 4Taking the computation graph 400 shown as an example, the data nodes containing variable data in this computation graph 400 are: "input" data node, "hidden" data node, and "output" data node. That is, the number of data nodes containing variable data in the computation graph 400 is 3. Since the shape of the tensor corresponding to each data node is 4th order, the required tensor shape storage space for the data nodes containing variable data can be calculated to be 3 * 4 = 12. Then, based on the calculated tensor shape storage space size, the following shape area allocation code can be generated:

[0091] int shapes

[12] ;

[0092] Among them, shapes[0], shapes[1], shapes[2], shapes[3], shapes[4]...shapes

[11] are the addresses of the tensor shape storage area.

[0093] After requesting the tensor shape storage area as described above, the tensor shape storage area address index can be added or updated in the label data of the data node containing variable data. This allows the shape representation of the tensor in the current data node to be determined based on the index. The range of values ​​for this address index can be determined based on the size of the tensor shape storage space required by the data node in the computation graph. Continuing the example above, if the required tensor shape storage space for the data node is 3 * 4 = 12, then the range of values ​​for the address index can be 0-11. Furthermore, the address index recorded in the label of the data node can be the index of the starting address of the tensor shape. For example, the address index in the data label of the "input" data node can be index=0, and the tensor shape of the "input" data node can be represented by shapes[0], shapes[1], shapes[2], and shapes[3]. The address index in the data label of the "hidden" data node can be index=4, indicating that the tensor shape storage area of ​​the "hidden" data node starts to be calculated from index=4. Therefore, based on this index, the tensor shape of the "hidden" data node can be represented by shapes[4], shapes[5], shapes[6], and shapes[7]. The address index in the data label of the "output" data node can be index=8, indicating that the tensor shape storage area of ​​the "output" data node starts to be calculated from index=8. The tensor shape of the "output" data node can be represented by shapes[8], shapes[9], shapes

[10] , and shapes

[11] .

[0094] In some optional implementations of this embodiment, the label data of the data nodes in the computation graph further includes category data for indicating the categories of neural network data. The category data indicates one or more of the following: input neurons, constant weights, constant neurons, hidden neurons, and output neurons. Here, category data refers to Neural Data Classification (NDC). Categories of neural network data include input neurons (NDC = IN), constant weights (NDC = CW), constant neurons (NDC = CN), hidden neurons (NDC = HN), output neurons (NDC = ON), input weights (NDC = IW), output weights (NDC = OW), hardware instructions (NDC = Inst), and auxiliary data (NDC = Aux), etc. Different categories of data have different data characteristics. For example, the data of hardware instructions (NDC=Inst), auxiliary data (NDC=Aux), constant weights (NDC=CW), and constant neurons (NDC=CN) are constant data; the data of input neurons (NDC=IN), hidden neurons (NDC=HN), output neurons (NDC=ON), input weights (NDC=IW), and output weights (NDC=OW) are variable data. Figure 6 A computational graph 600 is shown, containing category data for indicating neural network data categories according to embodiments of this disclosure. Embodiments of this disclosure improve the compilation efficiency of neural networks by rapidly processing label data through reading category data from label data at nodes in the computational graph.

[0095] In some optional implementations of this embodiment, the operator code may further include hidden neuron space allocation code and / or operator template code, wherein the hidden neuron space allocation code is generated by searching for hidden neurons connected to the computing node, and the operator template code is generated based on the attributes of the operators in the computing node. Figure 6 Taking the computation graph 600 shown as an example, we can search for data nodes connected to the "convolution" computation node and extract the tensor shape at the node where NDC = HN (i.e., obtain the hidden neuron connected to the "convolution" computation node). Since the tensor shape at the "hidden" data node is of order 4, we can allocate 4 tensor shape storage addresses for this hidden neuron and generate the following hidden neuron space allocation code:

[0096] int temp[shape[4]*shape[5]*shape[6]*shape[7]];

[0097] Among them, shape[4], shape[5], shape[6], and shape[7] are the storage addresses of tensor shapes.

[0098] The operator template code is related to the operator's properties, and is used in this context. Figure 7 Taking the computation graph 700 shown as an example, the input nodes connected to the convolution operator (the operator corresponding to the "convolution" computation node) include: the "input" data node, the "convolution kernel" data node, and the "bias" data node. The continuous output nodes of the convolution operator include the "hidden" data node. shape[0], shape[1], shape[2], and shape[3] are the tensor shape storage addresses at the "input" data node, and KN, KH, KW, SH, and SW are the defined constants at the "convolution kernel" data node, the "bias" data node, and the "convolution" computation node. Therefore, template code for the convolution operator can be generated:

[0099] conv(input,kernel,bias,temp,shape[0],shape[1],shape[2],shape[3],KN,KH,KW,SH,SW).

[0100] After introducing the methods of exemplary embodiments of this disclosure, the following references are made. Figure 8 A compiler for neural networks, based on exemplary embodiments of the present disclosure, is described.

[0101] Figure 8 A functional block diagram of a compiler for a neural network according to an embodiment of this disclosure is schematically shown. Figure 8 As shown, the compiler 800 may include: an acquisition unit 810, which is used to acquire a computation graph of a neural network, wherein the computation graph includes computation nodes and data nodes and their respective label data, wherein the label data includes shape parameters for describing the shape of tensors; an assignment unit 820, which is used to assign values ​​to the shape parameters in the label data of the data nodes; and an operator generation unit 830, which is used to generate operator code for the corresponding operator in the computation nodes based on the assigned shape parameters and the operator output shape formula library.

[0102] In some embodiments, the data node includes an input data node and an output data node, and the compiler further includes a pre-code generation unit configured to generate pre-code based on the input data node, the output data node, and the label data before assigning values ​​to the shape parameters in the label.

[0103] In some embodiments, the preamble includes constant definition code and function header code.

[0104] In some embodiments, the pre-code generation unit includes a constant code generation unit configured to generate definition constant code based on constant data in the tag data.

[0105] In some embodiments, the pre-code generation unit includes a function header code generation unit configured to generate function header code based on variable data in the input data node, output data node, and label data.

[0106] In some embodiments, the label data of data nodes in the computation graph further includes category data for indicating the categories of neural network data, which are used to indicate one or more of input neurons, constant weights, constant neurons, hidden neurons, and output neurons.

[0107] In some embodiments, the constant code generation unit includes: a reading unit configured to read a first constant from the label data of at least one input data node and a second constant from the label data of the corresponding operator; a naming unit configured to name the first constant and the second constant; and a generation unit configured to generate code for defining the named first constant and the second constant, so as to obtain constant definition code.

[0108] In some embodiments, the naming unit is further configured to merge and name the duplicate constant elements in response to detecting that there are duplicate constant elements in the first constant and / or the second constant.

[0109] In some embodiments, the function header code includes a function name and parameter code, wherein the function header code generation unit is further configured to generate the function name and parameter code based on data in the label data of at least one input data node and at least one output data node.

[0110] In some embodiments, the function header code further includes shape area request code, wherein the function header code generation unit is further configured to generate shape area request code based on the number of data nodes containing variable data in the computation graph and the size of the tensor shape storage area determined by the data corresponding to each data node.

[0111] In some embodiments, the operator code further includes hidden neuron space allocation code and / or operator template code, wherein the operator generation unit is further configured to generate hidden neuron space allocation code by searching for hidden neurons connected to the computing node, and to generate operator template code according to the properties of the operators in the computing node.

[0112] It should be understood that the units and references described in Compiler 800 Figure 1 and Figure 1 The steps in the described method correspond accordingly. Therefore, the operations and features described above for the compilation method used in neural networks also apply to compiler 800 and the units contained therein, and will not be repeated here.

[0113] Figure 9This is a structural diagram illustrating a combined processing apparatus according to an embodiment of the present disclosure. It will be understood that the combined processing apparatus disclosed herein can be used to perform the compilation operations described for the foregoing of this disclosure in conjunction with the accompanying drawings. In some scenarios, the combined processing apparatus may include the compiler described for the foregoing of this disclosure in conjunction with the accompanying drawings. In other scenarios, the combined processing apparatus may be linked to the compiler described for the foregoing of this disclosure in conjunction with the accompanying drawings to execute an executable program obtained after compilation by the compiler.

[0114] like Figure 9 As shown, the combined processing device 900 includes a computing processing device 902, an interface device 904, other processing devices 906, and a storage device 908. Depending on the application scenario, the computing processing device may include one or more computing devices 910, which can be configured to perform various computing operations, such as the various operations involved in machine learning in the field of artificial intelligence.

[0115] In different embodiments, the computing processing apparatus disclosed herein can be configured to perform user-specified operations. In exemplary applications, the computing processing apparatus can be implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Thus, the operator code described above in conjunction with the accompanying drawings can be executed by the intelligent processor. Similarly, one or more computing devices included within the computing processing apparatus can be implemented as an artificial intelligence processor core or a portion of the hardware structure of an artificial intelligence processor core. When multiple computing devices are implemented as artificial intelligence processor cores or portions of the hardware structure of artificial intelligence processor cores, the computing processing apparatus disclosed herein can be considered to have a single-core structure or a homogeneous multi-core structure.

[0116] In exemplary operation, the computing processing device disclosed herein can interact with other processing devices through an interface device to jointly complete user-specified operations. Depending on the implementation, the other processing devices disclosed herein may include one or more types of processors such as a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and an artificial intelligence processor, both general-purpose and / or dedicated-purpose processors. These processors may include, but are not limited to, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, the computing processing device disclosed herein can be considered to have a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and other processing devices are considered together, they can be considered to form a heterogeneous multi-core structure.

[0117] In one or more embodiments, the other processing device may serve as an interface between the computing processing device disclosed herein (which may be specifically embodied in artificial intelligence, such as neural network operations) and external data and control, performing basic controls including but not limited to data transfer, starting and / or stopping the computing device. In another embodiment, the other processing device may also cooperate with the computing processing device to jointly complete computational tasks.

[0118] In one or more embodiments, the interface device can be used to transfer data and control instructions between a computing processing device and other processing devices. For example, the computing processing device can obtain input data from other processing devices via the interface device and write it to on-chip storage (or memory) of the computing processing device. Further, the computing processing device can obtain control instructions from other processing devices via the interface device and write them to on-chip control cache of the computing processing device. Alternatively or optionally, the interface device can also read data from the storage device of the computing processing device and transmit it to other processing devices. In some scenarios, the interface device can also be implemented as an application programming interface between the computing processing device and other processing devices, including, for example, a driver interface, to transfer various instructions and programs to be executed by the computing processing device between them.

[0119] Additionally or optionally, the combined processing apparatus disclosed herein may further include a storage device. As shown in the figures, the storage device is connected to both the computing processing device and the other processing device. In one or more embodiments, the storage device may be used to store data from the computing processing device and / or the other processing device. For example, the data may be data that cannot be fully stored in the internal or on-chip storage of the computing processing device or other processing device.

[0120] In some embodiments, this disclosure also discloses a chip (e.g. Figure 10 The chip shown is 1002. In one implementation, this chip is a system-on-a-chip (SoC). This chip can interface with external devices (such as...) Figure 10 The external interface device 1006 shown is connected to other related components. These related components may be, for example, a camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. In some applications, the chip may integrate other processing units (e.g., video codecs) and / or interface modules (e.g., DRAM interfaces). In some embodiments, this disclosure also discloses a chip package structure that includes the aforementioned chip. In some embodiments, this disclosure also discloses a board that includes the aforementioned chip package structure. The following will be combined with… Figure 10 This board is described in detail.

[0121] Figure 10 This is a schematic diagram illustrating the structure of a board 1000 according to an embodiment of this disclosure, which may include the intelligent processor architecture described in conjunction with the accompanying drawings. Figure 10 As shown, the board includes a storage device 1004 for storing data, which includes one or more storage cells 1010. This storage device can be connected and transmit data with the controller 1008 and the aforementioned chip 1002 via, for example, a bus. Furthermore, the board also includes an external interface device 1006, configured for data relay or switching between the chip (or a chip in a chip package) and an external device 1012 (e.g., a server or computer). For example, data to be processed can be transferred from the external device to the chip via the external interface device. Alternatively, the calculation results of the chip can be transmitted back to the external device via the external interface device. Depending on the application scenario, the external interface device can have different interface forms; for example, it can adopt a standard PCIe interface.

[0122] In one or more embodiments, the controller in the disclosed board can be configured to regulate the state of the chip. Therefore, in one application scenario, the controller may include a microcontroller (MCU) for regulating the operating state of the chip.

[0123] Based on the above combination Figure 9 and Figure 10 Based on the description, those skilled in the art will understand that this disclosure also discloses an electronic device or apparatus that may include one or more of the aforementioned boards, one or more of the aforementioned chips, and / or one or more of the aforementioned combined processing apparatus.

[0124] Depending on the application scenario, the electronic devices or apparatus disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablets, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound machines, and / or electrocardiographs. The electronic devices or apparatus disclosed herein can also be applied in fields such as the Internet, IoT, data centers, energy, transportation, public management, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and healthcare. Furthermore, the electronic devices or apparatus disclosed herein can also be used in application scenarios related to artificial intelligence, big data, and / or cloud computing, such as cloud computing, edge computing, and terminal applications. In one or more embodiments, the high-computing-power electronic devices or apparatuses according to the present disclosure can be applied to cloud devices (e.g., cloud servers), while the low-power electronic devices or apparatuses can be applied to terminal devices and / or edge devices (e.g., smartphones or cameras). In one or more embodiments, the hardware information of the cloud devices and the hardware information of the terminal devices and / or edge devices are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud devices to simulate the hardware resources of the terminal devices and / or edge devices based on the hardware information of the terminal devices and / or edge devices, so as to complete the unified management, scheduling and collaborative work of end-to-cloud or cloud-edge-end integration.

[0125] It should be noted that, for the sake of brevity, this disclosure describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solutions disclosed herein are not limited by the order of the described actions. Therefore, based on the disclosure or teachings of this document, those skilled in the art will understand that some steps can be performed in a different order or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this disclosure can be considered optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions disclosed herein. In addition, depending on the solution, the description of some embodiments in this disclosure may have different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this disclosure can also be referred to the relevant descriptions of other embodiments.

[0126] In terms of specific implementation, based on the disclosure and teachings of this document, those skilled in the art will understand that several embodiments disclosed herein can also be implemented in other ways not disclosed herein. For example, regarding the various units in the electronic device or apparatus embodiments described above, this document divides them based on logical functions, but in actual implementation, there may be other division methods. As another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. Regarding the connection relationships between different units or components, the connections discussed above in conjunction with the accompanying drawings can be direct or indirect couplings between units or components. In some scenarios, the aforementioned direct or indirect couplings involve communication connections utilizing interfaces, where the communication interface can support electrical, optical, acoustic, magnetic, or other forms of signal transmission.

[0127] In this disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network units. Furthermore, depending on actual needs, some or all of the units can be selected to achieve the purpose of the solution described in the embodiments of this disclosure. Additionally, in some scenarios, multiple units in the embodiments of this disclosure may be integrated into one unit or each unit may exist physically independently.

[0128] In some implementation scenarios, the integrated unit described above can be implemented as a software program module. If implemented as a software program module and sold or used as an independent product, the integrated unit can be stored in a computer-readable storage device (CMSDD). Therefore, when the solution disclosed herein is embodied in the form of a software product (e.g., a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to cause a computer device (e.g., a personal computer, server, or network device) to execute some or all of the steps of the method described in the embodiments of this disclosure. The aforementioned memory may include, but is not limited to, various media capable of storing program code, such as USB flash drives, flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0129] In other implementation scenarios, the integrated units described above can also be implemented in hardware, i.e., as specific hardware circuits, which may include digital circuits and / or analog circuits. The physical implementation of the circuit's hardware structure may include, but is not limited to, physical devices, which may include, but are not limited to, transistors or memristors. Therefore, the various devices described herein (e.g., computing devices or other processing devices) can be implemented using appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage units or storage devices can be any suitable storage medium (including magnetic storage media or magneto-optical storage media), such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), ROM, and RAM.

[0130] The foregoing can be better understood in accordance with the following terms:

[0131] Clause A1, A compilation method for neural networks, comprising:

[0132] Obtain the computation graph of the neural network, wherein the computation graph includes computation nodes and data nodes and their respective label data, wherein the label data includes shape parameters for describing the shape of the tensor;

[0133] Assign values ​​to the shape parameters in the label data of the data nodes; and

[0134] The operator code for the corresponding operator in the computation node is generated based on the assigned shape parameters and the operator output shape formula library.

[0135] Clause A2, the method according to Clause A1, wherein the data node includes an input data node and an output data node, the method further includes:

[0136] Before assigning values ​​to the shape parameters in the label, a pre-code is generated based on the input data node, the output data node, and the label data.

[0137] Clause A3, the method described in Clause A2, wherein the pre-code includes constant definition code and function header code.

[0138] Clause A4, the method according to Clause A3, wherein the defined constant code is generated based on constant data in the tag data.

[0139] Clause A5, the method according to A3, wherein function header code is generated based on variable data in the input data node, output data node and label data.

[0140] Clause A6. The method according to Clause A1, wherein the label data of the data nodes in the computation graph further includes category data for indicating the category of neural network data, wherein the category data is used to indicate one or more of input neurons, constant weights, constant neurons, hidden neurons, and output neurons.

[0141] Clause A7. The method described in Clause A4, wherein generating the definition constant code based on constant data in the tag data includes:

[0142] Read the first constant from the label data of at least one input data node and the second constant from the label data of the corresponding operator;

[0143] Name the first constant and the second constant; and

[0144] Generate code for defining the named first and second constants to obtain the constant definition code.

[0145] Clause A8, the method described in Clause A7, wherein naming the first constant and the second constant further includes:

[0146] In response to the detection of duplicate constant elements in the first constant and / or the second constant, the duplicate constant elements are merged and named.

[0147] Clause A9. The method according to Clause A5, wherein the function header code includes a function name and parameter codes, wherein generating the function header code based on the input data node, the output data node, and the variable data in the label data includes:

[0148] Generate a function name and parameter code based on the label data from at least one input data node and at least one output data node.

[0149] Clause A10, the method according to Clause A9, wherein the function header code further includes request shape area code, wherein generating the function header code based on variable data in the input data node, output data node, and label data includes:

[0150] Based on the number of data nodes containing variable data in the computation graph and the size of the tensor shape storage area determined by the data corresponding to each data node, a shape area application code is generated.

[0151] Clause A11. The method according to Clause A1, wherein the operator code further includes hidden neuron space allocation code and / or operator template code, wherein the hidden neuron space allocation code is generated by searching for hidden neurons connected to the computing node, and the operator template code is generated based on the properties of the operators in the computing node.

[0152] Clause A12. A compiler for a neural network, comprising: an acquisition unit for acquiring a computation graph of the neural network, wherein the computation graph includes computation nodes and data nodes and their respective label data, wherein the label data includes shape parameters for describing the shape of a tensor; an assignment unit for assigning values ​​to the shape parameters in the label data of the data nodes; and an operator generation unit for generating operator code for corresponding operators in the computation nodes based on the assigned shape parameters and an operator output shape formula library.

[0153] Clause A13, the compiler as described in Clause A12, wherein the data nodes include input data nodes and output data nodes, the compiler further includes: a pre-code generation unit configured to generate pre-code based on the input data nodes, output data nodes and label data before assigning values ​​to the shape parameters in the label.

[0154] Clause A14, the compiler as described in Clause A13, wherein the pre-code includes constant definition code and function header code.

[0155] Clause A15, the compiler as described in Clause A14, wherein the pre-code generation unit includes a constant code generation unit configured to generate definition constant code based on constant data in the tag data.

[0156] Clause A16. The compiler as described in Clause A14, wherein the pre-code generation unit includes a function header code generation unit configured to generate function header code based on variable data in the input data nodes, output data nodes, and label data.

[0157] Clause A17, the compiler as described in Clause A12, wherein the label data of the data nodes in the computation graph further includes category data for indicating the categories of neural network data, wherein the category data is used to indicate one or more of input neurons, constant weights, constant neurons, hidden neurons, and output neurons.

[0158] Clause A18. The compiler according to Clause A15, wherein the constant code generation unit comprises: a reading unit configured to read a first constant from the label data of at least one input data node and a second constant from the label data of the corresponding operator; a naming unit configured to name the first constant and the second constant; and a generation unit configured to generate code for defining the named first constant and the second constant to obtain constant definition code.

[0159] Clause A19, the compiler as described in Clause A18, wherein the naming unit is further configured to: merge and name the duplicate constant elements in response to the detection of duplicate constant elements in the first constant and / or the second constant.

[0160] Clause A20, the compiler as described in Clause A16, wherein the function header code includes a function name and parameter code, wherein the function header code generation unit is further configured to generate the function name and parameter code based on data in the label data of at least one input data node and at least one output data node.

[0161] Clause A21. The compiler according to Clause A20, wherein the function header code further includes shape area allocation code, wherein the function header code generation unit is further configured to generate shape area allocation code based on the number of data nodes containing variable data in the computation graph and the size of the tensor shape storage area determined by the data corresponding to each data node.

[0162] Clause A22, the compiler as described in Clause A12, wherein the operator code further includes hidden neuron space allocation code and / or operator template code, wherein the operator generation unit is further configured to generate hidden neuron space allocation code by searching for hidden neurons connected to the computing node, and to generate operator template code based on the properties of the operators in the computing node.

[0163] Clause A23. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any of the first aspects of this disclosure.

[0164] Clause A24. A computer program product comprising a computer program that, when executed by a processor, implements any of the methods of the first aspect of this disclosure.

[0165] Clause A25, a system for real-time compilation of a variable neural network model, comprising: an interconnected host and a device, wherein the host includes a compiler according to a second aspect of this disclosure; wherein the device is configured to execute code compiled by the compiler.

[0166] It should be understood that the terms "first," "second," "third," and "fourth," etc., in the claims, specification, and drawings of this disclosure are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this disclosure indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0167] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0168] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0169] While numerous embodiments of this disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and intent of this disclosure. It should be understood that various alternatives to the embodiments of this disclosure described herein may be employed in the practice of this disclosure. The appended claims are intended to define the scope of this disclosure and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A compilation method for neural networks, applied during the compilation phase, comprising: Obtain the computation graph of the neural network, wherein the computation graph includes computation nodes and data nodes and their respective label data, wherein the label data includes shape parameters for describing the shape of the tensor; Assigning values ​​to the shape parameters in the label data of the data node, wherein assigning values ​​to the shape parameters includes generating assignment code for assigning values ​​to the shape parameters. For variable data in the shape parameters, the assignment code represents the value of the shape parameters as a variable in the generated code, and the value of the variable is provided when the generated code is run. as well as The operator code for the corresponding operator in the computation node is generated based on the assigned shape parameters and the operator output shape formula library, wherein the shape of the operator output tensor in the operator code is represented by the value of the shape parameter represented by the assigned code.

2. The method according to claim 1, wherein the data node includes an input data node and an output data node, and the method further includes: Before assigning values ​​to the shape parameters in the label data, pre-code is generated based on the input data node, the output data node, and the label data.

3. The method according to claim 2, wherein the pre-code includes constant definition code and function header code.

4. The method of claim 3, wherein the defined constant code is generated based on constant data in the tag data.

5. The method of claim 3, wherein the function header code is generated based on the variable data in the input data node, the output data node, and the label data.

6. The method of claim 1, wherein the label data of the data nodes in the computation graph further includes category data for indicating the category of neural network data, wherein the category data is used to indicate one or more of input neurons, constant weights, constant neurons, hidden neurons, and output neurons.

7. The method of claim 4, wherein generating the definition constant code based on the constant data in the tag data comprises: Read the first constant from the label data of at least one input data node and the second constant from the label data of the corresponding operator; Name the first constant and the second constant; as well as Generate code for defining the named first and second constants to obtain the constant definition code.

8. The method of claim 7, wherein naming the first constant and the second constant further comprises: In response to the detection of duplicate constant elements in the first constant and / or the second constant, the duplicate constant elements are merged and named.

9. The method according to claim 5, wherein the function header code includes a function name and parameter codes, wherein generating the function header code based on the variable data in the input data node, the output data node, and the label data includes: Generate a function name and parameter code based on the label data from at least one input data node and at least one output data node.

10. The method of claim 9, wherein the function header code further includes requesting shape area code, wherein generating the function header code based on the variable data in the input data node, the output data node, and the label data includes: Based on the number of data nodes containing variable data in the computation graph and the size of the tensor shape storage area determined by the data corresponding to each data node, a shape area application code is generated.

11. The method according to claim 1, wherein the operator code further includes hidden neuron space allocation code and / or operator template code, wherein the hidden neuron space allocation code is generated by searching for hidden neurons connected to the computing node, and the operator template code is generated according to the attributes of the operators in the computing node.

12. A compiler for neural networks, applied during the compilation phase, comprising: An acquisition unit is used to acquire a computation graph of a neural network, wherein the computation graph includes computation nodes and data nodes and their respective label data, wherein the label data includes shape parameters for describing the shape of a tensor; An assignment unit is used to assign values ​​to the shape parameters in the label data of the data node, wherein assigning values ​​to the shape parameters includes generating assignment code for assigning values ​​to the shape parameters. For variable data in the shape parameters, the assignment code represents the value of the shape parameters as a variable in the generated code, and the value of the variable is provided when the generated code is run. as well as An operator generation unit is used to generate operator code for the corresponding operator in the computation node based on the assigned shape parameters and the operator output shape formula library, wherein the shape of the operator output tensor in the operator code is represented by the value of the shape parameter represented by the assigned code.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, the instructions being... The at least one processor executes such that it is capable of performing the method according to any one of claims 1-11.

14. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

15. A system for real-time compilation of variable neural network models, comprising: Interconnected hosts and devices, The host computer includes the compiler according to claim 12; The device is configured to execute code compiled by the compiler.

Images