Neural network model processing method and related equipment

Abstract

The invention discloses a neural network model processing method in the field of artificial intelligence, and the method comprises the steps: obtaining an operation process of a neural network model, and the operation process is represented by at least one operator of a first type and a plurality of operators of a second type; in the operation process, the operator of the first type comprises a boundary identifier, and the computational logic of the operator of the first type is represented by a group of operators of a second type. For any one operator of the first type, the range of the operator of the second type included in any one operator of the first type is indicated by the boundary identifier in any one operator of the first type. And obtaining a first calculation graph of the neural network model according to the operation process. Through the technical scheme provided by the embodiment of the invention, the computational logic of the first type of operator is known for the deep learning framework, so that the computational graph has more optimization opportunities, and the deep learning framework can better optimize the computational graph.

Description

[0001] This application is a divisional application. The application number of the original application is 202010232059.9, and the original application date is March 27, 2020. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of communication technology, and in particular to a processing method for a neural network model and related equipment. Background Art

[0003] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, basic AI theory, etc.

[0004] With the rapid development of AI, neural networks (for example, deep neural networks) have made great achievements in the processing and analysis of various media signals such as images, videos, and voices in recent years. The deep learning framework converts the neural network model into a computational graph, optimizes the computational graph, and then compiles the optimized computational graph into instructions for the hardware platform, thus completing the compilation of the neural network model on the hardware platform. Through graph optimization, the instructions compiled by the model can be executed efficiently on the hardware platform. Therefore, how to better optimize the computational graph needs to be solved urgently. Summary of the invention

[0005] The embodiment of the present application provides a method for processing a neural network model, which can enable the deep learning framework to learn the calculation logic of the operators included in the calculation graph, so that the calculation graph has more optimization opportunities, and thus the calculation graph can be better optimized.

[0006] To achieve the above objectives, the present application provides the following technical solutions:

[0007] The first aspect of the present application provides a method for processing a neural network model, which may include: a deep learning framework obtains an operation flow of the neural network model. The operation flow is represented by at least one operator of the first type and multiple operators of the second type. The first type of operator and the second type of operator can be regarded as functions that can realize a certain specific function. The first type of operator and the second type of operator can perform mathematical operations on the input data. For example, when the first type of operator and the second type of operator are convolution operators, the convolution kernel can be used to perform convolution calculations on the local area of ​​the input feature image, and the output features are obtained by linear calculations of the data in the input feature image. For another example, when the first type of operator and the second type of operator are fully connected operators, all input features can be linearly combined using matrix multiplication. The operation flow of the neural network model mentioned in the present application refers to a process of performing mathematical operations on the input data by at least the first type of operator and multiple second type operators to obtain the output data of the neural network model.

[0008] The difference between the first type of operator and the second type of operator is that the deep learning framework can know the calculation logic of the first type of operator, but cannot know the calculation logic of the second type of operator. This is because in the operation process, the calculation logic of the first type of operator is represented by a group of second type of operators. Specifically, the first type of operator includes a boundary identifier. For any first type of operator, the calculation logic of the first type of operator is represented by a group of second type of operators, and the range of the second type of operator included in any first type of operator is indicated by the boundary identifier in any first type of operator. The range of the second type of operator included in the first type of operator in this scheme is the second type of operator included in the calculation logic of the first type of operator. For example, assuming that the operation process includes a first type of operator and three second type of operators, where the first type of operator is assumed to be A operator, and the three second type of operators are B operator, C operator and D operator. Among them, the calculation logic of the A operator is represented by the B operator and the C operator, then the range of the second type of operator included in the A operator refers to the B operator and the C operator, but does not include the D operator. As mentioned above, the first type of operator and the second type of operator can be regarded as functions that can realize certain specific functions. In this solution, the calculation logic of the first type of operator and the second type of operator can be understood as the expression of the function. In this solution, the operation process of the neural network model may include the calculation logic of at least one first type of operator, the calculation logic of the first operator is represented by a group of second type of operators, and the calculation logic of the second type of operator can be realized by a kernel function kernel corresponding to the name of the second type of operator.

[0009] The deep learning framework obtains the first calculation graph of the neural network model according to the calculation process. In the deep learning framework, a mathematical calculation process will first be compiled and converted into a calculation graph. The deep learning framework compiles the above-mentioned calculation process to obtain the first calculation graph.

[0010] From the first aspect, it can be seen that the operation process of the neural network is represented by two different types of operators. The calculation logic of the first type of operator can be represented by a group of second type of operators. Since the deep learning framework can obtain the calculation logic of the first type of operator, the calculation logic of the first type of operator is no longer unknown to the deep learning framework, which makes the calculation graph have more optimization opportunities, and the deep learning framework can better optimize the calculation graph.

[0011] Optionally, in combination with the first aspect above, in a first possible implementation, the first computational graph may include a main graph and a subgraph, and the first computational graph of the neural network model determined according to the operation flow may include: determining the main graph and the subgraph of the neural network model according to the operation flow, the first type of operator in the main graph is indicated by a boundary marker, the second type of operator in the main graph is indicated by the name of the second type of operator, the main graph is used to output the result of the operation flow, the subgraph may include the name of the second type of operator that may be included in any first type of operator, the subgraph is used to output the result of a first type of operator, and a subgraph represents the computational logic of a first type of operator. As can be seen from the first possible implementation of the first aspect, a specific structure of the first computational graph is given, which increases the diversity of the scheme.

[0012] Optionally, in combination with the above-mentioned first aspect or the first possible implementation of the first aspect, in a second possible implementation, the method may further include: optimizing the first computation graph with the first type of operator as the processing granularity to obtain a second computation graph. In this scheme, optimizing the first computation graph with the first type of operator as the processing granularity means that some graph optimization processing can be performed inside the first type of operator or between the first type of operators. For example, assuming that the first computation graph includes the first type of operators OP1, OP4 and OP6, optimizing the first computation graph with the first type of operator as the processing granularity means that any one of the subgraph corresponding to OP1, the subgraph corresponding to OP4 and the subgraph corresponding to OP6 can be selected for optimization processing, or any two of the subgraph corresponding to OP1, the subgraph corresponding to OP4 and the subgraph corresponding to OP6 can be selected for optimization processing, or the subgraph corresponding to OP1, the subgraph corresponding to OP4 and the subgraph corresponding to OP6 can be selected for optimization processing. Figure 1From the second possible approach of the first aspect, it can be seen that the first type of operator is used as the processing granularity to ensure that the boundary of the first type of operator is not broken, that is, to ensure that the first type of operator can be processed as a whole, the problem of too large a search space will not occur, which will further lead to the problem of uncontrollable compilation time.

[0013] Optionally, in combination with the second possible implementation of the first aspect, in a third possible implementation, the first type of operator may include a third operator and a fourth operator, the third operator and the fourth operator may include the same computing logic, and the first computing graph is optimized with the first type of operator as the processing granularity, which may include: fusing the subgraph corresponding to the third operator and the subgraph corresponding to the fourth operator to obtain a fused subgraph, and the second computing graph may include a fused subgraph. From the third possible implementation of the first aspect, it can be seen that a specific graph optimization method is given. Since the deep learning framework can know the computing logic of the first type of operator, the computing graph has more optimization opportunities, and the deep learning framework can better optimize the computing graph. Specifically, the deep learning framework can fuse the subgraphs of operators with the same computing logic.

[0014] Optionally, in combination with the second or third possible implementation of the first aspect above, in a fourth possible implementation, the first type of operator may include a fifth operator and a sixth operator, and the intermediate calculation result of the fifth operator is the same as the intermediate calculation result of the sixth operator. The first calculation graph is optimized with the first type of operator as the processing granularity, which may include: using the intermediate calculation result of the fifth operator as the input parameter of the sixth operator. The intermediate calculation result refers to the output result of the second type of operator included in the first type of operator. For example, assuming that the fifth operator and the sixth operator both include the C operator, the input of the C operator includes two, namely the output of the A operator and the output of the B operator, the input of the A operator is x, and the input of the B operator is y. Since the input of the C operator of the fifth operator and the sixth operator is the same, the output result must also be the same, then the output of the C operator of the fifth operator can be directly used as the input parameter of the sixth operator, and the sixth operator does not need to repeatedly calculate the output of the C operator. From the fourth possible implementation method of the first aspect, it can be seen that a specific graph optimization method is given. Since the deep learning framework can know the calculation logic of the first type of operator, the calculation graph has more optimization opportunities. The deep learning framework can better optimize the calculation graph. Specifically, when the intermediate calculation results of two operators are the same, the intermediate result of one of the operators can be used as the input parameter of the other operator.

[0015] Optionally, in combination with the fourth or fifth possible implementation of the first aspect above, in the fifth possible implementation, the third operator is a forward operator, the fourth operator is a reverse operator corresponding to the third operator, or the fourth operator is a forward operator, and the third operator is a reverse operator corresponding to the fourth operator. The forward operator is an operator used in the process of forward propagation in a neural network. Forward propagation, also known as forward propagation, refers to the process of gradually calculating low-level features into abstract high-level features from the input feature vector to the output and finally to the cost function of the neural network until the loss is obtained. Most forward operators have a corresponding reverse operator, and the reverse operator is an operator used in the process of back propagation in the neural network. Through back propagation, the parameters of each layer of the neural network can be trained to minimize the error. From the fifth possible implementation method of the first aspect, it can be seen that a specific optimization scenario is given. Since the reverse operator can usually be understood as the derivative of the forward operator plus certain operations, and the derivatives of many operators in the field of deep learning have a lot of repeated operation logic themselves, this solution can better optimize the forward operator and reverse operator included in the calculation graph.

[0016] Optionally, in combination with the first aspect or the first to fifth possible implementations of the first aspect, in a sixth possible implementation, the method may further include: determining a second IR of the first type of operator according to a first intermediate representation IR of the second type of operator and a computational logic of the first type of operator. Determine a set of kernel functions corresponding to the first type of operator according to the second IR.

[0017] Optionally, in combination with the first aspect or the first to sixth possible implementations of the first aspect, in the seventh possible implementation, the input of the operation process is tensor data. In this solution, a tensor is a description of the characteristics of the stored data, such as a tensor can record information such as the shape and type of the data. The solution provided in the present application does not limit the type of neural network model, such as an image processing neural network model, a speech recognition neural network model, or a neural network model for processing a video. For different neural network models, the input data of the neural network model is often different. For example, for a speech recognition neural network model, the input data may be pure speech and noise, and for an image processing neural network model, the input data may be image data. In this solution, the tensor data can be used to describe the input data of different neural network models. It should be noted that the solution provided in the present application can be applied to any scenario related to neural network model processing, exemplarily including but not limited to the following scenarios: including speech recognition, computer vision (computer vision, CV), video processing, image recognition and natural language processing (nature language process, NLP).

[0018] The second aspect of the present application provides a processing device for a neural network model, which may include: a programming interface module, which is used to obtain the operation flow of the neural network model, the operation flow is represented by at least one first-type operator and multiple second-type operators, in the operation flow, the first-type operator may include a boundary identifier, the calculation logic of the first-type operator is represented by a group of second-type operators, and for any first-type operator, the range of the second-type operator that any first-type operator can include is indicated by the boundary identifier in any first-type operator. A calculation graph processing module, which is used to obtain the first calculation graph of the neural network model according to the operation flow obtained by the programming interface module.

[0019] Optionally, in combination with the above-mentioned second aspect, in a first possible implementation method, the first computational graph may include a main graph and a sub-graph, and the computational graph processing module is specifically used to: determine the main graph and the sub-graph of the neural network model according to the operation flow, the first type of operator in the main graph is indicated by a boundary marker, the second type of operator in the main graph is indicated by the name of the second type of operator, the main graph is used to output the result of the operation flow, the sub-graph may include the name of the second type of operator that can be included in any first type of operator, the sub-graph is used to output the result of a first type of operator, and a sub-graph represents the computational logic of a first type of operator.

[0020] Optionally, in combination with the above-mentioned second aspect or the first possible implementation of the second aspect, in the second possible implementation, the computational graph processing module is further used to: optimize the first computational graph with the first type of operator as the processing granularity to obtain a second computational graph.

[0021] Optionally, in combination with the second possible implementation method of the second aspect above, in a third possible implementation method, the first type of operator may include a third operator and a fourth operator, the third operator and the fourth operator may include the same calculation logic, and the calculation graph processing module is specifically used to: fuse the subgraph corresponding to the third operator and the subgraph corresponding to the fourth operator to obtain a fused subgraph, and the second calculation graph may include the fused subgraph.

[0022] Optionally, in combination with the second or the third possible implementation of the second aspect above, in a fourth possible implementation, the first type of operator may include a fifth operator and a sixth operator, the intermediate calculation result of the fifth operator is the same as the intermediate calculation result of the sixth operator, and the calculation graph processing module is specifically used to: use the intermediate calculation result of the fifth operator as an input parameter of the sixth operator.

[0023] Optionally, in combination with the fourth or fifth possible implementation of the second aspect above, in the fifth possible implementation, the third operator is a forward operator, the fourth operator is a reverse operator corresponding to the third operator, or the fourth operator is a forward operator, and the third operator is a reverse operator corresponding to the fourth operator.

[0024] Optionally, in combination with the above-mentioned second aspect or the first to the fifth possible implementations of the second aspect, in a sixth possible implementation, the device may further include an operator compilation module and a kernel module, wherein the operator compilation module is used to determine a second IR of an operator of the first type according to a first intermediate representation IR of an operator of the second type and a calculation logic of an operator of the first type. The kernel module is used to determine a set of kernel functions kernel corresponding to an operator of the first type according to the second IR determined by the operator compilation module.

[0025] Optionally, in combination with the above-mentioned second aspect or the first to the sixth possible implementations of the second aspect, in the seventh possible implementation, the input of the operation process is tensor data. In this solution, a tensor is a description of the characteristics of the stored data, such as a tensor can record information such as the shape and type of the data. The solution provided in the present application does not limit the type of neural network model, for example, it can be an image processing neural network model, a speech recognition neural network model, or a neural network model for processing videos. For different neural network models, the input data of the neural network model is often different. For example, for a speech recognition neural network model, the input data may be pure speech and noise, and for an image processing neural network model, the input data may be image data. In this solution, the tensor data can be used to describe the input data of different neural network models. It should be noted that the solution provided in the present application can be applied to any scenario related to the processing of a neural network model, exemplarily including but not limited to the following scenarios: including speech recognition, image recognition, CV and NLP.

[0026] The third aspect of the present application provides a processing device for a neural network model, which may include: a memory for storing computer-readable instructions. It may also include a processor coupled to the memory for executing the computer-readable instructions in the memory to perform the method described in the first aspect or any possible implementation of the first aspect.

[0027] A fourth aspect of the present application provides a computer-readable storage medium, which, when instructions are executed on a computer device, enables the computer device to execute the method described in the first aspect or any possible implementation of the first aspect.

[0028] A fifth aspect of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0029] In a sixth aspect, the present application provides a chip system, which may include a processor for supporting a terminal device to implement the functions involved in the method described in the first aspect or any possible implementation manner of the first aspect.

[0030] Optionally, in combination with the sixth aspect above, in a first possible implementation, the chip system may further include a memory, which is used to store program instructions and data necessary for the terminal device. The chip system may be composed of a chip, or may include a chip and other discrete devices. Among them, the chip system may include an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices. Furthermore, the chip system may also include an interface circuit, etc.

[0031] It should be noted that the beneficial effects brought about by the implementation methods of the second to sixth aspects of the present application can be understood by referring to the implementation method of the first aspect, and therefore they are not repeated.

[0032] Through the technical solution provided by the present application, the operation process of the neural network is represented by two different types of operators, and the calculation logic of the first type of operators can be represented by a group of second type of operators. Since the deep learning framework can obtain the calculation logic of the first type of operators, the calculation logic of the first type of operators is no longer unknown to the deep learning framework, so that the calculation graph has more optimization opportunities, and the deep learning framework can better optimize the calculation graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a deep learning framework provided in an embodiment of the present application;

[0034] Figure 2 A schematic diagram of an operator registration process provided in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of a processing method of a neural network model provided in an embodiment of the present application;

[0036] Figure 4 A schematic diagram of a first type of operator with a boundary marker provided in an embodiment of the present application;

[0037] Figure 5A schematic diagram of a first calculation graph provided in an embodiment of the present application;

[0038] Figure 6 Another method for processing a neural network model provided in an embodiment of the present application;

[0039] Figure 7 The first computational graph provided for the embodiment of the present application includes a schematic diagram of boundary identification;

[0040] Figure 8 A schematic diagram of the first computation graph including and excluding boundary markers;

[0041] Fig. 9 A schematic diagram of another processing method of a neural network model provided in an embodiment of the present application;

[0042] Fig.10 A schematic diagram of another processing method of a neural network model provided in an embodiment of the present application;

[0043] Fig.11 A schematic diagram of a process for generating a kernel file provided in an embodiment of the present application;

[0044] Fig.12 A schematic diagram of a process for generating a kernel file provided in an embodiment of the present application;

[0045] Fig.13 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0046] Fig.14 A schematic diagram of the structure of an execution device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. It should be understood that the terms "include" and "comprise" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections. It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application.

[0048] In order to better understand the technical solutions described in this application, the key technical terms involved in the embodiments of this application are explained below:

[0049] Since the embodiments of the present application involve a large number of neural network applications, in order to facilitate understanding, the relevant terms and related concepts such as neural networks involved in the embodiments of the present application are first introduced below.

[0050] (1) Neural Network

[0051] A neural network can be composed of neural units. A neural unit can be a s The output of the operation unit with the intercept 1 as input can be expressed as follows:

[0052]

[0053] Where s=1, 2, ...n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the characteristics of the local receptive field. The local receptive field can be an area composed of several neural units.

[0054] There are many types of neural networks, for example, deep neural network (DNN), also known as multi-layer neural network, that is, a neural network with multiple hidden layers; another example is convolutional neural network (CNN), which is a deep neural network with a convolutional structure. This application does not limit the specific type of neural network involved.

[0055] (2) Operator

[0056] An operator refers to a function that implements a specific function. For example, taking the reshape operator as an example, this operator is used to reinterpret the shape of tensor data. For another example, taking the transpose operator as an example, this operator is used to adjust the dimensional order of tensor data. In this application, the commonly used functions used to build deep learning model algorithms are collectively referred to as operators, and any operation performed on any function can be considered an operator. For example, convolution is a mathematical method of integral transformation. For example, function f3 is generated by two functions f1 and f2, then f1, f2 and the convolution result f3 can be regarded as an operator respectively.

[0057] (3) Computational Graph

[0058] A computational graph is a way to describe a computational process using a graph structure. If the computation has obvious modularity and there are obvious temporal and logical dependencies between modules, a directed graph structure can usually be used to describe it. In practical applications, there are two basic elements of a graph structure, namely nodes and directed edges. In practical applications, a neural network is abstracted as a directed graph structure consisting of tensor data and operators. Nodes are also called operators. In other words, a directed graph is a directed graph constructed by input data, mathematical operations, and output data. Generally speaking, describing a neural network model using a computational graph is conducive to an overall grasp of the entire neural network computing task. At the same time, the expression of the computational graph also facilitates the scheduling and parallel execution of computing tasks. In a deep learning framework, a mathematical calculation process will first be compiled and converted into a computational graph.

[0059] (4) Graph Optimization

[0060] Graph optimization refers to the process of optimizing the computational graph. The purpose of graph optimization is to enable the instructions compiled by the algorithm to be efficiently executed on the hardware platform. Graph optimization methods can include optimization of expressions of addition and multiplication by 0, function inlining, expression reuse, and other optimization methods.

[0061] (5) Operator kernel function

[0062] Kernel is the binary program that can be executed on the hardware corresponding to the operator. The deep learning framework uses operators as the specific elements to implement computing tasks, and provides a kernel for each operator to be executed on the CPU or artificial intelligence processor. According to the calculation graph, the deep learning framework schedules the execution of the kernel function corresponding to each operator in the calculation graph to complete the calculation of the entire neural network.

[0063] (6) Operator Registration

[0064] Each operator in the deep learning framework needs to have a kernel implementation of the corresponding hardware. Therefore, the kernel of the corresponding hardware implementation needs to be registered in the deep learning framework. When running the model, the operator kernel implementation of the corresponding hardware is called according to the operator name.

[0065] (7) Forward operator

[0066] The forward operator is an operator used in the forward propagation process of a neural network. Forward propagation, also known as forward propagation, refers to the process of gradually calculating low-level features into abstract high-level features from the input feature vector to the output and finally to the cost function until the loss is obtained.

[0067] (8) Reverse operator

[0068] Most forward operators have a corresponding reverse operator, which is used in the process of back propagation in the neural network. Through back propagation, the parameters of each layer of the neural network can be trained to minimize the error.

[0069] (9) Artificial Intelligence Processor

[0070] Artificial intelligence processors, also known as dedicated processors, in the embodiments of the present application, artificial intelligence processors refer to processors for specific applications or fields. For example: graphics processing unit (GPU), also known as display core, visual processor, display chip, is a dedicated processor for image computing on personal computers, workstations, game consoles and some mobile devices (such as tablets, smart phones, etc.). Another example: neural network processor (neural processing unit, NPU), is a dedicated processor for matrix multiplication operations in applications in the field of artificial intelligence, using the "data-driven parallel computing" architecture, and is particularly good at processing massive multimedia data such as videos and images.

[0071] (10) Deep Learning Framework

[0072] In order to meet the growing demand for neural networks, deep learning frameworks have emerged. Through deep learning frameworks, researchers only need to focus on the network structure of deep learning algorithms. By writing simple python (a cross-platform computer programming language) scripts to write network structures, they can complete a complex deep learning network task, thereby realizing model reasoning and training on hardware. In other words, deep learning frameworks are used to lower the development threshold in the field of deep learning, provide a basic computing framework for deep learning, and quickly build deep learning applications. The current mainstream deep learning frameworks in the industry mainly include Tensorflow, Torch, Mxnet, Thenao, Caffe, etc. Taking the convolutional neural network framework Caffe as an example, in actual applications, Caffe supports multiple types of deep learning architectures, image classification and image segmentation, and can also support convolutional neural networks (CNN), convolutional neural networks for target detection (region-CNN, RCNN), long short-term memory neural networks (LSTM) and fully connected neural network designs. Deep learning frameworks can support multiple types of basic operators. Specifically, the multiple types of basic operators involved here can include: common neural network operators. For example, common neural network operators include: convolution / deconvolution operators, pooling operators, activation operators, classifier (softmax) operators, and fully connected operators. Among them, activation operators may include but are not limited to ReLU, Sigmoid, Tanh, and other operators that can be implemented by interpolation.

[0073] (11) Black box operator

[0074] A black-box operator is an operator whose input and output types and dimensions are known only to the deep learning framework. The deep learning framework cannot know the internal implementation details of the black-box operator, for example, the calculation logic of the operator.

[0075] (12) White-box operator

[0076] A white-box operator is an operator whose input and output types and dimensions can be known by the deep learning framework. The deep learning framework can also know the internal implementation details of the white-box operator, for example, the calculation logic of the operator. Of course, the calculation logic known is an expression of the calculation function of the white-box operator. The calculation function of some operators can be expressed in multiple ways. These multiple expressions are functionally equivalent. Functional equivalence can be understood as the output results obtained for the same input should be equal.

[0077] Figure 1 As shown, it is a schematic diagram of a deep learning framework provided by an embodiment of the present application. Figure 1 As shown, the deep learning framework may include a framework layer and an operator layer, wherein the framework layer may include a programming interface module 101 and a computational graph processing module 102. Among them, the computational graph processing module 102 may further include a graph compilation unit 1021 and a graph optimization unit 1022. The operator layer may further include an operator compilation module 103, an operator management module 104, an intermediate representation (IR) module 105 and a kernel module 106. The deep learning framework may receive program code input by the user through the programming interface module 101, such as an algorithm model designed by the user. It should be noted that in this application, unless otherwise specified, the algorithm, algorithm model and neural network model have the same meaning. For example, a neural network model may receive input data and generate a predicted output based on the received input data and the current model parameters. In practical applications, the neural network model can be a regression model, a deep neural network (DNN), a convolutional neural network model (CNN), a recurrent neural network model (RNN), etc., which is not specifically limited in the embodiments of the present application. For example, under the caffe framework, the neural network model has a hierarchical structure, and the operator can be the corresponding layer in the neural network model. The user can design the connection relationship between each operator through the programming interface module 101 according to the needs. For example, the face recognition neural network model contains many different types of operators (convolution operator, pooling operator, fully connected operator), among which the connection relationship between each operator can be: convolution operator 1-pooling operator 1-convolution operator 2-pooling operator 2-fully connected operator 1-fully connected operator 2. For another example, the license plate character recognition neural network model contains a variety of different types of operators (convolution operator, pooling operator, activation operator, softmax operator, etc.), where the connection relationship between the operators can be: convolution operator 1-activation function Relu-maximum pooling operator 1-convolution operator 2-activation function Relu-maximum pooling operator 2-convolution operator 3-activation function Relu-maximum pooling operator 3-convolution operator 4-activation function-maximum pooling operator 4-convolution operator 5-activation function-maximum pooling operator 5-fully connected operator 1-softmax operator-output operator. In other words, the user can describe the calculation logic of the algorithm model by calling the programming interface module 101. The program obtained by the programming interface module 101 can be compiled by the graph compilation unit 1021, and the compiled algorithm model will be converted into a calculation graph.

[0078] The graph optimization unit 1022 can perform graph optimization on the calculation graph obtained by the graph compilation unit 1021 to obtain an optimized calculation graph. The deep learning framework then compiles the optimized calculation graph through the operator layer to generate a kernel file. Specifically, the deep learning framework queries the operator management module 104 for the IR function pointer corresponding to the operator through the operator compilation module 103 according to the name of the operator. After the operator management module 104 queries the IR function pointer corresponding to the operator according to the name of the operator, it sends the IR function pointer to the operator compilation module 103. The operator compilation module 103 then queries the IR module for the IR corresponding to the operator according to the obtained IR function pointer. The IR module pre-stores the IR of the relevant operator. IR is a professional term in the field of editors. A bridge is needed between a large number of different programming languages ​​and an increasing number of hardware architectures. IR plays such a bridging role, so that if you need to support new programming languages ​​or new device platforms, you only need to develop the corresponding front-end and back-end.

[0079] It should be noted that the operator compilation module 103 can obtain the IR corresponding to the operator according to the name of the operator because the correspondence between the operator name and the operator IR function name is pre-established, that is, the IR corresponding to the operator is registered with the deep learning framework through the registration mechanism provided by the deep learning framework. Figure 2 Describe the registration process. Figure 2 FIG. 1 is a schematic diagram of an operator registration process provided by an embodiment of the present application. During the registration process, the IR module 105 registers with the deep learning framework through the operator IR function name, which may specifically include the following steps:

[0080] 1010. The IR module 105 initiates the operator IR registration process to the operator management module 104.

[0081] The IR module 105 may send the IR function name of the operator to the operator management module 104 .

[0082] 1020. The operator management module 104 establishes a correspondence between the operator name and the operator IR function name.

[0083] After the operator management module 104 receives the registration process sent by the IR module 105, it can create a mapping map (map), in which the mapping key (key) corresponds to the operator name, and the mapping value (value) corresponds to the operator IR function name, so that when the operator compilation module 103 compiles, it can query the operator IR function name in the operator management module 104 according to the operator name, and then load it into the IR function pointer, and obtain the IR corresponding to the operator according to the loaded IR function pointer.

[0084] In a specific implementation, it may also include 1030 , where the operator management module 104 returns a notification of successful registration to the IR module 105 .

[0085] In order to facilitate a better understanding of the present application, the research ideas of the technical solutions described in the present application are specifically described below:

[0086] The amount of computation of a deep learning algorithm (sometimes referred to as a deep learning algorithm in this application, when the difference between the two is not emphasized, the two express the same meaning) is generally very large, and it often takes hours or even days to complete the training of an algorithm. Therefore, improving the computational efficiency of deep learning has become the focus of various deep learning frameworks, and the optimization of the computational graph has become the top priority. Programmers describe the computational logic of deep learning algorithms through programming interfaces. The deep learning algorithm is then compiled through a deep learning framework to convert it into a computational graph. After the computational graph is optimized and compiled, an executable kernel program is generated. The kernel program will link the kernels of multiple operators included in the computational graph according to the optimized computational graph. Specifically, there is an obvious logical dependency between the operators in the computational graph, and a directed graph structure can be used to describe this dependency, and the kernels corresponding to the operators can be linked according to this dependency. The kernel program can be executed on a hardware platform, and during execution, the operator kernels can be executed in sequence according to the link order of the operators in the kerenl program. It should be noted that the existing process of optimizing the computational graph can usually only perform some basic optimizations on the computational graph, such as the optimization of expressions including addition and multiplication by 0. There are very few opportunities to optimize the computational graph. The kernel program obtained by compiling the optimized computational graph through the existing optimization scheme cannot meet our expectation of efficient execution of the kernel program on the hardware platform.

[0087] Specifically, when the deep learning framework compiles the calculation graph, it calls the kernel corresponding to the operator name through the operator name (also called the calculation node name) in the calculation graph to implement the function of the operator. In the above process, the deep learning framework is completely unaware of the internal logic of the operator, or the implementation details of the operator. The deep learning framework only needs to obtain the result based on the input data and the operator. Since the calculation logic of each operator is unknown to the deep learning framework, this will lose the opportunity to optimize some graphs. In addition, in the deep learning framework, each time a new operator is added, it is necessary to declare the operator interface in the deep learning framework, and then pre-configure the kernel corresponding to the operator in the hardware, and finally register the kernel corresponding to the operator with the deep learning framework through the registration mechanism provided by the deep learning framework. This process is relatively cumbersome. When a large number of operators in the deep learning framework need to be added, this process is even more cumbersome. It should be noted that the addition of a new operator here means that before the operator is successfully registered, there is no kernel of the operator in the deep learning framework, and the kernel of the operator cannot be obtained through the name of the operator. The registration process of a new operator is to establish a mapping relationship between the name of the new operator and the kernel of the operator, so that the deep learning framework can obtain the kernel of the operator according to the name of the operator.

[0088] At present, almost all graph optimizations are optimized based on black-box operators. Black-box operators are operators whose internal computing logic described above is invisible to the deep learning framework. The deep learning framework can only perform graph optimization based on the granularity of black-box operators. If the deep learning framework can know the internal computing logic of the operator, the deep learning framework will have more graph optimization opportunities. In addition, the degree of expansion of the internal computing logic of an operator must also be considered. For example, if the logic of each operator in the calculation graph is fully expanded, the calculation graph will include too many operators. During the graph optimization process, the compilation time may be uncontrollable due to the long traversal time. In addition, in the prior art, there are about thousands of black-box operators, that is, operators whose computing logic is invisible to the deep learning framework. As mentioned above, when a large number of operators are added, the registration process is cumbersome and more kernels need to be pre-configured.

[0089] Based on the above analysis and description, an embodiment of the present application provides a processing method for a neural network model. The computational logic of the first type of operator is represented by a group of second type operators. This method enables the deep learning framework to know the internal logic of the first type of operator, solving the problem of losing graph optimization opportunities caused by graph optimization by black box operators. This is illustrated with an example below. Assuming that the computational graph is regarded as an article, the operator in the computational graph in the prior art is equivalent to a word in the article, and each letter in the word represents the computational logic of the operator. In the prior art, the deep learning framework does not perceive each letter in the word, that is, the deep learning framework does not perceive the computational logic of the operator. The solution provided by the present application includes two types of operators in the computational graph, wherein the first type of operator can be equivalent to a word in the article, and the second type of operator can be equivalent to multiple letters constituting the word. Through the solution provided by the present application, the deep learning framework can perceive each letter in the word, that is, the deep learning framework can obtain the computational logic of the first type of operator. In this way, the internal logic of more operators is included in the computational graph, so that the computational graph can have more optimization opportunities.

[0090] In addition, the present application can select some commonly used deep learning operators as the second type of operators according to the statistical results, and the present application only pre-configures the kernels of these commonly used operators. Most of the other operators can be obtained by combining these operators. For example, the commonly used deep learning operators selected may be addition operators (add), subtraction operators (sub), multiplication operators (mul), exponential operators (exp), square root operators (sqrt) of non-negative real numbers, and so on. The kernels of these commonly used operators selected are registered with the deep learning framework through the registration mechanism provided by the deep learning framework. Using the example of the words above, for example, 26 English letters can be selected as the second type of operators, and other words can be obtained by combining 26 English letters. Other words can be regarded as the first type of operators, that is, the calculation logic of the first type of operators can be represented by the second type of operators. This method of combining commonly used operators into new operators will not increase the registration procedure of the newly added operators, and solves the above-mentioned problem that when a large number of operators are added, the registration process is cumbersome and more kernels need to be pre-configured.

[0091] In addition, the solution provided by this application optimizes the computational graph using the first type of operator as the processing granularity. Optimizing the computational graph using the first type of operator as the processing unit solves the problem that after the logic of the operator is expanded, the compilation time may be uncontrollable due to the long traversal time during the graph optimization process.

[0092] Based on the above research ideas, the technical solution provided by this application is introduced in detail below.

[0093] like Figure 3 , which is a schematic diagram of a processing method of a neural network model provided in an embodiment of the present application.

[0094] like Figure 3 As shown, a method for processing a neural network model provided in an embodiment of the present application may include the following steps:

[0095] 301. Obtain an operation flow of a neural network model, where the operation flow is represented by at least one operator of the first type and multiple operators of the second type.

[0096] An operation flow is represented by at least one first type of operator and multiple second type operators. In the operation flow, the first type of operator includes a boundary identifier, and the calculation logic of the first type of operator is represented by a group of second type operators. For any first type of operator, the range of the second type of operator included in any first type of operator is indicated by the boundary identifier in any first type of operator. The operation flow referred to in this application includes the calculation logic referred to in this application.

[0097] Specifically, the operation flow of the neural network model mentioned in the present application refers to a process of performing mathematical operations on input data through at least a first type of operator and multiple second type operators to obtain output data of the neural network model. The operation flow may refer to the calculation logic between multiple first type operators and multiple second type operators. For example, the operation flow may include the calculation logic between the first type operator and the first type operator, the calculation logic between the second type operator and the second type operator, and the calculation logic between the first type operator and the second type operator.

[0098] The scope of the second type of operators included in the first type of operators in this solution is the second type of operators included in the calculation logic of the first type of operators. For example, assume that the calculation process includes an operator of the first type: operator A, and three operators of the second type: operator B, operator C, and operator D. Among them, the calculation logic of operator A is represented by operator B and operator C, then the scope of the second type of operators included in operator A refers to operator B and operator C, but does not include operator D. It is mentioned above that the first type of operators and the second type of operators can be regarded as functions that can realize certain specific functions. In this solution, the calculation logic of the first type of operators and the second type of operators can be understood as function expressions.

[0099] In the implementation scheme provided in the present application, the internal calculation logic of the first type of operator is visible to the deep learning framework, and the internal calculation logic of the second type of operator is invisible to the deep learning framework. The second type of operator may be some basic operators provided by the deep learning framework, where the basic operators may refer to some commonly used deep learning operators, and basically most operators can be composed of these basic operators. Exemplarily, the basic operators may include addition operators (add), subtraction operators (sub), multiplication operators (mul), exponential operators (exp), square root operators (sqrt) of non-negative real numbers, and the like. It should be noted that the above-mentioned basic operators are only intended to illustrate, and do not represent the limitation of the category of the second type of operator. In the process of actual application, suitable basic operators can be selected according to actual needs, such as basic operators can also include maximum operators (maximum), re-adjustment of the number of rows, columns, and dimension operators (reshape) of the matrix, and the like.

[0100] In the prior art, there are about thousands of black box operators, that is, operators whose computing logic is invisible to the deep learning framework. The deep learning framework needs to provide a kernel for each black box operator to be executed on the CPU or artificial intelligence processor. In order to make the operator correspond to the kernel, it is necessary to register the kernel corresponding to the operator with the deep learning framework through the registration mechanism provided by the deep learning framework. The specific registration process has been described in the above text of this application, and it will not be repeated here. This application does not need thousands of black box operators, but only a small number of basic operators. By combining these basic operators (this application sometimes also refers to the basic operators as basic black box operators, and when the difference between the two is not emphasized, the two have the same meaning), the computing logic of another part of the more complex operators can be expressed. Therefore, in the technical solution provided by this application, it is only necessary to register the kernels corresponding to these basic operators with the deep learning framework through the registration mechanism provided by the deep learning framework. The basic operators here are the second type of operators mentioned in the embodiment of this application.

[0101] The following first describes a specific example. Figure 4 As shown, the sigmoid operator is taken as an example to illustrate a first type of operator with boundary identification provided by the embodiment of the present application. In the prior art, sigmoid is a black box operator, and the deep learning framework cannot know the calculation logic of the sigmoid operator. Figure 4As shown in a in , the deep learning framework can only know that the input data of the sigmoid operator is x, and the output data of the sigmoid operator is sigmoid(x). As for how to obtain the output data sigmoid(x) based on the input data x, that is, the calculation logic of sigmoid(x), the deep learning framework does not know. In the technical solution provided by this application, the calculation logic of the sigmoid operator can be expressed through basic operators, such as Figure 4 As shown in b, the basic operators are sub, exp, add and div, which have pre-configured kernels. The calculation logic of the sigmoid(x) operator can be expressed through these basic operators. Specifically, the calculation logic of sigmoid is as follows:

[0102]

[0103] like Figure 4 As shown in c in , the user can add boundary markers to the first type of operators so that the deep learning framework can process the first type of operators as a whole during the graph compilation of the operation flow.

[0104] After completing the above registration process, users can customize operators through the programming interface module of the deep learning framework. Specifically, the deep learning framework provides users with basic operator primitives through the programming interface module, and allows users to freely combine basic operators to obtain new operators, that is, to obtain the first type of operator by combining the second type of operators. In this way, the deep learning framework can know the calculation logic of the first type of operator. When the user inputs the operation flow of the neural network model, he can freely use the first type of operator and the second type of operator. That is, the operation flow of the neural network model finally obtained is composed of the second type of operator and the second type of operator.

[0105] The first type of operator includes a boundary identifier. For a first type of operator, the boundary identifier indicates a group of second type operators, and the group of second type operators is used to represent the calculation logic of the first type operator where the boundary identifier is located.

[0106] 302. Determine a first calculation graph of the neural network model according to the operation process.

[0107] In the deep learning framework, a mathematical calculation process will first be compiled and converted into a calculation graph. In the embodiment of the present application, the deep learning framework compiles the operation process mentioned in step 301 above to obtain a first calculation graph. After the neural network model is compiled, it can be regarded as a complex calculation graph composed of hundreds or even thousands of operators.

[0108] In a specific implementation, each operator of the first type includes a boundary identifier, and during the compilation process, the boundary identifier of each operator of the first type is kept unchanged to obtain the first computation graph.

[0109] Combine the following Figure 5 To explain, Figure 5 Schematic diagram of the first calculation graph provided in the embodiment of the present application. Figure 5 As shown, the first computation graph may include a main graph and a subgraph. The main graph represents the computation flow by using the boundary markers of the first type of operators and the names of the second type of operators. For example, Figure 5 In the example shown, OP1 represents the first type of operator. Specifically, OP1 includes a boundary mark, which is used to indicate that when processing the main graph, OP1 should be considered as a whole. OP2 and OP3 represent the second type of operator. Specifically, OP2 and OP3 are the names of the operators. The calculation process of the main graph is that OP1 has two inputs, namely data x and data y, OP2 has two inputs, namely data z and the output of OP1, OP3 has one input, the input of OP3 is the output of OP2, and the output of OP3 is the output result of the main graph.

[0110] The subgraph is used to represent the computational logic of the first type of operator, such as Figure 5 In the example shown, the computational logic of the subgraph is that the sub operator includes two inputs, namely data x and data y, the output of the sub operator is the input of the exp operator, and the output result of the exp operator is the output result of OP1, wherein the sub operator and the exp operator are operators of the second type, that is, pre-selected commonly used operators. The subgraph is provided with a boundary identifier, and in a specific embodiment, the boundary identifier may be a scope. The scope may represent a set of all other second-type operators associated with the computational logic of the first-type operator. It should be noted that the categories of the boundary identifiers included in each first-type operator may be exactly the same or different. In the embodiment of the present application, the boundary identifier included in the first-type operator is only used to indicate that in the process of processing the first computational graph, the first-type operator is to be processed as a whole.

[0111] like Figure 5 As shown, the main graph includes the first type of operator OP1, specifically, the main graph includes boundary markers, and the second type of operators OP2 and OP3, specifically, the main graph includes the names of OP2 and OP3. x, y, z represent input data, and c represents output data. The deep learning framework cannot know the calculation logic of OP2 and OP3, but can obtain the calculation logic of OP1 through the subgraph. Figure 5The sub-graph shown is the calculation logic of OP1. OP1 is composed of basic operators, that is, OP1 is composed of operators of the second type, such as Figure 5 As shown in , OP1 is composed of the sub operator and the exp operator. After the subgraph obtains the execution result of OP1, it returns the result obtained by the subgraph to the main graph.

[0112] Depend on Figure 5 It can be seen from the corresponding embodiments that the operation process of the neural network is represented by two different types of operators, and the calculation logic of the first type of operator is represented by the second type of operator. The second type of operator is pre-configured with the corresponding kernel. The technical solution provided by the present application can obtain the first type of operator by combining the basic second type of operator. Without increasing the registration process, the deep learning framework can obtain the calculation logic of the first type of operator, so that the calculation graph has more optimization opportunities.

[0113] It should be understood that when the deep learning framework can know the calculation logic of the operator, the opportunities for graph optimization will be greatly increased. The solution provided in this application can be used in various graph optimization scenarios. For example, the following is an example of how to use Figure 3 It should be noted that in addition to the several scenarios mentioned in this application, the graph optimization scenarios can also include other application scenarios, and other schemes for graph optimization using the calculation graph obtained by the scheme provided by this application should also fall within the scope of protection of this application.

[0114] Figure 6 Another method for processing a neural network model is provided in an embodiment of the present application.

[0115] like Figure 6 As shown, another processing method of a neural network model provided in an embodiment of the present application may include:

[0116] 601. Obtain an operation flow of a neural network model, where the operation flow is represented by a first type of operator and a second type of operator.

[0117] 602. Determine a first calculation graph of the neural network model according to the operation process.

[0118] Step 601 and step 602 can refer to Figure 3 The steps 301 and 302 of the corresponding embodiments are understood and will not be repeated here.

[0119] 603. Optimize the first computation graph using the first type of operator as the processing granularity to obtain a second computation graph.

[0120] The solution provided by the present application uses the first type of operator as the processing granularity when optimizing the computational graph, that is, the first type of operator is treated as a whole for processing. Specifically, the first type of operator can be used as the processing granularity in a variety of implementation methods. For example, one of the methods is that when the programmer inputs the computational flow of the neural network model, the first type of operator is annotated, and when the first computational graph of the neural network model is generated according to the computational flow, the boundary information of the first type of operator is added according to the annotation, such as in Figure 3 The scope introduced in the corresponding embodiment, when generating the first calculation graph of the neural network model according to the operation process, can add a scope to the first type of operator according to the annotation.

[0121] The embodiment of the present application uses the first type of operator as the processing granularity to ensure that the boundary of the first type of operator is not broken, that is, to ensure that the first type of operator can be processed as a whole. If the boundary of the first type of operator is broken, then all operators included in the first calculation graph will be merged into one graph, which may cause the search space to be too large when various graph optimizations are performed later. Figure 7 as well as Figure 8 For example, Figure 7 The first computational graph shown in includes a main graph and a subgraph. All operators of the first type in the main graph include boundary markers. For example, OP1, OP4, and OP6 represent operators of the first type. OP1, OP4, and OP6 all include boundary markers. The subgraph corresponding to OP1 further describes the computational logic of OP1, the subgraph corresponding to OP4 further describes the computational logic of OP1, and the subgraph corresponding to OP6 further describes the computational logic of OP6 (the specific internal logic is not shown in the figure). The main graph also includes operators of the second type, such as OP2, OP3, and OP5. x, y, z, m, and n all represent input data. The embodiment of the present application uses the first type of operator as the processing granularity. Figure 7 In the scenario shown, you can choose to optimize any one of the subgraphs corresponding to OP1, OP4, and OP6, or you can choose to optimize any two of the subgraphs corresponding to OP1, OP4, and OP6, or you can choose to optimize the subgraphs corresponding to OP1, OP4, and OP6 together. In other words, with the first type of operator as the processing granularity, that is, the first type of operator includes a boundary identifier, some graph optimization processing can be performed inside the first type of operator or between the first type of operators, so as to avoid the problem of too large a search space, which further leads to the problem of uncontrollable compilation time.

[0122] Figure 8The first computational graph does not include boundary markers. Figure 7 The first computational graph given includes a schematic diagram of boundary identification, Figure 8 In the first computation graph shown, the boundaries of the first type of operators, such as OP1, OP4, and OP6, are broken, that is, OP1, OP4, and OP6 do not include boundary markers, which will result in a large number of operators in the first computation graph, such as Figure 8 As shown in , it includes multiple exp operators, add operators, mul operators, and sub operators. This will lead to the problem that the operator granularity is too fine, and the search space is too large in the subsequent graph optimization process, resulting in the problem of too long compilation time. However, if the first type of operators is used as the processing granularity, that is, the first type of operators include boundary identifiers, some graph optimization processing can be performed between the first type of operators to avoid the problem of too large search space.

[0123] The following describes how to optimize the first computation graph using the first type of operator as the processing granularity to obtain the second computation graph in combination with several specific optimization schemes.

[0124] In a specific implementation, assuming that the first type of operator includes a third operator and a fourth operator, and the third operator and the fourth operator include the same calculation logic, optimizing the first calculation graph with the first type of operator as the processing granularity may include:

[0125] The subgraph corresponding to the third operator and the subgraph corresponding to the fourth operator are fused to obtain a fused subgraph, and the second computation graph includes the fused subgraph.

[0126] Combine the following Figure 7 To explain, Figure 7 As shown in , the first type of operator includes OP1, OP4 and OP6, wherein any one of the first type of operators can be regarded as the third operator, and any one of the operators in the first calculation graph except the selected third operator can be regarded as the fourth operator. When the first calculation graph is optimized with the first type of operator as the processing granularity, if any two or more first type of operators include the same calculation logic, the subgraphs corresponding to any two or more first type of operators can be fused to obtain a fused subgraph, which can simultaneously return the output results of two or more first type of operators. Assuming that OP1 and OP4 include the same calculation logic, the subgraphs corresponding to OP1 and OP4 can be fused to obtain a fused subgraph, which can simultaneously return the output results corresponding to OP1 and OP4. Or assuming that OP1, OP4 and OP6 include the same calculation logic, the subgraphs corresponding to OP1, OP4 and OP6 can be fused to obtain a fused subgraph, which can simultaneously return the output results corresponding to OP1, OP4 and OP6.

[0127] The following takes two first-type operators including the same calculation logic as an example to further illustrate how, when determining that the third operator and the fourth operator include the same calculation logic, the subgraph corresponding to the third operator and the subgraph corresponding to the fourth operator are fused to obtain a fused subgraph. Assume that the calculation logic of the third operator and the fourth operator is as follows:

[0128]

[0129] We can regard sigmoid(x) as the third operator and sigmoid'(x) as the fourth operator, or we can regard sigmoid(x) as the fourth operator and sigmoid'(x) as the third operator. Fig. 9 FIG. 2 is a schematic diagram of another processing method of a neural network model provided in an embodiment of the present application. Fig. 9 As shown in a in the figure, it is a subgraph of the sigmoid operator, that is, Fig. 9 The a in sigmoid is used to represent the calculation logic of sigmoid. The expression of sigmoid(x) has been given above and will not be repeated here. Fig. 9 As shown in b in the figure, it is the subgraph of the sigmoid'(x) operator, that is, Fig. 9 The b in is used to represent the calculation logic of the sigmoid'(x) operator. The expression of the sigmoid'(x) operator has been given above and will not be repeated here. Fig. 9 a and Fig. 9 As can be seen from b in the figure, the computational logic of the sigmoid operator is partially the same as that of sigmoid'(x). Therefore, we can modify the first computational graph to Fig. 9 The subgraph shown in a in Fig. 9 The subgraphs shown in b in are fused to obtain a fused subgraph, such as Fig. 9 The fused subgraph is shown in c. The make tuple operator outputs two results at the same time, that is, the subgraphs of the two operators are fused into one graph and the output results of the sigmoid operator and the sigmoid'(x) operator are returned at the same time.

[0130] In a specific implementation, the third operator may be a forward operator, and the fourth operator may be a reverse operator corresponding to the third operator, or the fourth operator is a forward operator, and the third operator is a reverse operator corresponding to the fourth operator. This is because the forward operator is an operator used in the process of forward signal propagation, and most forward operators require a corresponding reverse operator. In deep learning network training, the propagation of errors in each layer of the network is calculated through reverse operators and errors, and the parameters of each layer are trained to make the error extremely small. The reverse operator can usually be understood as the derivative of the forward operator plus certain operations, and the derivatives of many operators in the field of deep learning have many repeated operation logics with themselves.

[0131] In a specific implementation, the first type of operator includes a third operator and a fourth operator, the intermediate calculation result of the third operator is the same as the intermediate calculation result of the fourth operator, and the first calculation graph is optimized with the first type of operator as the processing granularity, including:

[0132] Determine the intermediate calculation result of the third operator as the input parameter of the fourth operator. The intermediate calculation result refers to the output result of the second type of operator included in the first type of operator. For example, assume that the fifth operator and the sixth operator both include the C operator, and the C operator has two inputs, namely the output of the A operator and the output of the B operator. The input of the A operator is x, and the input of the B operator is y. Since the output results of the C operator of the fifth operator and the sixth operator are the same, the output of the C operator of the fifth operator can be directly used as the input parameter of the sixth operator, and the sixth operator does not need to repeatedly calculate the output of the C operator.

[0133] Combine the following Figure 7 To explain, Figure 7As shown in , the first type of operators include OP1, OP4 and OP6, wherein any of the first type of operators can be regarded as the third operator, and any operator in the first calculation graph except the selected third operator can be regarded as the fourth operator. When the first calculation graph is optimized with the first type of operators as the processing granularity, when it is determined that the intermediate calculation result of the third operator is the same as the intermediate calculation result of the fourth operator, the intermediate calculation result of the third operator is determined as the input parameter of the fourth operator. For example, assuming that the intermediate calculation results of OP1 and OP4 are the same, the intermediate calculation result of OP1 can be used as the input parameter of OP4, or the intermediate calculation result of OP4 can be used as the input parameter of OP1. In other words, when the intermediate calculation results of two operators are the same, one of the operators can call the intermediate result of the other operator without secondary calculation. For another example, assuming that the intermediate calculation results of OP1, OP4 and OP6 are the same, the intermediate calculation result of OP1 can be used as the input parameter of OP4 and OP6, that is, OP4 and OP6 can directly call the intermediate calculation result of OP1 without secondary calculation, or the intermediate calculation result of OP4 can be used as the intermediate calculation result of OP1 and OP6, that is, OP1 and OP6 can directly call the intermediate calculation result of OP4 without secondary calculation, or the intermediate calculation result of OP6 can be used as the intermediate calculation result of OP1 and OP4, that is, OP1 and OP4 can directly call the intermediate calculation result of OP6 without secondary calculation.

[0134] The following example uses the same intermediate calculation results of two first-type operators to further illustrate how to determine the intermediate calculation result of the third operator as the input parameter of the fourth operator. Assume that the calculation logic of the third operator and the fourth operator is as follows:

[0135]

[0136]

[0137] The tanh operator can be considered as the third operator and the tanh'(x) operator as the fourth operator, or the tanh'(x) operator can be considered as the third operator and the tanh operator as the fourth operator. Fig.10 FIG. 2 is a schematic diagram of another processing method of a neural network model provided in an embodiment of the present application. Fig.10 As shown, the above-mentioned tanh operator and tanh'(x) are used as examples. Fig.10 As shown in a in the figure, it is the subgraph of the tanh operator, that is, Fig.10 The a in tanh is used to represent the calculation logic of tanh. The expression of tanh has been given above and will not be repeated here. Fig.10 As shown in b, it is the subgraph of the tanh'(x) operator, that is, Fig.10The b in tanh'(x) is used to represent the calculation logic of the tanh'(x) operator. The expression of the tanh'(x) operator has been given above and will not be repeated here. Fig.10 a and Fig.10 As can be seen from b in , the intermediate calculation result in the calculation logic of the tanh′(x) operator is the same as the calculation result in the calculation logic of the tanh operator. Then, the first calculation graph can be modified so that one subgraph directly reuses the intermediate calculation result obtained by another subgraph. For example, Fig.10 As shown in c, the intermediate calculation result of the tanh operator is used as the input parameter of the tanh'(x) operator. The input parameter means that the tanh'(x) operator can directly call the intermediate calculation result of the tanh operator.

[0138] In a specific implementation, the third operator may be a forward operator, and the fourth operator may be a reverse operator corresponding to the third operator, or the fourth operator is a forward operator, and the third operator is a reverse operator corresponding to the fourth operator. This is because the forward operator is an operator used in the process of forward signal propagation, and most forward operators require a corresponding reverse operator. In deep learning network training, the propagation of errors in each layer of the network is calculated through reverse operators and errors, and the parameters of each layer are trained to make the error extremely small. The reverse operator can usually be understood as the derivative of the forward operator plus certain operations, and the derivatives of many operators in the field of deep learning have many repeated operation logics with themselves.

[0139] The above describes the processing method of the neural network model provided in the embodiment of the present application, and specifically, how to obtain the calculation graph and how to optimize the calculation graph. The deep learning framework needs to map the optimized calculation graph into instructions and machine code of the hardware platform, that is, generate a kernel file. This process is described in detail below.

[0140] like Fig.11 The figure is a schematic diagram of a process of generating a kernel file provided in an embodiment of the present application.

[0141] like Fig.11 As shown, a process of generating a kernel file may include the following steps:

[0142] 1101. Determine a second IR of an operator of the first type according to a first intermediate representation IR of an operator of the second type and a computation logic of an operator of the first type.

[0143] The deep learning framework determines which second-type operators the first-type operator includes based on the calculation logic of the first-type operator, that is, determines which second-type operators the first-type operator is composed of. Since the IR module registers with the deep learning framework through the operator name during the registration process, the deep learning framework can query the IR name of the second-type operator based on the name of the second-type operator, and then load the function pointer corresponding to the IR of the operator. The deep learning framework then obtains the IR corresponding to the second-type operator based on the function pointer.

[0144] 1102. Determine a kernel corresponding to the first type of operator according to the second IR.

[0145] After the deep learning framework obtains the IRs corresponding to all the second-type operators included in the first-type operators, the deep learning framework combines the IRs corresponding to all the second-type operators obtained to obtain the IRs corresponding to the first-type operators. The deep learning framework compiles the IRs corresponding to the first-type operators to obtain the kernel files corresponding to the first-type operators.

[0146] Combine the following Fig.12 The above process is illustrated with an example. Fig.12 As shown in FIG. 1 , a schematic diagram of a process for generating a kernel file provided in an embodiment of the present application is shown. Assume that the operator of the first type is Figure 8 As shown in OP1, the following description is made by taking the first type of operator OP1 as an example.

[0147] 1201. The operator compiling module 103 parses the computation logic of the first type of operator.

[0148] For example, if the first type of operator is OP1, the operator compilation module 103 parses the calculation logic of the first type of operator to obtain the calculation logic of OP1. The second type of operators included in OP1 are sub and exp. The sub operator includes two inputs, x and y. The output of the sub operator is the input of the exp operator, and the output of the exp operator is the output of OP1.

[0149] 1202. The operator compilation module 103 queries the operator management module 104 for the IR function pointer of the sub operator through the name of the sub operator.

[0150] The deep learning framework can query the name of the sub operator's IR through the name of the sub operator, and then load the function pointer to the sub operator's IR.

[0151] 1203 . The operator management module 104 returns the acquired IR function pointer of the sub operator to the operator compilation module 103 .

[0152] 1204. The operator compiling module 103 obtains the IR corresponding to the sub operator from the IR module 105 by calling the function pointer of the IR of the sub operator.

[0153] 1205 . The IR module 105 returns the IR of the sub operator to the operator compilation module 103 .

[0154] 1206. The operator compilation module 103 queries the operator management module 104 for the IR function pointer of the exp operator through the name of the exp operator.

[0155] The deep learning framework can query the name of the IR of the exp operator through the name of the exp operator, and then load the function pointer to the IR of the exp operator.

[0156] 1207 . The operator management module 104 returns the acquired IR function pointer of the exp operator to the operator compilation module 103 .

[0157] 1208. The operator compilation module 103 obtains the IR corresponding to the exp operator from the IR module 105 by calling the function pointer of the IR of the exp operator.

[0158] 1209 . The IR module 105 returns the IR of the exp operator to the operator compilation module 103 .

[0159] 1210. The operator compilation module 103 combines the IRs of the first type of operators.

[0160] Through the above steps 1202 to 1209, the operator compilation module 103 obtains the IR of the sub operator and the IR of the exp operator. The operator compilation module 103 can combine the IR of the sub operator and the IR of the exp operator according to the calculation logic of OP1 to obtain the IR of OP1.

[0161] 1211. The operator compiling module 103 sends the IR of the first type operator to the kernel module 106. This process is a process of compiling the IR of the first type operator.

[0162] 1212. The kernel module 106 returns the compiled kernel file of the first type of operator to the operator compilation module 103.

[0163] Above, a processing method for a neural network model provided in an embodiment of the present application is introduced. Through the scheme provided in the embodiment of the present application, the deep learning framework can learn the calculation logic of the first type of operator, so that the calculation graph has more optimization opportunities. It can be understood that in order to realize the above functions, the above-mentioned deep learning framework includes hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0164] Describing from the hardware structure, Figures 3 to 12 The deep learning framework can be implemented by one physical device, or by multiple physical devices, or it can be a logical function module within a physical device, and the embodiments of the present application do not specifically limit this.

[0165] For example, deep learning frameworks can be Fig.13 The computer device in the embodiment can also be called a processing device of the neural network model. Fig.13 The figure shows a hardware structure diagram of a computer device provided in an embodiment of the present application, including: a communication interface 1301 and a processor 1302 , and may also include a memory 1303 .

[0166] The communication interface 1301 may use any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0167] Processor 1302 includes, but is not limited to, one or more of a central processing unit (CPU), a network processor (NP), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Processor 1302 is responsible for communication lines 1304 and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 1303 can be used to store data used by processor 1302 when performing operations.

[0168] The memory 1303 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor 1302 via a communication line 1304. The memory 1303 may also be integrated with the processor 1302. If the memory 1303 and the processor 1302 are independent devices, the memory 1303 and the processor 1302 are connected, for example, the memory 1303 and the processor 1302 can communicate through a communication line. The communication interface 1301 and the processor 1302 can communicate through a communication line, and the communication interface 1301 can also be directly connected to the processor 1302.

[0169] The communication line 1304 may include any number of interconnected buses and bridges, and the communication line 1304 links together various circuits including one or more processors 1302 represented by the processor 1302 and a memory represented by the memory 1303. The communication line 1304 may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be further described in this application.

[0170] In a specific embodiment, the deep learning framework may include: a memory for storing computer-readable instructions. It may also include a communication interface coupled to the memory, for obtaining the operation flow of the neural network model, the operation flow is represented by at least one first type of operator and multiple second type of operators, in the operation flow, the first type of operator may include a boundary identifier, the calculation logic of the first type of operator is represented by a group of second type of operators, for any first type of operator, the range of the second type of operator that any first type of operator can include is indicated by the boundary identifier in any first type of operator. It also includes: a processor coupled to the communication interface, for executing the computer-readable instructions in the memory to perform the following operations: obtaining the first calculation graph of the neural network model according to the operation flow.

[0171] In a specific implementation, the first computational graph may include a main graph and a sub-graph, and the processor is specifically used to: determine the main graph and the sub-graph of the neural network model according to the computational process, the first type of operator in the main graph is indicated by a boundary marker, the second type of operator in the main graph is indicated by the name of the second type of operator, the main graph is used to output the result of the computational process, the sub-graph may include the name of the second type of operator that may be included in any one of the first type of operators, the sub-graph is used to output the result of a first type of operator, and a sub-graph represents the computational logic of a first type of operator.

[0172] In a specific implementation, the processor is further configured to: optimize the first computation graph using the first type of operator as a processing granularity to obtain a second computation graph.

[0173] In a specific implementation, the first type of operator may include a third operator and a fourth operator, and the third operator and the fourth operator may include the same computing logic, and the processor is specifically used to: fuse the subgraph corresponding to the third operator and the subgraph corresponding to the fourth operator to obtain a fused subgraph, and the second computing graph may include the fused subgraph.

[0174] In a specific implementation, the first type of operator may include a fifth operator and a sixth operator, the intermediate calculation result of the fifth operator is the same as the intermediate calculation result of the sixth operator, and the processor is specifically used to: use the intermediate calculation result of the fifth operator as an input parameter of the sixth operator.

[0175] In a specific implementation, the third operator is a forward operator, and the fourth operator is a reverse operator corresponding to the third operator, or the fourth operator is a forward operator, and the third operator is a reverse operator corresponding to the fourth operator.

[0176] In a specific implementation, the processor is further configured to determine a second IR of the first type of operator according to the first intermediate representation IR of the second type of operator and the computation logic of the first type of operator, and to determine a set of kernel functions corresponding to the first type of operator according to the second IR determined by the operator compilation module.

[0177] In the embodiment of the present application, the communication interface can be regarded as the programming interface module 101 of the deep learning framework, and the processor with processing function can be regarded as the processing module of the deep learning framework. Specifically, the processing module can include the calculation graph processing module 102 and the operator layer including the operator compilation module 103, the operator management module 104, the IR module 105 and the kernel module 106. The memory is regarded as the storage module of the deep learning framework (not shown in the figure).

[0178] After the operator compilation module 103 obtains the IR of the required operator, it sends the IR of the required operator to the kernel module 106. The kernel module 106 compiles the obtained IR to obtain the kernel file, and returns the kernel file to the operator compilation module 103. After the operator compilation module 103 obtains the kernel file, it can send the kernel file to the execution device, such as the hardware platform. This process is to map the optimized calculation graph into the instructions and machine code of the hardware platform (in the embodiment of the present application, the hardware platform is sometimes referred to as the execution device. When the difference between the two is not emphasized, the two express the same meaning). The deep learning framework provides each operator with a kernel function (kernel) executed on the CPU or artificial intelligence processor. The kernel function is the instruction and machine code of the hardware platform. The following is an introduction to an execution device provided in an embodiment of the present application.

[0179] See also Fig.14 , is a schematic diagram of the structure of an execution device provided in an embodiment of the present application. Fig.14As shown, the execution device may include a processor 1401, a memory 1402, a communication bus 1403, and a communication interface 1404, and an artificial intelligence processor 1405 is connected to the memory 1402 and the communication interface 1404 via the communication bus.

[0180] The processor 1401 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 1401 may be a microprocessor or any conventional processor, etc.

[0181] The processor 1401 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the neural network processing method of the present application may be completed by an integrated logic circuit of hardware in the processor 1401 or by instructions in the form of software.

[0182] The memory 1402 may be a read-only memory (ROM), a random access memory (RAM) or other memory. In the embodiment of the present application, the memory 1402 is used to store data and various software programs, such as a program for splitting a neural network model according to a determined target splitting path in the embodiment of the present application.

[0183] Optionally, in the embodiment of the present application, the memory may include a physical device for storing information, which is usually to digitize the information and then store it in a medium using electrical, magnetic or optical methods. The memory described in this embodiment may also include: a device that stores information using electrical energy, such as RAM, ROM, etc.; a device that stores information using magnetic energy, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information using optical means, such as a CD or a DVD. Of course, there are other types of memory, such as quantum memory, graphene memory, etc.

[0184] The communication interface 1404 uses a transceiver such as, but not limited to, a transceiver to implement communication between the execution device and other devices or a communication network. For example, a model file sent by other devices can be received through the communication interface 1404.

[0185] Optionally, the execution device may also include at least one artificial intelligence processor 1405 .

[0186] The artificial intelligence processor 1405 can be mounted on the host CPU as a coprocessor, and the host CPU assigns tasks to it. In practical applications, the artificial intelligence processor 1405 can implement one or more operations. For example, taking the neural network processor (network processing Unit, NPU) NPU as an example, the core part of the NPU is the operation circuit, which is controlled by the controller to extract the matrix data in the memory 1402 and perform multiplication and addition operations.

[0187] Optionally, the artificial intelligence processor 1405 may include 8 clusters, each cluster including 4 artificial intelligence processor cores.

[0188] Optionally, the artificial intelligence processor 1405 may be an artificial intelligence processor with a reconfigurable architecture. Here, the reconfigurable architecture means that if an artificial intelligence processor can utilize reusable hardware resources and flexibly change its own architecture according to different application requirements, so as to provide an architecture that matches each specific application requirement, then this artificial intelligence processor is called a reconfigurable computing system, and its architecture is called a reconfigurable architecture.

[0189] It should be understood that the execution device is only an example provided in the embodiment of the present application, and the execution device may have more or fewer components than those shown, may combine two or more components, or may have different configurations of components.

[0190] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0191] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0192] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, disk or CD, etc.

[0193] The processing method of the neural network model provided in the embodiment of the present application and the related equipment are introduced in detail above. The principles and implementation methods of the present application are explained in this article by using specific examples. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A processing method for a neural network model, characterized in that: include: Acquire an operation flow of a neural network model, the operation flow is represented by at least one first-type operator and a plurality of second-type operators, the first-type operator is an operator whose computational logic can be represented by a group of second-type operators, and for any one of the first-type operators, a range of the second-type operators included in the any one of the first-type operators is indicated by a boundary identifier in the any one of the first-type operators; The operation flow is compiled to obtain a first calculation graph of the neural network model, wherein a range indicated by a boundary identifier of the first type of operator remains unchanged in the calculation graph.

2. The processing method according to claim 1, characterized in that: The first computational graph includes a main graph and a subgraph, and the computational process is compiled to obtain a first computational graph of the neural network model, including: The main graph and sub-graph of the neural network model are determined according to the operation process, the first type of operator in the main graph is indicated by the boundary marker, the second type of operator in the main graph is indicated by the name of the second type of operator, the main graph is used to output the result of the operation process, the sub-graph includes the name of the second type of operator included in any one of the first type of operators, the sub-graph is used to output the result of an operator of the first type, and a sub-graph represents the calculation logic of an operator of the first type.

3. The processing method according to claim 1 or 2, characterized in that: The method further comprises: The first computation graph is optimized with the first type of operator as the processing granularity to obtain a second computation graph.

4. The processing method according to claim 3, characterized in that: The first type of operator includes a third operator and a fourth operator, the third operator and the fourth operator include the same calculation logic, and the optimizing processing of the first calculation graph using the first type of operator as the processing granularity includes: The subgraph corresponding to the third operator and the subgraph corresponding to the fourth operator are fused to obtain a fused subgraph, wherein the second computation graph includes the fused subgraph.

5. The processing method according to claim 3, characterized in that: The first type of operator includes a fifth operator and a sixth operator, an intermediate calculation result of the fifth operator is the same as an intermediate calculation result of the sixth operator, and the optimizing processing of the first calculation graph using the first type of operator as the processing granularity includes: The intermediate calculation result of the fifth operator is used as an input parameter of the sixth operator.

6. The processing method according to claim 4, characterized in that: The third operator is a forward operator, and the fourth operator is a reverse operator corresponding to the third operator, or the fourth operator is a forward operator, and the third operator is a reverse operator corresponding to the fourth operator.

7. The processing method according to any one of claims 1 to 2, characterized in that: The method further comprises: Determine a second IR of the operator of the first type according to the first intermediate representation IR of the operator of the second type and the calculation logic of the operator of the first type; A set of kernel functions kernel corresponding to the first type of operator is determined according to the second IR.

8. The processing method according to any one of claims 1 to 2, characterized in that: The input of the operation process is tensor data, which is used to describe the characteristics of data of at least one of the following scenarios, including speech recognition, computer vision CV, video processing, image recognition and natural language processing NPL.

9. A processing device for a neural network model, characterized in that: include: A programming interface module, used to obtain an operation flow of a neural network model, wherein the operation flow is represented by at least one first-type operator and a plurality of second-type operators, wherein the first-type operator is an operator whose computational logic can be represented by a group of second-type operators, and for any one of the first-type operators, a range of the second-type operators included in the any one of the first-type operators is indicated by a boundary marker in the any one of the first-type operators; A calculation graph processing module is used to compile the operation process obtained by the programming interface module to obtain a first calculation graph of the neural network model, wherein the range indicated by the boundary identifier of the first type of operator in the calculation graph remains unchanged.

10. The processing device according to claim 9, characterized in that The first computation graph includes a main graph and a sub-graph, and the computation graph processing module is specifically used to: The main graph and sub-graph of the neural network model are determined according to the operation process, the first type of operator in the main graph is indicated by the boundary marker, the second type of operator in the main graph is indicated by the name of the second type of operator, the main graph is used to output the result of the operation process, the sub-graph includes the name of the second type of operator included in any one of the first type of operators, the sub-graph is used to output the result of an operator of the first type, and a sub-graph represents the calculation logic of an operator of the first type.

11. The processing device according to claim 9 or 10, characterized in that: The computational graph processing module is further used for: The first computation graph is optimized using the first type of operator as a processing granularity to obtain a second computation graph.

12. The processing device according to claim 11, characterized in that The first type of operator includes a third operator and a fourth operator, the third operator and the fourth operator include the same calculation logic, and the calculation graph processing module is specifically used to: The subgraph corresponding to the third operator and the subgraph corresponding to the fourth operator are fused to obtain a fused subgraph, wherein the second computation graph includes the fused subgraph.

13. The processing device according to claim 11, characterized in that The first type of operator includes a fifth operator and a sixth operator, an intermediate calculation result of the fifth operator is the same as an intermediate calculation result of the sixth operator, and the calculation graph processing module is specifically used to: The intermediate calculation result of the fifth operator is used as an input parameter of the sixth operator.

14. The processing device according to claim 12, characterized in that The third operator is a forward operator, and the fourth operator is a reverse operator corresponding to the third operator, or the fourth operator is a forward operator, and the third operator is a reverse operator corresponding to the fourth operator.

15. The processing device according to any one of claims 9 to 10, characterized in that: The device also includes an operator compilation module and a kernel module. An operator compiling module, configured to determine a second IR of an operator of the first type according to a first intermediate representation IR of an operator of the second type and a computation logic of an operator of the first type; The kernel module is used to determine a group of kernel functions kernel corresponding to the first type of operator according to the second IR determined by the operator compilation module.

16. The processing device according to any one of claims 9 to 10, characterized in that: The input of the operation process is tensor data, which is used to describe the characteristics of data of at least one of the following scenarios, including speech recognition, computer vision CV, video processing, image recognition and natural language processing NPL.

17. A method for processing a neural network model, characterized in that: include: Acquire an operation flow of a neural network model, wherein the operation flow includes at least one operator of a first type; The operation flow is compiled to obtain a first calculation graph of the neural network model, wherein, during the compilation process, the first type of operator is expanded into a boundary identifier of the first type of operator and a group of second type operators corresponding to the first type of operator, and for any one of the first type of operators, the range of the second type of operators included in any one of the first type of operators is indicated by the boundary identifier in any one of the first type of operators.

18. The processing method according to claim 17, characterized in that: The first computational graph includes a main graph and a subgraph, and the computational process is compiled to obtain a first computational graph of the neural network model, including: The main graph and sub-graph of the neural network model are determined according to the operation process, the first type of operator in the main graph is indicated by the boundary marker, the second type of operator in the main graph is indicated by the name of the second type of operator, the main graph is used to output the result of the operation process, the sub-graph includes the name of the second type of operator included in any one of the first type of operators, the sub-graph is used to output the result of an operator of the first type, and a sub-graph represents the calculation logic of an operator of the first type.

19. The processing method according to claim 17 or 18, characterized in that: The method further comprises: The first computation graph is optimized with the first type of operator as the processing granularity to obtain a second computation graph.

20. The processing method according to claim 19, characterized in that The first type of operator includes a third operator and a fourth operator, the third operator and the fourth operator include the same calculation logic, and the optimizing processing of the first calculation graph using the first type of operator as the processing granularity includes: The subgraph corresponding to the third operator and the subgraph corresponding to the fourth operator are fused to obtain a fused subgraph, wherein the second computation graph includes the fused subgraph.

21. The processing method according to claim 19, characterized in that: The first type of operator includes a fifth operator and a sixth operator, an intermediate calculation result of the fifth operator is the same as an intermediate calculation result of the sixth operator, and the optimizing processing of the first calculation graph using the first type of operator as the processing granularity includes: The intermediate calculation result of the fifth operator is used as an input parameter of the sixth operator.

22. The processing method according to claim 20, characterized in that: The third operator is a forward operator, and the fourth operator is a reverse operator corresponding to the third operator, or the fourth operator is a forward operator, and the third operator is a reverse operator corresponding to the fourth operator.

23. The processing method according to any one of claims 17 to 18, characterized in that: The method further comprises: Determine a second IR of the operator of the first type according to the first intermediate representation IR of the operator of the second type and the calculation logic of the operator of the first type; A set of kernel functions kernel corresponding to the first type of operator is determined according to the second IR.

24. The processing method according to any one of claims 17 to 18, characterized in that: The input of the operation process is tensor data, which is used to describe the characteristics of data of at least one of the following scenarios, including speech recognition, computer vision CV, video processing, image recognition and natural language processing NPL.

25. The processing method according to any one of claims 17 to 18, characterized in that: The range indicated by the boundary identifier of the first type of operator in the computation graph remains unchanged.

26. A processing device for a neural network model, characterized in that: include: A programming interface module, used to obtain a computing process of a neural network model, wherein the computing process includes at least one operator of a first type; A computational graph processing module is used to compile the operation flow obtained by the programming interface module to obtain a first computational graph of the neural network model, wherein, during the compilation process, the first type of operator is expanded into a boundary identifier of the first type of operator and a group of second type operators corresponding to the first type of operator, and for any one of the first type of operators, the range of the second type of operators included in any one of the first type of operators is indicated by the boundary identifier in any one of the first type of operators.

27. The processing device according to claim 26, characterized in that The first computation graph includes a main graph and a sub-graph, and the computation graph processing module is specifically used to: The main graph and sub-graph of the neural network model are determined according to the operation process, the first type of operator in the main graph is indicated by the boundary marker, the second type of operator in the main graph is indicated by the name of the second type of operator, the main graph is used to output the result of the operation process, the sub-graph includes the name of the second type of operator included in any one of the first type of operators, the sub-graph is used to output the result of an operator of the first type, and a sub-graph represents the calculation logic of an operator of the first type.

28. The processing device according to claim 26 or 27, characterized in that The computational graph processing module is further used for: The first computation graph is optimized using the first type of operator as a processing granularity to obtain a second computation graph.

29. The processing device according to claim 28, characterized in that The first type of operator includes a third operator and a fourth operator, the third operator and the fourth operator include the same calculation logic, and the calculation graph processing module is specifically used to: The subgraph corresponding to the third operator and the subgraph corresponding to the fourth operator are fused to obtain a fused subgraph, wherein the second computation graph includes the fused subgraph.

30. The processing device according to claim 28, characterized in that The first type of operator includes a fifth operator and a sixth operator, an intermediate calculation result of the fifth operator is the same as an intermediate calculation result of the sixth operator, and the calculation graph processing module is specifically used to: The intermediate calculation result of the fifth operator is used as an input parameter of the sixth operator.

31. The processing device according to claim 29, characterized in that The third operator is a forward operator, and the fourth operator is a reverse operator corresponding to the third operator, or the fourth operator is a forward operator, and the third operator is a reverse operator corresponding to the fourth operator.

32. The processing device according to any one of claims 26 to 27, characterized in that The device also includes an operator compilation module and a kernel module. An operator compiling module, configured to determine a second IR of an operator of the first type according to a first intermediate representation IR of an operator of the second type and a computation logic of an operator of the first type; The kernel module is used to determine a group of kernel functions kernel corresponding to the first type of operator according to the second IR determined by the operator compilation module.

33. The processing device according to any one of claims 26 to 27, characterized in that The input of the operation process is tensor data, which is used to describe the characteristics of data of at least one of the following scenarios, including speech recognition, computer vision CV, video processing, image recognition and natural language processing NPL.

34. The processing device according to any one of claims 26 to 27, characterized in that The range indicated by the boundary identifier of the first type of operator in the computation graph remains unchanged.

35. A processing device for a neural network model, characterized in that: include: a memory for storing computer readable instructions; The invention also includes a processor coupled to the memory, configured to execute computer-readable instructions in the memory to perform the method as described in any one of claims 1 to 8 or 17 to 25.

36. A computer-readable storage medium, characterized in that: When the instructions are executed on a computer device, the computer device is caused to execute the method as described in any one of claims 1 to 8 or 17 to 25.

37. A computer program product, when executed on a computer, enables the computer to perform the method as described in any one of claims 1 to 8, or 17 to 25.

Images