Method, device and related products for generating operation data

Abstract

The present disclosure relates to a method, device, and related product for generating operational data. The product includes a controller unit, the controller unit comprising: an instruction cache unit, an instruction processing unit, and a storage queue unit; the instruction cache unit is used to store computational instructions associated with the artificial neural network operation; the instruction processing unit is used to parse the computational instructions to obtain multiple computational instructions; and the storage queue unit is used to store an instruction queue, the instruction queue comprising: multiple computational instructions or computational instructions to be executed in a sequential order within the queue. Through the above method, the present disclosure can improve the operational efficiency of related products when performing neural network model operations.

Description

Technical Field

[0001] The present disclosure relates to the field of deep learning, and in particular to a method and device for generating operation data and related products. Background Art

[0002] Neural network algorithms are a recently popular type of machine learning algorithm in artificial intelligence (AI), achieving remarkable results in various fields, such as image recognition, speech recognition, and natural language processing. As neural network algorithms develop, their complexity increases, and to improve recognition, the scale of the models is also gradually increasing. Summary of the Invention

[0003] In view of this, the present disclosure proposes a method, device and related products for generating operation data, which can improve the performance optimization effect of deep learning algorithms on corresponding hardware platforms.

[0004] According to a first aspect of the present disclosure, a method for generating operation data is provided, the method comprising: receiving a user instruction; and triggering a deep learning programming library interface to create or call operation data according to the user instruction, wherein the operation data comprises at least one of tensor data and an operation instruction.

[0005] According to a second aspect of the present disclosure, a device for generating operation data is provided, comprising: a user instruction receiving module for receiving user instructions; a triggering module for triggering a deep learning programming library interface to create or call operation data according to the user instructions, wherein the operation data includes at least one of tensor data and operation instructions.

[0006] According to a third aspect of the present disclosure, a deep learning operation device is provided, which includes the operation data generation device as described in the second aspect above, and the deep learning operation device is used to complete the set deep learning operation.

[0007] According to a fourth aspect of the present disclosure, a combined operation device is provided, which includes the deep learning operation device as described in the third aspect above, a universal interconnection interface and other processing devices; the deep learning operation device interacts with the other processing devices to jointly complete the user-specified computing operations.

[0008] According to a fifth aspect of the present disclosure, a deep learning chip is provided, comprising: an operating data generating device as described in the second aspect above; or, a deep learning computing device as described in the third aspect above; or, a combined computing device as described in the fourth aspect above.

[0009] According to a sixth aspect of the present disclosure, an electronic device is provided, comprising: an operating data generating device as described in the second aspect above; or, a deep learning computing device as described in the third aspect above; or, a combined computing device as described in the fourth aspect above; or, a deep learning chip as described in the fifth aspect above.

[0010] By creating or calling operation data based on user instructions received by the deep learning programming library interface, and the operation data can be used to subsequently obtain the binary code of the deep learning algorithm, the above process can provide users with a universal programming interface to achieve effective conversion between user instructions and machine instructions.

[0011] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0013] Figure 1 A flowchart illustrating a method for generating operation data according to an embodiment of the present disclosure is shown.

[0014] Figure 2 A schematic diagram illustrating the overall architecture of a neural algorithm programming library interface according to an embodiment of the present disclosure is shown.

[0015] Figure 3 A diagram illustrating the correspondence between attributes, classifications, and meanings of tensor data according to an embodiment of the present disclosure is shown.

[0016] Figure 4 A diagram showing the correspondence between tensor data and shape symbols according to an embodiment of the present disclosure.

[0017] Figure 5 A diagram illustrating build-in operation instructions supported by NCLAPI according to an embodiment of the present disclosure is shown.

[0018] Figure 6 A schematic diagram illustrating an implementation method of automatic precision optimization according to an embodiment of the present disclosure is shown.

[0019] Figure 7 A schematic diagram of a calculation model according to an embodiment of the present disclosure is shown.

[0020] Figure 8 A diagram illustrating a principle for generating a customized operation instruction according to an embodiment of the present disclosure is shown.

[0021] Figure 9A schematic diagram of an operation fusion result according to an embodiment of the present disclosure is shown.

[0022] Figure 10 A schematic diagram of an operation fusion result according to an embodiment of the present disclosure is shown.

[0023] Figure 11 A schematic diagram illustrating a related programming interface for operation fusion according to an embodiment of the present disclosure is shown.

[0024] Figure 12 A schematic diagram of a process of creating a fusion operation according to an embodiment of the present disclosure is shown.

[0025] Figure 13 A data flow diagram of a three-layer computing model according to an embodiment of the present disclosure is shown.

[0026] Figure 14 A block diagram illustrating an implementation of a hybrid programming model according to an embodiment of the present disclosure is shown.

[0027] Figure 15 A schematic diagram illustrating the difference between an offline mode and an online mode according to an embodiment of the present disclosure.

[0028] Figure 16 A schematic diagram illustrating an offline interface according to an embodiment of the present disclosure is shown.

[0029] Figure 17 An architectural diagram of TensorFlow according to an embodiment of the present disclosure is shown.

[0030] Figure 18 A schematic diagram showing a comparison between NCLAPI and mainstream deep learning programming library interfaces according to an embodiment of the present disclosure is shown.

[0031] Figure 19 A schematic diagram of the overall architecture of NCLA according to an embodiment of the present disclosure is shown.

[0032] Figure 20 A schematic diagram illustrating an implementation of a static operation pool according to an embodiment of the present disclosure is shown.

[0033] Figure 21 A schematic diagram of the CDUCA architecture according to an embodiment of the present disclosure is shown.

[0034] Figure 22 The form of the original computation graph according to an embodiment of the present disclosure is shown.

[0035] Figure 23 A workflow diagram of a computational graph engine according to an embodiment of the present disclosure is shown.

[0036] Figure 24 A schematic diagram illustrating a structure including substructures within an image classification network according to an embodiment of the present disclosure is shown.

[0037] Figure 25 A flowchart of a method for compiling a deep learning algorithm according to an embodiment of the present disclosure is shown.

[0038] Figure 26 A schematic diagram of an instruction pipeline according to an embodiment of the present disclosure is shown.

[0039] Figure 27 A diagram illustrating an implementation method for optimizing model data according to an embodiment of the present disclosure is shown.

[0040] Figure 28 A schematic diagram illustrating optimization after data splitting according to an embodiment of the present disclosure is shown.

[0041] Figure 29 A schematic diagram showing modules and functions of a runtime system according to an embodiment of the present disclosure is shown.

[0042] Figure 30 A block diagram illustrating an apparatus for generating operation data according to an embodiment of the present disclosure is shown.

[0043] Figure 31 A block diagram of a combined processing device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0044] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0045] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0046] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0047] Deep learning programming libraries offer advantages such as high efficiency and ease of use, and are already widely used in production. However, existing deep learning programming libraries fail to balance performance optimization and programming flexibility in their design, and lack a unified theoretical foundation to guide their design. Regarding performance optimization, some libraries lack support for operation fusion, end-to-end execution, operation specialization, and offline optimization, resulting in suboptimal performance. Regarding programming flexibility, some libraries do not support layer-by-layer execution or runtime variability of operation parameters, limiting the library's scope of application. Because existing deep learning programming libraries fail to balance performance optimization and programming flexibility in their programming model design, a universal library interface with flexible programming is urgently needed.

[0048] Based on the above problems, Figure 1 A flow chart illustrating a method for generating operation data according to an embodiment of the present disclosure is shown. As shown in the figure, the method may include:

[0049] Step S11, receiving user instructions.

[0050] Step S12: triggering the deep learning programming library interface to create or call operation data according to the user instruction, wherein the operation data includes at least one of tensor data and operation instructions.

[0051] By creating or calling operation data based on user instructions received by the deep learning programming library interface, and the operation data can be used to subsequently obtain the binary code of the deep learning algorithm, the above process can provide users with a universal programming interface to achieve effective conversion between user instructions and machine instructions.

[0052] In the above steps, the implementation of the deep learning programming library interface is not limited and can be flexibly selected based on actual circumstances. In one possible implementation, the deep learning programming library interface can be the Neural Calculation Library API (NCLAPI). The specific implementation of this interface can be determined based on actual circumstances and is not limited to the following disclosed embodiments. Figure 2 A schematic diagram of the overall architecture of the neural algorithm programming library interface according to an embodiment of the present disclosure is shown. As shown in the figure, in an example, the implementation method of the NCLAPI interface can be: to enable the NCLAPI interface to have good deep learning modeling capabilities by simulating neural algorithms; to flexibly support various performance optimizations by designing reshapeable operations and corresponding operation rules; to improve the flexibility of the programming model by designing a hybrid programming model; to simplify the design of data structures and interfaces, and to hide hardware details inside the data structures and interfaces.

[0053] In the disclosed embodiments described above, neural algorithms are a functional deep learning modeling approach. In one example, neural algorithms can use tensors to represent the input data, output data, and model parameters of the input layer, and functions to represent the deep learning processing layer. Functions can be combined according to certain rules to construct various deep learning computational models. Because functions are inherently composable and reusable, neural algorithms can effectively represent the composability and reusability of deep learning algorithms. Neural algorithms designed according to this approach possess powerful deep learning modeling capabilities. Currently known deep learning frameworks such as TensorFlow and MXNet all use directed graphs to model deep learning computational models. Experiments have demonstrated that any directed graph can be mapped into a combination of functions, and any combination of functions can be mapped into a directed acyclic graph. Therefore, neural algorithms possess the same deep learning modeling capabilities as directed graphs. In one example, the NCLAPI interface can be given advanced deep learning modeling capabilities by simulating neural algorithms. Therefore, under appropriate simulation methods, NCLAPI can achieve learning modeling capabilities equivalent to directed graphs.

[0054] pass Figure 2 As can be seen in one example, NCLAPI can have two data structures: tensors (nclTensor) and plastic operations (nclOperator). nclTensor is used to describe the input data, output data, and model parameters of deep learning processing layers, while nclOperator is used to describe deep learning processing layers. These two data structures mimic the tensors and functions of neural algorithms, with some modifications and extensions based on actual programming models.

[0055] Since NCLAPI can contain two data structures: tensors and plastic operations, in one possible implementation, the operation data passed by the deep learning programming library interface can be tensor data corresponding to the nclTensor data structure, operation instructions corresponding to the nclOperator data structure, or both tensor data and operation instructions.

[0056] In one possible implementation, tensor data in NCLAPI is an abstract representation of multidimensional data that can be used to represent the input and output data of deep learning processing layers, as well as model parameters. In one example, the input, output, and weight data of a convolutional layer can all be represented as tensor data. Therefore, in one possible implementation, tensor data can have the following characteristics: it contains multiple attributes; it can describe multidimensional data such as scalars, vectors, matrices, and tensors; it follows certain naming conventions; and it can describe a variety of data types.

[0057] The above-mentioned public embodiments propose that tensor data follows certain naming rules. This naming rule can be flexibly set according to actual conditions and is not limited to the following public embodiments. In one possible implementation method, the naming rules followed by tensor data can be: it can only be composed of letters, numbers, and underscores; the first character must be an English letter; it cannot contain punctuation marks and type specifiers.

[0058] Since deep learning computing models usually process data of fixed size, taking the image classification model AlexNet as an example, the input and output data shapes of each processing layer are fixed, while the data values ​​change frequently with the input. Therefore, the attributes of the data and the values ​​of the data have completely different update frequencies. From the perspectives of data structure reuse and programming flexibility, the attributes of the data and the values ​​of the data should be decoupled. Therefore, in one possible implementation, the tensor data in the embodiment of the present disclosure is only used to describe the attributes of the data, and the value of the data can be described by a pointer to the memory area. The neural calculation tensor is fully mapped through the combination of tensor data and pointers.

[0059] As mentioned in the above disclosed embodiments, tensor data can contain multiple attributes, and the specific attributes can be flexibly set and selected based on actual conditions. In one possible implementation, tensor data can include a shape attribute (shape), a logical data type attribute (dtype), a physical data type attribute (pdtype), and a physical layout attribute (layout).

[0060] By setting four attributes for tensor data, namely shape attributes, logical data type attributes, physical data type attributes, and physical layout attributes, various data in deep learning algorithms can be sufficiently and well described, so that subsequent compilation methods can better adapt to various data situations in deep learning algorithms and improve the versatility of compilation methods.

[0061] During the application process, the above attributes can be further classified based on the usage of tensor data. The specific classification method can also be set according to the actual situation. Figure 3 A diagram illustrating the correspondence between the attributes, classifications, and meanings of tensor data according to an embodiment of the present disclosure is shown. As shown in the figure, in one example, the four attributes described above can be divided into two categories: visible attributes and invisible attributes. Shape attributes and logical data type attributes can be classified as visible attributes. During actual use, visible attributes can be set through the tensor assignment interface. Physical data type attributes and physical layout attributes can be classified as invisible attributes. They can be maintained, modified, and used within the programming library, thereby shielding hardware details from the outside world and reducing programming complexity.

[0062] pass Figure 3 It can be seen that the physical data type attribute can be used to indicate the precision of data stored in the hardware device memory, while the logical data type attribute can be used to indicate the precision of data stored in the host memory. Therefore, the precision represented by the physical data type attribute and the logical data type attribute can be the same or different. In one possible implementation, the physical data type attribute and the logical data type attribute can be different. In this case, the compilation process can be set to implement the automatic precision optimization function, that is, during the compilation process, the data type with the fastest running speed can be automatically selected for calculation, and this process can be transparent to the user. The specific implementation process of the automatic precision optimization function can be determined according to actual conditions, and will be specifically explained in the subsequent public embodiments.

[0063] The above disclosed embodiments also propose that tensor data can describe multiple data types. The specific data types that can be described can be flexibly determined based on actual conditions. In one possible implementation, tensor data can describe data types such as low bit width and quantization. In order to enable tensor data to support low bit width and quantization, the disclosed embodiments design different data types (including logical data types and physical data types) for tensor data, including:

[0064] Double-precision floating point: double; single-precision floating point: float; half-precision floating point: half; fixed point: fxm.b (m represents the number of integer bits, b represents the total number of bits); quantization: Qts.b (s represents the scaling factor scale of the tensor, b represents the bias of the tensor); integer: intx; unsigned integer: uintx.

[0065] Deep learning algorithms can support per-channel quantization of images, with each channel having a different scale and bias. Although per-channel quantization cannot be described using Qts.b, it can be implemented using the scale and add operations provided by NCLAPI, thus maintaining NCLAPI's comprehensive expressive capabilities for quantization. Furthermore, considering the potential emergence of other data types in the future, NCLAPI can also support expansion of the data types used to describe tensor data.

[0066] As mentioned in the above disclosed embodiments, nclTensor is used to describe the input data, output data, and model parameters of the deep learning processing layer. Since tensor data corresponds to nclTensor, and the most common deep learning processing layers are convolution, pooling, and RNN, their input data, output data, and model parameters are all high-dimensional data, Figure 4A diagram showing the correspondence between tensor data and shape symbols according to an embodiment of the present disclosure is shown. As shown in the figure, in a possible implementation, the shape of tensor data operated in deep learning can be agreed upon based on the correspondence shown in the figure.

[0067] It has been proposed in the above-mentioned public embodiments that operation data can be created or called according to user instructions received by the deep learning programming library interface, and tensor data, as a possible implementation method of operation data, can be created or called. Tensor data must be created before use, and the specific creation and calling processes can be flexibly set according to actual conditions. In one possible implementation method, the calling process can be to assign values ​​to tensor data. In one possible implementation method, the creation process can be to initialize its visible attributes when creating tensor data. In one possible implementation method, since deep learning frameworks such as Tensor decouple data object creation and attribute setting, in order to avoid being integrated into the deep learning framework without destroying the code structure of the deep learning framework, the creation process can be to first create an uninitialized tensor data, and then call the tensor assignment interface (nclSet-TensorAttr) to assign attributes.

[0068] In one possible implementation, an operation instruction in NCLAPI is an abstract representation of a transformation. It can be used to represent deep learning processing layers or general computations. In one example, an operation instruction can be used to represent deep learning processing layers such as convolution, pooling, and fully connected layers. In the disclosed embodiments, the operations performed by the operation instructions are collectively referred to as plastic operations.

[0069] In one possible implementation, an operation instruction may consist of three parts, namely input parameters (inputparams), output parameters (output params), and operation type (OpType), wherein the input parameters correspond to the set of input tensors to be transformed, that is, they may be nclTensor and pointers corresponding to all input data. The output parameters correspond to the set of output tensors to be transformed, that is, they may be nclTensor and pointers corresponding to all output data. The implementation of input parameters and output parameters is not limited. In one example, it may be specified that an operation instruction allows zero or more (tensor data, pointer) as input parameters and one or more (tensor data, pointer) as output parameters.

[0070] Operation types are used to specify the data transformations that an operation performs. Users can specify different operation types when creating an operation. These types can express three types of data transformations: value transformation, attribute transformation, and null transformation. Therefore, operations can describe not only deep learning processing layers, but also general computations such as data segmentation, data concatenation, and resizing.

[0071] For the deep learning programming library interface, it can provide a series of pre-defined self-operation instructions. In the embodiment of the present disclosure, for the NCLAPI interface, these operation instructions can be called build-in operation instructions (build-in operators). Figure 5 A diagram of built-in operation instructions supported by NCLAPI according to an embodiment of the present disclosure is shown. As can be seen from the diagram, the operation instructions that support in-place algorithms are the built-in operation instructions supported by NCLAPI.

[0072] The nature of the operation instructions can be flexibly set according to actual conditions. In one possible implementation, in order to make the behavior of the program easier to analyze and predict, except for the build-in operation instructions, the remaining operation instructions can be unidirectional, non-intersecting, and idempotent. Among them, unidirectionality means that the operation instruction does not change the input parameters (including tensor data and data pointed to by pointers), non-intersecting means that the input parameters and output parameters of the operation instruction cannot have the same name, and idempotence means that the result of the operation instruction call depends only on the input parameters and is not affected by the number of calls.

[0073] As proposed in the aforementioned disclosed embodiments, operation data can be created or called based on user instructions received by the deep learning programming library interface. Operation instructions, as a possible implementation of operation data, can therefore be created or called, and the specific creation and calling processes can be flexibly set based on actual circumstances. In one possible implementation, an operation instruction needs to be created before being called. An operation instruction call refers to mapping an operation instruction to a deep learning processor for execution. By default, operation instructions support runtime variable operation parameters. Specifically, operation instruction creation only needs to be performed once, while operation instruction calls can be performed repeatedly, and different input and output parameters can be specified each time an operation instruction is called.

[0074] In one possible implementation, in order to optimize program performance, the NCLAPI interface can support two important functions when executing operation instruction calls, namely asynchronous execution and automatic precision selection.

[0075] In one example, asynchronous execution means that the host returns immediately after calling an instruction call function. This allows the CPU to perform other operations while the deep learning processor is performing calculations, thereby improving overall system utilization and program performance. To ensure that asynchronous calls are completed, NCLAPI provides a device synchronization interface, nclSyncDevice, which blocks CPU execution until the device completes the operation.

[0076] Figure 6 A schematic diagram of an implementation method for automatic precision optimization according to an embodiment of the present disclosure is shown. As shown in the figure, in one example, automatic precision optimization means that before an operation instruction is executed on a device, the programming library will automatically select the data type with the shortest execution time, convert the original data into the optimal format, and then perform the operation. Automatic precision optimization will collaboratively consider the time overhead of both data format conversion and operation instruction execution to ensure that the overall execution time of the operation is the shortest. In addition, in order to meet the unidirectionality and idempotence of the operation, the programming library will apply for temporary space to complete the data format conversion to ensure that the original input data is not overwritten.

[0077] In one possible implementation, operations can also support operation connections. Operation connections involve using the output parameters of one operation A as input parameters for another operation B. After A completes its computation, B processes A's output data. The necessary and sufficient condition for two operations A and B to be connected is that they each have at least one output tensor T1 and one input tensor T2, with the properties of T1 and T2 being identical. Operation connections are directional, from the operation that provides data to the operation that consumes the data.

[0078] In one possible implementation, a deep learning computing model can be represented as a function combination, where a function combination is a function sequence in which only one-way function connections exist (in the function combination, the direction of the function connection can only be from left to right). Since operation instructions can be obtained by function mapping, a deep learning computing model can be represented as an operation instruction sequence in which only one-way operation connections exist (in the operation instruction sequence, the direction of the operation instruction connection can only be from left to right). In the embodiment of the present disclosure, such an operation instruction sequence is referred to as a one-way operation sequence. A directed graph can be converted into a one-way operation sequence according to a certain algorithm. In addition, in order to avoid the in-place algorithm destroying the unidirectionality of the operation, the tensor aliasing technology can be used to eliminate the in-place operation.

[0079] In one example, the algorithm for converting a directed graph into a unidirectional operation instruction sequence may be: first, convert the directed graph g into a directed acyclic graph g'. Then, topological sorting is performed on the directed acyclic graph g' to obtain:

[0080] g"(V{vertex1,vertex2,...,vertexn},E)

[0081] Map the vertices in the graph g'' into a sequence of operation instructions in order; map the edges in g'' into tensors, add the directed edges flowing out of the vertex to the output parameters of the vertex, and add the edges flowing into the vertex to the input parameters of the vertex.

[0082] In an example, the tensor aliasing technique can be implemented as follows:

[0083]

[0084] Figure 7 A schematic diagram of a computing model according to an embodiment of the present disclosure is shown. As shown in the figure, the computing model can be represented as two unidirectional operation instruction sequences: (conv, pool, bn, relu, add) and (conv, pool, relu, bn, add). By calling the operation instructions in the order in which the operation instructions appear in the unidirectional operation instruction sequence, the computing model execution can be completed once. It should be noted that since the relu and add operations have an operation connection from right to left, the above computing model cannot be represented as an operation instruction sequence of (conv, pool, add, bn, relu).

[0085] In one possible implementation, since deep learning algorithms usually process data of fixed size, performance optimization can be achieved by fixing the parameters of the operation instructions. The operation instructions support all parameters to be variable at runtime by default. In order to optimize the operation performance by using fixed parameters, the embodiment of the present disclosure designs parameter binding (parameterbinding) and operator specialization (operator specialization) functions for the operation instructions. Parameter binding refers to fixing some or all input parameters of an operation instruction; operation specialization refers to converting an operation instruction that has been parameter bound into a new operation instruction, and the new operation instruction can be called a specialized operation instruction. The specialized operation instruction still meets the definition and properties of the operation instruction and supports all functions of the operation instruction (supporting specialization, fusion, etc.). The classification method of the specialized instruction can be flexibly set according to the actual situation. In one possible implementation, the operation specialized operation instruction can be divided according to the number of parameter bindings. In one example, the specialized operation instruction can include a fully specialized operation instruction, a partially specialized operation instruction, and a pseudo-specialized operation instruction; therefore, in one example, according to the user instruction, triggering the deep learning programming library interface to create the specialized operation instruction can include:

[0086] According to the user's instructions, trigger the deep learning programming library interface to bind all input parameters of the operation instruction and convert it to obtain a fully specialized operation instruction; and / or,

[0087] According to the user instruction, trigger the deep learning programming library interface to bind N input parameters to the operation instruction and then convert it to obtain a partially specialized operation instruction, where N is a positive integer less than the number of input parameters of the operation instruction; and / or,

[0088] According to the user's instructions, the deep learning programming library interface is triggered to directly convert the operation instructions without binding the input parameters to obtain pseudo-specialized operation instructions.

[0089] As can be seen from the above disclosed embodiments, in this disclosure, specialized operation instructions are divided into three categories: fully specialized operation instructions that bind all input parameters of an operation, partially specialized operation instructions that bind some input parameters, and pseudo-specialized operation instructions that do not bind any input parameters. Bound parameters can be removed from the input parameters of the operation instruction. Users do not need to specify the bound parameters when calling an operation instruction with bound parameters. Therefore, fully specialized operation instructions do not need to specify any input parameters.

[0090] Parameter binding and specializing operators can reduce the runtime of operators by optimizing partial evaluation during compilation. While the specific implementation of parameter binding and specializing operators is not limited, one possible implementation involves implementing specializing operators through the nclSpecializeOperator interface, which performs just-in-time compilation optimization and returns specialized operators with faster execution time on the hardware. In one example, a convolution operation might have completely predetermined input, output, and weight attributes. Parameter binding and specializing operators can be used to generate a faster convolution operation. These techniques are widely applicable in real-world computing models. For example, deep learning models typically process fixed-size data. Therefore, tensor shapes can be parameterized and then specialized to generate specialized operators, thereby optimizing program performance. In one example, for an inference application scenario, the weights are pre-trained constants, so the weight data of the operation can be bound to parameters, and then the operation can be specialized to obtain specialized operation instructions, thereby optimizing program performance.

[0091] In one possible implementation, deep learning programming libraries usually only support processing layers that are used frequently and take a long time (such as convolution, full connection, RNN, pooling, activation), which results in the programming library being unable to support end-to-end execution well. In order to solve the above problems, the embodiment of the present disclosure designs an operation customization function for the operation instruction to make it customizable. Operation customization refers to writing an operation in a domain-specific programming language and then inserting it into the programming library in the form of binary code. In the embodiment of the present disclosure, this operation is called a customized operation instruction (customized operator). Customized operation instructions still meet the definition and properties of operation instructions, and support all functions of operation instructions (support specialization, fusion, etc.).

[0092] Since custom operation instructions are inserted into the programming library in the form of binary code, it means that the custom operation instructions need to be compiled in advance to generate binary code. The generation process of the binary code corresponding to the custom operation instructions can be flexibly determined according to the actual situation of NCLAPI and the deep learning programming library. Figure 8 A schematic diagram illustrating a generation principle of a customized operation instruction according to an embodiment of the present disclosure is shown. As shown in the figure, in one possible implementation, the generation process of the binary code corresponding to the customized operation instruction may include:

[0093] According to the interface and data structure definition of the operation instruction, the user instruction corresponding to the operation instruction is encapsulated to obtain the encapsulated user instruction;

[0094] Compile the encapsulated user instructions to obtain the compilation results;

[0095] Insert the compiled result into the static operation pool in a dynamic or static linking manner to obtain the binary code corresponding to the customized operation instruction.

[0096] Therefore, in one possible implementation, triggering the deep learning programming library interface to create or call a custom operation instruction according to a user instruction may also include:

[0097] Compile the customized operation instructions to obtain compilation results;

[0098] Insert the compiled result into the static operation pool in a dynamic or static linking manner to obtain the binary code corresponding to the customized operation instruction.

[0099] The static operation pool in the above-mentioned disclosed embodiments is a storage area in the deep learning programming library, and its specific implementation method will be specifically described in subsequent disclosed embodiments.

[0100] By compiling the encapsulated user instructions to obtain the compilation results, and inserting the compilation results into the static operation pool of the deep learning programming library in a dynamic or static linking manner, the binary code corresponding to the custom operation instructions is obtained. Through this process, pre-compiled custom operation instructions can be generated and the compilation results of the custom operation instructions can be saved. The operation instructions that are repeated multiple times and are not provided in the library can be converted into a packaged custom operation instruction. Therefore, when implementing a certain deep learning algorithm, the operations to be performed can be directly implemented by calling the custom operation instructions, avoiding repeated and useless instruction editing. Moreover, since the compilation results of the custom operation instructions have been saved in the deep learning programming library, the binary code corresponding to the custom operation instructions can be directly called during compilation without repeated compilation, which effectively improves the compilation efficiency and shortens the compilation time.

[0101] In one example, based on the binary code generation process corresponding to the customized operation instruction proposed in the above-mentioned disclosed embodiment, the specific process of implementing the operation customization can be: implementing the customized transformation in a programming language to obtain the code to be inserted (insert code); encapsulating the insert code according to the interface and data structure definition of the operation instruction, and completing data format conversion and other tasks; compiling the insert code and inserting it into the deep learning programming library in a dynamic or static linking manner to complete the operation customization; and using the customized operation instruction normally like the operation instruction. It should be noted that the name of the customized operation instruction is specified by the user and cannot conflict with the operation name of the built-in operation.

[0102] In one possible implementation, the operator instructions proposed in the embodiments of the present disclosure may also support operator fusion. Operator fusion refers to combining multiple plastic operations into a new plastic operation in the order of calling. This new operation may be referred to as a fusion operator. Fusion operator instructions still meet the definition and properties of operator instructions and support all the functions of operator instructions (supporting specialization, fusion, etc.). In one example, operator fusion is formally represented as follows:

[0103] op fused =Fuse(op1,op2,...,op n )

[0104] In the embodiments of the present disclosure, operation fusion satisfies weak transformation equivalence: the calculation results of the fused operation instructions and the calculation results of the original operation instruction sequence can be regarded as equal within the allowable error range, which is formally expressed as error < epsilon, where epsilon is determined by the sensitivity of the application process to precision. In one example, the fused operation instructions can participate in operation fusion again, which is called high-order fusion, and the output obtained by high-order fusion is still a malleable operation, which is expressed as follows:

[0105] op fused2 = Fuse(op1, op2,..., op fused ,..., op n )

[0106] Operation fusion can bring two benefits: optimizing performance and simplifying programming. In terms of optimizing performance, the programming library can perform compilation optimization at the computational graph level inside the fused operation (for example, reducing the overall amount of computation and memory access through optimization techniques such as linear transformation and constant folding), thereby reducing the execution time of the operation on the device; in terms of simplifying programming, a single fused operation instruction can be used to represent common functional blocks (such as the residual block in ResNet) or even the entire computational model in deep learning algorithms. These highly abstract components can be reused repeatedly, thereby improving the efficiency of program development.

[0107] In one possible implementation, operation fusion needs to meet certain conditions. In one example, this condition can be that the operation instructions to be fused can be represented as a continuous subsequence in a unidirectional operation instruction sequence. As Figure 7 shown in the schematic diagram of the computational model, it has been proposed in the above-mentioned embodiments of the present disclosure that, in one example, this computational model can be represented as the following two unidirectional operation instruction sequences, namely seq1: (conv, pool, bn, relu, add) and seq2: (conv, pool, relu, bn, add). Any subsequence in these two operation instruction sequences can be fused for operation, Figure 9 shown in the schematic diagram of the operation fusion result according to an embodiment of the present disclosure. As shown in the figure, in one example, the three operation instructions conv, pool, and bn in seq1 can be fused, and at this time, the computational model (fusion, relu, add) can be obtained; Figure 10A schematic diagram of the operation fusion result according to an embodiment of the present disclosure is shown. As shown in the figure, in one example, the two operation instructions bn and add in seq2 can be fused, and the computational model (conv, pool, relu, fusion) can be obtained. In one example, the three operation instructions pool, relu, and add cannot be fused because they are neither continuous subsequences of seq1 nor seq2. If they are forcibly fused into the operation fusion, then no matter where fusion is inserted in the sequence (conv, bn), there will be a circular data dependency, that is, bn depends on the output result of fusion, and fusion also depends on the output result of bn. Therefore, operation fusion cannot be performed at this time.

[0108] According to the principle of the above-mentioned operation fusion, in a possible implementation, triggering the creation of a fusion operation instruction according to a user instruction may include:

[0109] Create the name of the fused operation instruction.

[0110] Determine the fusion operation sub-instruction according to the operation instruction to be fused.

[0111] According to the calling sequence of the operation instructions to be fused, the operation connection relationship between the fused operation sub-instructions is determined.

[0112] According to the operation connection relationship, the operation sub-instructions are connected and fused to obtain the connection result.

[0113] According to the user instruction corresponding to the fusion operation instruction, the input parameters and output parameters of the fusion operation instruction are set.

[0114] The name, connection result, input parameters and output parameters are packaged to obtain a fusion operation instruction.

[0115] Through the above-mentioned creation process of fused operation instructions, multiple operation instructions can be conveniently fused into one fused operation instruction, thereby effectively optimizing compilation performance and reducing the execution time of operation instructions on the device. At the same time, a single fused operation instruction can also be used to represent commonly used functional blocks or even entire computing models in deep learning algorithms. These abstract instructions can be used repeatedly, thereby improving program development efficiency.

[0116] The specific programming interface of the operation fusion can be flexibly set according to the actual situation. Figure 11 A schematic diagram of a programming interface related to operation fusion according to an embodiment of the present disclosure is shown. In one example, based on the programming interface shown in the figure and combined with the creation process of the above-mentioned fusion operation instruction, a process of creating a fusion operation can be obtained. Figure 12A schematic diagram of a process for creating a fusion operation according to an embodiment of the present disclosure is shown. As shown in the figure, in one example, the steps of creating a fusion operation may include:

[0117] Create a fusion operation and specify the operation type name of the fusion operation (which must not conflict with the build-in operation type).

[0118] Call the nclAddFusionOperator / nclSetFusionOperators interface to specify all sub-operators to be fused.

[0119] Call the nclLinkOperator interface to specify the operation connection relationship between sub-operations.

[0120] Call nclAddFusionInput and nclAddFusionOutput to set the input and output parameters of the fusion operation.

[0121] Call the nclFuseOperator interface to complete operation fusion.

[0122] The purpose of functionally connecting sub-operations in the above steps is to build a computational graph. The nclFuseOperator interface will perform timely compilation and optimization on the computational graph, thereby accelerating the execution of operations.

[0123] It can be seen from the above disclosed embodiments that the operation instructions proposed in the disclosed embodiments may include operation fusion instructions, and may also include other types of operation instructions such as build-in operation instructions and specialized operation instructions. The programming models implemented for different types of operation instructions are also different. In the disclosed embodiments, NCLAPI adopts a hybrid programming model, that is, it supports both imperative programming and declarative programming. In one possible implementation, a hybrid programming model can be designed based on operation fusion, that is, the programming model that does not use fusion operations is an imperative programming model, and the programming model that uses fusion operations is a declarative programming model. The two programming models can be used in combination.

[0124] The implementation of the programming model can be flexibly configured based on actual circumstances. In one possible implementation, the programming model can be designed based on three factors: data flow, execution flow, and control flow. Regarding data flow, to facilitate data transfer between the host and device, the disclosed embodiments design the data copy interface nclMemcpy for NCLAPI. Regarding control flow, to control device execution and synchronize between the host and device, the disclosed embodiments design the operation call interface nclInvokeOperator and the device synchronization interface nclSyncDevice for NCLAPI. Regarding execution flow, the disclosed embodiments categorize the execution of the computation model into three types: layer-by-layer invocation (calling all operations in the computation model one by one); fusion invocation (fusing operations on the entire computation model and then calling the fusion operation); and segmented fusion invocation (fusing operations on the computation model segment by segment and then calling the fusion operation segment by segment). These three execution methods distinguish the NCLAPI programming model: the layer-by-layer invocation execution method corresponds to the imperative programming model; the fusion invocation execution method corresponds to the declarative programming model; and the segmented fusion invocation execution method corresponds to the hybrid programming model. The above-mentioned disclosed embodiments have proposed that, in a possible implementation method, operation data is created or called according to user instructions received by the deep learning programming library interface. In one example, based on the execution methods of the three computing models of the above-mentioned disclosed embodiments, it can be known that triggering the call of operation instructions in the deep learning programming library interface according to user instructions can include: calling all corresponding operation instructions one by one according to user instructions; or, according to user instructions, fusing all corresponding operation instructions to obtain fused operation instructions, and calling the fused operation instructions; or, according to user instructions, segmenting all corresponding operation instructions to obtain segmented results, fusing each segmented result separately to obtain the corresponding segmented fused operation instructions, and calling the segmented fused operation instructions in sequence.

[0125] In one possible implementation, these three calling methods can be used to distinguish NCLAPI programming models. The execution method of layer-by-layer calling can correspond to the imperative programming model; the execution method of fusion calling can correspond to the declarative programming model; and the execution method of segmented fusion calling can correspond to hybrid programming.

[0126] Figure 13A data flow diagram of a three-layer computing model according to an embodiment of the present disclosure is shown. As shown in the figure, in an example, based on the programming model in the figure, the process of calling operation instructions through different calling methods can be: layer-by-layer calling: calling the nclInvokeOperator interface three times to perform conv, pool, and fc operations respectively; fusion calling: first fusion the three operations conv, pool, and fc into a single operation, and then calling the nclInvokeOperator interface once to perform the fusion operation; segmented fusion calling: fusion of the conv and pool operations, and then calling the nclInvokeOperator interface twice to perform the fusion operation and the fc operation respectively.

[0127] Figure 14 A block diagram of an implementation of a hybrid programming model according to an embodiment of the present disclosure is shown. As shown in the figure, in one example, the hybrid programming model is composed of an imperative programming model and a declarative programming model. In one example, the complete programming process of the imperative programming model can be: initialization: initialize the device and the operating environment; create an operation: create a single operation, optionally bind parameters and specialize the operation; call an operation: prepare operation parameters (including creating a Tensor and assigning a device address), copy the host-side input data to the device memory, call the operation, synchronize the device, and read the output result from the device memory; release operation resources: destroy resources that are no longer used in the first two steps, including tensors, operations, memory, etc.; repeat the creation, calling, and release operations until all operations in the computing model are executed; exit: shut down the device and destroy the operating environment. In an example, the complete programming process of the declarative programming model can be: initialization: initialize the device and operating environment; create sub-operations: create all sub-operations that need to participate in the fusion, and optionally bind parameters to the sub-operations; create fusion operations: create a fusion operation, add sub-operations to be fused, specify the operation connection relationship between sub-operations, set the input and output parameters of the fusion operation, perform operation fusion, and optionally perform operation specialization; call the fusion operation: prepare operation parameters (including creating Tensor and allocating device addresses), copy the host side input data to the device memory, call the operation, synchronize the device, and read the output results from the device memory; release operation resources: release sub-operation resources, release fusion operation resources; exit: shut down the device and destroy the operating environment.

[0128] The aforementioned disclosed embodiments have proposed that specialized and fused operation instructions can optimize compilation performance. However, in one possible implementation, both types of operation instructions must be compiled via just-in-time compilation, which can increase host-side runtime and the overall program runtime. Therefore, in one possible implementation, an offline mode can be used to further optimize compilation time and improve efficiency.

[0129] Since deep learning algorithms are highly reusable, optimized computing models can be used repeatedly for reasoning. Therefore, for the same computing model, specialized operation instructions and fused operation instructions usually only need to be created once, and the operation instruction calls can be repeated. The more times the operation is called, the higher the benefits of specialized operations and operation fusion. Therefore, in one possible implementation, the embodiment of the present disclosure proposes an offline mode (offline mode) for NCLAPI to eliminate the secondary compilation overhead of operation specialization and operation fusion on the host side. The implementation method of the offline mode can be flexibly set according to actual conditions. In one example, the operation instruction can be optimized in advance using operation specialization or operation fusion in a separate program and used directly in another program. In one possible implementation, the operation instruction optimized in advance can be called an offline operation (offline operator). Corresponding to the offline mode is the online mode (online mode), Figure 15 A schematic diagram illustrating the difference between offline mode and online mode according to an embodiment of the present disclosure is shown. As shown in the figure, in one example, the online mode may first perform operation specialization or operation fusion in the same program, and then call the specialization operation instruction or the fusion operation instruction.

[0130] It can be seen from the above disclosed embodiments that in a possible implementation, offline operation instructions and the binary codes compiled therefrom need to be saved in advance to facilitate subsequent direct use. Therefore, in the disclosed embodiments, an offline cache is also designed for NCLAPI. The implementation of the offline cache can be flexibly determined according to the actual situation. In a possible implementation, the offline cache includes an offline file and an index table; wherein the offline file is used to save the pre-compiled results of the offline operation instructions; and the index table is used to indicate the position of the pre-compiled results of the offline operation instructions in the offline file. The specific implementation of the offline file and the index table can also be flexibly selected according to the actual situation. In one example, the offline operation is saved in the offline file, and the position in the offline file is indexed by the index table. The index table is implemented using a key-value pair (Key, Value), where the Key is the name of the offline operation and the Value is a pointer to the binary code corresponding to the offline operation in the offline file. The specific interface of the offline operation instruction can be set according to the actual situation, Figure 16A schematic diagram of an offline interface according to an embodiment of the present disclosure is shown. As shown in the figure, in one example, the interface implementation method of the offline operation instruction can be: the nclSaveOperator interface saves the specified operation instruction to the offline cache, and uses the string specified by op_type as the name and index key of the operation. The usage method of the offline operation instruction is exactly the same as the build-in operation instruction, except that the operation type is different. Therefore, users can still use the nclCreateOperator interface to create offline operation instructions. NCLAPI can prioritize matching the build-in operation instruction based on the given operation name. If there is no match, it will then search the offline cache for the corresponding offline operation instruction.

[0131] As can be seen from the aforementioned disclosed embodiments, NCLAPI can transmit operational data and has good adaptability to deep learning algorithms. Therefore, in one possible implementation, NCLAPI can be integrated into a deep learning framework. In one possible implementation, a class extension can be implemented for the deep learning framework, encapsulating tensor data and operational instructions within the deep learning framework's data, thereby integrating the deep learning framework with the deep learning programming library interface. In actual applications, the specific implementation of the deep learning framework may vary, and the implementation method of integrating NCLAPI into the deep learning framework may also vary accordingly.

[0132] In one example, the deep learning framework can be Caffe, because Caffe contains three key data structures: Blob, Layer, and Net. Blob is mainly used to store data, complete data copying between the host and the device, and provide a data access interface. Layer is used to represent operations (such as convolution, pooling, etc.), which takes Blob as input and output. Caffe has designed an inheritance system for Layer, and different operations can be implemented by writing Layer subclasses. Layer has three key methods: Setup, Forward, and Backward, which are responsible for operation initialization, forward calculation, and reverse calculation, respectively. In order to support different devices, the same Layer subclass can contain multiple Forward and Backward methods.

[0133] Net stores all blobs and layers, and uses a directed acyclic graph composed of layers to express the complete computing model. Net has three key methods: Init, Forward, and Backward. The Init method converts the computing model defined by NetParameter (converted from prototxt) into blobs and layers, and calls the Setup method to initialize all layers. The Forward method performs forward reasoning on the entire computing model, and the Backward method performs reverse training on the computing model. Caffe uses prototxt to model deep learning computing models. Users describe processing layers, data, and processing layer connection relationships according to the syntax of prototxt. Caffe receives the prototxt file, converts it into blobs, layers, and nets, and then executes it.

[0134] According to the composition of Caffe, in one example, the embodiment of the present disclosure integrates NCLAPI into Caffe in the following manner:

[0135] Expand the Blob class and encapsulate the nclTensor data structure and related interfaces (such as nclCreateTensor, nclSetTensorAttr, and nclMemcpy) into Blob;

[0136] Expand the Layer class and encapsulate the NCLAPI nclOperator data structure and related interfaces into Layer. Specifically, encapsulate interfaces such as nclCreateOperator, nclSpecializeOperator, and nclBlindOutputTensor into Layer's Setup method, and encapsulate nclInvokeOperator into Layer's Forward and Backward methods. To support new devices and operators without destroying existing Layers, this embodiment adopts the implementation method of adding a new subclass to Layer.

[0137] The Net class is expanded and the operation fusion interface of NCLAPI is encapsulated into Net. Since all layers can be obtained in Net, Net is most suitable as a carrier for operation fusion. The embodiment of the present disclosure adds an operation fusion module to Net, which can perform segmented or complete fusion of computing models.

[0138] In one example, the deep learning framework may be TensorFlow. Figure 17The following diagram illustrates the architecture of TensorFlow according to an embodiment of the present disclosure. As shown in the figure, in one example, TensorFlow was designed with architectural scalability in mind. It provides mechanisms for adding operators and registering devices, and provides detailed official documentation. Therefore, TensorFlow itself is relatively easy to integrate with third-party deep learning programming libraries and deep learning processors. Because TensorFlow's distributed master is responsible for partitioning and allocating tasks for computational subgraphs, in one example, NCLAPI operations can be integrated into the distributed master module to perform operational fusion on the subgraphs.

[0139] In one example, the disclosed embodiment integrates NCLAPI into TensorFlow in the following manner:

[0140] Expand the Tensor class and encapsulate the nclTensor data structure and related interfaces (such as nclCreateTensor, nclSetTensorAttr, and nclMemcpy) into Tensor;

[0141] Register new devices (deep learning processors) and NCLAPI operators according to the official TensorFlow documentation;

[0142] Integrate the operation fusion function into the distributed master module.

[0143] It can be seen from the above-mentioned disclosed embodiments that NCLAPI uses tensors to represent multi-dimensional data such as scalars, vectors, and matrices, and uses operation instructions to represent deep learning processing layers. Operation instructions support operation fusion, operation customization, operation specialization, variable operation parameters at runtime, offline optimization, and hybrid programming models (imperative + declarative), thereby effectively solving the problems of performance optimization and programming flexibility. In an example, a specific implementation of NCLAPI can be deployed on the DaDianNao deep learning processor platform, and it has also been integrated into mainstream deep learning frameworks such as Caffe and TensorFlow. Practice has proved that NCLAPI can run mainstream deep learning algorithms including image classification, target detection, and natural language processing, and it has strong versatility and flexibility. In addition, NCLAPI simulates neural calculus in the design of data structures and interfaces, thereby proving that neural calculus can serve as a theoretical basis for guiding the design of deep learning programming libraries.

[0144] Figure 18A schematic diagram comparing NCLAPI and mainstream deep learning programming library interfaces according to an embodiment of the present disclosure is shown. As can be seen from the figure, compared with mainstream deep learning programming library interfaces, NCLAPI can support hybrid programming models, meeting both performance optimization and programming flexibility requirements; it can support operation customization, possesses strong operational scalability, and can better support end-to-end execution performance optimization; it can support operation fusion, operation specialization, and offline mode, optimizing program performance from multiple perspectives. Furthermore, TensorFlow integrated with NCLAPI can support a large number of deep learning computational models end-to-end (without using any CPU operations).

[0145] According to the above disclosed embodiments, the operation data generation method proposed in the embodiment of the present disclosure can also be compiled based on the generated operation data. Therefore, in one possible implementation, the operation data generation method proposed in the embodiment of the present disclosure can also include:

[0146] Step S13: The deep learning programming library architecture reads the operation instructions transmitted by the deep learning programming library interface.

[0147] In step S14, the deep learning programming library architecture determines the instruction type of the operation instruction, and performs a compilation operation corresponding to the instruction type according to the determination result to obtain a binary code of the deep learning algorithm.

[0148] In the above disclosed embodiments, the binary code is a hardware instruction for guiding the hardware device to execute the deep learning algorithm. The specific hardware devices to be guided and the specific content of the hardware instructions are not limited in the disclosed embodiments and can be flexibly selected according to actual conditions.

[0149] By receiving the operation data transmitted by the deep learning programming library interface, and executing the compilation operation corresponding to the instruction type according to the instruction type of the operation instruction in the operation data, the binary code of the deep learning algorithm is obtained. According to the compilation method, device and related products of the deep learning algorithm of the various embodiments of the present disclosure, the compilation process can be adaptively changed according to the type of the operation instruction, thereby greatly improving the flexibility and efficiency of the compilation, effectively improving the performance optimization effect of the deep learning algorithm for the corresponding hardware platform, and then improving the processing performance of the deep learning processor.

[0150] Therefore, based on the compilation method further proposed in the above-mentioned disclosed embodiment, the disclosed embodiment also proposes a deep learning programming library architecture adapted to the compilation method - the neural calculus library architecture (NCLA). Figure 19The overall architecture diagram of NCLA according to an embodiment of the present disclosure is shown. As shown in the figure, in one possible implementation, NCLA can be composed of a just-in-time compilation system (NCLCS), a static operator pool (NCLSOPP), and a runtime system (NCLRT). In addition, NCLA can be integrated with NCLAPI and compiled according to the operation data transmitted by NCLAPI. The just-in-time compilation system can perform calculations and data co-compilation optimization on any operation instruction at runtime to generate efficient binary code; the static operator pool can be used to store the optimized binary code, thereby eliminating the secondary compilation overhead; the runtime system can provide basic functions such as device management, memory management, operation execution, and device synchronization, so that the operation instructions can be deployed end-to-end on the deep learning processor for execution.

[0151] It has been proposed in the above-mentioned disclosed embodiments that program optimization in the field of deep learning is extremely sensitive to algorithm and hardware changes. Therefore, in the implementation process, it is difficult to meet the performance requirements of different algorithms and different hardware platforms through ahead-of-time compilation optimization (AOT). Therefore, in one possible implementation, the disclosed embodiment can use just-in-time compilation optimization (JIT) to design NCLA. Just-in-time compilation optimization can dynamically adjust the optimization strategy for different algorithms and different hardware platforms at runtime to achieve universal performance optimization. However, just-in-time compilation introduces additional runtime overhead. A common means to alleviate this problem is to introduce a just-in-time compilation cache. Deep learning algorithms are highly reusable, so the cache can play a huge role. In one possible implementation, the disclosed embodiment uses a static operation pool as a cache for the just-in-time compilation system. Due to the extremely high coupling between operations and data, optimizing operations or data alone cannot fully utilize the performance of the deep learning processor. Therefore, in one possible implementation, a compilation framework for collaborative optimization of operations and data can be used to design just-in-time compilation.

[0152] Based on the above principles, the operation instructions contained in the operation data transmitted by NCLAPI can be divided into different types. The specific division method can be flexibly selected according to the actual situation. In one possible implementation, the operation instructions can be divided into static operation instructions and dynamic operation instructions. Dynamic operation instructions can trigger NCLA to perform just-in-time compilation, while static operation instructions can trigger NCLA to perform search operations. Therefore, in one possible implementation, step S12 may include:

[0153] According to user instructions, trigger the deep learning programming library interface to create or call dynamic operation instructions; and / or,

[0154] According to user instructions, trigger the deep learning programming library interface to create or call static operation instructions.

[0155] What specific instructions the static operation instructions and dynamic operation instructions include can be determined based on the actual situation of the operation instructions transmitted by the deep programming interface, and is not limited to the following disclosed embodiments.

[0156] In one possible implementation, the static operation instruction may include one or more of a customized operation instruction, a build-in operation instruction, and an offline operation instruction. Therefore, in one possible implementation, triggering the deep learning programming library interface to create or call the static operation instruction according to a user instruction may include:

[0157] According to the user instruction, trigger the deep learning programming library interface to create or call a custom operation instruction, wherein the custom operation instruction is obtained by encapsulating the user instruction corresponding to the operation instruction according to the interface and data structure definition of the operation instruction; and / or,

[0158] According to user instructions, trigger the deep learning programming library interface to create or call build-in operation instructions, wherein the build-in operation instructions are the own operation instructions included in the deep learning programming library interface; and / or,

[0159] According to the user instruction, the deep learning programming library interface is triggered to create or call an offline operation instruction, wherein the offline operation instruction is a pre-compiled dynamic operation instruction, and the pre-compiled result is stored in the offline cache.

[0160] The specific implementation methods of customized operation instructions, build-in operation instructions and offline operation instructions have been explained in the above-mentioned disclosed embodiments and will not be repeated here.

[0161] In one possible implementation, the dynamic operation instruction may include a specialized operation instruction and / or a fusion operation instruction. Therefore, in one possible implementation, triggering the deep learning programming library interface to create or call a dynamic operation instruction based on a user instruction may include:

[0162] According to user instructions, trigger the deep learning programming library interface to create or call a specialized operation instruction, wherein the specialized operation instruction is obtained by binding input parameters to the operation instruction and then converting it; and / or,

[0163] According to user instructions, the deep learning programming library interface is triggered to create or call a fusion operation instruction, wherein the fusion operation instruction is obtained by combining multiple operation instructions according to the calling order.

[0164] The specific implementation process of the specialization operation instruction and the fusion operation instruction has been explained in the above-mentioned disclosed embodiments and will not be repeated here.

[0165] Therefore, in a possible implementation, step S14 may include:

[0166] Step S141: determine the instruction type of the operation instruction.

[0167] Step S142: When the instruction type is a static operation instruction, the corresponding binary code is searched in the static operation pool according to the name of the static operation instruction, and used as the binary code of the deep learning algorithm.

[0168] Based on the principles proposed in the above-mentioned public embodiments, it can be seen that by searching for the corresponding binary code in the static operation pool according to the name of the static operation instruction when the instruction type is a static operation instruction, as the binary code of the deep learning algorithm, the corresponding binary code can be directly searched for reusable operation instructions, avoiding multiple repeated compilations, eliminating the secondary compilation overhead, and improving compilation efficiency.

[0169] Furthermore, in a possible implementation, step S142 may include:

[0170] According to the name of the static operation instruction, the binary code corresponding to the name is searched in the static operation pool.

[0171] When the search result is successful, a binary code is returned as the binary code of the deep learning algorithm.

[0172] In a possible implementation, step S142 may further include:

[0173] When the search result is failure, the static operation instruction is used as a dynamic operation instruction and compiled in real time.

[0174] The above-mentioned disclosed embodiments have proposed that, in a possible implementation method, customized operation instructions can be written by the user using a deep learning field programming language, which is first compiled in advance to generate binary code, and then inserted into the static operation pool in a dynamic link or static link manner. The binary code of the offline operation instruction can be generated by the just-in-time compilation system, and the user inserts it into the static operation pool by calling the nclSaveOperator interface. The build-in operation instruction is an operation that comes with NCLAPI. Considering that program optimization in the field of deep learning is extremely sensitive to algorithms and hardware, in order to reduce development costs, the disclosed embodiment does not use manual optimization to implement the build-in operation instruction. Instead, the just-in-time compilation system is used in advance to pseudo-specialize the operation instruction (specialization without binding any input parameters) to generate the binary code corresponding to the build-in operation instruction, and then insert it into the static operation pool.

[0175] It can be seen from the above disclosed embodiments that the static operation pool can be used to store binary code corresponding to static operation instructions. Its specific implementation method can be flexibly set according to actual conditions and is not limited to the following disclosed embodiments. In one possible implementation method, the static operation pool may include a static operation source file, and the static operation source file includes a static code segment, a static data segment, a dynamic code segment, and a dynamic data segment; wherein the static code segment is used to store the binary code corresponding to the build-in operation instruction; the dynamic code segment is used to store the binary code corresponding to the custom operation instruction; the static data segment is used to store the tensor data corresponding to the build-in operation instruction; and the dynamic data segment is used to store the tensor data corresponding to the custom operation instruction.

[0176] It is also proposed in the above disclosed embodiments that the binary code corresponding to the offline operation instruction can be kept in the offline cache. Therefore, in a possible implementation, the static operation pool can also include the offline cache.

[0177] Based on the above disclosed embodiments, Figure 20A schematic diagram illustrating an implementation of a static operation pool according to an embodiment of the present disclosure is shown. As shown in the figure, in one example, four new segments can be added to the source file (.so) of a deep learning programming library: a static code segment, a static data segment, a dynamic code segment, and a dynamic data segment. The static code segment and the static data segment are used to store the binary code and constant data corresponding to the build-in operation instructions, while the dynamic code segment and the dynamic data segment are used to store the binary code and constant data corresponding to the custom operation instructions. The reason for distinguishing between dynamic and static segments is as follows: custom operation instructions are written by users and will continue to expand, so the present disclosure embodiment designs a dynamic segment of variable size for them; whereas build-in operation instructions are included with the deep learning programming library and will not change, so the present disclosure embodiment designs a static segment of fixed size for them. In addition, the present disclosure embodiment does not embed the binary code corresponding to offline operation instructions into the source file (.so) of the deep learning programming library. Because offline operation instructions are typically heavyweight computational models (such as AlexNet and ResNet), they occupy a huge amount of storage space. Therefore, the present disclosure embodiment designs a separate offline cache for offline operations, using the file system to store offline operation instructions. The offline cache consists of an index table and offline files. The index table is implemented using key-value pairs (Key, Value). The Key is the type name of the offline operation instruction, and the Value is the binary code reference pointer corresponding to the offline operation instruction.

[0178] Based on the implementation of the static operation pool proposed in the above disclosed embodiment, in one possible implementation, searching the static operation pool for the binary code corresponding to the static operation instruction name may include:

[0179] According to the name specified when the static operation instruction is created, the binary code corresponding to the custom operation, the binary code corresponding to the build-in operation, and the binary code corresponding to the offline operation are searched in the static operation pool in turn to obtain the binary code corresponding to the name.

[0180] In an example, the user can specify the name of the operation when calling the operation creation interface (nclCreateOperator). NCLA uses the name of the operation as an index to search the static operation pool for the corresponding binary code. The search order is custom operation instructions, build-in operation instructions, and offline operation instructions. If the search hits, the corresponding binary code reference is returned, otherwise it triggers just-in-time compilation.

[0181] The above-mentioned disclosed embodiments illustrate the specific method of compilation when the operation instruction is a static operation instruction. It can also be known from the above-mentioned disclosed embodiments that the operation instruction can also be a dynamic operation instruction, which can trigger NCLA to perform real-time compilation. Therefore, in a possible implementation method, step S14 can also include step S143: when the instruction type is a dynamic operation instruction, the dynamic operation instruction is compiled in real time to obtain a real-time compilation result as the binary code of the deep learning algorithm.

[0182] From the above process, it can be seen that if the NCLAPI interface passes dynamic operation instructions such as fusion operation instructions or specialized operation instructions, just-in-time compilation (also known as real-time compilation) will be triggered when the operation fusion or operation specialization interface (nclFuseOperator, nclSpecializeOperator) is called. The just-in-time compilation system generates highly optimized binary code for the operation and hands it to the runtime system for execution when the operation is called. In one possible implementation, the user can also call the nclSaveOperator interface to save the optimized operation instruction (such as the fusion operation instruction). The binary code corresponding to the operation instruction can then be saved to the offline cache and its operation name used as the search index. Otherwise, to ensure that the size of the programming library does not expand rapidly, the binary code corresponding to the unsaved operation instruction will be discarded after the program exits.

[0183] The specific process of real-time compilation of dynamic operation instructions is not limited and can be flexibly selected according to actual conditions. In the embodiment of the present disclosure, a deep learning compilation framework (computation and data unified compilation architecture, CDUCA) for collaborative optimization of computation and data is involved to implement real-time compilation of dynamic operation instructions. Figure 21 The following is a schematic diagram of the CDUCA architecture according to one embodiment of the present disclosure. As shown in the figure, in one possible implementation, CDUCA includes three components: a computation graph engine, a code generator, and a data optimizer. It collaboratively optimizes operations and data at multiple levels to generate efficient binary code. The specific implementation of these three components is not unique. In one possible implementation, the three components can implement the following functions:

[0184] Computational graph engine: uses optimization technologies such as linear transformation and constant folding to perform algorithm-oriented advanced optimization on the original computational graph and constant data, generating optimized computational graphs and constant data;

[0185] Code Generator: This uses a heuristic search strategy based on a cost model to perform operations and data co-compilation optimization on computation graphs and constant data, generating efficient target platform code and data descriptors.

[0186] Data optimizer: parses data descriptors, performs optimizations such as splitting, reordering, and precision conversion on constant data for the target platform, and then packages the optimized constant data and target platform code (address relocation, etc.) to generate the final binary code.

[0187] Based on the architecture of the above disclosed embodiment, in a possible implementation, step S143 may include:

[0188] Step S1431: According to the dynamic operation instruction, the original calculation graph and original model data corresponding to the dynamic operation instruction are obtained.

[0189] Step S1432: Perform collaborative processing for the deep learning algorithm based on the original calculation graph and the original model data to obtain a first calculation graph and first model data.

[0190] Step S1433: Generate hardware instructions and data descriptors according to the first computation graph.

[0191] Step S1434: Perform hardware platform-oriented processing on the first model data according to the data descriptor to obtain second model data.

[0192] Step S1435: Obtain a binary code of a deep learning algorithm according to the hardware instruction and the second model data.

[0193] It can be seen from the above disclosed embodiments that the input of CDUCA is a computation graph and constant data. Therefore, in one possible implementation, it is necessary to obtain the original computation graph and original model data corresponding to the dynamic instruction through step S1431. The implementation of step S1431 is not limited. In one possible implementation, step S1431 may include:

[0194] By parsing the dynamic operation instructions, the original calculation graph corresponding to the dynamic operation instructions is obtained.

[0195] The original model data is obtained according to the parameters of the dynamic operation instruction.

[0196] Based on the above disclosed embodiments, in one example, the method for obtaining the original computation graph and original model data corresponding to the dynamic instruction can be: the original model data can be directly obtained according to the parameters of the dynamic operation instruction, and the original computation graph can be generated by NCLA parsing the dynamic operation instruction. The specific parsing process can be flexibly determined according to the actual situation and is not limited here. Figure 22The form of the original computation graph according to an embodiment of the present disclosure is shown. As shown in the figure, in one example, the original computation graph includes two graph nodes, Tensor and Operator, which correspond to the input and output data and data transformation of the operation instructions respectively.

[0197] It can be seen from the above disclosed embodiments that step S1432 may correspond to the computation graph engine component in CDUCA. The implementation of the computation graph engine is not limited and can be flexibly determined according to actual conditions.

[0198] The nodes in the computational graph can be used to represent the operations performed during the deep learning process. Common operations in deep learning algorithms include convolution, full connection, activation, pooling, batch normalization, scaling, etc. These operations can be divided into two categories according to their specific implementation: linear transformation operations and nonlinear change operations. Among them, all linear transformation operations can be expressed in the form of vector or matrix multiplication and addition. Therefore, the general expression of linear transformation operations can be:

[0199] Y=X*W+B

[0200] Among them, X and Y are variables, and W and B are model data constants.

[0201] Any linear transformation operation can be expressed as the above-mentioned general expression form. Therefore, operations that cannot be expressed as the above-mentioned general expression form are nonlinear transformation operations. In one example, among the common operations of the deep learning algorithm listed above, convolution, full connection, batch normalization, and scaling operations are linear transformation operations, and pooling and activation are nonlinear transformation operations. In one example, when the fully connected operation is expressed through the above-mentioned general expression form, X and Y represent the fully connected input neuron matrix and output neuron matrix, respectively, W represents the weight matrix, and B represents the bias matrix. The specific expression of other linear transformation operations through the general expression form will not be repeated here.

[0202] For linear transformation operations, if there are two consecutive linear transformation operations, they are:

[0203] Y1=X1*W1+B1

[0204] Y2=Y1*W2+B2

[0205] Since the linear transformation operation satisfies the distributive law and the associative law, and W and B are constants in the general expression of the linear transformation operation, the two linear transformation operations can perform the following equivalent linear transformations:

[0206] Y2=(X1*W1+B1)*W2+B2

[0207] Y2=X1*W1*W2+B1*W2+B2

[0208] W'=W1*W2,B'=B1*W2+B2

[0209] Y2=X1*W'+B'

[0210] Through the above equivalent linear transformations, the original linear transformation operation can be comprehensively optimized through linear transformation optimization techniques and constant folding optimization techniques, ultimately simplifying it to a single linear transformation operation. This optimization reduces the amount of computation while compressing the model data. This not only reduces the storage overhead of the model data, but also reduces the amount of memory access during runtime.

[0211] Therefore, in one possible implementation, the specific implementation form of collaborative processing for deep learning algorithms can be linear transformation and constant folding.

[0212] Figure 23 A workflow diagram of a computation graph engine according to an embodiment of the present disclosure is shown. As shown in the figure, in one possible implementation, step S1432 may include:

[0213] Step S14321, read the original calculation graph and original model data.

[0214] Step S14322, identifying the continuous linear transformation operation nodes in the original computation graph.

[0215] Step S14323: Process the continuous linear transformation operation nodes through linear transformation and constant folding to obtain a first calculation graph and first model data.

[0216] From the above content, it can be seen that two consecutive linear transformation operations can be simplified into one linear transformation operation through linear transformation and constant folding. When there are three consecutive linear transformation operations, the first two consecutive linear transformation operations can be simplified into one linear transformation operation through linear transformation and constant folding. Then, this simplified linear transformation operation and the remaining linear transformation operation are again simplified into one linear transformation operation through linear transformation and constant folding. Similarly, when there are more consecutive linear transformation operations, these linear transformation operations can be merged and simplified into at least one linear transformation operation through linear transformation and constant folding.

[0217] Because each linear transformation operation corresponds to a corresponding linear transformation operation node in the computation graph, in one possible implementation, the continuous linear transformation operation nodes may include at least two continuous linear transformation operation nodes. Thus, the continuous linear transformation operation nodes may correspond to at least two continuous linear transformation operations. The number of continuous linear transformation operation nodes is not limited and can be determined based on the actual situation of the computation graph.

[0218] In a possible implementation, the specific process of step S14323 may include:

[0219] The continuous linear transformation operation nodes in the original computation graph are linearly transformed and constant folded, and the continuous linear transformation operation nodes are merged to obtain a first computation graph.

[0220] The model data corresponding to the consecutive linear transformation operation nodes are merged to obtain the first model data.

[0221] In one example, the continuous linear transformation operation nodes in the original computational graph may be nodes 1, 2, 3, and 4 connected in sequence, and the corresponding model data combination may be model data group 1. Through linear transformation and constant folding, nodes 1, 2, 3, and 4 may be merged into 1, and finally node 5 is obtained. In this process, the model data contained in model data group 1 may be merged due to constant folding, and finally a merged model data group is obtained, which can be called model data group 2. In one example, the continuous linear transformation operation nodes in the original computational graph may be nodes 1, 2, 3, and 4 connected in sequence, and the corresponding model data combination may be model data group 1. The combination can be model data group 1. Through linear transformation and constant folding, nodes 1, 2 and 3 can be merged into one node 6, without merging node 6 and node 4. At this time, nodes 4 and 6 can be finally obtained. In this process, since nodes 1, 2 and 3 have undergone constant folding, the corresponding model data combination in model data group 1 may have been merged, and finally a merged model data group was obtained, which can be called model data group 3. Since the model data corresponding to the original node 4 has not undergone constant folding, the model data group 3 and the model data corresponding to node 4 are combined to obtain model data group 4 corresponding to the current overall linear transformation operation. By analogy with the above two examples, when the number of continuous linear transformation operation nodes in the original computational graph changes, the specific process of step S13323 may also change accordingly, which will not be listed one by one here. In one example, the neural network corresponding to the deep learning algorithm can be the classic image classification network Resnet, Figure 24A structural schematic diagram of a substructure included in an image classification network according to an embodiment of the present disclosure is shown. As can be seen from the figure, the Resnet network may include a substructure of convolution + batch normalization + scaling. Since convolution, batch normalization and scaling are all linear transformation operations, in the computational graph corresponding to the Resnet network, the three operation nodes corresponding to this substructure can be linearly transformed and constant folded through the linear transformation operation as described above, and finally merged into one operation node, and the model data involved in these three operation nodes are also merged.

[0222] Through any of the above processes, the operation process of the deep learning algorithm can be optimized to obtain the first calculation graph and first model data after collaborative processing for the deep learning algorithm, thereby reducing the amount of memory access when running this deep learning algorithm and reducing the storage overhead of the model data when storing it.

[0223] Based on the first computation graph and the first model data, hardware instructions and data descriptors can be generated through step S1433. The specific implementation of step S1433 is not limited, and any process that can generate hardware instructions based on the computation graph can be used as an implementation of step S1433.

[0224] The main purpose of step S1433 is to generate hardware instructions readable by the corresponding hardware platform based on the first calculation graph optimized in step S1432. In order to optimize the memory access performance of the hardware platform, an on-chip cache is often designed in the part of the hardware platform close to the calculation location. In one example, this part can be the location close to the deep learning processor computing unit. When accessing the hardware platform, the speed of accessing the on-chip cache is often faster than the speed of accessing other locations. In one example, the other location can be an off-chip double data rate synchronous dynamic random access memory (DDR). Affected by the location and function of the on-chip cache, the capacity of the on-chip cache is limited. Therefore, the utilization rate of the on-chip cache can directly affect the performance optimization effect of the deep learning algorithm for the corresponding hardware platform. However, the collaborative processing method for the deep learning algorithm in step S1432 can achieve optimization of the operation, but cannot improve the utilization rate of the on-chip cache. In one possible implementation, the process of generating hardware instructions can be optimized to improve on-chip cache utilization, thereby enhancing the performance optimization effect of deep learning algorithms on corresponding hardware platforms. Therefore, in one possible implementation, step S1433 may include: processing the first computation graph according to the cost model, combined with a heuristic search strategy, to obtain hardware instructions and data descriptors.

[0225] The first computation graph is processed according to the cost model and combined with the heuristic search strategy to obtain the hardware instructions and data descriptors. The specific process can be flexibly selected according to actual conditions. Figure 25 A flowchart of a method for compiling a deep learning algorithm according to an embodiment of the present disclosure is shown. As shown in the figure, in one possible implementation, step S1433 may include:

[0226] Step S14331: Model the first computation graph using a cost model to generate a search space and an objective function.

[0227] Step S14332: Search in the search space using a heuristic search strategy. When the target function reaches a threshold, hardware instructions and data descriptors for the hardware platform are generated.

[0228] There are many ways to implement the method of generating hardware instructions based on the computational graph. For example, the computational graph can be used as input, and a model that can generate hardware instructions is used, and the optimization is performed in the output results of the model to obtain the final hardware instructions for the hardware platform. There is no limitation on which model that can generate hardware instructions is used. In one possible implementation method, the first computational graph can be modeled by a cost model. The cost model estimates the total time (including overhead such as data format conversion) for an operation to be executed on a deep learning processor. The main factors it considers are the memory access time, calculation time, and the overlap rate of the two operations. These three factors directly determine the performance of the program. Specifically, the deep learning processor includes independent computing units and memory access units. The computing instructions and memory access instructions can be executed in a pipeline or overlapped. Figure 26 A schematic diagram of an instruction flow according to an embodiment of the present disclosure is shown in the figure. To minimize the total runtime of a program, it is necessary to reduce computation time, memory access time, and increase the overlap between the two. Therefore, after modeling the first computation graph using a cost model, the generated hardware instructions are actually a search space composed of a combination of multiple possible hardware instructions. The hardware instructions contained in the search space can indicate various options for the hardware platform during operation: in one example, the hardware instructions can instruct the hardware platform to load a complete block of data in several steps and place it in the on-chip cache; in another example, the hardware instructions can instruct the hardware platform to load a complete block of data in several steps and swap it in and out between the on-chip cache and the off-chip DDR as needed; in another example, the hardware instructions can indicate the amount of data the hardware platform needs to process when performing a vector operation. Since there are multiple hardware instructions in the search space, the specific application instructions that are ultimately applied require searching the search space to find the optimal instruction combination, which will serve as the final hardware instruction.

[0229] As can be seen from the above, the cost model can provide an estimated execution time for hardware instructions. Therefore, after passing the first computation graph through the cost model, a corresponding objective function can also be generated. The specific form of the objective function is not limited here and can be flexibly set according to actual conditions. This objective function can indicate how much time the hardware instructions generated by searching within the search space consume when running. Therefore, when the objective function reaches a threshold, it indicates that the generated hardware instructions meet the runtime requirements, and further indicates that the generated hardware instructions can improve the performance of the hardware instructions when running on the hardware platform. Since the specific implementation form of the objective function is not limited, the corresponding threshold of the objective function is also not limited and can be flexibly set according to actual conditions.

[0230] There are also no restrictions on how to search within the generated search space. In one possible implementation, a brute force search can be used. However, such a search process is time-consuming and may extend the compilation process, which in turn may reduce the performance optimization effect of the deep learning algorithm on the corresponding hardware platform. In another possible implementation, a heuristic search strategy can be used to search within the search space.

[0231] The purpose of the heuristic search strategy is to improve the efficiency of searching within the search space. Therefore, it is not limited to a specific search method and can be flexibly selected according to actual circumstances. In one possible implementation, the heuristic search strategy may include: a search strategy that improves the utilization of the on-chip cache within the hardware platform; or a search strategy that reduces the computational granularity and memory access granularity while ensuring the utilization of the computing units and access units within the hardware platform. In one example, the purpose of the heuristic search strategy may be to maximize the utilization of the on-chip cache, so the search strategy can be defined as a search strategy that improves the utilization of the on-chip cache within the hardware platform; in another example, the purpose of the heuristic search strategy may be to maximize the utilization of the computing units and access units within the hardware platform, so as to select instruction combinations with relatively small computational and access granularity, thereby improving the coverage of calculations and memory accesses; in another example, the search strategy can be defined as a search strategy that reduces the computational granularity and memory access granularity while ensuring the utilization of the computing units and access units within the hardware platform; in another example, the heuristic search strategy can also be a balanced strategy of the above two strategies, that is, a search strategy that achieves the best overall result of the above two strategies.

[0232] Through any of the above-mentioned search strategies, hardware instructions for the hardware platform can be generated after searching in the search space. Since the content of the hardware instructions contained in the search space can indicate the various selection methods of the hardware platform during operation, when searching in the search space, in addition to considering the operating performance of the above-mentioned hardware instructions, it is also possible to consider using the generated hardware instructions to guide the hardware platform to optimize the model data, so as to further improve the computing memory access speed and memory access efficiency of the hardware platform. Therefore, based on this consideration, step S14332 can generate a data descriptor while generating the hardware instructions, which is used to guide how to further optimize the model data.

[0233] Based on the data descriptor generated at the same time in step S1433, the first model data can be further optimized to obtain the second model data. The specific implementation form of how the data descriptor optimizes the model data is not limited. Figure 27 A diagram showing an implementation method for optimizing model data according to an embodiment of the present disclosure is shown. As shown in the figure, in one possible implementation method, step S1434 may include: according to the data descriptor, in accordance with the operation requirements within the hardware platform, performing data splitting on the first model data; according to the data descriptor, in accordance with the operation requirements within the hardware platform, performing data alignment on the first model data; or, according to the data descriptor, in accordance with the operation requirements within the hardware platform, performing dimension transformation on the first model data; or, according to the data descriptor, in accordance with the operation requirements within the hardware platform, performing precision selection on the first model data.

[0234] In the above-mentioned disclosed embodiments, in one possible implementation, for data tiling, loops, on-chip storage management, and instruction pipelines can be optimized; for data alignment, the deep learning processor supports vector and matrix operations, and therefore has certain requirements for data alignment. Aligned access can speed up memory access and reduce DDR bank conflicts; for data reordering, the input data of operations such as convolution is a multidimensional array. The arrangement order of the multidimensional array will affect the number of memory access jumps. Rearranging the data can improve memory access locality and reduce the MMU miss rate; for precision conversion, the deep learning processor usually supports low-precision operations, such as half-precision floating point and quark-bit quantization. Different precision operations have different performance. Performing appropriate precision conversion on the data can improve the overall performance of the program; precision conversion is a specific implementation of the automatic precision optimization function of NCLAPI.

[0235] The specific implementation method of data splitting is not limited. Figure 28A schematic diagram illustrating post-splitting optimization of data according to an embodiment of the present disclosure is shown. As shown in the figure, in one example, constant data with a channel count of 2 and a height and width of 4 can be optimized. Assume that the data descriptor requires the following data transformations: the height and width are simultaneously split into 2; the data arrangement order is adjusted from HWC to CHW; and the data calculation precision is adjusted from float32 to half. In accordance with the above description, the data optimizer performs physical transformations such as splitting, rearrangement, precision conversion, and alignment on the data from a DDR perspective, ultimately obtaining the optimized data.

[0236] In one example, when the hardware platform is performing operations according to hardware instructions, it may have certain requirements for data alignment. If this process is performed within the hardware platform, the performance of the hardware platform during operation will be greatly reduced. Therefore, the data to be aligned can be aligned in advance during the compilation process according to the data descriptor, thereby speeding up the memory access speed of the hardware platform when it is working. The specific alignment standards and methods are not limited here and can be determined according to the actual requirements of the hardware platform. In one example, when the hardware platform is performing operations according to hardware instructions, some algorithms, such as convolution algorithms, may need to interpret data as multiple arrays, and the arrangement order of multi-dimensional arrays will affect the number of memory access jumps within the hardware platform, thereby affecting the performance of the hardware platform during operation. Therefore, the model data can be aligned in advance according to the operation requirements during the compilation process according to the data descriptor. The row dimension is transformed to improve the locality of memory access, thereby minimizing the memory access jump of the hardware platform. The specific dimension transformation and the transformation method are not limited here, and can be determined according to the actual requirements of the hardware platform. In an example, since different operation precisions correspond to different operation speeds, the hardware platform may support different operation precision requirements. In an example, the hardware platform may support low-precision operations more. If high-precision operations are used, the running speed of the hardware platform may be reduced. Therefore, according to the data descriptor, the preferred model data precision can be selected in advance according to the precision requirements of the hardware platform during the compilation process. The specific selection of the data precision is not limited again, and can be determined according to the needs of the hardware platform. In an example, the preferred precision can be 16-bit quantization precision. In an example, the preferred precision can be 8-bit quantization precision.

[0237] The process of optimizing the first model data into the second model data through processing can be any of the four aforementioned methods, or any combination thereof, and can also include other methods that are beneficial for improving the speed of the hardware platform, which are not listed here. By pre-optimizing the model data during the compilation process, the performance of the hardware platform at runtime can be greatly improved, such as increasing memory access speed and efficiency.

[0238] Based on the above process, the binary code of the deep learning algorithm can be obtained. The content contained in this binary code can exist in various forms. In one possible implementation, step S1435 may include: packaging the hardware instructions and the second model data to obtain the binary code of the deep learning algorithm.

[0239] In one example, the hardware instructions may be hardware instructions generated according to the first computation graph, and the second model data may be model data obtained by subjecting the original model data to collaborative processing for the deep learning algorithm and processing for the hardware platform, i.e., the resulting binary code of the deep learning algorithm, obtained by sequentially performing steps S1432, S1433, and S1434. In another example, the hardware instructions may be hardware instructions generated directly according to the original computation graph, and the second model data may be model data obtained by subjecting the original model data to only processing for the hardware platform, i.e., the resulting executable file of the deep learning algorithm, obtained by sequentially performing steps S1433 and S1434. In another example, the hardware instructions may be hardware instructions generated according to the first computation graph, and the second model data may be the first model data, i.e., the resulting executable file of the deep learning algorithm, obtained by sequentially performing steps S1432 and S1433. In another example, the hardware instructions may be hardware instructions generated directly according to the original computation graph, and the second model data may be the original model data, i.e., the resulting executable file of the deep learning algorithm, obtained by sequentially performing steps S1433 and S1434. It can be seen from the above example that step S1432, step S1433 and step S1434 may not exist at the same time and can be flexibly combined according to actual conditions.

[0240] CDUCA, implemented through the aforementioned disclosed embodiments, can integrate previously mentioned compilation optimization technologies, such as memory reuse, operation fusion, latency hiding, linear algebraic transformations, common subexpression elimination, constant propagation, dead code elimination, and data parallelism. Furthermore, CDUCA's hierarchical design structure is highly scalable, allowing developers to integrate various compilation optimization technologies into various CDUCA modules. For example, operation aggregation technology can be integrated into the computational graph engine module, and polyhedron compilation optimization technology can be integrated into the code generator module. Real-time compilation of dynamic operation instructions through CDUCA can effectively improve compilation efficiency, thereby increasing the operating speed of hardware devices.

[0241] It can be seen from the above-mentioned disclosed embodiments that binary code of a deep learning algorithm can be generated through both just-in-time compilation and static search. However, it can also be seen from the above-mentioned disclosed embodiments that the overall architecture of NCLA also includes a runtime system. Therefore, in one possible implementation, the method proposed in the disclosed embodiments further includes: executing the binary code of the deep learning algorithm in a deep learning processor through the runtime system.

[0242] The implementation of the runtime system is not restricted. Figure 29 A schematic diagram of the modules and functions of a runtime system according to an embodiment of the present disclosure is shown. As shown in the figure, the runtime system can be responsible for the interaction between the host and the deep learning processor. It can encapsulate the device driver interface and provide device management, memory management, operation execution, device synchronization and other functions for the upper layer.

[0243] Through the above-mentioned disclosed embodiments, it is possible to compile deep learning algorithms through NCLA. Experiments have confirmed that a specific implementation of NCLA can be deployed on a deep learning processor platform and can support mainstream deep learning algorithms including image classification, object detection, and natural language processing. The disclosed embodiments used TensorFlow to experiment with several common deep learning applications. On average, the performance of the binary code generated by the NCLA just-in-time compilation system can reach 83.24% of the performance of manually optimized code. In the best case, NCLA can achieve at least 72.61% of the hardware peak performance. In addition, the success of NCLA further confirms the versatility of neural algorithms and the NCL API mentioned in the above-mentioned disclosed embodiments.

[0244] Figure 30 A block diagram of a compilation device for a deep learning algorithm according to an embodiment of the present disclosure is shown. As shown in the figure, the device 20 includes: a user instruction receiving module 21 for receiving user instructions; a triggering module 22 for triggering the deep learning programming library interface to create or call operation data according to the user instructions, wherein the operation data includes at least one of tensor data and operation instructions.

[0245] In one possible implementation, the trigger module includes: a dynamic operation instruction triggering unit, used to trigger the deep learning programming library interface to create or call dynamic operation instructions according to the user instruction; and / or, a static operation instruction triggering unit, used to trigger the deep learning programming library interface to create or call static operation instructions according to the user instruction.

[0246] In one possible implementation, the dynamic operation instruction triggering unit includes: a specialized operation instruction triggering sub-unit, which is used to trigger the deep learning programming library interface to create or call a specialized operation instruction according to the user instruction, wherein the specialized operation instruction is obtained by binding input parameters to the operation instruction and then converting it; and / or, a fusion operation instruction triggering sub-unit, which is used to trigger the deep learning programming library interface to create or call a fusion operation instruction according to the user instruction, wherein the fusion operation instruction is obtained by combining multiple operation instructions according to a calling order.

[0247] In one possible implementation, the specialized operation instruction triggering sub-unit is used to: according to the user instruction, trigger the deep learning programming library interface to bind all input parameters to the operation instruction and then convert it to obtain a fully specialized operation instruction; and / or, according to the user instruction, trigger the deep learning programming library interface to bind N input parameters to the operation instruction and then convert it to obtain a partially specialized operation instruction, wherein N is a positive integer less than the number of input parameters of the operation instruction; and / or, according to the user instruction, trigger the deep learning programming library interface to directly convert the operation instruction without binding the input parameters to obtain a pseudo-specialized operation instruction.

[0248] In one possible implementation, the fusion operation instruction triggering sub-unit is used to: create a name for the fusion operation instruction; determine a fusion operation sub-instruction based on the operation instruction to be fused; determine the operation connection relationship between the fusion operation sub-instructions based on the calling order of the operation instructions to be fused; connect the fusion operation sub-instructions based on the operation connection relationship to obtain a connection result; set the input parameters and output parameters of the fusion operation instruction based on the user instructions corresponding to the fusion operation instruction; package the name, connection result, input parameters and output parameters to obtain the fusion operation instruction.

[0249] In one possible implementation, the static operation instruction triggering unit includes: a customized operation instruction triggering subunit, used to trigger the deep learning programming library interface to create or call a customized operation instruction according to the user instruction, wherein the customized operation instruction is obtained by encapsulating the user instruction corresponding to the operation instruction according to the interface and data structure definition of the operation instruction; and / or, a build-in operation instruction triggering subunit, used to trigger the deep learning programming library interface to create or call a build-in operation instruction according to the user instruction, wherein the build-in operation instruction is an own operation instruction included in the deep learning programming library interface; and / or, an offline operation instruction triggering subunit, used to trigger the deep learning programming library interface to create or call an offline operation instruction according to the user instruction, wherein the offline operation instruction is a pre-compiled dynamic operation instruction, and the pre-compiled result is stored in an offline cache.

[0250] In one possible implementation, the binary code corresponding to the static operation instruction is stored in a static operation pool of the deep learning programming library, and the static operation pool includes a static operation source file, and the static operation source file includes a static code segment, a static data segment, a dynamic code segment and a dynamic data segment; wherein, the static code segment is used to store the binary code corresponding to the build-in operation instruction; the dynamic code segment is used to store the binary code corresponding to the custom operation instruction; the static data segment is used to store the tensor data corresponding to the build-in operation instruction; and the dynamic data segment is used to store the tensor data corresponding to the custom operation instruction.

[0251] In one possible implementation, the static operation pool also includes an offline cache, which includes an offline file and an index table; wherein the offline file is used to save the pre-compiled results of the offline operation instructions; and the index table is used to indicate the location of the pre-compiled results of the offline operation instructions in the offline file.

[0252] In one possible implementation, the customized operation instruction triggering subunit is further used to: compile the customized operation instruction to obtain a compilation result; insert the compilation result into the static operation pool in a dynamic link or static link manner to obtain the binary code corresponding to the customized operation instruction.

[0253] In one possible implementation, the trigger module is further used to: call all corresponding operation instructions one by one according to the user instruction; or, according to the user instruction, fuse all corresponding operation instructions to obtain a fused operation instruction, and call the fused operation instruction; or, according to the user instruction, segment all corresponding operation instructions to obtain a segmentation result, fuse each segmentation result separately to obtain a corresponding segmented fused operation instruction, and call the segmented fused operation instruction in sequence.

[0254] In one possible implementation, tensor data includes a shape attribute, a logical data type attribute, a physical data type attribute, and a physical layout attribute.

[0255] In one possible implementation, the device is also connected to a deep learning programming library architecture, which includes: an operation instruction reading module, used to read the operation instruction transmitted by the deep learning programming library interface; a compilation module, used to determine the instruction type of the operation instruction, and execute the compilation operation corresponding to the instruction type according to the determination result to obtain the binary code of the deep learning algorithm.

[0256] In one possible implementation, the compilation module includes: an operation instruction type judgment unit, used to judge the instruction type of the operation instruction; and a static search unit, used to search for the corresponding binary code in the static operation pool according to the name of the static operation instruction when the instruction type is a static operation instruction, as the binary code of the deep learning algorithm.

[0257] In one possible implementation, the static search unit is used to: search for a binary code corresponding to the name of the static operation instruction in the static operation pool; when the search result is successful, return the binary code as the binary code of the deep learning algorithm.

[0258] In a possible implementation, the static search unit is further configured to: when the search result is failure, use the static operation instruction as a dynamic operation instruction and perform real-time compilation.

[0259] In one possible implementation, the static search unit is further used to: according to the name specified when the static operation instruction is created, search in the static operation pool in sequence for the binary code corresponding to the custom operation, the binary code corresponding to the build-in operation, and the binary code corresponding to the offline operation to obtain the binary code corresponding to the name.

[0260] In one possible implementation, the compilation module further includes a dynamic compilation unit, which is used to: when the instruction type is a dynamic operation instruction, perform real-time compilation on the dynamic operation instruction to obtain a real-time compilation result as the binary code of the deep learning algorithm.

[0261] In one possible implementation, the dynamic compilation unit includes: an original data acquisition subunit, which is used to obtain the original calculation graph and original model data corresponding to the dynamic operation instruction according to the dynamic operation instruction; a collaborative processing subunit, which performs collaborative processing for the deep learning algorithm according to the original calculation graph and the original model data to obtain a first calculation graph and first model data; a hardware instruction generation subunit, which generates hardware instructions and data descriptors according to the first calculation graph; a model data processing subunit, which performs hardware platform-oriented processing on the first model data according to the data descriptor to obtain second model data; and a binary code generation subunit, which obtains the binary code of the deep learning algorithm according to the hardware instructions and the second model data.

[0262] In a possible implementation, the original data acquisition subunit is used to: obtain an original calculation graph corresponding to the dynamic operation instruction by parsing the dynamic operation instruction; and obtain the original model data according to the parameters of the dynamic operation instruction.

[0263] In one possible implementation, the collaborative processing sub-unit is used to: read the original computation graph and the original model data; identify the continuous linear transformation operation nodes within the original computation graph; and process the continuous linear transformation operation nodes through linear transformation and constant folding to obtain a first computation graph and first model data.

[0264] In a possible implementation, the hardware instruction generation subunit is used to: process the first computation graph according to a cost model, and combine it with a heuristic search strategy to obtain hardware instructions and data descriptors.

[0265] In one possible implementation, the model data processing subunit is used to: perform data alignment on the first model data according to the data descriptor and the computing requirements within the hardware platform; or, perform dimension transformation on the first model data according to the data descriptor and the computing requirements within the hardware platform; or, perform precision selection on the first model data according to the data descriptor and the computing requirements within the hardware platform.

[0266] In one possible implementation, the binary code generation subunit is used to: package the hardware instructions and the second model data to obtain the binary code of the deep learning algorithm.

[0267] In one possible implementation, the apparatus is further configured to: execute the binary code of the deep learning algorithm in a deep learning processor through a runtime system.

[0268] In one possible implementation, the device is further used to: perform class expansion on the deep learning framework, encapsulate the tensor data and the operation instructions within the data of the deep learning framework, and realize the integration of the deep learning framework and the deep learning programming library interface.

[0269] Figure 31 A block diagram of a combined processing device according to an embodiment of the present disclosure is shown. As shown in the figure, the combined processing device includes the above-mentioned deep learning computing device, a universal interconnection interface, and other processing devices.

[0270] The deep learning computing device interacts with other processing devices to jointly complete user-specified operations. Other processing devices include one or more types of processors such as central processing units (CPUs), graphics processing units (GPUs), neural network processors, and other general / special-purpose processors. There is no limit on the number of processors included in other processing devices. Other processing devices serve as interfaces between the deep learning computing device and external data and control, including data handling, and complete basic control of the deep learning computing device such as starting and stopping. Other processing devices can also collaborate with the deep learning computing device to jointly complete computing tasks. A universal interconnection interface is used to transmit data and control instructions between the deep learning computing device and other processing devices. The deep learning computing device obtains the required input data from other processing devices and writes it into the storage device on the deep learning computing device chip; it can obtain control instructions from other processing devices and write them into the control cache on the deep learning computing device chip; it can also read data from the storage module of the deep learning computing device and transmit it to other processing devices.

[0271] The combined processing device may further include a storage device, each connected to the deep learning computing device and the other processing device. The storage device is used to store data from the deep learning computing device and the other processing device, and is particularly suitable for data that cannot be fully stored in the internal storage of the deep learning computing device or other processing devices.

[0272] This combined processing device can be used as a system-on-chip (SoC) in devices such as mobile phones, robots, drones, and video surveillance equipment, effectively reducing the core area of ​​the control unit, increasing processing speed, and lowering overall power consumption. In this case, the combined processing device's universal interconnect interface connects to certain components of the device, such as a camera, display, mouse, keyboard, network card, and Wi-Fi interface.

[0273] In one possible implementation, the present disclosure further provides a deep learning chip, which includes the above-mentioned deep learning computing device or combined processing device.

[0274] In a possible implementation, the present disclosure further provides a chip packaging structure, which includes the above-mentioned chip.

[0275] In a possible implementation, the present disclosure further provides a board card, which includes the above-mentioned chip packaging structure.

[0276] In a possible implementation, the present disclosure further provides an electronic device including the above-mentioned board.

[0277] Electronic devices include data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, mobile phones, driving recorders, navigation systems, sensors, cameras, servers, cloud servers, cameras, camcorders, projectors, watches, headphones, mobile storage, wearable devices, vehicles, household appliances, and / or medical devices.

[0278] The transportation means include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging (MRI), ultrasound machines, and / or electrocardiographs.

[0279] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0280] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0281] In the several embodiments provided in the present disclosure, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0282] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0283] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0284] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned memory includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0285] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0286] The embodiments of the present disclosure are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core ideas of the present disclosure. At the same time, for those skilled in the art, according to the ideas of the present disclosure, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present disclosure.

[0287] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0288] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0289] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0290] The foregoing content can be better understood in accordance with the following terms:

[0291] Clause A1. A method for generating operation data, the method comprising:

[0292] Receive user instructions;

[0293] According to the user instruction, the deep learning programming library interface is triggered to create or call operation data, wherein the operation data includes at least one of tensor data and operation instructions.

[0294] Clause A2. The method according to clause A1, triggering the deep learning programming library interface to create or call an operation instruction according to the user instruction, including:

[0295] According to the user instruction, trigger the deep learning programming library interface to create or call dynamic operation instructions; and / or,

[0296] According to the user instructions, the deep learning programming library interface is triggered to create or call static operation instructions.

[0297] Clause A3. The method according to clause A2, wherein triggering the deep learning programming library interface to create or call a dynamic operation instruction according to the user instruction comprises:

[0298] According to the user instruction, trigger the deep learning programming library interface to create or call a specialized operation instruction, wherein the specialized operation instruction is obtained by binding input parameters to the operation instruction and then converting it; and / or,

[0299] According to the user instruction, the deep learning programming library interface is triggered to create or call a fusion operation instruction, wherein the fusion operation instruction is obtained by combining multiple operation instructions according to the calling order.

[0300] Clause A4. The method according to clause A3, wherein, according to the user instruction, triggering the deep learning programming library interface to create the specialized operation instruction comprises:

[0301] According to the user instruction, trigger the deep learning programming library interface to bind all input parameters to the operation instruction and then convert it to obtain a fully specialized operation instruction; and / or,

[0302] According to the user instruction, trigger the deep learning programming library interface to bind N input parameters to the operation instruction and then convert it to obtain a partially specialized operation instruction, where N is a positive integer less than the number of input parameters of the operation instruction; and / or,

[0303] According to the user instruction, the deep learning programming library interface is triggered to directly convert the operation instruction without binding the input parameters to obtain a pseudo-specialized operation instruction.

[0304] Clause A5. The method according to clause A3, wherein triggering the creation of the fusion operation instruction according to the user instruction comprises:

[0305] Create a name for the fusion operation instruction;

[0306] Determine a fusion operation sub-instruction according to the operation instruction to be fused;

[0307] Determining the operation connection relationship between the fused operation sub-instructions according to the calling order of the operation instructions to be fused;

[0308] Connecting the fusion operation sub-instructions according to the operation connection relationship to obtain a connection result;

[0309] Setting input parameters and output parameters of the fusion operation instruction according to a user instruction corresponding to the fusion operation instruction;

[0310] The name, connection result, input parameters and output parameters are packaged to obtain the fusion operation instruction.

[0311] Clause A6. The method according to clause A2, wherein triggering the deep learning programming library interface to create or call a static operation instruction according to the user instruction comprises:

[0312] According to the user instruction, trigger the deep learning programming library interface to create or call a custom operation instruction, wherein the custom operation instruction is obtained by encapsulating the user instruction corresponding to the operation instruction according to the interface and data structure definition of the operation instruction; and / or,

[0313] According to the user instruction, trigger the deep learning programming library interface to create or call a build-in operation instruction, wherein the build-in operation instruction is a built-in operation instruction included in the deep learning programming library interface; and / or,

[0314] According to the user instruction, the deep learning programming library interface is triggered to create or call an offline operation instruction, wherein the offline operation instruction is a pre-compiled dynamic operation instruction, and the pre-compiled result is stored in the offline cache.

[0315] Clause A7. The method according to Clause A6, wherein the binary code corresponding to the static operation instruction is stored in a static operation pool of the deep learning programming library, and the static operation pool includes a static operation source file, and the static operation source file includes a static code segment, a static data segment, a dynamic code segment, and a dynamic data segment; wherein,

[0316] The static code segment is used to store the binary code corresponding to the build-in operation instruction;

[0317] The dynamic code segment is used to store the binary code corresponding to the customized operation instruction;

[0318] The static data segment is used to store tensor data corresponding to the build-in operation instruction;

[0319] The dynamic data segment is used to store tensor data corresponding to the custom operation instruction.

[0320] Clause A8. The method according to clause A7, wherein the static operation pool further comprises an offline cache, wherein the offline cache comprises an offline file and an index table;

[0321] The offline file is used to store the pre-compiled results of the offline operation instructions;

[0322] The index table is used to indicate the position of the pre-compiled result of the offline operation instruction in the offline file.

[0323] Clause A9. The method according to clause A7, wherein triggering the deep learning programming library interface to create or call a custom operation instruction according to the user instruction further includes:

[0324] Compiling the customized operation instruction to obtain a compilation result;

[0325] The compilation result is inserted into the static operation pool in a dynamic link or static link manner to obtain a binary code corresponding to the customized operation instruction.

[0326] Clause A10. The method according to clause A1, wherein, according to the user instruction, triggering the calling of an operation instruction in the deep learning programming library interface comprises:

[0327] According to the user instruction, all corresponding operation instructions are called one by one; or,

[0328] According to the user instruction, all corresponding operation instructions are merged to obtain a fused operation instruction, and the fused operation instruction is called; or,

[0329] According to the user instruction, all corresponding operation instructions are segmented to obtain segmentation results, each segmentation result is fused respectively to obtain a corresponding segmentation fusion operation instruction, and the segmentation fusion operation instructions are called in sequence.

[0330] Clause A11. The method of clause A1, wherein the tensor data comprises a shape attribute, a logical data type attribute, a physical data type attribute, and a physical layout attribute.

[0331] Clause A12. The method according to clause A1, further comprising:

[0332] The deep learning programming library architecture reads the operation instructions transmitted by the deep learning programming library interface;

[0333] The deep learning programming library architecture determines the instruction type of the operation instruction, and performs a compilation operation corresponding to the instruction type according to the determination result to obtain the binary code of the deep learning algorithm.

[0334] Clause A13. The method according to clause A12, wherein determining the instruction type of the operation instruction and performing a compilation operation corresponding to the instruction type based on the determination result to obtain binary code of the deep learning algorithm comprises:

[0335] Determining the instruction type of the operation instruction;

[0336] When the instruction type is a static operation instruction, the corresponding binary code is searched in the static operation pool according to the name of the static operation instruction, and is used as the binary code of the deep learning algorithm.

[0337] Clause A14. The method according to Clause A13, wherein when the instruction type is a static operation instruction, searching a static operation pool for a corresponding binary code according to the name of the static operation instruction as the binary code of the deep learning algorithm includes:

[0338] According to the name of the static operation instruction, searching the static operation pool for a binary code corresponding to the name;

[0339] When the search result is successful, the binary code is returned as the binary code of the deep learning algorithm.

[0340] Clause A15. The method according to clause A14, wherein when the instruction type is a static operation instruction, searching for a corresponding binary code in a static operation pool according to the name of the static operation instruction as the binary code of the deep learning algorithm, further comprising:

[0341] When the search result is failure, the static operation instruction is used as a dynamic operation instruction and compiled in real time.

[0342] Clause A16. The method according to clause A14, wherein searching the static operation pool for a binary code corresponding to the name of the static operation instruction comprises:

[0343] According to the name specified when the static operation instruction is created, in the static operation pool, the binary code corresponding to the custom operation, the binary code corresponding to the build-in operation, and the binary code corresponding to the offline operation are searched in turn to obtain the binary code corresponding to the name.

[0344] Clause A17. The method according to clause A13, wherein determining the instruction type of the operation instruction and performing a compilation operation corresponding to the instruction type based on the determination result to obtain the binary code of the deep learning algorithm further comprises:

[0345] When the instruction type is a dynamic operation instruction, the dynamic operation instruction is compiled in real time to obtain a real-time compilation result as the binary code of the deep learning algorithm.

[0346] Clause A18. The method according to clause A17, wherein when the instruction type is a dynamic operation instruction, the dynamic operation instruction is compiled in real time to obtain a real-time compilation result as the binary code of the deep learning algorithm, including:

[0347] According to the dynamic operation instruction, an original calculation graph and original model data corresponding to the dynamic operation instruction are obtained;

[0348] Performing collaborative processing for the deep learning algorithm based on the original computation graph and the original model data to obtain a first computation graph and first model data;

[0349] Generate hardware instructions and data descriptors according to the first computation graph;

[0350] performing hardware platform-oriented processing on the first model data according to the data descriptor to obtain second model data;

[0351] According to the hardware instructions and the second model data, a binary code of the deep learning algorithm is obtained.

[0352] Clause A19. The method according to clause A18, wherein obtaining, based on the dynamic operation instruction, an original computation graph and original model data corresponding to the dynamic operation instruction comprises:

[0353] Obtaining an original computation graph corresponding to the dynamic operation instruction by parsing the dynamic operation instruction;

[0354] The original model data is obtained according to the parameters of the dynamic operation instruction.

[0355] Clause A20. The method according to clause A18, wherein the collaborative processing for the deep learning algorithm is performed based on the original computation graph and original model data of the deep learning algorithm to obtain the first computation graph and first model data, comprising:

[0356] Reading the original computation graph and the original model data;

[0357] Identifying continuous linear transformation operation nodes within the original computation graph;

[0358] The continuous linear transformation operation nodes are processed through linear transformation and constant folding to obtain a first computation graph and first model data.

[0359] Clause A21. The method of clause A18, wherein generating hardware instructions and data descriptors based on the first computation graph comprises:

[0360] The first computation graph is processed according to the cost model and combined with a heuristic search strategy to obtain hardware instructions and data descriptors.

[0361] Clause A22. The method of clause A18, wherein the processing of the first model data is performed on the first model data in a hardware platform-oriented manner according to the data descriptor to obtain the second model data, comprising:

[0362] According to the data descriptor, and in accordance with the operation requirements in the hardware platform, the first model data is aligned; or,

[0363] According to the data descriptor, and in accordance with the computational requirements within the hardware platform, the first model data is dimensionally transformed; or,

[0364] According to the data descriptor and in accordance with the computational requirements within the hardware platform, precision selection is performed on the first model data.

[0365] Clause A23. The method of clause A18, wherein obtaining a binary code of the deep learning algorithm based on the hardware instructions and the second model data comprises:

[0366] The hardware instructions and the second model data are packaged to obtain a binary code of the deep learning algorithm.

[0367] Item A24. The method according to Item A12 further comprises: executing the binary code of the deep learning algorithm in a deep learning processor through a runtime system.

[0368] Clause A25. The method of clause A1, further comprising:

[0369] The deep learning framework is extended in class, and the tensor data and the operation instructions are encapsulated in the data of the deep learning framework to achieve the integration of the deep learning framework and the deep learning programming library interface.

[0370] Clause A26. A device for generating operation data, comprising:

[0371] A user instruction receiving module, used for receiving user instructions;

[0372] A trigger module is used to trigger the deep learning programming library interface to create or call operation data according to the user instruction, wherein the operation data includes at least one of tensor data and operation instructions.

[0373] Clause A27. The apparatus of clause A26, wherein the triggering module comprises:

[0374] A dynamic operation instruction triggering unit, configured to trigger the deep learning programming library interface to create or call a dynamic operation instruction according to the user instruction; and / or,

[0375] The static operation instruction triggering unit is used to trigger the deep learning programming library interface to create or call static operation instructions according to the user instruction.

[0376] Clause A28. The apparatus according to clause A27, wherein the dynamic operation instruction triggering unit comprises:

[0377] A specialized operation instruction triggering subunit, configured to trigger the deep learning programming library interface to create or call a specialized operation instruction according to the user instruction, wherein the specialized operation instruction is obtained by converting the operation instruction after binding input parameters; and / or,

[0378] The fusion operation instruction triggering subunit is used to trigger the deep learning programming library interface to create or call a fusion operation instruction according to the user instruction, wherein the fusion operation instruction is obtained by combining multiple operation instructions according to the calling order.

[0379] Clause A29. The apparatus of clause A28, wherein the specialized operation instruction triggering subunit is configured to:

[0380] According to the user instruction, trigger the deep learning programming library interface to bind all input parameters to the operation instruction and then convert it to obtain a fully specialized operation instruction; and / or,

[0381] According to the user instruction, trigger the deep learning programming library interface to bind N input parameters to the operation instruction and then convert it to obtain a partially specialized operation instruction, where N is a positive integer less than the number of input parameters of the operation instruction; and / or,

[0382] According to the user instruction, the deep learning programming library interface is triggered to directly convert the operation instruction without binding the input parameters to obtain a pseudo-specialized operation instruction.

[0383] Clause A30. The apparatus of clause A28, wherein the fusion operation instruction triggering subunit is configured to:

[0384] Create a name for the fusion operation instruction;

[0385] Determine a fusion operation sub-instruction according to the operation instruction to be fused;

[0386] Determining the operation connection relationship between the fused operation sub-instructions according to the calling order of the operation instructions to be fused;

[0387] Connecting the fusion operation sub-instructions according to the operation connection relationship to obtain a connection result;

[0388] Setting input parameters and output parameters of the fusion operation instruction according to a user instruction corresponding to the fusion operation instruction;

[0389] The name, connection result, input parameters and output parameters are packaged to obtain the fusion operation instruction.

[0390] Clause A31. The apparatus according to clause A27, wherein the static operation instruction triggering unit comprises:

[0391] A customized operation instruction triggering subunit is used to trigger the deep learning programming library interface to create or call a customized operation instruction according to the user instruction, wherein the customized operation instruction is obtained by encapsulating the user instruction corresponding to the operation instruction according to the interface and data structure definition of the operation instruction; and / or,

[0392] A build-in operation instruction triggering subunit is configured to trigger the deep learning programming library interface to create or call a build-in operation instruction according to the user instruction, wherein the build-in operation instruction is an inherent operation instruction included in the deep learning programming library interface; and / or,

[0393] The offline operation instruction triggering subunit is used to trigger the deep learning programming library interface to create or call offline operation instructions according to the user instructions, wherein the offline operation instructions are pre-compiled dynamic operation instructions, and the pre-compiled results are saved in the offline cache.

[0394] Clause A32. The device according to clause A31, wherein the binary code corresponding to the static operation instruction is stored in a static operation pool of a deep learning programming library, wherein the static operation pool includes a static operation source file, and the static operation source file includes a static code segment, a static data segment, a dynamic code segment, and a dynamic data segment; wherein,

[0395] The static code segment is used to store the binary code corresponding to the build-in operation instruction;

[0396] The dynamic code segment is used to store the binary code corresponding to the customized operation instruction;

[0397] The static data segment is used to store tensor data corresponding to the build-in operation instruction;

[0398] The dynamic data segment is used to store tensor data corresponding to the custom operation instruction.

[0399] Clause A33. The apparatus according to clause A32, wherein the static operation pool further comprises an offline cache, wherein the offline cache comprises an offline file and an index table;

[0400] The offline file is used to store the pre-compiled results of the offline operation instructions;

[0401] The index table is used to indicate the position of the pre-compiled result of the offline operation instruction in the offline file.

[0402] Clause A34. The apparatus of clause A32, wherein the customized operation instruction triggering subunit is further configured to:

[0403] Compiling the customized operation instruction to obtain a compilation result;

[0404] The compilation result is inserted into the static operation pool in a dynamic link or static link manner to obtain a binary code corresponding to the customized operation instruction.

[0405] Clause A35. The apparatus of clause A26, wherein the trigger module is further configured to:

[0406] According to the user instruction, call all corresponding operation instructions one by one; or,

[0407] According to the user instruction, all corresponding operation instructions are merged to obtain a fused operation instruction, and the fused operation instruction is called; or,

[0408] According to the user instruction, all corresponding operation instructions are segmented to obtain segmentation results, each segmentation result is fused respectively to obtain a corresponding segmentation fusion operation instruction, and the segmentation fusion operation instructions are called in sequence.

[0409] Clause A36. An apparatus according to clause A26, wherein the tensor data comprises a shape attribute, a logical data type attribute, a physical data type attribute, and a physical layout attribute.

[0410] Clause A37. The apparatus of clause A26, further coupled to a deep learning programming library architecture, the deep learning programming library architecture comprising:

[0411] An operation instruction reading module, used to read the operation instructions transmitted by the deep learning programming library interface;

[0412] A compilation module is used to determine the instruction type of the operation instruction, and perform a compilation operation corresponding to the instruction type according to the determination result to obtain the binary code of the deep learning algorithm.

[0413] Clause A38. The apparatus of clause A37, wherein the compilation module comprises:

[0414] An operation instruction type determination unit, configured to determine the instruction type of the operation instruction;

[0415] A static search unit is used to search for the corresponding binary code in the static operation pool according to the name of the static operation instruction when the instruction type is a static operation instruction, as the binary code of the deep learning algorithm.

[0416] Clause A39. The apparatus of clause A38, wherein the static lookup unit is configured to:

[0417] According to the name of the static operation instruction, searching the static operation pool for a binary code corresponding to the name;

[0418] When the search result is successful, the binary code is returned as the binary code of the deep learning algorithm.

[0419] Clause A40. The apparatus of clause A39, wherein the static search unit is further configured to:

[0420] When the search result is failure, the static operation instruction is used as a dynamic operation instruction and compiled in real time.

[0421] Clause A41. The apparatus of clause A39, wherein the static lookup unit is further configured to:

[0422] According to the name specified when the static operation instruction is created, in the static operation pool, the binary code corresponding to the custom operation, the binary code corresponding to the build-in operation, and the binary code corresponding to the offline operation are searched in turn to obtain the binary code corresponding to the name.

[0423] Clause A42. The apparatus of clause A38, wherein the compilation module further comprises a dynamic compilation unit configured to:

[0424] When the instruction type is a dynamic operation instruction, the dynamic operation instruction is compiled in real time to obtain a real-time compilation result as the binary code of the deep learning algorithm.

[0425] Clause A43. The apparatus of clause A42, wherein the dynamic compilation unit comprises:

[0426] The original data acquisition subunit is used to obtain the original calculation graph and original model data corresponding to the dynamic operation instruction according to the dynamic operation instruction;

[0427] A collaborative processing subunit performs collaborative processing for the deep learning algorithm based on the original computation graph and the original model data to obtain a first computation graph and first model data;

[0428] A hardware instruction generation subunit generates hardware instructions and data descriptors according to the first computation graph;

[0429] a model data processing subunit, performing hardware platform-oriented processing on the first model data according to the data descriptor to obtain second model data;

[0430] The binary code generation subunit obtains the binary code of the deep learning algorithm according to the hardware instruction and the second model data.

[0431] Clause A44. The apparatus of clause A43, wherein the raw data acquisition subunit is configured to:

[0432] Obtaining an original computation graph corresponding to the dynamic operation instruction by parsing the dynamic operation instruction;

[0433] The original model data is obtained according to the parameters of the dynamic operation instruction.

[0434] Clause A45. The apparatus of clause A43, wherein the co-processing subunit is configured to:

[0435] Reading the original computation graph and the original model data;

[0436] Identifying continuous linear transformation operation nodes within the original computation graph;

[0437] The continuous linear transformation operation nodes are processed through linear transformation and constant folding to obtain a first computation graph and first model data.

[0438] Clause A46. The apparatus of clause A43, wherein the hardware instruction generation subunit is configured to:

[0439] The first computation graph is processed according to the cost model and combined with a heuristic search strategy to obtain hardware instructions and data descriptors.

[0440] Clause A47. The apparatus of clause A43, wherein the model data processing subunit is configured to:

[0441] According to the data descriptor, and in accordance with the operation requirements in the hardware platform, the first model data is aligned; or,

[0442] According to the data descriptor, and in accordance with the computational requirements within the hardware platform, the first model data is dimensionally transformed; or,

[0443] According to the data descriptor and in accordance with the computational requirements within the hardware platform, precision selection is performed on the first model data.

[0444] Clause A48. The apparatus of clause A43, wherein the binary code generation subunit is configured to:

[0445] The hardware instructions and the second model data are packaged to obtain a binary code of the deep learning algorithm.

[0446] Item A49. The apparatus according to Item A37 is further configured to execute the binary code of the deep learning algorithm in a deep learning processor through a runtime system.

[0447] Clause A50. The apparatus of clause A26, further comprising:

[0448] The deep learning framework is extended in class, and the tensor data and the operation instructions are encapsulated in the data of the deep learning framework to achieve the integration of the deep learning framework and the deep learning programming library interface.

[0449] Item A51. A deep learning computing device, comprising one or more operation data generating devices as described in any one of Items A26-A50, and the deep learning computing device is used to complete a set deep learning operation.

[0450] Item A52. A combined computing device comprising one or more deep learning computing devices as described in Item A51, a universal interconnect interface, and other processing devices;

[0451] The deep learning computing device interacts with the other processing devices to jointly complete the computing operations specified by the user.

[0452] Clause A53. A deep learning chip, comprising:

[0453] An operating data generating device as described in any of clauses A26-A50; or

[0454] A deep learning computing device as described in clause A51; or

[0455] A combined operation device as described in clause A52.

[0456] Clause A54. An electronic device, comprising:

[0457] An operating data generating device as described in any of clauses A26-A50; or

[0458] A deep learning computing device as described in clause A51; or

[0459] A combined operation device as described in clause A52; or

[0460] A deep learning chip as described in clause A53.

[0461] The embodiments of the present disclosure are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core ideas of the present disclosure. At the same time, changes or modifications made by those skilled in the art based on the ideas of the present disclosure, on the specific implementation methods and application scope of the present disclosure, all fall within the scope of protection of the present disclosure. In summary, the contents of this specification should not be understood as limiting the present disclosure.

Claims

1. A method for generating operation data, characterized in that: The method comprises: Receive user instructions; According to the user instruction, trigger the deep learning programming library interface to create or call operation data, wherein the operation data includes at least one of tensor data and operation instructions; The method further includes: the deep learning programming library architecture reading the operation instruction transmitted by the deep learning programming library interface; the deep learning programming library architecture determining the instruction type of the operation instruction, and performing a compile operation corresponding to the instruction type according to the determination result to obtain the binary code of the deep learning algorithm; The method of determining the instruction type of the operation instruction and performing a compilation operation corresponding to the instruction type according to the determination result to obtain the binary code of the deep learning algorithm includes: determining the instruction type of the operation instruction; when the instruction type is a dynamic operation instruction, performing real-time compilation on the dynamic operation instruction to obtain a real-time compilation result as the binary code of the deep learning algorithm; Among them, when the instruction type is a dynamic operation instruction, the dynamic operation instruction is compiled in real time to obtain a real-time compilation result as the binary code of the deep learning algorithm, including: according to the dynamic operation instruction, obtaining the original calculation graph and original model data corresponding to the dynamic operation instruction; according to the original calculation graph and the original model data, performing collaborative processing for the deep learning algorithm to obtain a first calculation graph and first model data; according to the first calculation graph, generating hardware instructions and data descriptors; according to the data descriptor, performing hardware platform-oriented processing on the first model data to obtain second model data; according to the hardware instructions and the second model data, obtaining the binary code of the deep learning algorithm, and the binary code is a hardware instruction for guiding the hardware device to execute the deep learning algorithm.

2. The method according to claim 1, characterized in that According to the user instructions, trigger the deep learning programming library interface to create or call operation instructions, including: According to the user instruction, trigger the deep learning programming library interface to create or call dynamic operation instructions; and / or, According to the user instructions, the deep learning programming library interface is triggered to create or call static operation instructions.

3. The method according to claim 2, characterized in that The triggering of the deep learning programming library interface to create or call a dynamic operation instruction according to the user instruction includes: According to the user instruction, trigger the deep learning programming library interface to create or call a specialized operation instruction, wherein the specialized operation instruction is obtained by binding input parameters to the operation instruction and then converting it; and / or, According to the user instruction, the deep learning programming library interface is triggered to create or call a fusion operation instruction, wherein the fusion operation instruction is obtained by combining multiple operation instructions according to the calling order.

4. The method according to claim 3, characterized in that According to the user instruction, triggering the deep learning programming library interface to create the specialized operation instruction includes: According to the user instruction, trigger the deep learning programming library interface to bind all input parameters to the operation instruction and then convert it to obtain a fully specialized operation instruction; and / or, According to the user instruction, trigger the deep learning programming library interface to bind N input parameters to the operation instruction and then convert it to obtain a partially specialized operation instruction, where N is a positive integer less than the number of input parameters of the operation instruction; and / or, According to the user instruction, the deep learning programming library interface is triggered to directly convert the operation instruction without binding the input parameters to obtain a pseudo-specialized operation instruction.

5. The method according to claim 3, characterized in that Triggering creation of the fusion operation instruction according to the user instruction includes: Create a name for the fusion operation instruction; Determine a fusion operation sub-instruction according to the operation instruction to be fused; Determining the operation connection relationship between the fused operation sub-instructions according to the calling order of the operation instructions to be fused; Connecting the fusion operation sub-instructions according to the operation connection relationship to obtain a connection result; Setting input parameters and output parameters of the fusion operation instruction according to a user instruction corresponding to the fusion operation instruction; The name, connection result, input parameters and output parameters are packaged to obtain the fusion operation instruction.

6. The method according to claim 2, characterized in that The triggering of the deep learning programming library interface to create or call a static operation instruction according to the user instruction includes: According to the user instruction, trigger the deep learning programming library interface to create or call a custom operation instruction, wherein the custom operation instruction is obtained by encapsulating the user instruction corresponding to the operation instruction according to the interface and data structure definition of the operation instruction; and / or, According to the user instruction, trigger the deep learning programming library interface to create or call a build-in operation instruction, wherein the build-in operation instruction is a built-in operation instruction included in the deep learning programming library interface; and / or, According to the user instruction, the deep learning programming library interface is triggered to create or call an offline operation instruction, wherein the offline operation instruction is a pre-compiled dynamic operation instruction, and the pre-compiled result is stored in the offline cache.

7. The method according to claim 6, characterized in that The binary code corresponding to the static operation instruction is stored in the static operation pool of the deep learning programming library, and the static operation pool includes a static operation source file, and the static operation source file includes a static code segment, a static data segment, a dynamic code segment, and a dynamic data segment; wherein, The static code segment is used to store the binary code corresponding to the build-in operation instruction; The dynamic code segment is used to store the binary code corresponding to the customized operation instruction; The static data segment is used to store tensor data corresponding to the build-in operation instruction; The dynamic data segment is used to store tensor data corresponding to the custom operation instruction.

8. The method according to claim 7, characterized in that The static operation pool also includes an offline cache, which includes offline files and index tables; wherein, The offline file is used to store the pre-compiled results of the offline operation instructions; The index table is used to indicate the position of the pre-compiled result of the offline operation instruction in the offline file.

9. The method according to claim 7, characterized in that The triggering of the deep learning programming library interface to create or call a custom operation instruction according to the user instruction further includes: Compiling the customized operation instruction to obtain a compilation result; The compilation result is inserted into the static operation pool in a dynamic link or static link manner to obtain a binary code corresponding to the customized operation instruction.

10. The method according to claim 1, characterized in that According to the user instruction, triggering the calling of the operation instruction in the deep learning programming library interface includes: According to the user instruction, call all corresponding operation instructions one by one; or, According to the user instruction, all corresponding operation instructions are merged to obtain a fused operation instruction, and the fused operation instruction is called; or, According to the user instruction, all corresponding operation instructions are segmented to obtain segmentation results, each segmentation result is fused respectively to obtain a corresponding segmentation fusion operation instruction, and the segmentation fusion operation instructions are called in sequence.

11. The method according to claim 1, wherein The tensor data includes shape attributes, logical data type attributes, physical data type attributes, and physical layout attributes.

12. The method according to claim 1, characterized in that The step of determining the instruction type of the operation instruction and performing a compile operation corresponding to the instruction type according to the determination result to obtain a binary code of the deep learning algorithm further includes: When the instruction type is a static operation instruction, the corresponding binary code is searched in the static operation pool according to the name of the static operation instruction, and is used as the binary code of the deep learning algorithm.

13. The method according to claim 12, characterized in that When the instruction type is a static operation instruction, searching for a corresponding binary code in a static operation pool according to the name of the static operation instruction as the binary code of the deep learning algorithm includes: According to the name of the static operation instruction, searching the static operation pool for a binary code corresponding to the name; When the search result is successful, the binary code is returned as the binary code of the deep learning algorithm.

14. The method according to claim 13, characterized in that When the instruction type is a static operation instruction, searching for a corresponding binary code in a static operation pool according to the name of the static operation instruction as the binary code of the deep learning algorithm further includes: When the search result is failure, the static operation instruction is used as a dynamic operation instruction and compiled in real time.

15. The method according to claim 13, characterized in that The step of searching, according to the name of the static operation instruction, for a binary code corresponding to the name in the static operation pool includes: According to the name specified when the static operation instruction is created, in the static operation pool, the binary code corresponding to the custom operation, the binary code corresponding to the build-in operation, and the binary code corresponding to the offline operation are searched in turn to obtain the binary code corresponding to the name.

16. The method according to claim 1, wherein The step of obtaining, according to the dynamic operation instruction, an original calculation graph and original model data corresponding to the dynamic operation instruction includes: Obtaining an original computation graph corresponding to the dynamic operation instruction by parsing the dynamic operation instruction; The original model data is obtained according to the parameters of the dynamic operation instruction.

17. The method according to claim 1, wherein The step of performing collaborative processing for the deep learning algorithm based on the original computation graph and the original model data to obtain a first computation graph and first model data includes: Reading the original computation graph and the original model data; Identifying continuous linear transformation operation nodes within the original computation graph; The continuous linear transformation operation nodes are processed through linear transformation and constant folding to obtain a first computation graph and first model data.

18. The method according to claim 1, wherein Generating hardware instructions and data descriptors according to the first computation graph includes: The first computation graph is processed according to the cost model and combined with a heuristic search strategy to obtain hardware instructions and data descriptors.

19. The method according to claim 1, wherein According to the data descriptor, the first model data is processed in a hardware platform-oriented manner to obtain second model data, including: According to the data descriptor, and in accordance with the operation requirements in the hardware platform, the first model data is aligned; or, According to the data descriptor, and in accordance with the computational requirements within the hardware platform, the first model data is dimensionally transformed; or, According to the data descriptor and in accordance with the computational requirements within the hardware platform, precision selection is performed on the first model data.

20. The method according to claim 1, wherein Obtaining a binary code of the deep learning algorithm according to the hardware instruction and the second model data includes: The hardware instructions and the second model data are packaged to obtain a binary code of the deep learning algorithm.

21. The method according to claim 1, wherein The method further includes executing the binary code of the deep learning algorithm in a deep learning processor through a runtime system.

22. The method according to claim 1, wherein The method further comprises: The deep learning framework is extended in class, and the tensor data and the operation instructions are encapsulated in the data of the deep learning framework to achieve the integration of the deep learning framework and the deep learning programming library interface.

23. A device for generating operation data, characterized in that: include: A user instruction receiving module, used for receiving user instructions; a trigger module, configured to trigger the deep learning programming library interface to create or call operation data according to the user instruction, wherein the operation data includes at least one of tensor data and operation instructions; The device is also connected to a deep learning programming library architecture, which includes: an operation instruction reading module for reading the operation instruction transmitted by the deep learning programming library interface; a compilation module for determining the instruction type of the operation instruction and performing a compilation operation corresponding to the instruction type based on the determination result to obtain a binary code of the deep learning algorithm; The compilation module includes: an operation instruction type determination unit for determining the instruction type of the operation instruction; a dynamic compilation unit for performing real-time compilation of the dynamic operation instruction when the instruction type is a dynamic operation instruction, and obtaining a real-time compilation result as the binary code of the deep learning algorithm; Among them, the dynamic compilation unit includes: an original data acquisition subunit, which is used to obtain the original calculation graph and original model data corresponding to the dynamic operation instruction according to the dynamic operation instruction; a collaborative processing subunit, which performs collaborative processing for the deep learning algorithm according to the original calculation graph and the original model data to obtain a first calculation graph and first model data; a hardware instruction generation subunit, which generates hardware instructions and data descriptors according to the first calculation graph; a model data processing subunit, which performs hardware platform-oriented processing on the first model data according to the data descriptor to obtain second model data; a binary code generation subunit, which obtains the binary code of the deep learning algorithm according to the hardware instructions and the second model data, and the binary code is a hardware instruction used to guide the hardware device to execute the deep learning algorithm.

24. The device according to claim 23, characterized in that The trigger module includes: A dynamic operation instruction triggering unit, configured to trigger the deep learning programming library interface to create or call a dynamic operation instruction according to the user instruction; and / or, The static operation instruction triggering unit is used to trigger the deep learning programming library interface to create or call static operation instructions according to the user instruction.

25. The device according to claim 24, characterized in that The dynamic operation instruction triggering unit includes: A specialized operation instruction triggering subunit, configured to trigger the deep learning programming library interface to create or call a specialized operation instruction according to the user instruction, wherein the specialized operation instruction is obtained by converting the operation instruction after binding input parameters; and / or, The fusion operation instruction triggering subunit is used to trigger the deep learning programming library interface to create or call a fusion operation instruction according to the user instruction, wherein the fusion operation instruction is obtained by combining multiple operation instructions according to the calling order.

26. The device according to claim 25, characterized in that The special operation instruction triggering subunit is used to: According to the user instruction, trigger the deep learning programming library interface to bind all input parameters to the operation instruction and then convert it to obtain a fully specialized operation instruction; and / or, According to the user instruction, trigger the deep learning programming library interface to bind N input parameters to the operation instruction and then convert it to obtain a partially specialized operation instruction, where N is a positive integer less than the number of input parameters of the operation instruction; and / or, According to the user instruction, the deep learning programming library interface is triggered to directly convert the operation instruction without binding the input parameters to obtain a pseudo-specialized operation instruction.

27. The device according to claim 25, characterized in that The fusion operation instruction triggering subunit is used to: Create a name for the fusion operation instruction; Determine a fusion operation sub-instruction according to the operation instruction to be fused; Determining the operation connection relationship between the fused operation sub-instructions according to the calling order of the operation instructions to be fused; Connecting the fusion operation sub-instructions according to the operation connection relationship to obtain a connection result; Setting input parameters and output parameters of the fusion operation instruction according to a user instruction corresponding to the fusion operation instruction; The name, connection result, input parameters and output parameters are packaged to obtain the fusion operation instruction.

28. The device according to claim 24, characterized in that The static operation instruction triggering unit includes: A customized operation instruction triggering subunit is used to trigger the deep learning programming library interface to create or call a customized operation instruction according to the user instruction, wherein the customized operation instruction is obtained by encapsulating the user instruction corresponding to the operation instruction according to the interface and data structure definition of the operation instruction; and / or, A build-in operation instruction triggering subunit is configured to trigger the deep learning programming library interface to create or call a build-in operation instruction according to the user instruction, wherein the build-in operation instruction is an inherent operation instruction included in the deep learning programming library interface; and / or, The offline operation instruction triggering subunit is used to trigger the deep learning programming library interface to create or call offline operation instructions according to the user instructions, wherein the offline operation instructions are pre-compiled dynamic operation instructions, and the pre-compiled results are saved in the offline cache.

29. The device according to claim 28, characterized in that The binary code corresponding to the static operation instruction is stored in the static operation pool of the deep learning programming library, and the static operation pool includes a static operation source file, and the static operation source file includes a static code segment, a static data segment, a dynamic code segment, and a dynamic data segment; wherein, The static code segment is used to store the binary code corresponding to the build-in operation instruction; The dynamic code segment is used to store the binary code corresponding to the customized operation instruction; The static data segment is used to store tensor data corresponding to the build-in operation instruction; The dynamic data segment is used to store tensor data corresponding to the custom operation instruction.

30. The device according to claim 29, characterized in that The static operation pool also includes an offline cache, which includes offline files and index tables; wherein, The offline file is used to store the pre-compiled results of the offline operation instructions; The index table is used to indicate the position of the pre-compiled result of the offline operation instruction in the offline file.

31. The device according to claim 29, characterized in that The customized operation instruction triggering subunit is further used for: Compiling the customized operation instruction to obtain a compilation result; The compilation result is inserted into the static operation pool in a dynamic link or static link manner to obtain a binary code corresponding to the customized operation instruction.

32. The device according to claim 23, characterized in that The trigger module is further configured to: According to the user instruction, call all corresponding operation instructions one by one; or, According to the user instruction, all corresponding operation instructions are merged to obtain a fused operation instruction, and the fused operation instruction is called; or, According to the user instruction, all corresponding operation instructions are segmented to obtain segmentation results, each segmentation result is fused respectively to obtain a corresponding segmentation fusion operation instruction, and the segmentation fusion operation instructions are called in sequence.

33. The device according to claim 23, characterized in that The tensor data includes shape attributes, logical data type attributes, physical data type attributes, and physical layout attributes.

34. The device according to claim 23, wherein The compilation module also includes: A static search unit is used to search for the corresponding binary code in the static operation pool according to the name of the static operation instruction when the instruction type is a static operation instruction, as the binary code of the deep learning algorithm.

35. The device according to claim 34, characterized in that The static search unit is used to: According to the name of the static operation instruction, searching the static operation pool for a binary code corresponding to the name; When the search result is successful, the binary code is returned as the binary code of the deep learning algorithm.

36. The device according to claim 35, characterized in that The static search unit is further configured to: When the search result is failure, the static operation instruction is used as a dynamic operation instruction and compiled in real time.

37. The device according to claim 35, characterized in that The static search unit is further configured to: According to the name specified when the static operation instruction is created, in the static operation pool, the binary code corresponding to the custom operation, the binary code corresponding to the build-in operation, and the binary code corresponding to the offline operation are searched in turn to obtain the binary code corresponding to the name.

38. The device according to claim 23, characterized in that The raw data acquisition subunit is used for: Obtaining an original computation graph corresponding to the dynamic operation instruction by parsing the dynamic operation instruction; The original model data is obtained according to the parameters of the dynamic operation instruction.

39. The device according to claim 23, characterized in that The collaborative processing subunit is used for: Reading the original computation graph and the original model data; Identifying continuous linear transformation operation nodes within the original computation graph; The continuous linear transformation operation nodes are processed through linear transformation and constant folding to obtain a first computation graph and first model data.

40. The device according to claim 23, wherein The hardware instruction generation subunit is used for: The first computation graph is processed according to the cost model and combined with a heuristic search strategy to obtain hardware instructions and data descriptors.

41. The device according to claim 23, wherein The model data processing subunit is used for: According to the data descriptor, and in accordance with the operation requirements in the hardware platform, the first model data is aligned; or, According to the data descriptor, and in accordance with the computational requirements within the hardware platform, the first model data is dimensionally transformed; or, According to the data descriptor and in accordance with the computational requirements within the hardware platform, precision selection is performed on the first model data.

42. The device according to claim 23, characterized in that The binary code generating subunit is used for: The hardware instructions and the second model data are packaged to obtain a binary code of the deep learning algorithm.

43. The device according to claim 23, characterized in that The device is also used to: execute the binary code of the deep learning algorithm in the deep learning processor through the runtime system.

44. The device according to claim 23, characterized in that The device is also used for: The deep learning framework is extended in class, and the tensor data and the operation instructions are encapsulated in the data of the deep learning framework to achieve the integration of the deep learning framework and the deep learning programming library interface.

45. A deep learning computing device, characterized in that The deep learning operation device includes one or more operation data generating devices as described in any one of claims 23-44, and the deep learning operation device is used to complete the set deep learning operation.

46. ​​A combined operation device, characterized in that The combined operation device includes one or more deep learning operation devices as claimed in claim 45, a universal interconnection interface and other processing devices; The deep learning computing device interacts with the other processing devices to jointly complete the computing operations specified by the user.

47. A deep learning chip, characterized in that The deep learning chip includes: The operation data generating device according to any one of claims 23 to 44; or The deep learning computing device according to claim 45; or A combinatorial operation device as claimed in claim 46.

48. An electronic device, characterized in that The electronic device comprises: The operation data generating device according to any one of claims 23 to 44; or The deep learning computing device according to claim 45; or The combinatorial computing device as claimed in claim 46; or the deep learning chip as claimed in claim 47.

Images