A method for creating and executing deep learning model representations

Abstract

This invention discloses a method for creating a deep learning model representation, comprising the steps of: creating one or more node entries, each node entry corresponding to a processing node in the deep learning model, wherein for each node entry: creating one or more input entries, each input entry representing a processing node that sends data to its corresponding processing node for processing; creating operator entries, representing the operations performed in the corresponding processing node; creating a first attribute entry, representing the output data type and / or output data format of the corresponding processing node; and creating a second attribute entry, representing the various processing parameters required for the processing performed by the corresponding processing node. This invention also discloses a method for performing deep learning model operations using the created deep learning model representation, a data processing device using these methods, and a computing device.

Description

Technical Field

[0001] This invention relates to the field of machine learning, and more particularly to the field of deep learning and the description of deep learning models. Background Technology

[0002] With the rapid development of artificial intelligence in recent years, artificial intelligence, especially deep learning algorithms, has been widely used in fields such as visual image, speech recognition and natural language processing.

[0003] To facilitate the development of machine learning models, especially deep learning models, various deep learning frameworks have emerged (Google's TensorFlow, Facebook's pyTorch, Apache MXNet, Caffe developed by Berkeley AI Research Center and community contributors, etc.). These deep learning frameworks provide various deep learning operator libraries. This allows developers to easily construct deep learning models using pre-provided operator libraries. The constructed deep learning models can be represented by human-readable or machine-interpretable model representations (IR) and can be easily transferred between different machines, thus significantly simplifying the deep learning model development process.

[0004] Furthermore, the increasing computational power required for artificial intelligence has led to large-scale deep learning model computation scenarios. These scenarios demand substantial computing resources, prompting the development of acceleration units such as neural network processors (NPUs) to speed up computations for artificial intelligence, particularly deep learning models.

[0005] Since various deep learning frameworks have not developed acceleration units specifically for them, when using acceleration units to load and execute deep learning models created by the front-ends of various deep learning frameworks, adaptive modifications to the machine learning frameworks are required. When the acceleration unit needs to support a large number of deep learning frameworks, this consumes a significant amount of time and manpower, thus hindering the efficient use of the acceleration unit.

[0006] Therefore, a new model representation for deep learning models is needed that can be adapted to both deep learning frameworks and hardware acceleration units, thereby facilitating the efficient use of hardware acceleration units to perform deep learning model operations. Summary of the Invention

[0007] Therefore, the present invention provides a method and computing system for creating and executing representations of deep learning models, in an attempt to solve or at least alleviate at least one of the problems mentioned above.

[0008] According to one aspect of the present invention, a method for creating a representation of a deep learning model is provided. The deep learning model includes multiple processing nodes. The method includes the steps of: creating one or more node entries, each node entry corresponding to a processing node in the deep learning model, wherein for each node entry: creating one or more input entries, each input entry representing a processing node that sends data to its corresponding processing node for processing; creating operator entries representing the operations performed in the corresponding processing node; creating a first attribute entry representing the output data type and / or output data format of the corresponding processing node; and creating a second attribute entry representing various processing parameters required for the processing performed by the corresponding processing node.

[0009] Optionally, the method according to the invention further includes the step of: pre-setting an operator list. Each operator in the list corresponds to an operation suitable for execution in the acceleration unit, and each operator has one or more attributes. The step of creating a second attribute entry further includes: if the operation to be executed in the corresponding processing node is suitable for loading into the acceleration unit for execution, then creating a second attribute entry by referring to the attributes of the corresponding operator in the operator list.

[0010] Optionally, in the method according to the invention, the step of creating a second attribute entry by referring to the attribute of the corresponding operator includes: if the attribute is defined in the processing parameters of the corresponding processing node, then setting the attribute value of the created second attribute entry to the parameter value; and if the attribute is not defined in the processing parameters of the corresponding processing node, then setting the attribute value of the created second attribute entry to a default value.

[0011] Optionally, in the method according to the invention, the step of creating the second attribute entry further includes: if the operation performed in the corresponding processing node is not suitable for loading into the acceleration unit for execution, then the second attribute entry is created with reference to the processing parameters already provided by the operation.

[0012] Optionally, in the method according to the invention, each node entry has a corresponding node name, and the step of creating an input entry includes: setting the name of the input entry to the node name of the node entry representing the processing node, so as to construct a graph representing the deep learning model based on each node entry and the corresponding input entry.

[0013] Optionally, in the method according to the invention, the first attribute entry also characterizes the input data type and / or input data format of the corresponding processing node.

[0014] Optionally, in the method according to the invention, the first attribute entry further characterizes the quantized data format of the input data of the corresponding processing node.

[0015] Optionally, the method according to the invention further includes the steps of: generating a name for a deep learning model representation; and generating version information of the model representation to indicate the model framework that generated the deep learning model.

[0016] According to another aspect of the present invention, a method for creating a deep learning model representation is provided, comprising the steps of: obtaining a first deep learning model representation created by a model framework for a deep learning model; analyzing the first deep learning model representation to obtain processing node information of the deep learning model; and processing the processing node information using the method according to the present invention to create a second deep learning model representation. The second deep learning model representation is suitable for processing in an acceleration unit.

[0017] According to another aspect of the present invention, a method for executing a deep learning model is provided, comprising the steps of: acquiring node entries; acquiring output data generated by the processing nodes corresponding to each input entry of the node entries as input data; determining processing parameters according to second attribute entries in the node entries; and performing processing on the acquired input data according to the determined processing parameters and the processing determined according to the operator entries in the node entries.

[0018] According to another aspect of the present invention, a data processing apparatus is provided, including an acceleration unit and a processor. The acceleration unit includes one or more acceleration unit cores, each acceleration unit core having a corresponding on-chip memory. The processor is adapted to perform the method according to the present invention to create a deep learning model representation, and to utilize the acceleration unit to execute a deep learning model corresponding to the created deep learning model representation.

[0019] According to another aspect of the invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing any of the methods described above.

[0020] According to the present invention, each processing node in the deep learning model is converted into a corresponding node entry, and sub-entries representing the operation method, input data source, input and output data type and format, and processing parameters required for the operation are set in the node entry. This creates a model representation (intermediate representation, IR) that can uniformly represent the deep learning model table, thereby representing different deep learning models created using various deep learning frameworks as a consistent model representation, simplifying the subsequent execution process for the deep learning model.

[0021] Furthermore, according to the present invention, the model representation is created by referencing various operations that the hardware acceleration unit can support. This allows for the differentiation between operations that the hardware acceleration unit can and cannot support within the created model representation, and the addition of hardware acceleration unit-specific parameters for the supported operations. When this model representation is executed on a computing device equipped with a hardware acceleration unit, it can be easily loaded into the hardware acceleration unit for processing without requiring modification to the model representation or learning framework, thereby improving the efficiency of the hardware acceleration unit.

[0022] Furthermore, according to the present invention, when creating each node entry of the model representation, a first attribute entry characterizing the output data type and / or format of that node is created. Thus, in the model graph representation established by interrelation of input entries in each node entry, the data transmitted between processing nodes has a defined type and format, thereby providing an accurate definition of the deep learning model.

[0023] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0024] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of this disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout this disclosure, the same reference numerals generally refer to the same parts or elements.

[0025] Figure 1A A schematic diagram of an example computer system according to an embodiment of the present invention is shown;

[0026] Figure 1B A schematic diagram of a deep neural network as a machine learning model according to an embodiment of the present invention is shown;

[0027] Figure 2A An internal structural diagram of a computing device according to an embodiment of the present invention is shown;

[0028] Figure 2B A connection diagram of a scheduling unit and an acceleration unit inside a computing device according to an embodiment of the present invention is shown;

[0029] Figure 3An internal structural diagram of an acceleration unit core according to an embodiment of the present invention is shown;

[0030] Figure 4A A schematic diagram illustrating the deployment and execution of a deep learning model in a computing device according to an embodiment of the present invention is shown.

[0031] Figure 4B A schematic diagram illustrating the deep learning model representation transformation and execution in a computing device according to an embodiment of the present invention is shown.

[0032] Figure 5A A schematic diagram of the structure of a deep learning model representation according to an embodiment of the present invention is shown;

[0033] Figure 5B A schematic diagram illustrating the transformation and execution of deep learning pattern representation according to an embodiment of the present invention is shown;

[0034] Figure 6 A schematic diagram of a method for creating a deep learning model representation according to an embodiment of the present invention is shown;

[0035] Figure 7 A schematic diagram illustrating an example of a deep learning model according to an embodiment of the present invention is shown; and

[0036] Figure 8 A method for executing a deep learning model according to an embodiment of the present invention is illustrated. Detailed Implementation

[0037] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0038] Figure 1A A block diagram of an example computing system 9100 according to an example embodiment of the present disclosure is depicted. System 9100 includes a user computing device 9110, a server computing system 9130, and a training computing system 9150 communicatively coupled via a network 9180.

[0039] User computing device 9110 can be any type of computing device, including but not limited to personal computing devices (e.g., laptops or desktop computers), mobile computing devices (smartphones or tablets), game consoles or controllers, wearable computing devices, embedded computing devices, edge computing devices, or any other type of computing device. User computing device 9110 can be deployed as an end-smart device at the user's location and interact with the user to process user input.

[0040] User computing device 9110 may store or include one or more machine learning models 9120. Machine learning models 9120 may be designed to perform various tasks, such as image classification, object detection, speech recognition, machine translation, content filtering, etc. Machine learning models 9120 may be neural networks (e.g., deep neural networks) or other types of machine learning models, including nonlinear and / or linear models. Examples of machine learning models 9120 include, but are not limited to, various types of deep neural networks (DNNs), such as feedforward neural networks, recurrent neural networks (RNNs, e.g., Long Short-Term Memory Recurrent Neural Networks (LSTM), Transformer neural networks with or without attention mechanisms), convolutional neural networks (CNNs), or other forms of neural networks. Machine learning model 9120 may include one machine learning model or may be a combination of multiple machine learning models.

[0041] Figure 1B The diagram illustrates a deep learning model, namely a neural network, as a machine learning model 9120 according to some implementations. The neural network has a hierarchical architecture, with each network layer having one or more processing nodes (called neurons or filters) for processing. In a deep neural network, the output of the previous layer after processing becomes the input of the next layer, where the first layer in the architecture receives network input for processing, and the output of the last layer is provided as the network output. Figure 1B As shown, the machine learning model 9120 includes network layers 9122, 9124, 9126, etc., where network layer 9124 receives network input and network layer 9126 provides network output.

[0042] In deep neural networks, the main processing operations within the network are interwoven linear and nonlinear transformations. These processes are distributed across various processing nodes. Figure 1B A magnified view of node 9121 in model 9120 is also shown. Node 9121 receives multiple input values ​​a1, a2, a3, etc., and processes the input values ​​based on corresponding processing parameters (such as weights w1, w2, w3, etc.) to generate output z. Node 171 can be designed to process the input using an activation function, which can be represented as:

[0043] z = σ(w) T α+b) (1)

[0044] Where α∈R N This represents the input vector of node 9121 (which includes elements a1, a2, a3, etc.); w∈R N This represents the weight vector (including elements w1, w2, w3, etc.) in the processing parameters used by node 9121, where each weight is used to weight the corresponding input; N represents the number of input values; b∈R N This represents the bias vector (including elements b1, b2, b3, etc.) in the processing parameters used by node 9121. Each bias is used to bias the corresponding input and the weighted result. σ() represents the activation function used by node 9121, which can be a linear or nonlinear function. Commonly used activation functions in neural networks include the sigmoid function, ReLU function, tanh function, maxout function, etc. The output of node 9121 can also be called the activation value. Depending on the network design, the output (i.e., the activation value) of each network layer can be provided as input to one, several, or all nodes of the next layer.

[0045] Each network layer in the machine learning model 9120 may include one or more nodes 9121. When the processing in the machine learning model 9121 is viewed in terms of network layers, the processing of each network layer can also be represented in the form of formula (1) or formula (2), where a represents the input vector of the network layer and w represents the weight of the network layer.

[0046] It should be understood that Figure 1B The architecture of the machine learning model shown, along with the number of network layers and processing nodes, is illustrative. In different applications, the machine learning model can be designed with other architectures as needed.

[0047] Continue to refer to Figure 1A In some implementations, the user computing device 9110 can receive the machine learning model 9120 from the server computing system 130 via the network 9180, store it in the memory of the user computing device, and use or implement it by an application in the user computing device.

[0048] In other implementations, user computing device 9110 may invoke machine learning module 9140 stored and implemented in server computing system 9130. For example, machine learning model 9140 may be implemented by server computing system 9130 as part of a web service, so user computing device 9110 may invoke machine learning model 9140 as a web service implementation, for example, via network 9180 and according to a client-server relationship. Therefore, machine learning modules that can be used at user computing device 102 include machine learning model 9120 stored and implemented at user computing device 9110 and / or machine learning model 9140 stored and implemented at server computing system 9130.

[0049] Server computing system 9130 may include one or more server computing devices. When server computing system 9130 includes multiple server computing devices, these server computing devices may operate according to sequential computing architecture, parallel computing architecture, or some combination thereof.

[0050] As described above, the server computing system 9130 can store or include one or more machine learning models 9140. Similar to machine learning model 9120, machine learning model 9140 can be designed to perform various tasks, such as image classification, object detection, speech recognition, machine translation, content filtering, and so on. Model 9140 can include various machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.

[0051] User computing device 9110 and / or server computing system 9130 can train models 9120 and / or 9140 via interaction with training computing system 9150, which is communicatively coupled via network 9180. Training computing system 9150 may be separate from server computing system 9130, or it may be part of server computing system 9130.

[0052] Similar to server computing system 9130, training computing system 9150 may include one or more server computing devices or be implemented by one or more server computing devices in other ways.

[0053] The training computing system 9150 may include a model trainer 9160 that trains machine learning models 9120 and / or 9140 stored at user computing device 9110 and / or server computing system 9130 using various training or learning techniques, such as, for example, backpropagation of error. In some implementations, performing backpropagation of error may include performing truncated backpropagation through time. The model trainer 9160 may perform various generalization techniques (e.g., weight decay, loss of weight, etc.) to improve the generalization ability of the model being trained.

[0054] Specifically, model trainer 9160 can train machine learning models 9120 and / or 9140 based on a set of training data 9162. Training data 9162 may include multiple different training data sets, each of which, for example, helps train machine learning models 9120 and / or 9140 to perform multiple different tasks. For example, training data sets may include datasets that help machine learning models 9120 and / or 9140 perform object detection, object recognition, object segmentation, image classification, and / or other tasks.

[0055] Additionally, in some implementations, the model trainer 9160 can modify the machine learning model 9140 in the server computing system 9130 to obtain a machine learning model 9120 suitable for use in the user computing device 9110. These modifications include, for example, reducing the number of various parameters in the model, storing parameter values ​​with lower precision, so that the trained machine learning models 9120 and / or 9140 are suitable for operating considering the different processing performance of the server computing system 9130 and the user computing device 9110.

[0056] Model trainer 9160 includes computer logic for providing the desired functionality. Model trainer 9160 can be implemented using hardware, firmware, and / or software that controls a general-purpose processor. For example, in some implementations, model trainer 9160 includes a program file stored on a storage device, loaded into memory, and executed by one or more processors. In other implementations, model trainer 9160 includes a set of one or more computer-executable instructions stored in a tangible computer-readable storage medium such as RAM, a hard disk, or an optical or magnetic medium. In some implementations, model trainer 9160 can be copied and / or distributed across multiple different devices.

[0057] Network 9180 can be any type of communication network, such as a local area network (e.g., intranet), a wide area network (e.g., the Internet), or a combination thereof, and can include any number of wired or wireless links. Typically, communication over Network 9180 can be carried via any type of wired and / or wireless connection, using various communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML, and JSON), and / or protection schemes (e.g., VPN, HTTPS, SSL).

[0058] Figure 1A The user computing device 9110, server computing system 9130, and training computing system 9150 in the example computing system 9100 shown can all be implemented using the computing device 9200 described below. Figure 2A A schematic diagram of a computing device 200 according to an embodiment of the present invention is shown.

[0059] Figure 2A A structural block diagram of the internal structure of a computing device 200 according to an embodiment of the present disclosure is shown. The computing device 200 includes a bus-connected memory 210, a scheduling unit cluster 270, and an acceleration unit cluster 280. The scheduling unit cluster 270 includes a plurality of scheduling units 220. The acceleration unit cluster 280 includes a plurality of acceleration units 230. In this embodiment of the present disclosure, the acceleration units are specialized processing units designed to accelerate the computational processing speed of neural network models, and can be embodied in processing units specifically designed for neural network operations (NPU), graphics processing units (GPUs), application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs), etc. The scheduling unit is a processing unit that schedules the acceleration units and allocates the sequence of instructions to be executed to each acceleration unit, and can take various forms such as a central processing unit (CPU), an application-specific integrated circuit (ASIC), and a field-programmable gate array (FPGA).

[0060] Traditional central processing unit (CPU) architectures often dedicate significant space to control and storage units, leaving insufficient space for computation units. While efficient for logic control, this approach is inefficient for large-scale parallel computing. Therefore, various specialized acceleration units have been developed to improve computational speed for different functions and domains. The acceleration unit proposed in this invention is specifically designed to accelerate the computational speed of neural network models. It employs a data-driven parallel computing architecture to handle the numerous computations (e.g., convolution, pooling) performed by each neural network node. Since the data and intermediate results from these computations are closely linked and frequently used throughout the computation process, existing CPU architectures suffer from inefficiencies due to the limited internal memory of the CPU core, necessitating frequent access to external memory. By employing this acceleration unit specifically designed to speed up the computation of neural network models, and because each core has on-chip (internal) memory with storage capacity suitable for neural network computation, frequent access to external memory is avoided, thus greatly improving processing efficiency and computational performance.

[0061] Acceleration unit 230 must accept scheduling from scheduling unit 220. For example... Figure 2A As shown, memory 210 stores various neural network models, including the nodes and weight data of these models. These neural network models are used when needed. Figure 2A A scheduling unit 220 is deployed to an acceleration unit 230. Specifically, the scheduling unit 220 can send the addresses of the model's parameters (such as the weights of each node) in memory 210 to the acceleration unit 230 via instructions. When the acceleration unit 230 actually uses the neural network model for computation, it directly addresses these parameters (e.g., weights) in memory 210 based on their addresses and temporarily stores them in its on-chip memory. When the acceleration unit 230 actually uses the neural network model for computation, the scheduling unit 220 also sends the model's input parameters to the acceleration unit 230 via instructions, which are then temporarily stored in the acceleration unit 230's on-chip memory. In this way, the acceleration unit 230 can perform inference computation based on these input parameters and the model's parameters (e.g., weights).

[0062] Internal structure of scheduling unit and acceleration unit

[0063] The following is combined Figure 2B The internal structure diagram of the scheduling unit 220 and the acceleration unit 230 is shown in detail, illustrating how the scheduling unit 220 schedules the acceleration unit 230 to perform its work.

[0064] like Figure 2B As shown, the scheduling unit 220 includes multiple processor cores 222 and a cache 221 shared by the multiple processor cores 222. Each processor core 222 includes an instruction fetch unit 203, an instruction decode unit 224, an instruction issue unit 225, and an instruction execution unit 226.

[0065] Instruction fetch unit 223 is used to move the instruction to be executed from memory 210 to instruction register (which may be...). Figure 3 The instruction is stored in one of the registers in the register file 229 shown, and the next fetch address is received or calculated according to the fetch algorithm, which may include, for example, incrementing or decrementing the address based on the instruction length.

[0066] After the instruction is fetched, the scheduling unit 220 enters the instruction decoding stage. The instruction decoding unit 224 decodes the fetched instruction according to a predetermined instruction format to obtain the operand fetch information required by the fetched instruction, thereby preparing for the operation of the instruction execution unit 225. Operand fetch information includes, for example, pointers to immediate values, registers, or other software / hardware that can provide source operands.

[0067] The instruction issuing unit 225 is located between the instruction decoding unit 224 and the instruction execution unit 226. It is used for instruction scheduling and control to efficiently allocate each instruction to different instruction execution units 226, making parallel operation of multiple instructions possible.

[0068] After instruction issuing unit 225 sends an instruction to instruction execution unit 226, instruction execution unit 226 begins executing the instruction. However, if instruction execution unit 226 determines that the instruction should be executed by an acceleration unit, it forwards it to the corresponding acceleration unit for execution. For example, if the instruction is a neural network inference instruction, instruction execution unit 226 will not execute the instruction but will instead send it to acceleration unit 230 via the bus for execution.

[0069] The acceleration unit 30 includes multiple acceleration unit cores 236 (four cores are shown in Figure 2, but those skilled in the art should understand that the acceleration unit 230 may also contain other numbers of cores 236), an instruction processor 237, a direct memory access mechanism 235, and a bus channel 231.

[0070] Bus channel 231 is the channel through which instructions enter and exit the acceleration unit 230 via the bus.

[0071] Direct Memory Access (DMA) mechanism 235 is a feature provided by some computer bus architectures that allows data to be written directly from an external device to the computer motherboard's memory. This method significantly improves data access efficiency compared to the previous method where all data transfers between devices had to go through a scheduling unit. Because of this mechanism, the core of acceleration unit 230 can directly access memory 210 and read parameters (such as the weights of each node) from the neural network model, greatly improving data access efficiency.

[0072] Command processor 237 distributes instructions sent from scheduling unit 220 to acceleration unit 230 to core 236 for execution. Instruction execution unit 226 sends the sequence of instructions to be executed to acceleration unit 230. This sequence of instructions, after entering from bus channel 231, is cached in command processor 237, which then selects core 236 and assigns the instruction sequence to it for execution. Additionally, command processor 237 is also responsible for synchronization operations between cores 236.

[0073] Acceleration unit core

[0074] Figure 3 This is an internal structural diagram of an acceleration unit core 236 according to an embodiment of the present disclosure.

[0075] In one embodiment, such as Figure 3 As shown, the acceleration unit core 236 includes a tensor engine 310, a pooling engine 320, a memory copy engine 330, a sequencer 350, an instruction cache 340, an on-chip memory 360, and a constant buffer 370.

[0076] The instruction sequence allocated by the instruction processor 237 to the acceleration unit core 236 is first cached in the instruction register 340. Then, the sequencer 350 fetches instructions from the instruction register 340 in a first-in, first-out (FIFO) order and allocates them to the tensor engine 310, pooling engine 320, or memory copy engine 330 for execution, depending on the nature of the instruction. The tensor engine 310 handles convolution and matrix multiplication operations in the neural network model. The pooling engine 320 handles pooling operations in the neural network model. The memory copy engine 330 copies operands stored in the on-chip memory 360 within the core 236 to shared memory between cores 236 or to the on-chip memory 360 within other cores 236. The sequencer 350 determines whether to allocate the fetched instruction to the tensor engine 310, pooling engine 320, or memory copy engine 330 based on the nature of the operation (convolution, matrix multiplication, pooling, or operand copying).

[0077] On-chip memory 360 is the core memory that stores the weight parameters of the neural network model, as well as the input parameters and various intermediate results during actual use of the neural network model. Constant buffer 370 is a buffer that stores other constant parameters (e.g., hyperparameters) in the neural network model besides the weight parameters. As described above, during the process of scheduling unit 220 pre-configuring the neural network model in acceleration unit 230, scheduling unit 220 sends the addresses of the model's parameters in memory 210 to acceleration unit 230 in the form of instructions. These parameters include node weights and other parameters (e.g., hyperparameters). For weights, acceleration unit 230 retrieves them from the corresponding locations in memory 210 and places them in on-chip memory 360 during actual neural network model computation. For other parameters, acceleration unit 230 retrieves them from the corresponding locations in memory 210 and places them in constant buffer 370 during actual neural network model computation. Furthermore, when the instruction to actually begin inference is allocated to core 236 by command processor 237 for execution, the input parameters (inputs to the neural network model) in the instruction are also stored in on-chip memory 360. In addition, after the tensor engine 310 and the pooling engine 320 perform convolution or pooling operations, the various intermediate results are also stored in the on-chip memory 360.

[0078] Figure 4A A schematic diagram illustrating the deployment and execution of a deep learning model in a computing device 200 according to an embodiment of the present invention is shown. Figure 4A The deployment and execution of a deep learning model in computing device 200 is illustrated using a software stack approach. For example... Figure 4A As shown, the application that uses machine learning models 9120 / 9140 for inference is called machine learning application 9602. As mentioned above, machine learning application 9602 can achieve any type of machine intelligence, including but not limited to: image recognition, mapping and localization, autonomous navigation, speech synthesis, medical imaging, or language translation.

[0079] The deep learning framework 9604 provides a library of deep learning operation units (OMUs). OMUs are the basic units of operation typically performed by deep learning algorithms, constituting the processing nodes in a deep learning model. Exemplary OMUs include tensor convolution, activation functions, and pooling, which are computational operations performed in convolutional neural networks (CNNs). Deep learning models can be built using the OMUs provided by the deep learning framework 9604, significantly simplifying the development process. Commercially known deep learning frameworks 9604 include, for example, Google's TensorFlow, Facebook's pyTorch, Apache's MXNet, and Caffe developed by the Berkeley AI Research Center and community contributors. This invention is not limited to any specific deep learning framework 9604; any deep learning framework that facilitates the implementation of machine learning models is within the scope of this invention.

[0080] The deep learning framework 9604 can generate deep learning model representations 9605A and 9605B (also known as intermediate model representations, IRs) that represent deep learning models. Since a deep learning model can be represented as a graph of connections between processing nodes in the model, such intermediate model representations can also be called graph intermediate representations (Graph IRs). A deep learning framework 9604 can generate one intermediate model representation 9605A, and intermediate model representation 9605A can be converted into another intermediate model representation 9605B.

[0081] The computing framework 9606 abstracts the low-level instructions provided to the underlying hardware driver 9608, enabling the deep learning framework 9604 to utilize the hardware acceleration capabilities provided by the hardware 9610 (e.g., acceleration unit 230 in Figure 2) without needing to be very familiar with the architecture of the hardware 9610. The computing framework 9606 is closely associated with the underlying acceleration hardware; for certain hardware 9610s (especially the latest neural network processors (NPUs), a dedicated computing framework 9606 is required. This invention is not limited to any specific computing framework 9606; any computing framework capable of abstracting the instructions of the hardware driver 9608 and utilizing the hardware acceleration capabilities of the hardware 9610 is within the scope of this invention.

[0082] The machine learning framework 9604 provides an interface to the computation framework 9606, allowing the corresponding computation framework 9606 to process the intermediate representation of the model. As shown in Figure 4, the computation framework 9606A is adapted to process the intermediate representation 9605A of the model so as to perform deep learning model inference and / or training computations using the hardware 9610A. Correspondingly, the computation framework 9606B is adapted to process the intermediate representation 9605B of the model so as to perform deep learning model inference and / or training computations using the hardware 9610B.

[0083] According to one implementation, the computing framework 9606 and the deep learning framework 9604 can be implemented as separate applications (in which case the deep learning framework 9604 is referred to as the framework front-end or front-end framework, and the computing framework 9606 is referred to as the processing back-end), or incorporated into various applications 9222. All such configurations are illustrative and are within the scope of this invention.

[0084] Figure 4B A schematic diagram illustrating the deep learning model representation transformation and execution in a computing device according to an embodiment of the present invention is shown. The computing device is, for example, a... Figure 2A The computing device 200 shown. (As shown) Figure 4B As shown, various front-end frameworks 9604 generate multiple model representations 9605A-9605B. For example, model representation 9605A is derived from the reference... Figure 4A The front-end framework 9604A is generated without considering the computations supported by the acceleration unit 230. When a deep learning model represented by model representation 9605A needs to be executed (including inference and training) in the computing device 200, the scheduling unit 220 converts model representation 9605A into model representation 9605C. Model representation 9605C has a specific format that takes into account the computations supported by the acceleration unit 230. Subsequently, the scheduling unit 220 sends model representation 9605C to the acceleration unit 230 so that the acceleration unit 230 can load and execute the deep learning model corresponding to model representation 9605C (the same deep learning model represented by model representation 9605A). In this way, the deep learning model generated by the front-end framework can be executed more quickly in the acceleration unit without worrying about the details of the acceleration unit.

[0085] Figure 5A A schematic diagram of the structure of a deep learning model representation (IR) 500 according to an embodiment of the present invention is shown. Figure 5A As shown, IR 500 includes a name 510 indicating the name of the IR, multiple nodes 520, attributes 530, and a version 540. Attribute 530, for example, is in key-value pair format and defines the properties of the entire model, such as model creation time and matching accelerator units. Considering the future development of IR 500 and the conversion between various IRs, version 540 also includes the current version of IR 500 (the version defined in the format) and the learning framework information that generated the IR.

[0086] Node 520 is an important part of IR 500. Each node 520 corresponds to a processing node in the deep learning model (e.g., convolution, pooling, multiplication, etc. in a certain model layer). Considering that deep learning models generally have a large number of processing nodes, IR 500 usually has multiple nodes 520.

[0087] Node 520 includes a node name 522, an operator 524, an input item 526, and an attribute item 528. The node name 522 indicates the name of the node. When generating processing nodes, the computation framework 9606 assigns a unique name to each processing node to uniquely identify it in the model representation. For example, two nodes 520 that perform convolutional processing can be assigned node names such as "Conv2D_1" and "Conv2D_2".

[0088] Operator 524 indicates the operation performed in the processing node corresponding to this node. As mentioned above, deep learning models have various basic operational units, such as normalization, pooling, convolution, multiplication, and maximum value extraction. These basic operational units are defined in IR 500. Operator 524 identifies the basic operational unit corresponding to this processing node. According to one implementation, operator 524 stores the unique name of the operator.

[0089] Input item 526 indicates other nodes, i.e., input nodes, that provide data to be processed by node 520. A node 520 typically has at least one input node (for the first node in the model, there is no input node) to obtain the output data of these input nodes as input data for processing. Since the node name 522 uniquely identifies the node, according to one embodiment, each input item 526 includes a node name 5262 to uniquely identify the input node. According to another embodiment, considering that the input node may output different data to different nodes, the input item 526 may also include an index identifier 5264 to indicate which data the input node outputs to this node. In addition, according to one embodiment, when there are multiple inputs, the input item 526 may also include a control identifier 5266 to indicate whether the input is a control input.

[0090] Attribute item 528 generally uses key-value pairs to indicate various attributes possessed by node 520. The attributes defined in attribute item 528 include at least two categories. One category indicates the data attributes processed by node 520. For example, output data type 5281 indicates the type of output data (integer, floating-point number, etc.); output data format 5282 indicates the format of the output data (single value, one-dimensional or multi-dimensional array of predetermined length, etc.). Optionally, attribute item 528 may also include input data type 5283 and input data format 5284 to indicate the type and format of input data, respectively.

[0091] According to one implementation, considering that the acceleration unit 230 supported by IR 500 performs quantization on the input data before processing to accelerate processing in the acceleration unit core 236, attribute item 528 also includes quantization data format 5285 to indicate the quantized data format of the input data after quantization.

[0092] The attributes defined in attribute item 528 also include the parameters required for the operations to be performed by node 520, i.e., the operations defined by operator 524. Such attributes are included in attribute 5286 defined in the generation of the deep learning model using the computation framework 9606. For example, for Conv2D convolution computation, the computation framework 9606 sets parameters such as filter_layout and padding for convolution computation.

[0093] In addition, IR 500 predefines processing-specific attributes 5286 in the acceleration unit core 236 for the operations supported by the acceleration unit core 236. For example, for the convolution calculation Conv2D, the attributes num_group and strides are also defined in IR 500.

[0094] Considering that IR 500 may be derived from other model representations, attribute item 528 may also include other attributes 5286 from other models, but not classified as attributes 5285 and 5286.

[0095] Figure 5B A schematic diagram illustrating the transformation and execution of a deep learning model representation according to an embodiment of the present invention is shown. Figure 5B The process shown can be found in the reference. Figure 2A The processing is performed in the computing device 200. When the front-end 592 receives a processing request, it determines the node in the deep learning model representation corresponding to the request and uses the conversion unit 594 to convert the node into an operation defined by IR 500. According to the definition of IR 500, the model representation includes operations 596 supported by the hardware acceleration unit and general operations 598 not supported by the acceleration unit. For operations 596 supported by the acceleration unit, the corresponding request is sent to the acceleration unit 230 for processing and a result is returned. For general operations 598, the scheduling unit 220 can directly process and return a result. Subsequently, the results are returned to the front-end 592 for further processing.

[0096] Figure 6 A schematic diagram of a method 600 for creating a deep learning model representation according to an embodiment of the present invention is shown. Method 600 is adapted to create a deep learning model representation (IR) 500 as shown in FIG. 5. Figure 6As shown, method 600 begins with step S610. In step S610, node items 520 are created. Each node item 520 corresponds to a processing node in the deep learning model. As mentioned above, each deep learning model generally has multiple processing nodes. When creating node items 520 in step S610, a node name 522 can be created to uniquely identify the node item 520.

[0097] In subsequent steps S612-618, for the created node item 520, specific sub-entries are created in the node item 520 according to the characteristics of the corresponding processing node.

[0098] Specifically, in step S612, an input item 526 is created under the node item 520. As described above, each input item 526 represents a processing node that sends data to the processing node corresponding to the node item 520 for processing. For a deep learning model, there may be processing nodes that do not have input items 526 (e.g., the first node in the model), or there may be a node graph composed of multiple processing nodes due to the relationship between input and output data. Therefore, according to one embodiment of the present invention, a node name sub-item 5262 can be set for the created input item 526 in step S612 to store the unique node name of the represented input processing node. In this way, a graph representing the deep learning model can be constructed based on each node item and the corresponding input item. Alternatively, if the input node to the node item 520 has multiple outputs, an index identifier sub-item 5264 can be set in the input item 526 in step S612 to identify which data is output to this node. Optionally, if node item 520 has multiple input nodes to that node, then in step S612, a control identifier 5266 may also be set to indicate whether the current input item 526 is a control input in a multiplexed input.

[0099] Subsequently, in step S614, an operator item 524 is created within the node item 520 to characterize the computation performed in the processing node represented by the node item 520. According to one embodiment, in step S614, a name sub-item is also created under the operator item 524 to store the operator that uniquely represents the operation. Since the IR 500 has already defined a list of various computational units used in the deep learning model, the corresponding operation can be uniquely represented using this name, and the various parameters required for the operation can be obtained.

[0100] As described above with reference to Figure 5, the attribute item 528 in node item 520 includes a first attribute item indicating the data attributes to be processed by the corresponding processing node, and a second attribute item indicating the parameters required for the processing performed by the corresponding node.

[0101] Therefore, in step S616, a first attribute item is created in node item 520 to indicate the data attributes to be processed by the corresponding processing node. Subsequently, in step S618, a second attribute item is created in node item 520 to indicate the various processing parameters required for the processing performed by the corresponding processing node.

[0102] Since the input data to the processing node is generally the output data of other processing nodes, it does not need to be specifically defined. Therefore, the first attribute item created in step S616 includes an attribute item indicating the output data type and / or output data format of the corresponding processing node. Optionally, considering that the input processing node may have multiple outputs, the first attribute item created in step S616 also includes an attribute item indicating the input data type and / or input data format. According to one embodiment, considering that the underlying hardware acceleration unit performs data quantization before data processing, the first attribute item created in step S616 also includes an attribute item indicating the quantized data format of the input data.

[0103] According to one implementation, an operator list is pre-defined in IR 500 for various operations in the deep learning model supported by the underlying hardware acceleration unit. This list defines multiple processing parameter settings for each operation supported by the acceleration unit. If the operation (name of operator item 524) in the node 520 created in step S610 is in this operator list, then in step S618, the processing parameter settings of the predefined operation are referenced to the second attribute item.

[0104] Specifically, according to one implementation, for parameters already defined for this processing in the deep learning model, in step S618, these parameters are used to create corresponding second attribute items, that is, the attribute value of the second attribute item is set to the defined parameter value. For parameters not defined in the deep learning model but defined in a pre-set operator (operator list), the parameters in the pre-set operator are used to create corresponding second attribute items. Optionally, the attribute value of the created second attribute item can also be set to the default value of the parameter in the pre-set operator.

[0105] In addition, for operations not listed in the operator list, in step S618, the corresponding second attribute item is created only based on the parameters already defined for the processing in the deep learning model.

[0106] In this way, when creating the second attribute item related to the processing to be performed by the node in step S618, the specific characteristics of the underlying hardware acceleration unit can be fully considered and corresponding parameters can be added. These parameters are not closely related to the deep learning model, but setting these parameters allows the hardware acceleration unit to perform the processing. In this way, the adaptation problem between the model representation generated by the front-end framework and the underlying hardware acceleration unit is solved.

[0107] Alternatively, after creating the node item and its various sub-items for each node using steps S610 and the corresponding steps S612-S618, method 600 further includes step S620 to generate the name of the IR and step S630 to generate the version information of the IR.

[0108] Considering the differences between different model frameworks, the version information generated in step S630 also includes items indicating the model framework used for the deep learning model. This way, when a computation in a node cannot be supported by the underlying hardware acceleration unit, the computation framework corresponding to that model framework can be used to perform the computation.

[0109] The following is combined Figure 7 Given Figure 6 An example of how method 600 is executed. Figure 7 A schematic diagram of a deep learning model example 700 according to an embodiment of the present invention is shown. Figure 7 As shown, this deep learning model 700 includes six operation nodes: the first operation node (tf_placeholder), the second operation node (Conv2D, the first convolutional node), the third operation node (Conv2D, the second convolutional node), the fourth operation node (Identity), the fifth operation node (Mul), and the sixth operation node (Maximum). Utilizing... Figure 6 The method 600 shown can generate the IR given in Figure 5 for each computing node.

[0110] Specifically, for the third computation node (Conv2D), the generated IR portion includes the node name "Conv2D_1" and the operator "Conv2D" (indicating convolution operation). The computation node has two inputs, "INT8 / Const_1" and "Conv2D". The output data format is defined as a 1*10*10*3 array in the attribute "_output_shapes", and the output data type is defined as a floating-point array in the attribute "_output_type". The parameters required for convolution operations are set in the attributes "data_format", "dilations", "filler_layout", "num_group", "padding", and "strides". When performing convolution operations using an acceleration unit, these parameters can be configured specifically for the acceleration unit, enabling the acceleration unit to perform the corresponding processing for this computation node. The specific content of this computation node is given below:

[0111]

[0112]

[0113]

[0114]

[0115] The above has been referenced Figure 6 A method 600 for creating deep learning model representations (IRs) is presented. It should be noted that method 600 can be used to directly analyze and process deep learning models to obtain model representations (IRs) suitable for compilation and execution on underlying hardware acceleration units. This method can also be used for conversion between different IRs.

[0116] According to one implementation, a deep learning model representation created by a deep learning model framework (framework front-end) can be obtained. This model representation is then analyzed to determine the deep learning model corresponding to it and the attributes of each processing node within it. Subsequently, a reference... Figure 6 The method 600 can obtain a new model representation (IR) suitable for a specific hardware acceleration unit. Using this method, model representations created by existing framework front-ends can be converted into new model representations (IRs) suitable for a specific hardware acceleration unit, enabling high-performance loading and execution within the hardware acceleration unit. For newly developed hardware acceleration units, the method proposed in this disclosure allows for efficient and easy execution of deep learning models not designed for that specific hardware acceleration unit, thereby facilitating the widespread adoption of hardware acceleration units.

[0117] Figure 8 A method 800 for executing a deep learning model according to an embodiment of the present invention is illustrated. Method 800 is used to execute a deep learning model utilizing the above-mentioned reference. Figure 6 The deep learning model created by the method described in 600 is characterized by a deep learning model representation (IR) with a format defined in reference Figure 5.

[0118] Method 800 begins with step S810. In step S810, the IR is analyzed to construct a node graph, and then node items are obtained from the graph one by one according to the execution order of the data. For each node item, in step S820, the output data generated after processing by the processing node identified by each input entry in that node item is obtained as input data. In step S830, the processing parameters to be processed are determined according to the attribute settings of the second attribute item in that node item. Subsequently, in step S840, using the processing parameters determined in step S830, the input data determined in step S820 is processed according to the operator entries in that node item to obtain output data.

[0119] As described above, the type and format of the output data are defined in the first attribute of the node item. Therefore, according to one embodiment, the output obtained in step S840 has the type and format defined in the first attribute of the node item.

[0120] Alternatively, when the type and format of input data are defined in the first attribute of a node item, the data obtained in step S820 is checked against the first attribute. If the two do not match, it indicates that the input data provided to the node item is incorrect, and therefore the execution of method 800 is terminated. In another embodiment, if the two do not match, the data obtained in step S820 can be converted into the input data type and / or format defined by the first attribute.

[0121] According to another implementation, the first attribute of the node item also defines the quantized data format of the input data. In this case, before processing the input data in step S840, it is necessary to quantize the acquired input data according to the quantized data format defined in the first attribute.

[0122] According to one implementation, various operations in the deep learning model supported by the underlying hardware acceleration unit are pre-configured in the IR 500 to define multiple processing parameters and corresponding default configurations for each operation supported by the acceleration unit. In this implementation, when processing the input data in step S840, it is necessary to determine whether the operation indicated by the operator entry in the node item is a pre-configured operation supported by the acceleration unit. If the operation is included in the operator list pre-configured in the IR 500, the processing parameters determined in step S830 and the input data obtained in step S820 are loaded into the acceleration unit to perform the operation. Optionally, before loading the operation by the hardware acceleration unit, it can also be determined whether the second attribute setting of the node item matches the attribute pre-configured in the IR 5000 (e.g., whether the attribute pre-configured in the IR is missing). If they do not match, it indicates that the model representation to be processed is not set for the corresponding underlying hardware unit or the model representation is corrupted, and the execution of method 800 can be exited. Only when the two match will the processing in step S840 proceed.

[0123] Additionally, if the operation to be performed in step S840 is not included in the operator list pre-set by the IR 500, it indicates that the operation is not supported by the hardware acceleration unit. According to one implementation, in this case, if the model representation is derived from a model representation generated by another front-end framework, the node can be converted back to a node in the previous model representation and processed by a back-end computing framework that matches the previous front-end framework.

[0124] For example, when the hardware acceleration unit is an NPU, the front-end framework is Tensorflow, and the computing framework is Nvidia's CUDA, when processing a model representation with the format defined by IR 500, which is converted from an existing Tensorflow model IR, the NPU directly executes operations that it supports, based on the definition of the corresponding node. For operations that the NPU does not support, the node can be converted back to the node definition in the existing model IR, and the corresponding computing framework (CUDA) can then process the operations within that node.

[0125] Using the above method, a model representation IR adapted to hardware acceleration units is provided, and deep learning models created using existing frameworks can be easily transferred to hardware acceleration units for efficient execution.

[0126] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0127] When the program code is executed on a programmable computer, the computing device generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the method of the present invention according to instructions in the program code stored in the memory.

[0128] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0129] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing embodiments of the invention.

[0130] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0131] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0132] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0133] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0134] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0135] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the methods or method elements forms means for implementing the methods or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by elements for the purposes of carrying out the invention.

[0136] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

[0137] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the invention is illustrative rather than restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. A method for creating a representation of a deep learning model, the deep learning model comprising multiple processing nodes, the method comprising the steps of: One or more node entries are created based on the deep learning model generated by the front-end framework. Each node entry corresponds to a processing node in the model, where for each node entry: Create one or more input entries, each input entry representing the processing node to which data is sent for processing; Create operator entries to represent the operations performed in the corresponding processing node; Create a first attribute entry to characterize the output data type and / or output data format of the corresponding processing node; as well as Create a second attribute entry to represent the various processing parameters required for the processing performed by the corresponding processing node; The method further includes the following steps: A list of operators is pre-defined, each operator in the list corresponding to an operation supported in the acceleration unit, and each operator has one or more attributes; The step of creating the second attribute entry also includes: If the operation performed in the corresponding processing node is an operation supported by the acceleration unit, then a second attribute entry is created by referring to the attributes of the corresponding operator in the operator list. Different types of acceleration units support different operations. If the operation performed in the corresponding processing node is an operation that the acceleration unit does not support, then the second attribute entry is created with reference to the processing parameters provided by the operation. The processing parameters are parameters provided by the front-end framework that can be executed by the back-end computing framework.

2. The method of claim 1, wherein the step of creating a second attribute entry by referring to the attributes of the corresponding operator includes: If the attribute is already defined in the processing parameters of the corresponding processing node, then the attribute value of the created second attribute entry is set to the processing parameter; as well as If the attribute is not defined in the processing parameters of the corresponding processing node, the attribute value of the created second attribute entry will be set to the default value.

3. The method of claim 1, wherein each node entry has a corresponding node name, and the step of creating an input entry includes: The name of the input entry is set as the node name of the node entry of the processing node represented, so as to construct a graph representing the deep learning model based on each node entry and the corresponding input entry.

4. The method of claim 1, wherein the first attribute entry further characterizes the input data type and / or input data format of the corresponding processing node.

5. The method of claim 1, wherein the first attribute entry further characterizes the quantized data format of the input data of the corresponding processing node.

6. The method of claim 1, further comprising the step of: Generate the name of the model representation; and Version information of the model representation is generated, and the version information indicates the model framework for generating the deep learning model.

7. A method for creating representations of deep learning models, characterized in that, The method includes the following steps: Obtain the first deep learning model representation created by the model framework for the deep learning model; The first deep learning model representation is analyzed to obtain the processing node information of the deep learning model; as well as The processing node information is processed using the method described in any one of claims 1-6 to create a second deep learning model representation, which is suitable for processing in an acceleration unit.

8. A method for executing a deep learning model, characterized in that, The method includes the following steps: Obtain node entries, which are created based on the deep learning model generated under the front-end framework; The output data generated by the processing node corresponding to each input entry of the node entry is obtained as the input data; The processing parameters are determined based on the second attribute entry in the node entry; as well as Based on the determined processing parameters, the acquired input data is processed according to the operator entries in the node entries; The determined processing of the acquired input data includes: Determine whether the determined process is included in a pre-set list of operators, where each operator in the list corresponds to an operation supported by the acceleration unit, and each operator has one or more attributes; If the determined processing corresponds to one of the operators in the operator list, the determined processing parameters and the acquired input data are loaded into the acceleration unit to perform the determined processing; different types of acceleration units support different operations; If the operation performed in the corresponding processing node is an operation not supported by the acceleration unit, the node entry is converted into the corresponding node in the first deep learning model representation so that the processing engine in the backend computing framework can be used to perform the determined processing.

9. The execution method of claim 8, wherein the output data generated by the determined process has the data type and format defined by the first attribute entry in the node entry.

10. The execution method of claim 9, wherein the first attribute entry further characterizes the input data type and / or input data format of the corresponding processing node, and the method further comprises the step of: If the acquired input data does not match the input data type and / or format represented by the first attribute entry, the execution will terminate.

11. The execution method of claim 9, wherein the first attribute entry further characterizes the quantized data format of the input data of the corresponding processing node; the method further includes the step of: The acquired input data is quantized according to the quantized data format represented by the first attribute entry.

12. The execution method of claim 8, further comprising the step of: prior to performing the determined processing in the acceleration unit. Determine whether the second attribute entry of the node entry conforms to the attribute value set by the corresponding operator; If it does not meet the requirements, the process will terminate.

13. A data processing device, comprising: An acceleration unit includes one or more acceleration unit cores, each acceleration unit core having a corresponding on-chip memory; as well as A processor adapted to perform the method as described in any one of claims 1-7 to create a deep learning model representation, and to perform the method as described in any one of claims 8-12 to utilize the acceleration unit to execute a deep learning model corresponding to the deep learning model representation.

14. A computing device, comprising: At least one processor; and A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the method as described in any one of claims 1-12.

Images