Model preprocessing method for computing-in-memory intelligent computing architecture

Abstract

The present invention relates to the technical field of computing-in-memory intelligent computing architectures, and in particular to a model preprocessing method for a computing-in-memory intelligent computing architecture. The method comprises: step S1: reading and parsing an ONNX source file, to be specific, upon reading an ONNX source file, traversing indexes and nodes of the ONNX source file, checking whether a Sigmoid operation completely unsupported by a computing-in-memory system exists, converting an operation type of each node into a corresponding operator, recording a predecessor node, a successor node, and edge information of each node, and then classifying the operation types of the operators; step S2: partitioning a host operator graph and a device operator graph; and step S3: further processing and optimizing the device operator graph. The present invention implements deployment of an ONNX pre-trained model on a computing-in-memory platform, and greatly improves the adaptability of the computing-in-memory platform to existing artificial intelligence models.

Description

Model preprocessing method of storage-computation integrated intelligent computing architecture TECHNICAL FIELD

[0001] The present application relates to the technical field of storage-computation integrated intelligent computing architecture, in particular to a model preprocessing method of storage-computation integrated intelligent computing architecture. BACKGROUND

[0002] In recent years, artificial intelligence has developed rapidly. Artificial intelligence algorithms represented by neural networks belong to computing and memory-intensive tasks. Most hardware resources in traditional CPUs (processors) are used for general control rather than computing, so they cannot provide enough computing power for the deployment of artificial intelligence algorithms. GPGPU (general-purpose graphics processing unit) provides more computing resources at the cost of general control performance, providing an ideal deployment platform for artificial intelligence algorithms. However, GPGPU still belongs to the Von Neumann architecture, and its computing and storage are separated, so in GPGPU, data transfer between neural network layers must pass through the memory. With the improvement of GPGPU computing speed, the time-consuming of executing one calculation is becoming smaller and smaller, which makes the proportion of data exchange time-consuming between the computing unit and the memory increasing, even more than the computing time-consuming, forming a memory constraint (Memory Bound). Memory constraint means that memory access has become the bottleneck of computing. At this time, simply expanding the scale of the computing unit has already been difficult to achieve the expansion of computing power.

[0003] In order to eradicate the memory constraint, it is necessary to reduce or avoid data movement between the computing unit and the memory, but in the Von Neumann architecture where computing and storage are separated, the memory cannot be bypassed by data, so it is necessary to go out of the Von Neumann architecture and explore a computing architecture that integrates storage and computing, namely the storage-computation integrated architecture. In the storage-computation integrated architecture, the memory is endowed with the function of computing, so the data generated by the memory can be calculated on the spot, thereby greatly reducing or even avoiding data movement during computing and eliminating the memory constraint. TECHNICAL PROBLEM

[0004] The current memory-compute integrated system is mainly divided into three categories according to the type of in-memory computing device based thereon: SRAM (static random access memory) memory-compute integration, DRAM (dynamic random access memory) memory-compute integration and NVM (non-volatile memory) memory-compute integration. Due to the low storage density of SRAM, SRAM memory-compute integration is difficult to realize large capacity, which is insufficient to support the deployment of large-scale artificial intelligence algorithms. DRAM has an extremely complex and closed hardware model and protocol stack, so only memory manufacturers have the conditions to conduct DRAM memory-compute integration research. NVM, such as ReRAM, MRAM, PCM, NOR-FLASH, not only has high storage density, but also has a simple and unified hardware model, strong universality. In addition, NVM also has the advantage of low power consumption. Therefore, how to design an NVM memory-compute integrated intelligent computing architecture to realize a high energy-efficient artificial intelligence algorithm deployment platform has become an open and valuable research topic.

[0005] In the NVM memory-compute integrated intelligent computing architecture (hereinafter referred to as the memory-compute integrated intelligent computing architecture), a plurality of computing units (Tiles) are usually grouped into a computing array, and a special interconnection structure is provided in the computing array for communication between the plurality of computing units. When executing an algorithm task, each Tile is responsible for executing a part of matrix multiplication operation, and a plurality of Tiles can execute computation in a parallel manner and transmit input data and transfer operation results by relying on the interconnection network, so as to finally realize the forward inference process of a neural network.

[0006] In the memory-compute integrated intelligent computing architecture, it is necessary to improve the support of the system for pre-trained models. The ONNX (Open Neural Network Exchange) file format is one of the common pre-trained model storage methods in the industry today, and manufacturers such as Huawei, Intel, AMD and Qualcomm all provide support for this format. Therefore, by supporting pre-trained models of this format, the compatibility and ease of use of the memory-compute integrated system can be greatly improved.

[0007] However, the ONNX model needs to be converted before being mapped to each Tile of the memory-compute integrated system, and the memory-compute integrated system does not support some operation operations in the model. This requires the development of an algorithm to efficiently map the operations supported by the memory-compute integrated system in the model to the Tiles, and to hand over the operation operations not supported by the memory-compute integrated system to external hardware. Technical solutions

[0008] The purpose of the present application is to provide a model preprocessing method for a memory-compute integrated intelligent computing architecture, aiming to develop a conversion method suitable for a memory-compute integrated system to realize the adaptation of the system to ONNX models.

[0009] To solve the above technical problems, the application provides a model preprocessing method of a storage-computing integrated intelligent computing architecture, comprising the following steps:

[0010] Step S1: reading and analyzing the ONNX source file; that is, after reading the ONNX source file, the indexes and various nodes are traversed, it is checked whether there is a Sigmoid operation completely not supported by the storage-computing integrated system, the operation types of various nodes are converted into corresponding operators, the predecessor and successor nodes of the nodes and the edge information are recorded, and then the operation types of various operators are classified into convolution, matrix multiplication and pooling types, and the operators of the splicing type in the model of the ONNX source file are configured;

[0011] Step S2: splitting the host operator graph and the device operator graph; that is, through a depth-first search algorithm, operations unsuitable for being executed by the storage-computing integrated system are found, and the device operator graph and the host operator graph are split at the operations;

[0012] Step S3: further processing and optimizing the device operator graph; that is, the operators in the redundant host operator graph in the device operator graph are deleted, and then according to the model category of the ONNX source file, the running efficiency of the device operator graph on the storage-computing integrated architecture is further optimized by fusing part of the operations in the device operator graph;

[0013] Step S4: saving the host operator graph and the device operator graph after processing and optimization.

[0014] Preferably, the ONNX source file comprises indexes of various nodes, data flow graphs and various elements in the model;

[0015] In the ONNX source file, each calculation operation in the calculation graph of the deep learning algorithm is taken as a node, all nodes in the algorithm form a data flow graph without a ring in a topological sorting manner, nodes are linked by edges, and the edges represent the data flow between different nodes.

[0016] The node comprises a name, an attribute, a called operator, input and output elements; wherein the operator specifies the operation type of the node, the operation type comprises Relu and Conv, and the input and output specify the input and output node information of the node.

[0017] The indexes comprise the numbers and positions of various operators, input and output elements in each node, to help locating and processing various components in the model and facilitate accessing various operations and intermediate results of the model.

[0018] Preferably, the step S1 configures the splicing type operator in the model of the ONNX source file, specifically including: if an operator is of the splicing type, the input of the operator needs to be configured according to the given order to ensure that the splicing is performed according to the order given by the original model.

[0019] Preferably, the step S1 further includes: simplifying the processed operator graph, removing useless operators, and normalizing the non-standard convolution and pooling operators in the modified part of the model.

[0020] Preferably, the model type of the ONNX source file in the step S3 includes: ResNet, GoogLeNet, VGG, YOLOv3 and SqueezeNet models.

[0021] Preferably, when the model type of the ONNX source file is ResNet type, the device operator graph is shaped, specifically including the following steps:

[0022] (a) find the activation function node in the device operator graph, fuse the activation function node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the activation function node;

[0023] (b) find the pooling function node in the device operator graph, fuse the pooling function node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the pooling function node;

[0024] (c) find the Add operation node in the device operator graph, fuse the Add operation node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the Add operation node;

[0025] (d) according to the dependency relationship between different convolution layers, the time of data reaching each layer is calculated using a recursive method.

[0026] Preferably, the ONNX source file is a general pre-training model storage method, supporting Linux, Mac and Windows operating systems, and supporting different hardware architecture to read the model and perform inference. Advantages

[0027] Compared with the prior art, the present application has the following advantages:

[0028] 1、The preprocessing method realizes the deployment of the ONNX pre-training model on the memory-computing integrated platform, greatly improving the adaptability of the memory-computing integrated platform to existing artificial intelligence models.

[0029] 2、The preprocessing method of the application is targeted at the characteristics of the storage-computing integrated platform, and further optimizes the performance of the ONNX pre-training model on the storage-computing integrated platform through the way of fusing part of nodes. BRIEF DESCRIPTION OF DRAWINGS

[0030] Fig. 1 is a flowchart of a model preprocessing method of a storage-computing integrated intelligent computing architecture according to the application. Embodiments of the application

[0031] The application will be further described in detail below in combination with the drawings and specific embodiments. The advantages and features of the application will be clearer according to the following description. It should be noted that the drawings are very simplified and use non-precise proportions, and are only used to facilitate and clarify the purpose of assisting in the description of the embodiments of the application.

[0032] The application provides a model preprocessing method of a storage-computing integrated intelligent computing architecture. The ONNX file format is one of the common pre-training model storage methods in the industry today, supports Linux, Mac, Windows and other operating systems, and supports different hardware architectures to read models and perform inference. Such a file takes each calculation operation in the computation graph of a deep learning algorithm as a node, and all nodes in the algorithm form a loop-free data flow graph in topological order. The nodes are linked by edges, and the edges represent the data flow between different nodes. In the ONNX file, a node includes its name, attributes, called operators, inputs, outputs and other parts. Among them, the operator specifies the operation type of the node, including Relu, Conv, etc.; the input and output specify the input and output node information of the node. In addition to the data flow graph and each node, the ONNX file also includes the index of each element in the model, including the number and position of each element (such as input, output, operator, etc.) in each node, which helps to locate and process each component in the model, and facilitates access to each operation and intermediate result of the model.

[0033] The process of preprocessing the ONNX model file for the storage-computing integrated chip platform mainly includes four steps: first, parse the source file to obtain useful information of each node, and discard useless information, thereby obtaining an original operator graph; second, move out the operators in the original operator graph that are not suitable for the storage-computing integrated computing mode, to form a device operator graph for the storage-computing integrated system and a host operator graph for the host (CPU); third, further process the device operator graph to shape and structure it into an operator graph that can adapt to the storage-computing integrated computing mode; and finally, save the host operator graph and the device operator graph after the above steps. Next, each step will be briefly described.

[0034] In the process of parsing the source file, first, the ONNX model file will be read. Then, its index and each node are traversed to check whether there is an operation that is completely not supported by the storage-computing integrated system (such as the Sigmoid operation), and the operation type of each node is converted into the corresponding operator, and the predecessor and successor node information of the node is recorded. Then, the operators are classified, including convolution, matrix multiplication, pooling, etc., and the configuration is performed for the splicing type operator.

[0035] Next, using the depth-first search algorithm, the operation node that can only be performed by the host is found, and the original graph is divided into a host operator graph and a device operator graph at the node, and saved.

[0036] Then, the device operator graph is optimized, and according to the specific model type, some operators are fused to further optimize the performance of the device operator graph on the storage-computing integrated architecture, and then the time for data to reach each layer is calculated, and the processed host operator graph and device operator graph are saved.

[0037] Finally, the output host operator graph and device operator graph are written to the file system for saving.

[0038] As a further description of the technical scheme of the embodiment of the application, as shown in FIG. 1, the embodiment of the application specifically provides a model preprocessing method of a storage-computing integrated intelligent computing architecture:

[0039] Step S1: reading and analyzing the ONNX source file;

[0040] (1) reading the ONNX model file, traversing the index of the file and each node in the file, and checking whether there is an operation that is completely not supported by the storage-computing integrated system (such as the Sigmoid operation).

[0041] (2) converting the operation type of each node into the corresponding operator (Operator), and reading and recording the predecessor node and edge of each node. Because the Merge type node has multiple inputs, it is necessary to traverse all the inputs to complete this process. In addition, it is also necessary to read and record the successor node and edge of each node.

[0042] (3) classifying the operation type of each operator, according to the nature of the operation, which can be classified into Conv, Gemm, Pool, etc., so as to map the model to the storage-computing integrated chip in the subsequent process.

[0043] (4) configuring the Concat type operator in the model. If an operator is of the splicing type, the input of the operator needs to be configured in the given order to ensure that the splicing is performed in the order given by the original model.

[0044] (5) Simplify the processed operator graph, remove useless operators, and standardize the non-standard convolution and pooling operators in the modified part of the model.

[0045] Step S2: Split the host operator graph and the device operator graph; use the depth-first search algorithm on the operator graph generated in the previous step to split the part of the graph containing only host-executable operators into a host operator graph, and the rest into a device operator graph.

[0046] Step S3: Further processing and optimization of the device operator graph;

[0047] (1) Delete the operators in the host operator graph from the device operator graph.

[0048] (2) According to the specific type of the original input ONNX model (currently supporting ResNet, GoogLeNet, VGG, YOLOv3 and SqueezeNet models), reshape the device operator graph. Taking a ResNet type model as an example, the following steps are required:

[0049] (a) Find the activation (Act) function node in the device operator graph, fuse it with its predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the activation function node.

[0050] (b) Find the pooling (Pooling) function node in the device operator graph, fuse it with its predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the pooling function node.

[0051] (c) Find the Add operation node in the device operator graph, fuse it with its predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the Add operation node.

[0052] (d) According to the dependency relationship between different convolution layers, use a recursive method to calculate the time for data to reach each layer.

[0053] Step S4: Save the processed and optimized host operator graph and device operator graph.

[0054] The preprocessing method of the application converts, splits and optimizes the computation graph in ONNX file format into a host operator graph suitable for host execution and a device operator graph suitable for storage-computing integrated device execution, realizes the deployment of the pre-trained model on the storage-computing integrated platform, and greatly improves the universality of the platform.

[0055] The above description is only the description of the preferred embodiments of the present application, and is not any limitation on the scope of the present application. Any change and modification made by the person skilled in the art according to the above disclosure is within the protection scope of the claims.

Claims

1. A model preprocessing method for an in-memory computing integrated intelligent computing architecture, characterized in that, Includes the following steps: Step S1: Read and parse the ONNX source file; that is: after reading the ONNX source file, traverse its index and each node, check for the existence of Sigmoid operation that is not supported by the in-memory computing system, convert the operation type of each node into the corresponding operator, record the predecessor and successor nodes of the node, as well as the edge information, and then classify the operation type of each operator into convolution, matrix multiplication and pooling types, and configure the concatenation type operator in the model of the ONNX source file. Step S2: Segment the host operator graph and the device operator graph; that is: use a depth-first search algorithm to find operations in the operator graph that are not suitable for execution by the in-memory computing system, and segment the graph at these operations to form a device operator graph and a host operator graph; Step S3: Further process and optimize the device operator graph; that is: delete redundant operators in the host operator graph in the device operator graph, and then, based on the model type of the ONNX source file, further optimize the running efficiency of the device operator graph on the in-memory computing architecture by merging some operations in the device operator graph. Step S4: Save the processed and optimized host operator graph and device operator graph.

2. The model preprocessing method for an in-memory computing integrated intelligent computing architecture as described in claim 1, characterized in that, The ONNX source file includes: each node, data flow diagram, and index of each element in the model; In this process, each computational operation in the computation graph of the deep learning algorithm is treated as a node through the ONNX source file. All nodes in the algorithm form a data flow graph without loops by sorting them topologically. Nodes are connected by edges, which represent the data flow between different nodes. A node includes: name, attributes, invoked operator, input and output elements; wherein the operator specifies the operation type of the node, and the operation type includes ReLU and Conv; the input and output specify the input and output node information of the node; The index includes the number and position of each operator, input, and output element in each node to help locate and process the various components in the model, and to facilitate access to the various operations and intermediate results of the model.

3. The model preprocessing method for an in-memory computing integrated intelligent computing architecture as described in claim 1, characterized in that, In step S1, the operators of the splicing type in the model of the ONNX source file are configured. Specifically, if an operator is of the splicing type, the input of the operator needs to be configured in the given order to ensure that the splicing is performed in the order given by the original model.

4. The model preprocessing method for an in-memory computing integrated intelligent computing architecture as described in claim 3, characterized in that, Step S1 further includes: simplifying the processed operator graph, removing useless operators, and normalizing non-standard convolution and pooling operators in the modified part of the model.

5. The model preprocessing method for an in-memory computing integrated intelligent computing architecture as described in claim 1, characterized in that, The types of models in the ONNX source files in step S3 include: ResNet, GoogLeNet, VGG, YOLOv3, and SqueezeNet models.

6. The model preprocessing method for an in-memory computing integrated intelligent computing architecture as described in claim 5, characterized in that, When the ONNX source file is of ResNet type, the device operator graph is shaped, specifically including the following steps: (a) Find the activation function node in the device operator graph, merge the activation function node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the activation function node. (b) Locate the pooling function node in the device operator graph, merge the pooling function node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the pooling function node. (c) Locate the Add operation node in the device operator graph, merge the Add operation node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the Add operation node. (d) Based on the dependencies between different convolutional layers, use a recursive method to calculate the time it takes for data to reach each layer.

7. A model preprocessing method for an in-memory computing integrated intelligent computing architecture as described in any one of claims 1 to 6, characterized in that, The ONNX source files are a universal pre-trained model storage method that supports Linux, Mac, and Windows operating systems, as well as different hardware architectures for reading models and performing inference.

Images