A model preprocessing method for storage-computing integrated intelligent computing architecture
By preprocessing the ONNX model, including parsing, classification, and optimization, the adaptation problem of pre-trained models to in-memory computing systems is solved, achieving efficient model deployment and performance improvement.
Patent Information
- Application Number
- CN202410822123.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-25
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-06-25
AI Technical Summary
Existing in-memory computing systems have difficulty directly supporting ONNX pre-trained models, especially since some computational operations are not supported, resulting in low model deployment efficiency.
A model preprocessing method for an in-memory computing intelligent computing architecture is provided, including reading ONNX source files, parsing and classifying node operations, splitting host and device operator graphs using a depth-first search algorithm, further optimizing the device operator graphs, removing redundant and unsupported operations, and merging some nodes to adapt to the in-memory computing system.
It enables efficient deployment of ONNX pre-trained models on in-memory computing platforms, improves system adaptability and performance, and enhances support for existing artificial intelligence models.
Smart Images

Figure CN118394704B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of storage-computing integrated intelligent computing architecture, and in particular to a model preprocessing method for a storage-computing integrated intelligent computing architecture. Background Art
[0002] In recent years, artificial intelligence has developed rapidly. Artificial intelligence algorithms represented by neural networks are computationally and memory-intensive tasks. Most of the hardware resources in traditional CPUs (processors) are used for general control rather than computing, so they cannot provide sufficient computing power for the deployment of artificial intelligence algorithms. GPGPUs (general-purpose graphics processors) provide more computing resources at the expense of general control performance, providing an ideal deployment platform for artificial intelligence algorithms. However, GPGPUs still belong to the von Neumann architecture, and their computing and storage are separated. Therefore, in GPGPUs, data transfer between neural network layers must pass through memory. As the computing speed of GPGPUs increases, the time it takes to perform a calculation becomes smaller and smaller, which makes the proportion of the time spent on data exchange between computing units and memory continue to increase, even exceeding the computing time, forming a memory constraint. Memory constraints mean that memory access has become a bottleneck for computing. At this time, it is difficult to expand computing power simply by expanding the scale of computing units.
[0003] In order to eliminate memory constraints, it is necessary to reduce or avoid data movement between computing units and memory. However, in the von Neumann architecture where computing and storage are separated, memory cannot be bypassed by data. Therefore, it is necessary to go beyond the scope of the von Neumann architecture and explore a computing architecture that integrates storage and computing, that is, a storage-computing integrated architecture. In the storage-computing integrated architecture, the memory is given the function of computing, so the data generated by the memory can be calculated on the spot, thereby greatly reducing or even avoiding data movement during computing and eliminating memory constraints.
[0004] The current storage-computing integrated systems are mainly divided into three categories according to the type of in-memory computing devices they are based on: SRAM (static random access memory) storage-computing integrated, DRAM (dynamic random access memory) storage-computing integrated, and NVM (Non-Volatile Memory) storage-computing integrated. Due to the low storage density of SRAM, SRAM storage-computing integrated is difficult to achieve large capacity and is insufficient to support the deployment of large-scale artificial intelligence algorithms. DRAM has an extremely complex and closed hardware model and protocol stack, so only memory manufacturers have the conditions to conduct DRAM storage-computing integrated research. NVM, such as ReRAM, MRAM, PCM, and NOR-FLASH, not only has high storage density, but also has a simple and unified hardware model and strong universality. In addition, NVM also has the advantage of low power consumption. Therefore, how to design an NVM storage-computing integrated intelligent computing architecture to achieve an energy-efficient artificial intelligence algorithm deployment platform has become an open and valuable research topic.
[0005] In the NVM storage-computing integrated intelligent computing architecture (hereinafter referred to as the storage-computing integrated intelligent computing architecture), multiple computing units (Tiles) are usually combined into a computing array, and a special interconnection structure is provided in the computing array for communication between multiple computing units. When executing algorithm tasks, each Tile is responsible for performing a part of the matrix multiplication operation. Multiple Tiles can perform calculations in parallel and rely on the interconnection network to send input data and transmit calculation results, ultimately realizing the forward reasoning process of the neural network.
[0006] In the storage-computing integrated intelligent computing architecture, it is necessary to improve the system's support for pre-trained models. The ONNX (Open Neural Network Exchange) file format is one of the common pre-trained model storage methods in the industry today. Manufacturers including Huawei, Intel, AMD, and Qualcomm all support this format. Therefore, by supporting pre-trained models in this format, the compatibility and ease of use of the storage-computing integrated system can be greatly improved.
[0007] However, the ONNX model needs to be converted to each tile in the storage-computing system, and the storage-computing system does not support some operations in the model. This requires the development of an algorithm to efficiently map the operations supported by the storage-computing system in the model to the tile, and to hand over the operations that it does not support to external hardware. Summary of the invention
[0008] The purpose of the present invention is to provide a model preprocessing method for a storage-computing integrated intelligent computing architecture, aiming to develop a conversion method adapted to the storage-computing integrated system to achieve the adaptation of the system to the ONNX model.
[0009] In order to solve the above technical problems, the present invention provides a model preprocessing method for a storage-computing integrated intelligent computing architecture, comprising the following steps:
[0010] Step S1: Read and parse the ONNX source file; that is, after reading the ONNX source file, traverse its index and each node, check whether there is a Sigmoid operation that is completely unsupported by the storage and computing system, and convert the operation type of each node into the corresponding operator, record the predecessor and successor nodes of the node, and the edge information, and then classify the operation type of each operator into convolution, matrix multiplication and pooling types, and configure the splicing type operator in the model of the ONNX source file;
[0011] Step S2: Splitting the host operator graph and the device operator graph; that is, using a depth-first search algorithm, searching for operations in the operator graph that are not suitable for execution by the storage-computing integrated system, and splitting the operations at the operations to form a device operator graph and a host operator graph;
[0012] Step S3: further process and optimize the device operator graph; that is, delete the redundant operators in the host operator graph in the device operator graph, and then further optimize the operation efficiency of the device operator graph on the storage-computing integrated architecture by integrating some operations in the device operator graph according to the model type of the ONNX source file;
[0013] Step S4: Save the processed and optimized host operator graph and device operator graph.
[0014] Preferably, the ONNX source file includes: indexes of each node, data flow diagram and each element in the model;
[0015] Among them, through the ONNX source file, each computing operation in the calculation graph of the deep learning algorithm is regarded as a node, and all nodes in the algorithm form a data flow graph without a ring in a topological sorting manner, and the nodes are linked by edges, and the edges represent the data flow between different nodes;
[0016] A node includes: a name, attributes, a called operator, input and output elements; wherein the operator specifies the operation type of the node, and the operation types include Relu and Conv; the input and output standardize the input and output node information of the node;
[0017] The index includes: the number and position of each operator, input and output element in each node to help locate and process each component in the model and facilitate access to each operation and intermediate results of the model.
[0018] Preferably, in step S1, the operator of the splicing type in the model of the ONNX source file is configured, specifically including: if an operator is of the splicing type, the input of the operator needs to be configured in a given order to ensure that the splicing is performed in the order given by the original model.
[0019] Preferably, the step S1 also includes: simplifying the processed operator graph, removing useless operators, and normalizing the non-standard convolution and pooling operators in the modified part of the model.
[0020] Preferably, the model types of the ONNX source file in step S3 include: ResNet, GoogLeNet, VGG, YOLOv3 and SqueezeNet models.
[0021] Preferably, when the model type of the ONNX source file is a ResNet type, shaping the device operator graph specifically includes the following steps:
[0022] (a) Find the activation function node in the device operator graph, merge the activation function node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the activation function node;
[0023] (b) Find the pooling function node in the device operator graph, merge the pooling function node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the pooling function node;
[0024] (c) Find the Add operation node in the device operator graph, merge the Add operation node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the Add operation node;
[0025] (d) Based on the dependencies between different convolutional layers, a recursive method is used to calculate the time it takes for data to reach each layer.
[0026] Preferably, the ONNX source file is a universal pre-trained model storage method, supporting Linux, Mac and Windows operating systems, and supporting different hardware architectures to read models and perform reasoning.
[0027] Compared with the prior art, the present invention has the following beneficial effects:
[0028] 1. The preprocessing method of the present invention realizes the deployment of the ONNX pre-trained model on the storage and computing integrated platform, greatly improving the adaptability of the storage and computing integrated platform to the existing artificial intelligence model.
[0029] 2. The preprocessing method of the present invention aims at the characteristics of the storage-computing integrated platform and further optimizes the performance of the ONNX pre-trained model on the storage-computing integrated platform by fusing some nodes. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a flow chart of a model preprocessing method of a storage-computation-in-one intelligent computing architecture of the present invention. DETAILED DESCRIPTION
[0031] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the accompanying drawings are in very simplified form and in non-precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.
[0032] The present invention provides a model preprocessing method for a storage-computation integrated intelligent computing architecture. The ONNX file format is one of the common pre-trained model storage methods in the industry today, supporting operating systems such as Linux, Mac, and Windows, and supporting different hardware architectures to read models and perform reasoning. This file regards each computing operation in the calculation graph of the deep learning algorithm as a node. All nodes in the algorithm form a data flow graph without a ring in a topologically sorted manner. The nodes are linked by edges, and the edges represent the data flow between different nodes. In the ONNX file, a node includes its name, attributes, called operators, inputs, outputs, and other parts. Among them, the operator specifies the operation type of the node, including Relu, Conv, etc.; the input and output standardize the input and output node information of the node. In addition to the data flow graph and each node, the ONNX file also includes the index of each element in the model, including the number and position of each element (such as input, output, operator, etc.) in each node, which helps to locate and process each component in the model and facilitates access to each operation and intermediate result of the model.
[0033] The process of preprocessing the ONNX model file for the storage-computing integrated chip platform can be divided into four steps: first, parse the source file to obtain useful information of each node, discard useless information, and obtain the original operator graph; second, remove the operators in the original operator graph that are not suitable for the storage-computing integrated computing mode, and form a device operator graph for the storage-computing integrated system and a host operator graph for the host (CPU); third, further process the device operator graph, and reshape its shape and structure into an operator graph that can adapt to the storage-computing integrated computing mode; finally, save the host operator graph and device operator graph that have gone through the above steps. Next, each step will be briefly described.
[0034] When parsing the source file, the ONNX model file is read first. Then, its index and each node are traversed to check whether there are operations that are completely unsupported by the storage and computing system (such as Sigmoid operations), and the operation type of each node is converted to the corresponding operator, and the predecessor and successor node information of the node is recorded. Then, the operators are classified, including convolution, matrix multiplication, pooling and other types, and the splicing type operators are configured.
[0035] Next, a depth-first search algorithm is used to find operation nodes that can only be performed by the host. At this node, the original graph is split into a host operator graph and a device operator graph, and the graphs are saved.
[0036] Then, the device operator graph is optimized. According to the specific model type, some operators are fused to further optimize the performance of the device operator graph on the storage-computing integrated architecture. The time it takes for data to reach each layer is then calculated, and the processed host operator graph and device operator graph are saved.
[0037] Finally, the generated host operator graph and device operator graph are written to the file system for storage.
[0038] As a further elaboration of the technical solution of the embodiment of the present invention, Figure 1 As shown, the embodiment of the present invention specifically provides a model preprocessing method for a storage-computing integrated intelligent computing architecture:
[0039] Step S1: Read and parse the ONNX source file;
[0040] (1) Read the ONNX model file, traverse the index of the file and each node in the file, and check whether there are operations that are completely unsupported by the storage and computing system (such as Sigmoid operations).
[0041] (2) Convert the operation type corresponding to each node into the corresponding operator, and read and record the predecessor nodes and edges of each node. Because the fusion type node has multiple inputs, it is necessary to traverse all its inputs to complete this process. In addition, it is also necessary to read and record the successor nodes and edges of each node.
[0042] (3) Classify the operation type of each operator. According to the nature of the operation, it can be divided into types such as convolution (Conv), matrix multiplication (Gemm), and pooling (Pool), so that the model can be subsequently mapped to the storage and computing integrated chip.
[0043] (4) Configure the concatenation type operator in the model. If an operator is of the concatenation type, the input of the operator needs to be configured in the given order to ensure that the concatenation is performed in the order given in the original model.
[0044] (5) Simplify the processed operator graph, remove useless operators, and normalize the irregular convolution and pooling operators in the modified model.
[0045] Step S2: Split the host operator graph and the device operator graph; use a depth-first search algorithm on the operator graph generated in the previous step to split the part of the graph containing operators that can only be executed by the host into the host operator graph, and the rest into the device operator graph.
[0046] Step S3: further processing and optimizing the device operator graph;
[0047] (1) Delete operators in the host operator graph from the device operator graph.
[0048] (2) Reshape the device operator graph according to the specific type of the original input ONNX model (currently supports ResNet, GoogLeNet, VGG, YOLOv3 and SqueezeNet models). Taking the ResNet type model as an example, the following steps are required:
[0049] (a) Find the activation function node in the device operator graph, merge it with its predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the activation function node.
[0050] (b) Find the pooling function node in the device operator graph, merge it with its predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the pooling function node.
[0051] (c) Find the Add operation node in the device operator graph, merge it with its predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the Add operation node.
[0052] (d) Based on the dependencies between different convolutional layers, a recursive method is used to calculate the time it takes for data to reach each layer.
[0053] Step S4: Save the processed and optimized host operator graph and device operator graph.
[0054] The preprocessing method of the present invention realizes the deployment of the pre-trained model on the storage-computing integrated platform by converting, splitting and optimizing the calculation graph in the ONNX file format into a host operator graph suitable for host execution and a device operator graph suitable for storage-computing integrated device execution, thereby greatly improving the versatility of the platform.
[0055] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.
Claims
1. A model preprocessing method for a storage-computing integrated intelligent computing architecture, characterized in that: The steps include: Step S1: Read and parse the ONNX source file; that is, after reading the ONNX source file, traverse its index and each node, check whether there is a Sigmoid operation that is completely unsupported by the storage and computing system, and convert the operation type of each node into the corresponding operator, record the predecessor and successor nodes of the node, and the edge information, and then classify the operation type of each operator into convolution, matrix multiplication and pooling types, and configure the splicing type operator in the model of the ONNX source file; Step S2: Splitting the host operator graph and the device operator graph; that is, using a depth-first search algorithm, searching for operations in the operator graph that are not suitable for execution by the storage-computing integrated system, and splitting the operations at the operations to form a device operator graph and a host operator graph; Step S3: further process and optimize the device operator graph; that is, delete the redundant operators in the host operator graph in the device operator graph, and then further optimize the operation efficiency of the device operator graph on the storage-computing integrated architecture by integrating some operations in the device operator graph according to the model type of the ONNX source file; Step S4: Save the processed and optimized host operator graph and device operator graph.
2. The model preprocessing method of the storage-computation integrated intelligent computing architecture according to claim 1, characterized in that: The ONNX source file includes: indexes of each node, data flow diagram and each element in the model; Among them, through the ONNX source file, each computing operation in the calculation graph of the deep learning algorithm is regarded as a node, and all nodes in the algorithm form a data flow graph without a ring in a topological sorting manner, and the nodes are linked by edges, and the edges represent the data flow between different nodes; A node includes: a name, attributes, a called operator, input and output elements; wherein the operator specifies the operation type of the node, and the operation types include Relu and Conv; the input and output standardize the input and output node information of the node; The index includes: the number and position of each operator, input and output element in each node to help locate and process each component in the model and facilitate access to each operation and intermediate results of the model.
3. The model preprocessing method of the storage-computation integrated intelligent computing architecture according to claim 1, characterized in that: In the step S1, the operator of the splicing type in the model of the ONNX source file is configured, specifically including: if an operator is of the splicing type, the input of the operator needs to be configured in a given order to ensure that the splicing is performed in the order given by the original model.
4. The model preprocessing method of the storage-computation integrated intelligent computing architecture according to claim 3, characterized in that: The step S1 also includes: simplifying the processed operator graph, removing useless operators, and normalizing the non-standard convolution and pooling operators in the modified part of the model.
5. The model preprocessing method of the storage-computation integrated intelligent computing architecture according to claim 1, characterized in that: The model types of the ONNX source file in step S3 include: ResNet, GoogLeNet, VGG, YOLOv3 and SqueezeNet models.
6. A model preprocessing method for a storage-computation integrated intelligent computing architecture as claimed in claim 5, characterized in that: When the model type of the ONNX source file is a ResNet type, the device operator graph is reshaped, specifically including the following steps: (a) Find the activation function node in the device operator graph, merge the activation function node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the activation function node; (b) Find the pooling function node in the device operator graph, merge the pooling function node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the pooling function node; (c) Find the Add operation node in the device operator graph, merge the Add operation node with the predecessor node, change the edge from the original activation function node to the successor node to the edge from the predecessor node to the successor node, and then delete the Add operation node; (d) Based on the dependencies between different convolutional layers, a recursive method is used to calculate the time it takes for data to reach each layer.
7. A model preprocessing method for a storage-computation-in-one intelligent computing architecture as described in any one of claims 1 to 6, characterized in that: The ONNX source file is a universal way to store pre-trained models, supporting Linux, Mac, and Windows operating systems, as well as supporting model reading and reasoning on different hardware architectures.
