ONNX-based general model pruning adaptation method

Abstract

The present invention relates to the technical field of model compression and deployment. Disclosed is an ONNX-based general model pruning adaptation method, comprising: S1, automatic identification of an ONNX model structure, specifically comprising: S1.1, constructing a node attribute library on the basis of operator types of nodes in an ONNX model, wherein the nodes are classified into four attributes: pruned, next-no-process, next-process and stop-process, and the attribute pruned is classified as pruning nodes, and the attributes next-no-process, next-process, and stop-process are classified as associated nodes; and S1.2, on the basis of the node attribute library of S1.1, constructing node association trees for the pruning nodes; and S2, optimization of a pruning adaptation method by means of a tree-level evaluation method, specifically comprising: S2.1, performing tree-level importance evaluation on each node association tree; and S2.2, perform pruning and fine-tuning on the ONNX model. According to the present invention, the interoperability of pruning algorithms in end-side deployment applications is enhanced; the node association trees enable the present invention to adapt to different model structures; and the tree-level evaluation method evaluates different node connection structures without introducing additional components.

Description

A general model pruning and adaptation method based on ONNX Technical Field

[0001] The present invention belongs to the technical field of model compression and deployment, and in particular relates to a general model pruning and adaptation method based on ONNX. Background Art

[0002] In recent years, deep learning has achieved remarkable success in many fields. However, the large number of parameters and computational intensity of deep neural networks have limited their deployment on real-world devices. To alleviate these limitations and accelerate inference while maintaining model performance, researchers have designed numerous DNN model compression methods.

[0003] Among various model compression methods, model pruning is widely used due to its fast speed and strong portability. Model pruning methods can be divided into two main categories. One is unstructured pruning, which creates a sparse model by resetting the weights of less important filters to zero. However, this requires specialized hardware and software to effectively process sparse matrices. The other is structured pruning, which achieves model compression by comprehensively removing unimportant or redundant filters. This method is independent of the computing framework and hardware characteristics and has gained wider application.

[0004] However, structured pruning technology faces two main challenges when applied to end-side deployment. First, existing pruning algorithms are usually bound to specific deep learning frameworks and are difficult to convert on different platforms. For example, developers need to re-adapt the pruning algorithm for each framework (such as Keras, PyTorch, PaddlePaddle, etc.), then convert them into ONNX (Open NeuralNetwork Exchange) format, and then deploy them on the end-side device through the compiler. This process is time-consuming and labor-intensive. Second, due to the complex internal connections between nodes in DNN, developers must tailor pruning programs for different model structures. Although some studies have attempted to solve this problem by constructing a relationship chain between pruned nodes and associated nodes, they usually lack clear modeling and require the insertion of additional components and operations into the model.

[0005] To address these challenges, we propose a universal model pruning adaptation method based on ONNX, namely ONNXPruner, based on the ONNX development framework to enhance the interoperability of pruning algorithms in application systems. ONNX is a cross-platform deep learning model exchange format that can be easily converted and deployed between different deep learning frameworks and hardware platforms.

[0006] Moreover, the format has been supported by various deep learning frameworks and hardware acceleration platforms. Therefore, pruning research is conducted on the ONNX model to improve the flexibility of the application of pruning algorithms. Secondly, in order to solve the cascade effect between nodes, ONNXPruner constructs a node association tree for each pruned node in the model, so that the pruning algorithm can automatically adapt to various model structures. In addition, in structured pruning, the parameters of the pruned nodes and the associated nodes are removed at the same time. However, the existing pruning algorithms only evaluate the pruned nodes and cannot reflect their correctness. In order to solve this problem, the present invention designs a tree-level pruning method based on the node association tree, which can more safely remove unimportant weights and can handle complex node association structures without introducing any additional components or operations.

[0007] Summary of the Invention

[0008] Purpose of the Invention: This invention addresses the challenges faced by model pruning technology in on-device model deployment, such as the rigid binding to development frameworks and the non-universal nature of model structures. By proposing a universal model pruning and adaptation method based on ONNX, this method directly targets on-device deployment of deep learning models, unconstrained by the type of development framework. By constructing a node association tree and designing a tree-level evaluation method, this method enables automated pruning and optimization of any model structure.

[0009] In order to solve the technical problems existing in the above background technology, the present invention provides a general model pruning adaptation method based on ONNX, comprising:

[0010] S1, automatic identification of ONNX model structure, including:

[0011] S1.1. Construct a node attribute library based on the operator type of the node in the ONNX model. The nodes are divided into four attributes: pruned, next-no-process, next-process, and stop-process. The pruned attribute is classified as a pruned node, and the next-no-process, next-process, and stop-process attributes are classified as associated nodes.

[0012] S1.2, constructing a node association tree for the pruned nodes based on the node attribute library of S1.1;

[0013] S2, optimizing the pruning adaptation method by a tree-level evaluation method; specifically comprising:

[0014] S2.1, performing tree-level importance evaluation on each node association tree;

[0015] S2.2, prune and fine-tune the ONNX model.

[0016] Preferably, the pruned node refers to the layer that needs to be pruned in the ONNX model, and the associated node refers to the node that is modified accordingly after being pruned, as well as all nodes in the chain.

[0017] Preferably, the S1.2 specifically includes:

[0018] S1.2.1, constructing the pruned node as the root node of the node association tree;

[0019] S1.2.2. Find all the associated nodes of the pruned node according to the node attribute library, set the associated nodes with the attributes of next-no-process and next-process as child nodes of the node association tree; set the associated nodes with the attribute of stop-process as leaf nodes of the node association tree.

[0020] Preferably, the tree-level evaluation method uses a tree-level pruning structure on the node association tree, which is divided into four cases: one-to-one connection, one-to-many connection, many-to-one connection and many-to-many connection;

[0021] The one-to-one connection includes:

[0022] For the input feature F n , Perform convolution and output the i-th feature map as the calculation of the next layer;

[0023] The next convolutional layer for The filter corresponding to the output channel;

[0024] The calculation form for evaluating the pruned nodes is as follows:

[0025] in, is the i-th weight of the pruned node, represents the i-th channel of the k-th weight of the associated node, and j represents W n+1 The number of

[0026] The one-to-many connection refers to one pruning node corresponding to multiple associated nodes, including:

[0027] For the input feature F n , Perform convolution and output the feature map as the calculation for the next layer;

[0028] In the next convolutional layer, and yes The filter corresponding to the output channel;

[0029] The calculation form for evaluating the pruned nodes is as follows:

[0030] Where j1 is The number of j2 is The number of

[0031] The many-to-one connection includes:

[0032] For the input feature F n , there will be a coupling node F n-1 ,Will and Respectively with F n and F n-1 Perform convolution to generate two output feature maps;

[0033] Add the two generated output feature maps as the calculation of the next layer;

[0034] The calculation form for evaluating the pruned nodes is as follows:

[0035] where j is The number of

[0036] The many-to-many connection is a fusion of the one-to-many connection and the many-to-one connection, wherein the calculation form of the evaluation of the pruned node is as follows:

[0037] Beneficial effects of the present invention:

[0038] 1. This paper proposes a general model pruning adaptation method based on ONNX, which enhances the interoperability of pruning algorithms in end-to-end deployment applications. It is simple to use and highly efficient, allowing developers to quickly deploy pruning algorithms on end-to-end devices.

[0039] 2. A node association tree is proposed to explicitly construct the relationship between pruned nodes and associated nodes, enabling the present invention to handle different model structures;

[0040] 3. A tree-level evaluation method based on node association tree is proposed. This method can evaluate various node connection structures without introducing additional components and has better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] FIG1 is a flow chart of the present invention;

[0042] FIG2 is a schematic diagram of the construction of a node association tree in the present invention;

[0043] FIG3 is a schematic diagram of a tree-level evaluation method according to the present invention;

[0044] FIG4 is a schematic diagram of a tree-level pruning structure in the present invention;

[0045] FIG5 is a schematic diagram of filter index differences after embedding the pruning method of the present invention;

[0046] FIG6 is a schematic diagram of the experimental results of pruning the model layer by layer in the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] Example 1

[0049] This embodiment discloses a general model pruning adaptation method based on ONNX, namely ONNXPruner. ONNXPruner is based on ONNX as a development framework to enhance the interoperability of pruning algorithms in application systems. A node association tree is constructed for each pruned node in the model, so that the pruning algorithm can automatically adapt to various model structures. In addition, a tree-level evaluation method based on the node association tree is designed, which can handle complex node association structures without introducing any additional components or operations. The flowchart of the present invention is shown in Figure 1, including:

[0050] S1, automatic identification of ONNX model structure, including:

[0051] S1.1 Build a node attribute library based on the operator type of the node in the ONNX model. Nodes are divided into four attributes: pruned, next-no-process, next-process, and stop-process. The pruned attribute is classified as a pruned node; the next-no-process, next-process, and stop-process attributes are classified as associated nodes.

[0052] ONNXPruner identifies all pruned nodes and their associated nodes in the ONNX model. Pruned nodes are layers in the model that need to be pruned, and associated nodes are nodes that need to be modified after pruning a pruned node, as well as all nodes in the chain. To achieve automatic model pruning, it is necessary to explicitly find the associated nodes of the pruned node and their relationships. Generally, our goal is to find all the associated nodes of the pruned node and group them together. To achieve this goal, we first build a node attribute library that divides nodes into four attributes based on the operator type: pruned, next-no-process, next-process, and stop-process.

[0053] S1.2 builds a node association tree for all pruned nodes in the ONNX model based on the node attribute library in step S1.1, as shown in Figure 2;

[0054] A node association tree is constructed for all pruned nodes in the ONNX model. The node association tree is constructed as follows: First, the ONNX node graph is traversed, with the pruned node as the root of the node association tree. All nodes that take the root node's output as input are found, and these nodes are considered children of the root node. Each child node is then evaluated based on its properties. If the node is a next-no-process or next-process node, the process is repeated by treating each child node as a parent node. If it is a stop-process node, the search for its children is terminated and it is considered a leaf node. The above method can generate a node association tree for pruned nodes of any model.

[0055] S2, optimizes the pruning adaptation method through tree-level evaluation method; specifically includes:

[0056] S2.1 performs tree-level importance evaluation on each node association tree; Figure 3 gives an example of building a node association tree for each pruned node in SqueezeNet.

[0057] Existing pruning algorithms are adapted to the pruning adapter, and tree-level pruning is performed on each node association tree. Model pruning methods are primarily applied by pruning a pre-trained model and then fine-tuning the pruned model to recover performance losses caused by pruning. Although some studies have incorporated pruning methods into model training before and during training, the implementation of these methods has become difficult due to large data volumes and long training cycles. The tree-level pruning structure on the node association tree is divided into four cases: one-to-one connections, one-to-many connections, many-to-one connections, and many-to-many connections, as shown in Figure 4.

[0058] One-to-one connection is the most basic node connection structure in the DNN model. n , Perform convolution on it and output the i-th feature map as the calculation for the next layer. The next convolution layer for The filter corresponding to the output channel. Therefore, the calculation form of the evaluation of the pruned node in the one-to-one structure is as follows:

[0059] in is the i-th weight of the pruned node, represents the i-th channel of the k-th weight of the associated node, and j represents W n+1 The number of .

[0060] One-to-many connection means that one pruned node corresponds to multiple associated nodes. n , Instead of performing convolution, the output feature map is used as the calculation for the next layer. In the next convolutional layer, and yes The filter corresponding to the output channel. Therefore, the evaluation of the pruned node in the one-to-many structure is defined as

[0061] Where j1 is The number of j2 is The number of .

[0062] Many-to-one connections are also commonly found in DNN models, such as residual connections. For the input feature F n , there will be a coupling node F n-1 . and Respectively with F n and F n-1 Convolution is performed to generate two output feature maps. They are then added together as the calculation for the next layer. Therefore, the evaluation of the pruned node in the many-to-one structure is expressed as:

[0063] where j is The number of .

[0064] Many-to-many connections are actually a fusion of one-to-many connections and many-to-one connections. The evaluation of pruned nodes in the many-to-many structure is expressed as:

[0065] S2.2 Prune and fine-tune the model based on tree-level importance evaluation.

[0066] Based on the filter importance evaluated in S1.2, the model is pruned and fine-tuned. To evaluate and fine-tune the pruned ONNX model, ONNX Runtime is used for model inference and backpropagation training. ONNX Runtime is an engine that supports models based on the ONNX format. It can be integrated with hardware-specific libraries using a flexible interface, allowing efficient training and inference across various platforms and hardware on CPUs and GPUs. As it is an existing technology, it will not be described in detail here.

[0067] Example 2

[0068] This embodiment discloses the content and specific configuration of the present invention, and analyzes the experimental results to verify the effectiveness of the present invention.

[0069] ONNXPruner was evaluated on three benchmark datasets: CIFAR-10, CIFAR-100, and PASCAL VOC 2012. The CIFAR-10 and CIFAR-100 datasets contain the same number of images, a total of 60,000 RGB images, with 50,000 training images and 10,000 test images, representing 10 and 100 categories, respectively. PASCAL VOC 2012 is a semantic segmentation benchmark that includes 20 object classes and one background class, with 1,464 images for training, 1,449 for validation, and 1,456 for testing. The original dataset was augmented with a semantic boundary dataset, resulting in 10,582 images for training.

[0070] For the CIFAR dataset, evaluation is performed on multiple popular network models, including AlexNet, VGG16, VGG19, SqueezeNet, ResNet18, and ViT-B 16. For the PASCAL VOC 2012 dataset, ONNXPruner is evaluated on the FCN and PSPNet models.

[0071] To verify the effectiveness of tree-level evaluation, Figure 5 shows the difference in filter index between ONNXPruner (l1-norm) and l1-norm, and ONNXPruner (l2-norm) and l2-norm. The difference in filter index is calculated as: The structure shows that ONNXPruner (ln-norm) and ln-norm differ in filter indices at different layers. The difference becomes more significant when the pruning rate is small. In particular, when the pruning rate is less than 0.3, the difference exceeds 20%. Notably, the difference decreases as the pruning rate increases, which is mainly due to the increased coverage of the pruned channels.

[0072] To validate the tree-level evaluation, we further tested the accuracy of the pruned model, as shown in Figure 6. This experiment directly tested the pruned model to eliminate any potential effects of fine-tuning. All convolutional and fully connected layers in AlexNet were pruned using the CIFAR-10 dataset, with pruning rates set between 0.1 and 0.9. The results show that the proposed ONNXPruner method improves pruning accuracy at various pruning rates. As expected, when the pruning rate reaches 0.9, the accuracy approaches random chance due to the significant degradation of the model weights.

[0073] First, we validated the performance of ONNXPruner using different pruning rates on CIFAR10 and VGG16. The results for the CIFAR dataset are shown in Table 1. A value of 0.3 removes 30% of the filters in all convolutional and fully connected layers. All pruned models were fine-tuned for 10 epochs using ONNXPruner, with an initial learning rate of 1e-3. l1-norm, l2-norm, and Hrank were embedded in ONNXPruner as baseline algorithms, and tree-level filter importance evaluation was performed based on these algorithms. The results show that, without introducing any additional constraints, ONNXPruner can insert multiple weight evaluation algorithms and achieve superior performance at different pruning rates.

[0074] Table 1 Pruning results of VGG16 on CIFAR-10 at different pruning rates

[0075] To comprehensively evaluate ONNXPruner's performance on various models, we validated a larger number of models with a pruning rate of 0.5, as shown in Table 2. ONNXPruner was compared with Taylor-FO, GReg-2, and reimpl. Reimpl, TPP, and DepGraph primarily optimize filter importance evaluation. Because these methods require additional components and constraints, they were not integrated into ONNXPruner. The results show that ONNXPruner achieves superior performance to baseline methods on a variety of models. It also achieves superior or comparable accuracy compared to methods requiring additional components and constraints. "-" indicates that the corresponding algorithm is not suitable for the model, and the results cannot be displayed.

[0076] Table 2 Performance of ONNXPruner on image classification tasks

[0077] In addition to classification tasks, ONNXPruner was also applied to image segmentation tasks and validated on two commonly used segmentation models, FCN and PSPNet. The PASCAL VOC 2012 detection results are shown in Table 3. mIOU was used as the evaluation metric. Similarly, the baseline methods l1-norm, l2-norm, and Hrank were embedded in ONNXPruner.

[0078] The proposed ONNXPruner can quickly adjust the model structure to achieve pruning. In addition, because ONNXPruner adopts a tree-level pruning method, it can more accurately evaluate the importance of filters. Quantitative results show that ONNXPruner can achieve better results compared with the baseline method.

[0079] Table 3 Performance of ONNXPruner on image segmentation tasks

[0080] The above is a detailed introduction to a general model pruning and adaptation method based on ONNX provided by the present invention. It is worth noting that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical details recorded in the aforementioned embodiments or make equivalent replacements for some of the technical features therein. Any equivalent replacements, modifications, etc. made within the core ideas and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A general model pruning and adaptation method based on ONNX, characterized in that: include: S1, automatic identification of ONNX model structure, including: S1.

1. Construct a node attribute library based on the operator type of the node in the ONNX model. The nodes are divided into four attributes: pruned, next-no-process, next-process, and stop-process. The pruned attribute is classified as a pruned node, and the next-no-process, next-process, and stop-process attributes are classified as associated nodes. S1.2, constructing a node association tree for the pruned nodes based on the node attribute library of S1.1; S2, optimizing the pruning adaptation method by a tree-level evaluation method; specifically comprising: S2.1, performing tree-level importance evaluation on each node association tree; S2.2, prune and fine-tune the ONNX model.

2. A general model pruning and adaptation method based on ONNX according to claim 1, characterized in that: The pruned node refers to the layer that needs to be pruned in the ONNX model, and the associated node refers to the node that is modified accordingly after being pruned, as well as all nodes in the chain.

3. The general model pruning and adaptation method based on ONNX according to claim 1, characterized in that: Said S1.2 specifically includes: S1.2.1, constructing the pruned node as the root node of the node association tree; S1.2.

2. Find all the associated nodes of the pruned node according to the node attribute library, set the associated nodes with the attributes of next-no-process and next-process as child nodes of the node association tree; set the associated nodes with the attribute of stop-process as leaf nodes of the node association tree.

4. The ONNX-based general model pruning and adaptation method according to claim 1, wherein: The tree-level evaluation method uses a tree-level pruning structure on the node association tree, which is divided into four cases: one-to-one connection, one-to-many connection, many-to-one connection and many-to-many connection; The one-to-one connection includes: For the input feature F n , Perform convolution and output the i-th feature map as the calculation of the next layer; The next convolutional layer for The filter corresponding to the output channel; The calculation form for evaluating the pruned nodes is as follows: in, is the i-th weight of the pruned node, represents the i-th channel of the k-th weight of the associated node, and j represents W n+1 The number of The one-to-many connection refers to one pruning node corresponding to multiple associated nodes, including: For the input feature F n , Perform convolution and output the feature map as the calculation for the next layer; In the next convolutional layer, and yes The filter corresponding to the output channel; The calculation form for evaluating the pruned nodes is as follows: Where j1 is The number of j2 is The number of The many-to-one connection includes: For the input feature F n , there will be a coupling node F n-1 ,Will and Respectively with F n and F n-1 Perform convolution to generate two output feature maps; Add the two generated output feature maps as the calculation for the next layer; The calculation form for evaluating the pruned nodes is as follows: where j is The number of The many-to-many connection is a fusion of the one-to-many connection and the many-to-one connection, wherein the calculation form of the evaluation of the pruned node is as follows:

Images