Transformation method and deployment method of binary neural network model and computer
By replacing continuous operations by custom BNN combinatorial operators, a concise ONNX model is formed, which solves the problem of excessive storage and computing resources of binary neural network models in low-power chip scenarios, and achieves efficient deployment and computing efficiency.
Patent Information
- Application Number
- CN202510357711.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-24
- Publication Date
- 2025-08-01
AI Technical Summary
In the prior art, when deploying to a dedicated chip, the storage space occupies too high and the computing resources consumes too much, which seriously restricts its application in low-power chip scenarios.
Custom BNN combinatorial operators are adopted, including convolutional binarized combinatorial operators, binary convolutional binarized combinatorial operators and binarized fully connected combinatorial operators. By replacing multiple consecutive operations with matching BNN combinatorial operators, the final ONNX model is formed, and compiled and deployed using an AI compiler.
The computing efficiency and deployment flexibility of the binary neural network model are optimized, the computing overhead is reduced, and the inference performance is improved.
Smart Images
Figure CN120409574A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of binary neural network models, and in particular to a conversion method and a deployment method for binary neural network models based on the ONNX standard, a storage medium, and a computer.
Background Art
[0002] Currently, in the process of deploying a neural network model to a dedicated chip (Neural Processing Unit, abbreviated as NPU for example), it is usually necessary to convert a pre-trained model into an Open Neural Network Exchange format (abbreviated as ONNX) or a similar data format (such as tflite), and then use an AI (Artificial Intelligence) compiler to adapt to different hardware platforms to improve compatibility. However, after converting a binary neural network (BNN) model into the ONNX format according to the ONNX standard and compiling the ONNX format binary neural network model on an AI compiler, there are still problems of excessive storage space occupation and excessive consumption of computing resources during deployment, which seriously restricts its application in low-power chip scenarios. In view of the above defects, there is an urgent need to construct a general deployment framework for BNN to optimize the computing efficiency at the operator level, thereby reducing the hardware resource requirements.
Summary of the Invention
[0003] One of the objectives of the present invention is to provide a conversion method and a deployment method for binary neural network models based on the ONNX standard, a storage medium, and a computer, which can support the compilation and deployment of binary neural network models with lower hardware resources.
[0004] According to one aspect of the present invention, there is provided a conversion method for a binary neural network model based on the ONNX standard, which includes: converting the binary neural network model into an initial ONNX model based on the ONNX standard, where the ONNX standard includes one or more custom BNN combined operators, and each BNN combined operator includes two or more operations; and traversing each operation of the initial ONNX model, and replacing consecutive multiple operations in the initial ONNX model that match any one BNN combined operator with the matching BNN combined operator to form a final ONNX model.
[0005] According to another aspect of the present invention, the present invention provides a method for deploying a binary neural network model based on the ONNX standard, which includes: converting the binary neural network model into an initial ONNX model based on the ONNX standard, where the ONNX standard includes one or more custom BNN combination operators, and each BNN combination operator includes two or more operations; traversing each operation of the initial ONNX model, and replacing a continuous plurality of operations that match a BNN combination operator with the matching BNN combination operator to form a final ONNX model; using an AI compiler to compile the final ONNX model, where the compiler can recognize and compile the BNN combination operator in the final ONNX model; and deploying the compiled final ONNX model to run on a neural network processing unit, where the BNN combination operator in the compiled final ONNX model also runs on the neural network processing unit.
[0006] According to another aspect of the present invention, the present invention provides a storage medium that stores program instructions, and the program instructions are run to execute the above conversion method or the above deployment method.
[0007] According to another aspect of the present invention, the present invention provides a computer, which includes a processor and a memory, the memory stores program instructions, and the processor runs the program instructions to execute the above conversion method or the above deployment method.
[0008] Compared with the prior art, the present invention uses one or more custom BNN combination operators, and each BNN combination operator includes two or more operations, so that it can support the compilation and deployment of binary neural network models with lower hardware resources, improve the computing efficiency and deployment flexibility of binary neural network models in the ONNX framework, help optimize the inference performance and reduce the computing overhead.
Description of the Drawings
[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts. Among them:
[0010] Figure 1 It is a flowchart of the conversion method of the binary neural network model based on the ONNX standard in an embodiment of the present invention;
[0011] Figure 2 It is a flowchart of the deployment method of the binary neural network model based on the ONNX standard in an embodiment of the present invention;
[0012] Figure 3a It is an example of the convolutional binarization combination operator ConvBin;
[0013] Figure 3b It is an example of the binarized convolutional binarization combination operator BinConvBin;
[0014] Figure 3c It is an example of the binarized fully connected combination operator BinLinear;
[0015] Figure 4a It is an example of a binary neural network model in the present invention;
[0016] Figure 4b It is Figure 4a a conversion example of the binary neural network model in based on the ONNX standard, where the left figure is the initial ONNX model and the right figure is the final ONNX model.
Detailed Implementation Manner
[0017] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.
[0018] As used herein, "an embodiment" or "embodiment" refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in an embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments.
[0019] The present invention provides a method for converting a binary neural network model based on the ONNX standard, which can support the compilation and deployment of the binary neural network model with lower hardware resources, improve the computational efficiency and deployment flexibility of the binary neural network model in the ONNX framework, and help optimize the inference performance and reduce the computational overhead.
[0020] Figure 1 It is a flowchart of the method for converting a binary neural network model based on the ONNX standard in an embodiment of the present invention. As Figure 1 shown, the conversion method includes the following steps.
[0021] Step 110, converting the binary neural network model into an initial ONNX model based on the ONNX standard, where the ONNX standard includes one or more custom BNN combination operators, and each BNN combination operator includes two or more operations. In addition, the ONNX standard also includes basic ONNX standard operators, and since the ONNX standard operators belong to the prior art, they will not be elaborated in the present invention.
[0022] Step 120, traverse each operation of the initial ONNX model, and replace consecutive multiple operations in the initial ONNX model that match any one BNN combination operator with the matching BNN combination operator to form a final ONNX model.
[0023] In one embodiment, compile the final ONNX model using an AI compiler, where the AI compiler can recognize and compile the BNN combination operators in the final ONNX model; deploy the compiled final ONNX model to run on a neural network processing unit, where the BNN combination operators in the compiled final ONNX model also run on the neural network processing unit (NPU).
[0024] In one embodiment, the BNN combination operator includes one or more of the following operators: a convolutional binary combination operator; a binary convolutional binary combination operator; a binary fully connected combination operator. Of course, in other embodiments, other operators may also be included.
[0025] Specifically, the convolutional binary combination operator ConvBin can be as Figure 3a shown. As Figure 3a shown, the convolutional binary combination operator ConvBin is:
[0026] Data and parameters:
[0027] Input data: X
[0028] Internal weight: W
[0029] Internal bias: Bias
[0030] Calculation flow: {
[0031] Y = Conv(X, W, k, p, s)
[0032] Y = Y + Bias
[0033] Y = sign(Y + 10 -6 )
[0034] }
[0035] Output: Y
[0036] Among them, the input data is X, W is the convolution kernel weight (internal weight) after fixed-point quantization, and its data dimension is Co*k*Ci, where Co is the number of output channels of the convolution, k is the convolution kernel size, and Ci is the number of input channels of the convolution; Bias is the bias term (i.e., the convolution kernel bias term of fixed-point quantization), and its dimension is Co; s is the convolution stride strides, p is the padding option pads, k is the convolution kernel size ksize, and s, p, and k are convolution-related parameters.
[0037] The calculation flow included in the convolutional binarization combination operator includes:
[0038] Fixed-point convolution operation Conv;
[0039] Adding the bias term Bias to the output data stream after convolution;
[0040] Binarization operation sign,
[0041] where 10 -6 To prevent zero small amounts and prevent the numerical value 0 from being quantized to 0 by the sign function;
[0042] The convolutional binarization combination operator ConvBin integrates fixed-point convolution and binarization processing of the output result, avoiding the storage and transmission of intermediate data generated by step-by-step operations and improving the calculation efficiency.
[0043] Specifically, the binarized convolutional binarization combination operator BinConvBin can be as Figure 3b shown. As Figure 3b shown, the binarized convolutional binarization combination operator BinConvBin is:
[0044] Data and parameters:
[0045] Input data: X
[0046] Internal weight: BinW = {-1, 1}
[0047] Internal bias: Bias
[0048] Calculation flow: {
[0049] BinX = sign(X + 10 -6 )
[0050] Y = BinConv(BinX, BinW, k, p, s)
[0051] Y = Y + Bias
[0052] Y = sign(Y + 10 -6 )
[0053] }
[0054] Output: Y
[0055] Among them, the input data is X; BinW is the binarized weight (specifically the binarized convolutional kernel weight in this combination operator), its data dimension is Co * k * Ci, the value is ±1, Co is the number of output channels of the convolution, k is the size of the convolutional kernel, Ci is the number of input channels of the convolution; Bias is the bias term; s is the convolution stride, p is the padding option, k is the convolutional kernel size, and s, p, and k are convolution-related parameters.
[0056] The operation flow included in the binarized convolutional binarization combination operator BinConvBin includes:
[0057] Binarization Bin;
[0058] Binarized convolution operation BinConv;
[0059] Adding a bias term to the output data stream after binarized convolution;
[0060] Performing binarization quantization on the output result.
[0061] The binarized convolutional binarization combination operator BinConvBin is used to binarize the input data, then perform a binarized convolution operation and directly binarize the output result, so as to reduce the computational complexity and storage requirements, avoid the storage and transmission of intermediate data generated by step-by-step operations, and improve the computational efficiency.
[0062] Specifically, the binarized fully connected combination operator BinLinear can be as Figure 3c shown. As Figure 3c shown, the binarized fully connected combination operator BinLinear is:
[0063] Data and parameters:
[0064] Input data: X
[0065] Internal weight: BinW = {-1, 1}
[0066] Calculation flow: {
[0067] BinX = sign(X + 10 -6 )
[0068] Y = BinLinear(BinX, BinW)
[0069] Y = Y + Bias
[0070] }
[0071] Output: Y
[0072] Among them, the input data is X; BinW is the binarized weight (specifically, the binarized fully-connected layer weight in this combined operator), its data dimension is Co * Ci, the value is ±1, Co is the number of output channels of the convolution, and Ci is the number of input channels of the convolution; Bias is the bias term, and its dimension is Co; s is the convolution stride strides, p is the padding option pads, and k is the convolution kernel size ksize. s, p, and k are convolution-related parameters.
[0073] The operation flow included in the binarized fully-connected combined operator BinLinear includes:
[0074] Binarization Bin;
[0075] Binarized fully-connected operation BinLinear;
[0076] Adding a bias term to the output data stream after the binarized fully-connected operation.
[0077] The binarized fully-connected combined operator BinLinear is used to implement the binarized fully-connected layer operation, further reducing the storage and computing requirements of the fully-connected calculation.
[0078] Figure 4a This is an example of a binary neural network model in the present invention; Figure 4b is Figure 4a An example of the conversion of a binary neural network model based on the ONNX standard, where the left figure is the initial ONNX model and the right figure is the final ONNX model.
[0079] In this way, through one or more custom BNN combined operators in the present invention, the final ONNX model of the binary neural network model is made more concise. After such a final ONNX model is compiled and deployed, the intermediate data storage and transmission generated by the step-by-step operation are avoided, and the computing efficiency is improved. In this way, the binary neural network model can be compiled and deployed with lower hardware resources, that is, the computing efficiency and deployment flexibility of the binary neural network model in the ONNX framework are improved, which helps to optimize the inference performance and reduce the computing overhead.
[0080] Figure 2The flowchart of a deployment method of a binary neural network model based on the ONNX standard in an embodiment of the present invention.
[0081] As Figure 2 shown, the deployment method includes the following steps.
[0082] Step 210, convert the binary neural network model into an initial ONNX model based on the ONNX standard, where the ONNX standard includes one or more custom BNN combination operators, and each BNN combination operator includes two or more operations;
[0083] Step 220, traverse each operation of the initial ONNX model, and replace consecutive multiple operations that match a BNN combination operator with the matching BNN combination operator to form a final ONNX model;
[0084] Step 230, compile the final ONNX model using an AI compiler, where the compiler can recognize and compile the BNN combination operators in the final ONNX model; and
[0085] Step 240, deploy the compiled final ONNX model to run on a neural network processing unit, where the BNN combination operators in the compiled final ONNX model also run on the neural network processing unit.
[0086] Specifically, the BNN combination operator includes one or more of the following operators: convolution binarization combination operator; binarized convolution binarization combination operator; binarized fully connected combination operator. For the specific structures of the convolution binarization combination operator; the binarized convolution binarization combination operator; and the binarized fully connected combination operator, reference can be made to the above-mentioned conversion method, and the description will not be repeated here.
[0087] In this way, through one or more custom BNN combination operators of the present invention, the final ONNX model of the binary neural network model is made more concise. After the final ONNX model is compiled and deployed, the intermediate data storage and transmission generated by step-by-step operations are avoided, and the computing efficiency is improved. In this way, the deployment of the binary neural network model on the neural network processing unit can be achieved with lower hardware resources.
[0088] According to another aspect of the present invention, the present invention provides a storage medium that stores program instructions, and when the program instructions are executed, they are run to execute the above-mentioned conversion method or the above-mentioned deployment method. For simplicity, the specific contents of the conversion method of the binary neural network model based on the ONNX standard and the deployment method of the binary neural network model based on the ONNX standard are not repeated here.
[0089] According to another aspect of the present invention, the present invention provides a computer, which includes a processor and a memory, and program instructions are stored in the memory, and the processor runs the program instructions to execute the conversion method or the deployment method described above. For simplicity, the specific contents of the conversion method of the binary neural network model based on the ONNX standard and the deployment method of the binary neural network model based on the ONNX standard are not repeated here.
[0090] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples", etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.
[0091] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications and variations to the above embodiments within the scope of the present invention.
Claims
1. A conversion method for a binary neural network model based on the ONNX standard, characterized in that, It includes: Converting the binary neural network model into an initial ONNX model based on the ONNX standard, where the ONNX standard includes one or more custom BNN combination operators, and each BNN combination operator includes two or more operations; and Traversing each operation of the initial ONNX model, and replacing consecutive multiple operations in the initial ONNX model that match any one BNN combination operator with the matching BNN combination operator to form a final ONNX model.
2. The conversion method according to claim 1, characterized in that It further includes: Compiling the final ONNX model using an AI compiler, where the AI compiler can recognize and compile the BNN combination operators in the final ONNX model; and Deploying the compiled final ONNX model to run on a neural network processing unit, where the BNN combination operators in the compiled final ONNX model also run on the neural network processing unit.
3. The conversion method according to claim 1, wherein The BNN combination operator includes one or more of the following operators: Convolution binarization combination operator; Binarized convolution binarization combination operator; Binarized fully connected combination operator.
4. The conversion method according to claim 3, wherein The convolution binarization combination operator is: Data and parameters: Input data: X Internal weight: W Internal bias: Bias Calculation flow: { Y = Conv(X, W, k, p, s) Y = Y + Bias Y = sign(Y + 10 -6 ) } Output: Y where W is the convolutional kernel weight after fixed-point quantization, and its data dimension is Co * k * Ci, Co is the number of output channels of the convolution, k is the convolutional kernel size, and Ci is the number of input channels of the convolution; Bias is the bias term, and its dimension is Co, s is the convolution stride, p is the padding option, k is the convolutional kernel size, The calculation flow Calculation flow included in the convolution binarization combination operator includes: Fixed-point convolution operation Conv; Adding the bias term Bias to the output data stream after convolution; Binarization operation sign, Among them, 10 -6 To prevent small zero values and prevent the value 0 from being quantized to 0 by the sign function; The binarized convolution binarization combination operator is: Data and parameters: Input data: X Internal weight: BinW = {-1, 1} Internal bias: Bias Calculation flow: { BinX = sign(X + 10 -6 ) Y = BinConv(BinX, Binw, k, p, s) Y = Y + Bias Y = sign(Y + 10 -6 ) } Output: Y where BinW is the binarized convolutional kernel weight, and its data dimension is Co * k * Ci, and the value is ±1, and Bias is the bias term; The calculation flow included in the binarized convolution binarization combination operator includes: Binarization Bin; Binarized convolution operation BinConv; Adding the bias term to the output data stream after binarized convolution; Performing binarized quantization on the output result; The binarized fully connected combination operator is: Data and parameters: Input data: X Internal weight: BinW = {-1, 1} Calculation flow: { BinX = sign(X + 10 -6 ) Y = BinLinear(BinX, BinW) Y = Y + Bias } Output: Y Among them, BinW is the binarized weight, whose data dimension is Co * Ci, and the value is ±1 Bias is the bias term, whose dimension is Co, The calculation flow included in this binarized fully-connected combination operator includes: Binarization Bin; Binarized fully-connected operation BinLinear; Adding a bias term to the output data stream after the binarized fully-connected operation.
5. A deployment method for a binary neural network model based on the ONNX standard, characterized in that, It includes: Converting the binary neural network model into an initial ONNX model based on the ONNX standard, where the ONNX standard includes one or more custom BNN combination operators, and each BNN combination operator includes two or more operations; Traversing each operation of the initial ONNX model, and replacing a continuous plurality of operations matching a BNN combination operator with this matching BNN combination operator to form a final ONNX model; Compiling the final ONNX model using an AI compiler, where the compiler can recognize and compile the BNN combination operator in the final ONNX model; and Deploying the compiled final ONNX model to run on a neural network processing unit, where the BNN combination operator in the compiled final ONNX model also runs on the neural network processing unit.
6. The deployment method according to claim 5, wherein The BNN combination operator includes one or more of the following operators: Convolution binarization combination operator; Binarized convolution binarization combination operator; Binarized fully-connected combination operator.
7. A storage medium, characterized in that, It stores program instructions, and the program instructions are run to execute the conversion method according to any one of claims 1-4 or the deployment method according to any one of claims 5-6.
8. A computer, characterized in that, It includes a processor and a memory, and program instructions are stored in the memory, and the processor runs the program instructions to execute the conversion method according to any one of claims 1-4 or the deployment method according to any one of claims 5-6.