A simplified reasoning method, system, medium and device for binary neural network

By quantizing weights and input data in the binary neural network and performing logical operations in the calculation graph, the problem of slow inference speed of binary neural networks is solved, efficient inference and low computing resource occupation are achieved, and it is suitable for low-configuration equipment.

CN118052283BActive Publication Date: 2024-11-26ZHONGKE LINGWEI (NANJING) TECHNOLOGY CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202410170334.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-02-06
Publication Date
2024-11-26
Estimated Expiration
2044-02-06

AI Technical Summary

Technical Problem

When binary neural networks make inference calculations, binary data needs to be expanded into integer or floating point for multiplication and accumulation calculations, resulting in slow inference speed.

Method used

By reading the calculation graph and weight data from the system configuration file of the neural network, the weight and input data are quantized into a binary matrix based on the matrix quantization information, and logical or operations and logic and operations are performed in the calculation graph, eliminating the multiplication and Addition operation until all calculation nodes are traversed.

Benefits of technology

It realizes efficient inference of binary neural network model, improves inference speed, and consumes less computing resources, which will not cause computing power conflicts. It is suitable for use on low-configuration devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118052283B_ABST
    Figure CN118052283B_ABST
Patent Text Reader

Abstract

The present invention discloses a simplified reasoning method, system, medium and device for a binary neural network, including: reading a neural network calculation graph and weight-related data from a system configuration file of the neural network, and converting the input data and the corresponding weights into a binary matrix. Then, the reasoning calculation of the input matrix and the weight matrix in the neural network calculation graph is controlled according to the reasoning flow. In particular, in the reasoning calculation of the convolution calculation node, the multiplication operation is eliminated by using the logical XOR operation, and the addition operation is eliminated by using the logical AND operation and the character statistics method, so that the efficient reasoning of the binary neural network model can be realized, the reasoning speed can be improved, and the computing resources occupied are very small, and there will be no computing power conflict with low-configuration devices, and it can be deployed on a large scale for use on low-configuration devices.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a simplified reasoning method, system, medium and device for a binary neural network. Background Art

[0002] In recent years, deep learning technology has penetrated into various industries and fields, and the demand for deep learning technology on low-configuration devices such as mobile devices and wearable devices has become increasingly strong. Since deep learning models require a lot of computing and storage resources, direct application on low-configuration devices will cause computing power conflicts. Therefore, binary neural network models (BNN) are usually used to deal with the difficulty of applying deep learning models on low-configuration devices.

[0003] However, the applicant found in the research that although the binary neural network model can achieve maximum compression of the model by using a single bit to represent the input tensor and weights, it is still necessary to expand the binary data into integer or floating-point data for multiplication and accumulation calculations during the network model reasoning calculation, which weakens the reasoning speed of the binary neural network model. Summary of the invention

[0004] The present invention provides a simplified reasoning method, system, medium and device for a binary neural network to solve or partially solve the technical problem of slow reasoning speed of the binary neural network.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a simplified reasoning method of a binary neural network, characterized in that the method comprises:

[0006] Reading a neural network calculation graph and weight-related data from a system configuration file of the neural network; wherein the neural network calculation graph includes a number of calculation nodes and reasoning flows between nodes, and the weight-related data includes weight values ​​and matrix quantization information;

[0007] quantizing the weights into a binary weight matrix based on the matrix quantization information and loading the weights into corresponding computing nodes in the neural network computing graph;

[0008] quantizing the input data into a binary input matrix based on the matrix quantization information;

[0009] Controlling the inference calculation of the input matrix and the weight matrix in the neural network calculation graph according to the inference flow until all the calculation nodes of the neural network calculation graph are traversed to obtain output data;

[0010] Among them, in the inference calculation of the convolution calculation node among the several computing nodes, a logical XOR operation is performed on the sign bit of the node input matrix and the sign bit of the node weight matrix belonging to the convolution calculation node to obtain a first output matrix; a logical AND operation is cyclically performed in the first output matrix according to a set loop order to obtain a second output matrix; based on the output dimension belonging to the convolution calculation node, the second output matrix is ​​converted into a node output matrix to serve as the node input matrix of the next computing node.

[0011] Optionally, the weight is a four-dimensional tensor, which includes: number, number of channels, height, and width; and quantizing the weight into a binary weight matrix based on the matrix quantization information specifically includes:

[0012] Converting the weights from the four-dimensional tensor into a two-dimensional weight matrix;

[0013] The two-dimensional weight matrix is ​​quantized based on the quantization zero point in the matrix quantization information to obtain the weight matrix.

[0014] Optionally, the dimension of the input data is: IH×IW×IC; wherein IH represents the height of the input data, IW represents the width of the input data, and IC represents the number of channels of the input data; and quantizing the input data into a binary input matrix based on the matrix quantization information specifically includes:

[0015] According to the convolution calculation characteristics, the input data is expanded into a two-dimensional input matrix of (IH×IW)×(KH×KW×IC); wherein the expansion method is: KH represents the height of the convolution kernel, KW represents the height of the convolution kernel, and S represents the convolution stride. Indicates rounding down;

[0016] The two-dimensional input matrix is ​​quantized based on the quantization zero point in the matrix quantization information to obtain the input matrix.

[0017] Optionally, performing a logical XOR operation on the sign bit of the node input matrix and the sign bit of the node weight matrix belonging to the convolution calculation node to obtain the first output matrix specifically includes:

[0018] Extracting the sign bit of the node input matrix to obtain an input sign bit matrix;

[0019] Extracting the sign bit of the node weight matrix to obtain a weight sign bit matrix;

[0020] A logical XOR operation is performed on corresponding positions of the input sign bit matrix and the weight sign bit matrix to obtain a first output matrix.

[0021] Optionally, the step of cyclically performing a logical AND operation in the first output matrix according to a set cycle order to obtain a second output matrix specifically includes:

[0022] Traversing the first output matrix row by row, performing a logical AND operation in the first output matrix according to the set loop order according to the operation logic loop of "dividing each row into N groups row by row, dividing each group into K elements, and performing a logical AND operation within the K elements", to obtain an intermediate output matrix; N ≥ 2 and is a positive integer, 1 ≤ k < N and is a positive integer;

[0023] Perform character statistics on the intermediate output matrix, and generate the second output matrix based on the character statistics result.

[0024] Optionally, performing character statistics on the intermediate output matrix and generating the second output matrix based on the character statistics result specifically includes:

[0025] The number of characters "1" contained in the corresponding positions in the intermediate output matrix is ​​counted, and the corresponding positions in the second output matrix are filled in.

[0026] Optionally, after obtaining the output data, the method further includes:

[0027] The output data is converted into a target output matrix according to the output dimension information read from the system configuration file.

[0028] A second aspect of the present invention discloses a simplified reasoning system of a binary neural network, characterized in that the system comprises:

[0029] A reading module, used to read a neural network calculation graph and weight-related data from a system configuration file of the neural network; wherein the neural network calculation graph includes a number of calculation nodes and reasoning flows between nodes, and the weight-related data includes weights and matrix quantization information;

[0030] A first quantization module, configured to quantize the weight into a binary weight matrix based on the matrix quantization information and load the weight matrix into a corresponding computing node in the neural network computing graph;

[0031] A second quantization module, configured to quantize the input data into a binary input matrix based on the matrix quantization information;

[0032] An inference calculation module, used for controlling the inference calculation of the input matrix and the weight matrix in the neural network calculation graph according to the inference flow until all the calculation nodes of the neural network calculation graph are traversed to obtain output data;

[0033] Among them, in the inference calculation of the convolution calculation node among the several computing nodes, a logical XOR operation is performed on the sign bit of the node input matrix and the sign bit of the node weight matrix belonging to the convolution calculation node to obtain a first output matrix; a logical AND operation is cyclically performed in the first output matrix according to a set loop order to obtain a second output matrix; based on the output dimension belonging to the convolution calculation node, the second output matrix is ​​converted into a node output matrix to serve as the node input matrix of the next computing node.

[0034] According to a third aspect of the present invention, a computer-readable storage medium is disclosed, on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented.

[0035] According to a fourth aspect of the present invention, a computer device is disclosed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0036] Through one or more technical solutions of the present invention, the present invention has the following beneficial effects or advantages:

[0037] In the technical solution of the present invention, the neural network calculation graph and weight-related data are read from the system configuration file of the neural network, and the input data and the corresponding weights are converted into a binary matrix. Then, the reasoning calculation of the input matrix and the weight matrix in the neural network calculation graph is controlled according to the reasoning flow. In particular, in the reasoning calculation of the convolution calculation node, the multiplication operation is eliminated by using the logical XOR operation, and the addition operation is eliminated by using the logical AND operation and the character statistics method. Compared with the prior art, it can realize the efficient reasoning of the binary neural network model, improve the reasoning speed, and occupy very few computing resources, and will not cause computing power conflicts with low-configuration devices. It can be deployed on a large scale for use on low-configuration devices.

[0038] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.

[0040] In the attached picture:

[0041] Figure 1 A simplified reasoning method flow chart of a binary neural network according to an embodiment of the present invention is shown;

[0042] Figure 2 A schematic diagram showing an implementation principle of a logical XOR operation according to an embodiment of the present invention is shown;

[0043] Figure 3A A schematic diagram showing the implementation principle of a logic AND operation according to an embodiment of the present invention is shown;

[0044] Figure 3B A logical principle diagram of a logical AND operation of K elements according to an embodiment of the present invention is shown;

[0045] Figure 3C A specific numerical calculation example diagram of a logical AND operation according to an embodiment of the present invention is shown;

[0046] Figure 4 A schematic diagram of a simplified reasoning system of a binary neural network according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0047] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0048] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0049] First, as Figure 1 As shown, the simplified reasoning method of the binary neural network provided in the embodiment of this specification includes at least the following steps:

[0050] S101, read the neural network calculation graph and weight related data from the system configuration file of the neural network.

[0051] Among them, the system configuration file of the neural network contains system configuration information, input source setting information, neural network calculation graph and weight related data. Among them, the system configuration information defines the basic configuration of the operating system to which the neural network belongs. The input source setting information refers to the path of the specified input file or the I / O port of the acquisition device, which can be read from the file or from the acquisition device such as the sensor (such as the camera). The neural network calculation graph contains input dimension information, output dimension information, several computing nodes and the reasoning flow between nodes. The input dimension information defines the input dimension of the input data; the output dimension information defines the output dimension of the output data; the computing node can be understood as a "layer", and the several computing nodes include: convolution computing nodes and other nodes. This solution mainly processes the convolution computing nodes. The reasoning flow between nodes defines the calculation order of each node, and the node output data of the previous computing node is used as the node input data of the next computing node. The weight-related data includes weights and matrix quantization information. The weights involve relevant parameters such as data bit width, data type, and tensor dimension. Among them, data bit width refers to the number of bits used to represent integer data. For example, the binary data bit width is 1, that is, one bit is used to represent the data. The data type is used to indicate whether the data is floating point or integer, signed or unsigned. Before quantization, the data type of the weight is a signed floating point number (float32). The tensor dimension refers to the weight dimension information. For example, the dimension of the convolution weight is [16, 3, 3, 3], which means 16 in number, 3 in number of channels, 3 in height, and 3 in width. Quantization is a process that does not involve dimensional changes. Quantization information includes quantization zero point and binary quantization method. The quantization zero point is the standard for quantization. For example, 0 is the quantization zero point, and the binary quantization method is: quantization is 1 if it is greater than zero, otherwise it is quantized to 0. Of course, the quantization zero point can also be other values, such as 0.5, which is not a limitation.

[0052] S102, quantizing the weights into a binary weight matrix based on the matrix quantization information and loading the matrix into a corresponding computing node in the neural network computing graph.

[0053] Among them, the weight is a four-dimensional tensor, which are: number, number of channels, height, and width. In the specific quantization process, the weight is converted from a four-dimensional tensor to a two-dimensional weight matrix. For example, the Im2Col function is used to convert the weight from a four-dimensional tensor to a two-dimensional weight matrix. Of course, the row priority or column priority can be used during the conversion. Then, the two-dimensional weight matrix is ​​quantized based on the quantization zero point in the matrix quantization information to obtain a weight matrix. For the convenience of describing and explaining the present invention, for example: the two-dimensional weight matrix is (floating point number), taking 0 as the quantization zero point, and quantizing using matrix quantization information, the resulting binary weight matrix is:

[0054] S103: quantize the input data into a binary input matrix based on the matrix quantization information.

[0055] First, perform preprocessing operations such as gamma transformation and normalization on the input data. The dimension of the input data is: IH×IW×IC; where IH represents the height of the input data, IW represents the width of the input data, and IC represents the number of channels of the input data;

[0056] In the quantization process, the input data is expanded into a two-dimensional input matrix of (IH×IW)×(KH×KW×IC) according to the convolution calculation characteristics; the expansion method is: KH represents the height of the convolution kernel, KW represents the height of the convolution kernel, and S represents the convolution stride. Indicates rounding down; after expansion, refer to Im2Col to convert the matrix into a column vector and fill in the specific value of the input data, but it does not form a limitation. After obtaining the two-dimensional input matrix, refer to the quantization method with the same weight, quantize the two-dimensional input matrix based on the quantization zero point in the matrix quantization information to obtain a binary input matrix. For the convenience of describing and explaining the present invention, for example: the two-dimensional input matrix is Taking 0 as the quantization zero point and using matrix quantization information for quantization, the resulting binary input matrix is:

[0057] S104, controlling the inference calculation of the input matrix and the weight matrix in the neural network calculation graph according to the inference flow until all the calculation nodes of the neural network calculation graph are traversed to obtain output data.

[0058] Among them, if the computing node of the neural network computing graph is a convolution computing node, the input data needs to be expanded into a binary input matrix as mentioned above. If it is another type of computing node, it remains unchanged.

[0059] Furthermore, in the inference calculation of the convolution calculation node among several computing nodes, a logical XOR operation is performed on the sign bit of the node input matrix and the sign bit of the node weight matrix belonging to the convolution calculation node to obtain a first output matrix; a logical AND operation is performed in the first output matrix according to a set loop order to obtain a second output matrix; based on the output dimension belonging to the convolution calculation node, the second output matrix is ​​converted into a node output matrix to serve as the node input matrix of the next computing node. It is worth noting that the sign bit of the node input matrix and the sign bit of the node weight matrix are both the positions corresponding to the first digit. In actual operation, the node input matrix and the node weight matrix can also be directly used to perform logical XOR operations. For example, since the node input matrix is Then the corresponding input sign bit matrix is ​​also: The actual node input matrix and the corresponding input sign bit matrix are the same, so the node input matrix and the node weight matrix can be directly used to perform logical XOR operations.

[0060] In the process of performing logical XOR operations, the sign bit of the node input matrix is ​​extracted to obtain the input sign bit matrix; the sign bit of the node weight matrix is ​​extracted to obtain the weight sign bit matrix; the first output matrix is ​​initialized according to the dimension information of the input sign bit matrix and the weight sign bit matrix. At this time, the first output matrix is ​​an all-zero matrix, and the dimension is equal to the dimension after the input sign bit matrix and the weight sign bit matrix are multiplied. For example, the weight sign bit matrix is: M×K, and the input sign bit matrix is: K×N, then the first output matrix is ​​initialized to an M×N matrix of all zeros. Perform logical XOR operations on the corresponding positions of the input sign bit matrix and the weight sign bit matrix to obtain the first output matrix. Determine the corresponding positions according to the matrix operation rules. For example:

[0061]

[0062] In the matrix of the above example, a 11 ⊙b 11 Perform logical XOR operation, and the other positions are similar. Further, the input sign bit matrix and the weight sign bit matrix are actually matrices composed of binary digits "0" and "1", so according to the operation principle of "same is 1, different is 0" in the logical XOR operation, the first output matrix is ​​obtained. The first output matrix is ​​also a binary output matrix.

[0063] See also Figure 2 , is the implementation logic diagram of the logical XOR operation. When the corresponding positions of the input sign bit matrix and the weight sign bit matrix are subjected to the logical XOR operation, the first output matrix M×K×N is obtained.

[0064] Taking the specific binary values ​​"0" and "1" as an example, the weight symbol bit matrix is: 4×3, and the input symbol bit matrix is: 3×4, then the matrix multiplication is used to obtain the 4×4 first output matrix. Among them, the values ​​in brackets () need to be logically exclusive or ORed, and after performing the logical exclusive or ORed operation, fill in the corresponding positions to obtain the final schematic 4×4 first output matrix.

[0065] Furthermore, a logical AND operation is performed in the first output matrix. During the operation, the first output matrix is ​​traversed row by row, and the logical AND operation is performed in the first output matrix according to the set loop order according to the operation logic loop of "dividing each row into N groups row by row, dividing each group into K elements, and performing logical AND operation inside the K elements" to obtain an intermediate output matrix; N ≥ 2 and is a positive integer. It is worth noting that N can be the total number of elements in each row. 1≤K<N, and is a positive integer. Character statistics are then performed on the intermediate output matrix, and a second output matrix is ​​generated based on the character statistics results. When performing character statistics, the number of characters "1" contained in the corresponding positions in the intermediate output matrix is ​​counted, and the number is filled in the corresponding positions of the second output matrix.

[0066] See also Figure 3A , is a logical diagram of the logical AND operation. Figure 3A In the example, the dimension of the first output matrix is: M×K×N, each row is divided into N groups, each group is divided into K elements, and logical AND operation is performed inside the K elements. After the division, M data are obtained, each of which contains N groups, and each group has K elements. In the process of performing the logical AND operation, according to the operation rule of "0 is 0, all 1s are 1", the logical AND operation is performed on two elements as a group. The first execution of the logical AND operation is represented as a 1-order operation. At this time, the operation result has N groups, each with K / 2 elements. Exemplarily, if the K elements are: c0, c1, c2, c3, then after performing a logical AND operation in pairs, c_1=c0&c1, c_2=c2&c3 are obtained. Assuming that the loop order is set to 1 order, the elements in each group are filled into the corresponding positions of the intermediate output matrix, for example, r1=c_1c_2. Count the number of characters "1" contained in the corresponding position in the intermediate output matrix, and fill the number into the corresponding position of the second output matrix. At this time, r1=c_1c_2 in the second output matrix. Assuming that the loop order is set to 2, perform the logical AND operation r1=c_1&c_2 again and fill in the corresponding position of the intermediate output matrix, count the number of characters "1" contained in the corresponding position in the intermediate output matrix, and fill the number into the corresponding position of the second output matrix.

[0067] It is worth noting that the K elements can be an odd number or an even number. Regardless of whether they are odd or even numbers, they are first grouped in pairs to perform a logical AND operation. When the set loop order is not met, the remaining single elements of the K elements that have not participated in the logical AND operation after grouping directly participate in the next logical AND operation; when the set loop order is not met, the remaining single elements of the K elements that have not participated in the logical AND operation after grouping are filled in the corresponding positions of the intermediate output matrix. Figure 3B , is the logical principle diagram of the logical AND operation of K elements.

[0068] See also Figure 3C For the specific numerical calculation in , the dimension of the first output matrix is: 4×3×4. When grouping, each row is divided into 4 groups, and the 3 elements in each group perform logical AND operation. If the loop order is set to 1 order, after executing the first logical AND operation, fill the elements in each group into the corresponding position of the intermediate output matrix, and then count the number of characters "1" contained in the corresponding position of the intermediate output matrix, and fill the number into the corresponding position of the second output matrix. If the loop order is set to 2 order, after executing the first logical AND operation, continue to perform the second logical AND operation, fill the elements in each group into the corresponding position of the intermediate output matrix, and then count the number of characters "1" contained in the corresponding position of the intermediate output matrix, and fill the number into the corresponding position of the second output matrix.

[0069] The above is the processing logic of the convolution calculation node. After traversing the neural network calculation graph to obtain the output data, the output data is converted into the target output matrix according to the output dimension information read from the system configuration file. For example, if the output dimension information is [batch size B, number of channels C, height H, width W], the final target output matrix is ​​obtained based on this conversion. Figure 3B Taking the second output matrix in as an example, it can be reshaped into the target output matrix of [1,4,2,2], and the inference result is obtained by looking up the table and output.

[0070] It is worth noting that since the most common operation in neural networks is multiplication and accumulation, that is, summing the products of each element, after multiplying each element of a binary network, its output is only 0 and 1. The results of "summing the results of 0 and 1" and "counting the character 1" are the same.

[0071] When performing classification, what is needed is not the specific numerical value of the operation, but the subscript value of the maximum value (generally the maximum value) of the output vector group. Therefore, after executing the logical XOR operation and logical AND operation in this application, there is no need to convert into floating point numbers. It is only necessary to "count the character 1", reshape the target output matrix, and then look up the table to get the subscript value of the corresponding statistical result. Lookup refers to querying the label value corresponding to the data processing result after "counting the character 1" from the classification dictionary.

[0072] Exemplarily, the table structure of the classification dictionary described in this application is as follows:

[0073] {

[0074] 1: label value 1,

[0075] 2: label value 2,

[0076] ……}

[0077] The label value (classification result) is related to the specific dataset, that is, it is provided by the dataset. For example, for MNIST, the table is:

[0078] {

[0079] 0:0,

[0080] 1:1,

[0081] ...,

[0082] 9:9,

[0083] }

[0084] Use the maximum subscript value to look up the table to get the corresponding label value.

[0085] For example, if the target output matrix is ​​[0,0,0,5,3,4,7,4,1,1], the maximum value is 7, and the subscript value of 7 is 6. By looking up the table, we know that the inference result is the number 6.

[0086] The method disclosed in the technical solution can be applied to low-end devices for calculations, and can also be used as a verification method to verify the calculation results of binary neural networks.

[0087] Furthermore, in order to verify the feasibility of the technical solution, an embodiment is provided: LeNet is modified to obtain a Bin_Mnist_Mixed_Big network for MNIST (handwritten digit recognition) test.

[0088] Bin_Mnist_Mixed_Big network, the number of parameters is 47.746K and the number of multiplication and accumulation calculations is 158.368K. The network structure is shown in Table 1. The network consists of 2 convolution units (Conv1, Conv2) with a convolution kernel size of 5×5 and a stride of 2, 1 fully connected layer and 1 binary fully connected unit; ReLU is used as the activation function.

[0089] Table 1 Bin_Mnist_Mixed_Big network structure table

[0090]

[0091]

[0092] The Bin_Mnist_Mixed_Big network is trained with mixed precision, with 32-bit single-bit floating-point weights in the backbone and 1-bit (0 and 1) weights in the output layer (FC2 layer).

[0093] When performing inference, the first three layers of the network, namely Conv1, Conv2, and FC1, are deployed using ONNXRunTime, and the output layer is used for method verification. The selected test set consists of 10,000 handwritten font images, which are specifically composed as follows:

[0094] Table 2 MNIST official test set image distribution statistics

[0095]

[0096] The test results are shown in Table 3. The error of the first-order simplified reasoning is 0.65%, which is lower than 1%.

[0097] Table 3 Simplified matrix multiplication test results

[0098]

[0099]

[0100] Note:

[0101] 1) PyTorch refers to the test performed under the PyTorch framework;

[0102] 2) NumPy means that FC2 uses NumPy operations;

[0103] 3)MAC refers to the multiplication and accumulation operation used by FC2;

[0104] 4) Simplified_MAC refers to FC2 using the proposed binary matrix simplification calculation method.

[0105] This technical solution,can further reduce the number of addition operations by multiple applications.,Table 4 shows the high-level test results of the proposed,algorithm.

[0106] Table 4 High-order simplified matrix multiplication test results

[0107]

[0108] In the second aspect, based on the same inventive concept as the simplified reasoning method of the binary neural network provided in the embodiment of the first aspect, the embodiment of this specification also provides a simplified reasoning system of the binary neural network, see Figure 4 , the system includes:

[0109] The reading module 401 is used to read the neural network calculation graph and weight-related data from the system configuration file of the neural network; wherein the neural network calculation graph includes a number of calculation nodes and reasoning flows between nodes, and the weight-related data includes weights and matrix quantization information;

[0110] A first quantization module 402, configured to quantize the weight into a binary weight matrix based on the matrix quantization information and load the weight matrix into a corresponding computing node in the neural network computing graph;

[0111] A second quantization module 403, configured to quantize the input data into a binary input matrix based on the matrix quantization information;

[0112] An inference calculation module 404, used to control the inference calculation of the input matrix and the weight matrix in the neural network calculation graph according to the inference flow until all the calculation nodes of the neural network calculation graph are traversed to obtain output data;

[0113] Among them, in the inference calculation of the convolution calculation node among the several computing nodes, a logical XOR operation is performed on the sign bit of the node input matrix and the sign bit of the node weight matrix belonging to the convolution calculation node to obtain a first output matrix; a logical AND operation is cyclically performed in the first output matrix according to a set loop order to obtain a second output matrix; based on the output dimension belonging to the convolution calculation node, the second output matrix is ​​converted into a node output matrix to serve as the node input matrix of the next computing node.

[0114] It should be noted that the simplified reasoning system of the binary neural network provided in the embodiments of this specification, wherein the specific manner in which each module performs operations has been described in detail in the method embodiment provided in the above-mentioned first aspect, and the specific implementation process can refer to the method embodiment provided in the above-mentioned first aspect, and will not be elaborated here.

[0115] In the third aspect, based on the same inventive concept as the simplified reasoning method of binary neural network provided in the embodiment of the first aspect mentioned above, the embodiment of the present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0116] In the fourth aspect, based on the same inventive concept as the simplified reasoning method of binary neural network provided in the embodiment of the first aspect mentioned above, the embodiment of the present invention also discloses a computer device, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the methods described above when executing the program.

[0117] Through one or more embodiments of the present invention, the present invention has the following beneficial effects or advantages:

[0118] In the technical solution of the present invention, the neural network calculation graph and weight-related data are read from the system configuration file of the neural network, and the input data and the corresponding weights are converted into a binary matrix. Then, the reasoning calculation of the input matrix and the weight matrix in the neural network calculation graph is controlled according to the reasoning flow. In particular, in the reasoning calculation of the convolution calculation node, the multiplication operation is eliminated by using the logical XOR operation, and the addition operation is eliminated by using the logical AND operation and the character statistics method. Compared with the prior art, it can realize the efficient reasoning of the binary neural network model, improve the reasoning speed, and occupy very few computing resources, and will not cause computing power conflicts with low-configuration devices. It can be deployed on a large scale for use on low-configuration devices.

[0119] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0120] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0121] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0122] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0123] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0124] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components in a gateway, proxy server, or system according to an embodiment of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0125] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

Claims

1. A simplified inference method for binary neural networks, which is applied to low-end devices; The low-end equipment includes: A mobile device or a wearable device, characterized in that the method comprises: Reading a neural network calculation graph and weight-related data from a system configuration file of the neural network; wherein the neural network calculation graph includes a number of calculation nodes and reasoning flows between nodes, and the weight-related data includes weight values ​​and matrix quantization information; quantizing the weights into a binary weight matrix based on the matrix quantization information and loading the weights into corresponding computing nodes in the neural network computing graph; quantizing the input data into a binary input matrix based on the matrix quantization information; Controlling the inference calculation of the input matrix and the weight matrix in the neural network calculation graph according to the inference flow until all the calculation nodes of the neural network calculation graph are traversed to obtain output data; Among them, in the inference calculation of the convolution calculation node among the several computing nodes, a logical XOR operation is performed on the sign bit of the node input matrix and the sign bit of the node weight matrix belonging to the convolution calculation node to obtain a first output matrix; a logical AND operation is cyclically performed in the first output matrix according to a set loop order to obtain a second output matrix; based on the output dimension belonging to the convolution calculation node, the second output matrix is ​​converted into a node output matrix to serve as the node input matrix of the next computing node.

2. The method according to claim 1, characterized in that The weight is a four-dimensional tensor, which includes: number, number of channels, height, and width; quantizing the weight into a binary weight matrix based on the matrix quantization information specifically includes: Converting the weights from the four-dimensional tensor into a two-dimensional weight matrix; The two-dimensional weight matrix is ​​quantized based on the quantization zero point in the matrix quantization information to obtain the weight matrix.

3. The method according to claim 1 or 2, characterized in that The dimension of the input data is: IH×IW×IC; wherein IH represents the height of the input data, IW represents the width of the input data, and IC represents the number of channels of the input data; the step of quantizing the input data into a binary input matrix based on the matrix quantization information specifically includes: According to the convolution calculation characteristics, the input data is expanded into a two-dimensional input matrix of (IH×IW)×(KH×KW×IC); wherein the expansion method is: KH represents the height of the convolution kernel, KW represents the height of the convolution kernel, and S represents the convolution stride. Indicates rounding down; The two-dimensional input matrix is ​​quantized based on the quantization zero point in the matrix quantization information to obtain the input matrix.

4. The method according to claim 1, characterized in that The step of performing a logical XOR operation on the sign bit of the node input matrix and the sign bit of the node weight matrix belonging to the convolution calculation node to obtain a first output matrix specifically includes: Extracting the sign bit of the node input matrix to obtain an input sign bit matrix; Extracting the sign bit of the node weight matrix to obtain a weight sign bit matrix; A logical XOR operation is performed on corresponding positions of the input sign bit matrix and the weight sign bit matrix to obtain a first output matrix.

5. The method according to claim 1 or 4, characterized in that The step of cyclically performing a logical AND operation in the first output matrix according to a set cyclic order to obtain a second output matrix specifically includes: Traversing the first output matrix row by row, and performing a logical AND operation in the first output matrix according to the set loop order according to the operation logic loop of "dividing each row into N groups row by row, dividing each group into K elements, and performing a logical AND operation within the K elements", to obtain an intermediate output matrix; N ≥ 2 and is a positive integer, 1 ≤ k < N and is a positive integer; Perform character statistics on the intermediate output matrix, and generate the second output matrix based on the character statistics result.

6. The method according to claim 5, characterized in that The performing character statistics on the intermediate output matrix and generating the second output matrix based on the character statistics result specifically includes: Count the number of characters "1" contained in the corresponding positions in the intermediate output matrix, and fill them into the corresponding positions in the second output matrix.

7. The method according to claim 1, characterized in that After obtaining the output data, the method further includes: The output data is converted into a target output matrix according to the output dimension information read from the system configuration file.

8. A simplified reasoning system for a binary neural network, the system being applied to low-end devices; The low-end equipment includes: A mobile device or a wearable device, wherein the system comprises: A reading module, used to read a neural network calculation graph and weight-related data from a system configuration file of the neural network; wherein the neural network calculation graph includes a number of calculation nodes and reasoning flows between nodes, and the weight-related data includes weights and matrix quantization information; A first quantization module, configured to quantize the weight into a binary weight matrix based on the matrix quantization information and load the weight matrix into a corresponding computing node in the neural network computing graph; A second quantization module, configured to quantize the input data into a binary input matrix based on the matrix quantization information; An inference calculation module, used for controlling the inference calculation of the input matrix and the weight matrix in the neural network calculation graph according to the inference flow until all the calculation nodes of the neural network calculation graph are traversed to obtain output data; Among them, in the inference calculation of the convolution calculation node among the several computing nodes, a logical XOR operation is performed on the sign bit of the node input matrix and the sign bit of the node weight matrix belonging to the convolution calculation node to obtain a first output matrix; a logical AND operation is cyclically performed in the first output matrix according to a set loop order to obtain a second output matrix; based on the output dimension belonging to the convolution calculation node, the second output matrix is ​​converted into a node output matrix to serve as the node input matrix of the next computing node.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Method for partitioning large-scale static network based on graphics processor

    CN101977120A

  • Data processing method and device based on neural network model and electronic equipment

    CN112748953A

  • Unified binary model and methodology for knowledge representation and for data and information mining

    US20020087567A1

  • Unfolded convolution for fast feature extraction

    US20070086655A1

  • Fuzzy inference methods, and apparatuses, systems and apparatus using such inference apparatus

    US20130103630A1