A quantization method and system for pulse neural network for brain-like computing platform

Abstract

The present invention discloses a pulse neural network quantization method and system for a brain-like computing platform, including: loading and saving a pre-trained full-precision float32 pulse neural network model, recorded as a pulse neural network model T1; modifying the pulse neural network model T1 to obtain a pulse neural network model T2; selecting a calibration data set to train the pulse neural network model T2 in a quantization process, so as to select a suitable scaling quantization factor scale and a zero point zp to minimize the precision loss before and after quantization; deploying the trained pulse neural network model T2 to a brain-like computing platform; and realizing truly asynchronous pulse neural network calculation without precision loss.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence model compression, and in particular relates to a pulse neural network quantization method and system. Background Art

[0002] The spiking neural network is a third-generation neural network. Its principle is to simulate the pulse emission mode of biological neurons to achieve information interaction between network layers. Due to its pulse characteristics, convolution and full-connection multiplication and addition operations are directly converted into addition operations, which can greatly reduce the amount of calculation. Since the current CPU and GPU both belong to the synchronous computing architecture, it is difficult to give full play to the advantages of spiking neural networks.

[0003] WenTian is a massively parallel, general-purpose super-brain-like computer based on spiking neural networks. WenTian is based on the computing architecture of ARM chips. The int8 quantized model can not only play an acceleration role, but also achieve precision lossless migration deployment in combination with the hardware int8 architecture. However, due to the sparsity of spiking neural networks, the loss of model accuracy has a serious impact on the overall accuracy of the model. Summary of the invention

[0004] The present invention provides a pulse neural network quantization method for a brain-like computing platform to solve the technical problem of precision loss caused by insufficient representation accuracy.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] The first aspect of the present invention provides a pulse neural network quantization method for a brain-like computing platform, comprising:

[0007] Load and save the pre-trained full-precision float32 spiking neural network model, denoted as spiking neural network model T1;

[0008] The process of modifying the spiking neural network model T1 to obtain the spiking neural network model T2 is:

[0009] Insert pseudo-quantization nodes into the input layer and output layer of the spiking neural network model T1. The input layer quantizes the input features from the full-precision float32 format to the int8 format, and the output layer dequantizes the output features from the int8 format to the full-precision float32 format.

[0010] Define the linear operator in the spiking neural network model T1 to the pseudo-quantization node, dequantize the feature parameters input to the pseudo-quantization node from the int8 format to the full-precision float32 format, and then quantize them back to the int8 format after calculation by the pseudo-quantization node;

[0011] The spiking neural network model T1 is changed into the Euler numerical solution form, and then the convolution operator, batch normalization processing, and LIF neuron operator are folded, and the weights and activation functions of each layer in the spiking neural network model T1 are inserted into observation nodes, and the observation nodes are used to calculate and save the maximum and minimum values ​​and quantization range of each group of weights under each channel, so as to obtain the spiking neural network model T2;

[0012] A calibration data set is selected to train the quantization process of the pulse neural network model T2, so as to select the appropriate scaling factor scale and zero point zp to minimize the accuracy loss before and after quantization; the trained pulse neural network model T2 is deployed to the brain-like computing platform.

[0013] Furthermore, the pulse neural network model T1 is converted into the expression formula of Euler numerical solution:

[0014]

[0015] Where V(t) is the membrane potential at time t, V reset is the reset potential, V threshold is the threshold potential, X(t) is the input, S(t) is the pulse at time t, Φ is the step function, and τ is the time decay constant.

[0016] Furthermore, the method of folding the convolution operator, batch normalization processing, and LIF neuron operator includes:

[0017] The convolution operator, batch normalization processing, and LIF neuron operator expression formula of the spiking neural network model T1 are:

[0018]

[0019] The expression formula for folding the convolution operator, batch normalization, and LIF neuron operator is:

[0020]

[0021]

[0022]

[0023] In the formula, w n represents the weight of the nth layer, w n+1 represents the weight of the n+1th layer, b n+1 Bias of the n+1th layer, x n+2 (t) represents the output value of the n+2th layer at time t, x n+1 (t) represents the output value of the n+1th layer at time t, x n-1 (t) represents the output value of the n-1th layer at time t, xn (t) represents the output value of the nth layer at the tth moment, σ 2 is the variance, μ is the mean, ∈=1e-5;.

[0024] Furthermore, the calibration data set is selected to train the quantization process of the pulse neural network model T2, so as to select a suitable scaling factor scale and zero point zp to minimize the accuracy loss before and after quantization. The method includes:

[0025] The pulse neural network model T2 is trained using the calibration data set. The step function Φ in the back propagation process of the pulse neural network model T2 is set to the sigmoid function. The maximum and minimum values ​​of each group of weights under each channel and the quantization range are calculated and saved by observing the nodes, and then the scaling quantization factor scale and the zero point zp are calculated. According to the scaling quantization factor scale and the zero point zp, the quantization operation from the full-precision float32 format to the int8 format or the dequantization operation from the int8 format to the full-precision float32 format is performed.

[0026] Furthermore, a quantization operation is performed from the full-precision float32 format to the int8 format, and the expression formula is:

[0027]

[0028] In the formula, x Q To quantize the output features, x is represented as the quantized input features. clamp(·) quantizes the input features x through Quantized truncated 0 to N levels -1 range; N levels The maximum value expressed as int8 format.

[0029] Furthermore, a dequantization operation is performed from the int8 format to the full-precision float32 format, and the expression formula is:

[0030] x float =(x Q -zp)scale

[0031] In the formula, x float Represented as the inverse quantized output feature; x Q To quantify the output features.

[0032] Furthermore, the method for deploying the trained spiking neural network model T2 to the brain-like computing platform includes: creating a population in a one-to-one ratio according to the trained spiking neural network model T2, deploying the trained spiking neural network model T2 to the brain-like computing platform, connecting the neurons in a one-to-one manner by specifying the neuron numbers using FromListConnector, and assigning the folded weights w to the connections. n+1′ ; Set the neuron's bias current I_offset to b n+1′ , and the neuron membrane capacitance C is set to τ.

[0033] In a second aspect, the present invention provides a pulse neural network quantization system for a brain-like computing platform, comprising:

[0034] The loading module is used to load and save the pre-trained full-precision float32 spiking neural network model, denoted as spiking neural network model T1;

[0035] A model modification module is used to insert pseudo-quantization nodes into the input layer and output layer of the pulse neural network model T1, and the input layer quantizes the input features from the full-precision float32 format to the int8 format, and the output layer dequantizes the output features from the int8 format to the full-precision float32 format; the linear operator in the pulse neural network model T1 is defined to the pseudo-quantization node, the feature parameters of the input pseudo-quantization node are dequantized from the int8 format to the full-precision float32 format, and then quantized back to the int8 format after calculation by the pseudo-quantization node; the pulse neural network model T1 is changed to the Euler numerical solution form, and then the convolution operator, batch normalization processing, and LIF neuron operator are folded, and the weights and activation functions of each layer in the pulse neural network model T1 are inserted into the observation node, and the observation node is used to calculate and save the maximum and minimum values ​​of each group of weights under each channel and the quantization range, so as to obtain the pulse neural network model T2;

[0036] The training module is used to select a calibration data set to train the quantization process of the pulse neural network model T2, so as to select a suitable scaling factor scale and zero point zp to minimize the accuracy loss before and after quantization;

[0037] The execution module is used to deploy the trained pulse neural network model T2 to the brain-like computing platform.

[0038] Furthermore, the training module is used to select a calibration data set to train the quantization process of the pulse neural network model T2, so as to select a suitable scaling factor scale and zero point zp to minimize the accuracy loss before and after quantization. The method includes:

[0039] The spiking neural network model T2 is trained using the calibration data set. The maximum and minimum values ​​of each group of weights under each channel and the quantization range are calculated and saved by the observation node, and then the scaling quantization factor scale and the zero point zp are calculated. According to the scaling quantization factor scale and the zero point zp, the quantization operation from the full-precision float32 format to the int8 format or the dequantization operation from the int8 format to the full-precision float32 format is performed.

[0040] In a third aspect, the present invention provides an electronic device, comprising a storage medium and a processor; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the method described in the first aspect.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention selects a calibration data set to train the quantization process of the pulse neural network model T2, so as to select a suitable scaling quantization factor scale and zero point zp to minimize the precision loss before and after quantization; the trained pulse neural network model T2 is deployed to a brain-like computing platform to realize truly asynchronous pulse neural network calculation without precision loss.

[0043] The convolution or fully connected multiplication and addition operation, the batch normalization multiplication and addition operation, and the LIF neuron multiplication and addition operation of the present invention are all linear multiplication and addition calculations, so the three operators can be directly folded into one layer, avoiding the precision loss caused by repeated quantization and dequantization operations during the quantization process; at the same time, since the three layers are folded into one layer, the network reasoning speed is greatly shortened. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of a pulse neural network quantization method for a brain-like computing platform provided in an embodiment. DETAILED DESCRIPTION

[0045] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0046] Example 1

[0047] like Figure 1 As shown, a pulse neural network quantization method for a brain-like computing platform includes:

[0048] Load and save the pre-trained full-precision float32 spiking neural network model, denoted as spiking neural network model T1;

[0049] The process of modifying the spiking neural network model T1 to obtain the spiking neural network model T2 is:

[0050] Insert pseudo-quantization nodes into the input layer and output layer of the spiking neural network model T1. The input layer quantizes the input features from the full-precision float32 format to the int8 format, and the output layer dequantizes the output features from the int8 format to the full-precision float32 format.

[0051] Define the linear operator in the spiking neural network model T1 to the pseudo-quantization node, dequantize the feature parameters input to the pseudo-quantization node from the int8 format to the full-precision float32 format, and then quantize them back to the int8 format after calculation by the pseudo-quantization node;

[0052] The expression formula of converting the pulse neural network model T1 into the Euler numerical solution form is:

[0053]

[0054] Where V(t) is the membrane potential at time t, V reset is the reset potential, V threshold is the threshold potential, X(t) is the input, S(t) is the pulse at time t, Φ is the step function, and τ is the time decay constant.

[0055] Methods for folding the convolution operator, batch normalization, and LIF neuron operator include:

[0056] The convolution operator, batch normalization processing, and LIF neuron operator expression formula of the spiking neural network model T1 are:

[0057]

[0058] The expression formula for folding the convolution operator, batch normalization, and LIF neuron operator is:

[0059]

[0060]

[0061]

[0062] In the formula, w n represents the weight of the nth layer, w n+1 represents the weight of the n+1th layer, b n+1 Bias of the n+1th layer, x n+2 (t) represents the output value of the n+2th layer at time t, x n+1 (t) represents the output value of the n+1th layer at time t, x n-1 (t) represents the output value of the n-1th layer at time t, x n (t) represents the output value of the nth layer at the tth moment, σ2 is the variance, μ is the mean, ∈=1e-5; avoid the loss of accuracy caused by repeated quantization and dequantization operations during the quantization process; at the same time, since the three layers are folded into one layer, the network inference speed is greatly shortened.

[0063] And insert observation nodes into the weights and activation functions of each layer in the pulse neural network model T1, wherein the observation nodes are used to calculate and save the maximum and minimum values ​​and the quantization range of each group of weights under each channel, so as to obtain the pulse neural network model T2;

[0064] The method of selecting the calibration data set to perform gradient training of the quantization process on the pulse neural network model T2 to select the appropriate scaling factor scale and zero point zp to minimize the accuracy loss before and after quantization includes:

[0065] The pulse neural network model T2 is trained using the calibration data set. The step function Φ in the back propagation process of the pulse neural network model T2 is set to the sigmoid function. The maximum and minimum values ​​of each group of weights under each channel and the quantization range are calculated and saved by observing the nodes, and then the scaling quantization factor scale and the zero point zp are calculated. According to the scaling quantization factor scale and the zero point zp, the quantization operation from the full-precision float32 format to the int8 format or the dequantization operation from the int8 format to the full-precision float32 format is performed.

[0066] Perform quantization operation from full-precision float32 format to int8 format, expressed as:

[0067]

[0068] In the formula, x Q To quantize the output features, x is represented as the quantized input features. clamp(·) quantizes the input features x through Quantized truncated 0 to N levels -1 range; N levels The maximum value expressed as int8 format.

[0069] Perform the dequantization operation from int8 format to full-precision float32 format, expressed as:

[0070] x float =(x Q -zp)scale

[0071] In the formula, x float Represented as the dequantized output features.

[0072] The method for deploying the trained spiking neural network model T2 to the brain-like computing platform includes: by creating a population in a one-to-one ratio according to the trained spiking neural network model T2, the trained spiking neural network model T2 is deployed to the Wentian brain-like computing platform, which is a large-scale parallel, general-purpose super brain-like computer based on spiking neural networks. The Wentian brain-like computing platform is based on the computing architecture of ARM chips. The int8 quantized model can not only play an acceleration role, but also realize precision lossless migration deployment in combination with the hardware int8 architecture; and the architecture of the Wentian brain-like computing platform is very gentle with the working principle of the spiking neural network. When no pulse signal is received, the chip maintains a resting state, and only the location of the neuron that receives the pulse is calculated, which greatly reduces the energy consumption of the calculation.

[0073] The folded weights of the operators are connected to the neurons in the way that FromListConnector specifies the neuron numbers one by one and assign the folded weights w to the connections n+1′ ; Set the neuron's bias current I_offset to b n+1′ , and the neuron membrane capacitance C is set to τ; thus achieving truly asynchronous pulse neural network calculation without loss of precision.

[0074] Example 2

[0075] This embodiment provides a spiking neural network quantization system for a brain-like computing platform. The spiking neural network quantization system described in this embodiment can be applied to the spiking neural network quantization method described in Example 1. The spiking neural network quantization system includes:

[0076] The loading module is used to load and save the pre-trained full-precision float32 spiking neural network model, denoted as spiking neural network model T1;

[0077] A model modification module is used to insert pseudo-quantization nodes into the input layer and output layer of the pulse neural network model T1, and the input layer quantizes the input features from the full-precision float32 format to the int8 format, and the output layer dequantizes the output features from the int8 format to the full-precision float32 format; the linear operator in the pulse neural network model T1 is defined to the pseudo-quantization node, the feature parameters of the input pseudo-quantization node are dequantized from the int8 format to the full-precision float32 format, and then quantized back to the int8 format after calculation by the pseudo-quantization node; the pulse neural network model T1 is changed to the Euler numerical solution form, and then the convolution operator, batch normalization processing, and LIF neuron operator are folded, and the weights and activation functions of each layer in the pulse neural network model T1 are inserted into the observation node, and the observation node is used to calculate and save the maximum and minimum values ​​of each group of weights under each channel and the quantization range, so as to obtain the pulse neural network model T2;

[0078] The training module is used to select a calibration data set to train the quantization process of the pulse neural network model T2, so as to select a suitable scaling factor scale and zero point zp to minimize the accuracy loss before and after quantization;

[0079] The execution module is used to deploy the trained pulse neural network model T2 to the brain-like computing platform.

[0080] The training module is used to select a calibration data set to train the quantization process of the pulse neural network model T2, so as to select a suitable scaling quantization factor scale and zero point zp so that the accuracy loss before and after quantization is minimized. The method includes:

[0081] The spiking neural network model T2 is trained using the calibration data set. The maximum and minimum values ​​of each group of weights under each channel and the quantization range are calculated and saved by the observation node, and then the scaling quantization factor scale and the zero point zp are calculated. According to the scaling quantization factor scale and the zero point zp, the quantization operation from the full-precision float32 format to the int8 format or the dequantization operation from the int8 format to the full-precision float32 format is performed.

[0082] Example 3

[0083] This embodiment provides an electronic device, including a storage medium and a processor; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the method described in Embodiment 1.

[0084] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0085] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0088] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A pulse neural network quantization method for a brain-like computing platform, characterized in that: include: Load and save the pre-trained full-precision float32 spiking neural network model, denoted as spiking neural network model T1; The process of modifying the spiking neural network model T1 to obtain the spiking neural network model T2 is: Insert pseudo-quantization nodes into the input layer and output layer of the spiking neural network model T1. The input layer quantizes the input features from the full-precision float32 format to the int8 format, and the output layer dequantizes the output features from the int8 format to the full-precision float32 format. Define the linear operator in the spiking neural network model T1 to the pseudo-quantization node, dequantize the feature parameters input to the pseudo-quantization node from the int8 format to the full-precision float32 format, and then quantize them back to the int8 format after calculation by the pseudo-quantization node; The spiking neural network model T1 is changed into the Euler numerical solution form, and then the convolution operator, batch normalization processing, and LIF neuron operator are folded, and the weights and activation functions of each layer in the spiking neural network model T1 are inserted into observation nodes, and the observation nodes are used to calculate and save the maximum and minimum values ​​and quantization range of each group of weights under each channel, so as to obtain the spiking neural network model T2; The calibration data set is selected to train the quantization process of the pulse neural network model T2, so as to select the appropriate scaling factor scale and zero point zp to minimize the accuracy loss before and after quantization; The method for deploying the trained spiking neural network model T2 to the brain-like computing platform includes: creating a population in a one-to-one ratio according to the trained spiking neural network model T2, deploying the trained spiking neural network model T2 to the brain-like computing platform, connecting neurons according to the one-to-one specified neuron number method of FromListConnector, and assigning the folded weights to the connections. ; Set the neuron's bias current I_offset to , and the neuronal membrane capacitance Set to .

2. The pulse neural network quantization method according to claim 1, characterized in that: The expression formula of converting the pulse neural network model T1 into the Euler numerical solution form is: ； in, is the membrane potential at time t, To reset the potential, is the threshold potential, For input, is the pulse at time t, is a step function, is the time decay constant.

3. The pulse neural network quantization method according to claim 1, characterized in that: Methods for folding the convolution operator, batch normalization, and LIF neuron operator include: The convolution operator, batch normalization processing, and LIF neuron operator expression formula of the spiking neural network model T1 are: ； The expression formula for folding the convolution operator, batch normalization, and LIF neuron operator is: ； ； ； In the formula, represents the weight of the nth layer, represents the weight of the n+1th layer, The n+1th layer bias, represents the output value of the n+2th layer at time t, represents the output value of the n+1th layer at time t, represents the output value of the n-1th layer at time t, represents the output value of the nth layer at the tth moment, is the variance, is the mean, =1e-5; is the time decay constant.

4. The pulse neural network quantization method according to claim 1, characterized in that: The method of selecting a calibration data set to train the quantization process of the pulse neural network model T2 to select a suitable scaling factor scale and zero point zp to minimize the accuracy loss before and after quantization includes: The spiking neural network model T2 is trained using the calibration data set, and the step function in the back propagation process of the spiking neural network model T2 is Set to the sigmoid function, calculate and save the maximum and minimum values ​​of each group of weights under each channel and the quantization range through the observation node, and then calculate the scaling quantization factor scale and the zero point zp; perform the quantization operation from the full-precision float32 format to the int8 format or the dequantization operation from the int8 format to the full-precision float32 format according to the scaling quantization factor scale and the zero point zp.

5. The pulse neural network quantization method according to claim 4, characterized in that: Perform quantization operation from full-precision float32 format to int8 format, expressed as: ； In the formula, To quantify the output features, Represented as quantized input features, Quantize the input features go through After quantization, the truncation is 0 to scope; The maximum value expressed as int8 format.

6. The pulse neural network quantization method according to claim 4, characterized in that: Perform the dequantization operation from int8 format to full-precision float32 format, expressed as: ； In the formula, Represented as dequantized output features; To quantify the output features.

7. A pulse neural network quantization system for a brain-like computing platform, characterized in that: include: The loading module is used to load and save the pre-trained full-precision float32 spiking neural network model, denoted as spiking neural network model T1; A model modification module is used to insert pseudo-quantization nodes into the input layer and output layer of the pulse neural network model T1, and the input layer quantizes the input features from the full-precision float32 format to the int8 format, and the output layer dequantizes the output features from the int8 format to the full-precision float32 format; the linear operator in the pulse neural network model T1 is defined to the pseudo-quantization node, the feature parameters of the input pseudo-quantization node are dequantized from the int8 format to the full-precision float32 format, and then quantized back to the int8 format after calculation by the pseudo-quantization node; the pulse neural network model T1 is changed to the Euler numerical solution form, and then the convolution operator, batch normalization processing, and LIF neuron operator are folded, and the weights and activation functions of each layer in the pulse neural network model T1 are inserted into the observation node, and the observation node is used to calculate and save the maximum and minimum values ​​of each group of weights under each channel and the quantization range, so as to obtain the pulse neural network model T2; The training module is used to select a calibration data set to train the quantization process of the pulse neural network model T2, so as to select a suitable scaling factor scale and zero point zp to minimize the accuracy loss before and after quantization; The execution module is used to deploy the trained pulse neural network model T2 to the brain-like computing platform.

8. The pulse neural network quantization system according to claim 7, characterized in that: The training module is used to select a calibration data set to train the quantization process of the pulse neural network model T2, so as to select a suitable scaling factor scale and zero point zp to minimize the accuracy loss before and after quantization. The method includes: The spiking neural network model T2 is trained using the calibration data set. The maximum and minimum values ​​of each group of weights under each channel and the quantization range are calculated and saved by the observation node, and then the scaling quantization factor scale and the zero point zp are calculated. According to the scaling quantization factor scale and the zero point zp, the quantization operation from the full-precision float32 format to the int8 format or the dequantization operation from the int8 format to the full-precision float32 format is performed.

9. An electronic device comprising a storage medium and a processor; the storage medium is used to store instructions; characterized in that: The processor is configured to operate according to the instructions to execute the method according to any one of claims 1 to 6.

Images