A low-latency and low-power pulse neural network conversion method

Abstract

The present application provides a low-latency, low-power pulse neural network conversion method, including: building a convolutional neural network suitable for conversion, wherein the activation function is replaced by a finite soft step activation function, and the network weights are trained using a backpropagation algorithm; the weights are normalized using training set images; a soft reset IF neuron model is constructed; an event-driven maximum pooling layer is constructed; a pulse neural network consistent with the original ANN structure is constructed using a neuron model and a maximum pooling layer, and the weight parameters obtained by the original ANN training are reused; the input is repeatedly encoded, and the amplitude of the output calculated by the first convolutional layer is input into the above-mentioned replaced neuron, a pulse sequence of a specified time step is output, and the pulse sequence is input into the network to obtain a classification result. The present application improves the conversion accuracy of the network and reduces the accuracy loss of the model conversion.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a low-latency and low-power pulse neural network conversion method. Background Art

[0002] Spiking Neural Networks (SNNs) are a new generation of biologically inspired artificial neural network models. They differ from artificial neural networks primarily in the nature of their information processing. While traditional artificial neural networks use real-valued computations, such as signal amplitude, SNNs utilize the signal's temporal information, using an "integrate-and-fire" mechanism to receive and emit pulses to exchange information. This characteristic makes SNNs easy to deploy on relevant hardware. The neurons in SNNs are event-driven, meaning they are active only when receiving or emitting pulses, significantly reducing energy consumption.

[0003] Although spiking neural networks have many advantages, their discrete computational characteristics prevent them from being trained using direct gradient calculations like ANNs. Currently, there are three main training methods, including a series of methods based on STDP and its improvements, alternative gradient training methods, and conversion-based methods. Conversion-based methods can quickly produce usable spiking neural networks, but there is a loss of accuracy during the conversion process. Currently, some studies have proposed methods to address this issue. One method is to convert an ANN into an SNN by setting the threshold voltage equal to the maximum ReLU activation value, copy parameters, and bias of the corresponding ANN layer when the simulation step size T is large enough. Furthermore, when the initial membrane potential of each layer of neurons is half the threshold voltage, the conversion error can be minimized. Other studies have proposed Burstneurons, which allow pulses to be emitted between two time steps, thereby increasing the pulse firing rate, and have proposed Lateral Inhibition Pooling (LIPooling) to address the issues caused by maximum pooling during conversion.

[0004] However, for the sake of computational convenience, these methods use average pooling instead of maximum pooling, and use the floating-point number after the average pooling operation as the membrane potential increment of the neurons in this layer, so that the neurons in this layer update the membrane potential and determine whether to output pulses. This ignores the energy consumption cost and accuracy loss of this process during hardware implementation. Summary of the Invention

[0005] The present application provides a low-latency and low-power pulse neural network conversion method, which can be used to solve the technical problem of high energy consumption of pulse neural networks during training.

[0006] The present application provides a low-latency and low-power pulse neural network conversion method, the method comprising:

[0007] Step 1: Build a convolutional neural network suitable for conversion, where the activation function is replaced by a finite soft step activation function, and the backpropagation algorithm is used to train the network weights;

[0008] Step 2: Normalize the weights using the training set images.

[0009] Step 3, construct a soft reset IF neuron model;

[0010] Step 4: Construct an event-driven maximum pooling layer;

[0011] Step 5: Use the neuron model and the maximum pooling layer to build a spiking neural network with the same structure as the original ANN, and reuse the weight parameters obtained from the original ANN training;

[0012] Step 6: Repeat the encoding of the input, calculate the output amplitude through the first convolutional layer and input it into the replaced neuron, output the pulse sequence of the specified time step, and input the pulse sequence into the network to obtain the classification result.

[0013] Optionally, a convolutional neural network includes:

[0014] Several convolutional layers, maximum pooling layers, and fully connected layers; all layer biases are 0; the feature extraction network order is convolution, ReLU, and pooling; the bias gradient is disabled during training; the activation function is replaced by a finite soft step activation function, and the characteristic formula of the activation function is:

[0015]

[0016]

[0017]

[0018]

[0019] Among them, a in with a out are the input and output of the activation function respectively; a limit is the output upper limit of the activation function; K is the quantization coefficient; w controls the climbing slope between adjacent steps, and the value of w is adjusted by n. When w is larger, the slope is steeper, the rise is faster, and the quantization accuracy is higher. Without affecting the training, n takes the largest value possible; c is related to the quantization accuracy and is adjusted with a during training. limit Change and change; b i is the input activation value corresponding to the inflection point of the curve. The number of inflection points of the curve is related to K.

[0020] Optionally, normalize the weights using the training set images, including:

[0021] Step 2-1, evenly extract the data of each classification training set Batch input into ANN;

[0022] Step 2-2, record the maximum activation value corresponding to each layer in layers;

[0023] Step 2-3: Use the ratio of the maximum activation values ​​of the two layers corresponding to the weight parameter as the scaling factor to scale the weight. The specific method is as follows:

[0024]

[0025] Among them, w l represents the weight parameter of the ANN layer l, α l , α l-1 They represent the maximum activation values ​​of the l-1th layer and the lth layer respectively.

[0026] Optionally, a soft reset IF neuron model is as follows:

[0027]

[0028]

[0029] Among them, V thresh represents the threshold voltage of the neuron, t represents the current moment, T represents the total time step of the network, represents the membrane voltage of the i-th neuron in the l-th layer of the spiking neural network at time t, Indicates whether the i-th neuron in the l-th layer of the spiking neural network at time t emits a pulse. If it does, it is 1, otherwise it is 0.

[0030] Optionally, building an event-driven max pooling layer includes:

[0031] Step 4-1, initialize the event counter and maximum pooling threshold in the pooling area to 0;

[0032] Step 4-2: Input the event sequence. Determine whether the event counter needs to be updated based on the value of the event sequence. If the input is 0, the counter remains unchanged. If the input is 1, the counter value is updated.

[0033] In step 4-3, the maximum value of the counter whose current event is 1 is taken as the pre-activation value, and compared with the current threshold. If it exceeds the threshold, a pulse is issued and the threshold is updated to the pre-activation value. Otherwise, no pulse is issued and the threshold remains unchanged.

[0034] Optionally, the spiking neural network consists only of convolutional layers, pooling layers, and fully connected layers; the bias of each network layer is 0; the neurons of the convolutional layers and the fully connected layers use soft-reset IF neurons; and the pooling layer uses a maximum pooling layer built based on event-driven architecture.

[0035] Optionally, the input is repeatedly encoded, and the amplitude of the output is calculated by the first convolutional layer and input into the above-mentioned replaced neuron. The specific characteristics of the pulse sequence output at the specified time step include:

[0036] The input of T consecutive time steps is the same normalized image. The first convolutional layer encodes the features extracted by the convolution kernel. The calculation results of the convolutional layer are soft-reset to the IF neurons, and the output pulse sequence is considered to be the feature coding sequence of the image.

[0037] The finite soft step activation function designed in this application introduces the conversion quantization error into the network training link in advance, which can improve the conversion accuracy of the network; the event-driven maximum pooling layer of this application uses fixed-point number operations and binary output, which can reduce device power consumption and speed up calculation speed; this application uses trained parameters to encode the main features extracted from the receptive field. Compared with traditional Poisson coding, it reduces the randomness of the coding and further reduces the accuracy loss of the model conversion. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the implementation process provided in the embodiment of the present application;

[0039] Figure 2 A calculation flow chart of the event-driven pooling layer provided in an embodiment of the present application;

[0040] Figure 3 A flowchart of encoding an input image into a pulse sequence provided in an embodiment of the present application;

[0041] Figure 4 A graph of a finite soft step activation function under different slope conditions provided in an embodiment of the present application;

[0042] Figure 5 This is a graph showing the relationship between the conversion accuracy loss of the Cifar10 dataset and the pulse sequence length provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0044] The following first introduces the possible system architecture applicable to the embodiments of the present application with reference to the accompanying drawings.

[0045] The present invention provides a method for converting a spiking neural network into a low-latency and low-power spiking neural network. The method includes the following steps:

[0046] Step 1: Build a convolutional neural network suitable for conversion, where the activation function is replaced by a finite soft step activation function, and the backpropagation algorithm is used to train the network weights;

[0047] Step 2: Normalize the weights using the training set images.

[0048] Step 3, construct a soft reset IF neuron model;

[0049] Step 4: Construct an event-driven maximum pooling layer;

[0050] Step 5: Use the above neuron model and the above maximum pooling layer to build a spiking neural network with the same structure as the original ANN, and reuse the weight parameters obtained by the original ANN training;

[0051] Step 6: Repeat the encoding of the input, calculate the output amplitude through the first convolutional layer and input it into the replaced neuron, output the pulse sequence of the specified time step, and input the pulse sequence into the network to obtain the classification result.

[0052] Furthermore, in one embodiment, step 1 trains an ANN that meets the requirements.

[0053] Exemplarily, for the Cifar10 dataset, the VGG16 network structure is adopted, which does not include a BN layer, the pooling layer uses maximum pooling, the training uses the ADAM optimizer, the learning rate is 0.001, the activation function uses the finite soft step activation function of the present invention, the quantization coefficient K is designed to be 32, the slope coefficient n is 100, and the training obtains an ANN network with an accuracy of 95.3%.

[0054] Furthermore, in one embodiment, the weights are normalized using the training set images in step 2. The specific process includes:

[0055] Step 2-1: Uniformly extract the data of each classification training set Batch input into ANN;

[0056] Step 2-2: Record the maximum activation value of neurons in each layer of ANN in units of layers.

[0057] Step 2-3: Use the ratio of the maximum activation values ​​of the two layers corresponding to the weight parameter as the scaling factor to scale the weight. The characteristic formula is as follows:

[0058]

[0059] Among them, wl represents the weight parameter of the ANN layer l, α l , α l-1 They represent the maximum activation values ​​of the l-1th layer and the lth layer respectively.

[0060] Furthermore, in one embodiment, step 3 constructs a soft reset IF neuron model.

[0061] Before use, the neuron membrane potential is initialized. When receiving a pulse, the neuron membrane potential is updated. The characteristic formula is as follows:

[0062]

[0063] Among them, n l represents the number of neurons in layer l, represents the synaptic weight between the jth neuron in layer l-1 and the i-th neuron in layer l, Indicates whether the j-th neuron in layer l-1 emits a pulse at time t-1.

[0064] Determine whether the membrane potential exceeds the threshold voltage. If it is greater than the threshold voltage, a pulse is issued and the membrane potential is updated. Otherwise, no pulse is issued and the current membrane potential is maintained. The characteristic formula is as follows:

[0065]

[0066]

[0067] Furthermore, in one embodiment, the step 4 constructs an event-driven maximum pooling layer. The specific process of the event-driven maximum pooling layer includes:

[0068] Step 4-1: Initialize the event counter and the maximum pooling threshold in the pooling area to 0.

[0069] Step 4-2: Input the event sequence. Determine whether the event counter needs to be updated based on the value of the event sequence. If the input is 0, it remains unchanged. If the input is 1, the counter value is updated.

[0070] Step 4-3: Take the maximum value of the counter whose current event is 1 as the pre-activation value and compare it with the current threshold. If it exceeds the threshold, a pulse is issued and the threshold is updated to the pre-activation value. Otherwise, no pulse is issued and the threshold remains unchanged.

[0071] Taking the above Cifar10 dataset and VGG16 network as an example, when the pooling scope is 2*2, the initialization event counter and threshold are both 0. If the event sequence input at time t=1 is 0011, the updated event counters are 0, 0, 1, 1 respectively. The maximum value of the event counter corresponding to the current event is 1, which is greater than the threshold, so the pooling layer outputs 1 and updates the threshold to 1. If the time sequence input at time t=2 is 1000, the updated event counters are 1, 0, 1, 1 respectively. The maximum value of the event counter corresponding to the current event is 1, which is equal to the threshold, so the pooling layer outputs 0 and keeps the threshold unchanged. According to the above process, if the subsequent input time series is 0001, 1000, 1001, 0100, 0000, 0100, 1000, 1000, the pulse sequence finally output by the pooling layer is 1100010101, which can be verified to be consistent with the maximum pulse frequency of the input.

[0072] Furthermore, in one example, step 5 utilizes the aforementioned neuron model and the aforementioned maximum pooling layer to construct a spiking neural network with the same structure as the original ANN, reusing the weight parameters obtained from the original ANN training. Specifically, the network consists solely of convolutional layers, pooling layers, and fully connected layers; the bias of each network layer is 0; the neurons in the convolutional and fully connected layers are implemented using the aforementioned soft-reset IF neurons; and the pooling layer is implemented using the aforementioned event-driven maximum pooling layer.

[0073] Furthermore, in one embodiment, the input is repeatedly encoded as described in step 6, and the amplitude of the output is calculated by the first convolutional layer and input into the above-mentioned replaced neurons, and the specific features of the pulse sequence output at the specified time step include: the input of T consecutive time steps is the same normalized image, the first convolutional layer plays a role in encoding the features extracted by the convolution kernel, the calculation result of the convolution layer passes through the above-mentioned soft reset IF neurons, and the output pulse sequence is considered to be the feature encoding of the image.

[0074] Taking the above-mentioned Cifar10 dataset and VGG16 network as an example, experiments have verified that only no more than 30 time steps are needed and the accuracy loss can be reduced to 0.5%, that is, the classification task can be completed in fewer time steps, verifying the effectiveness of the present invention.

[0075] The finite soft step activation function designed in this application introduces the conversion quantization error into the network training link in advance, which can improve the conversion accuracy of the network; the event-driven maximum pooling layer of this application uses fixed-point number operations and binary output, which can reduce device power consumption and speed up calculation speed; this application uses trained parameters to encode the main features extracted from the receptive field. Compared with traditional Poisson coding, it reduces the randomness of the coding and further reduces the accuracy loss of the model conversion.

[0076] The above-described embodiments of the present application do not constitute a limitation on the scope of protection of the present application.

Claims

1. A low-latency and low-power pulse neural network conversion method, characterized in that: The method comprises: Step 1: Build a convolutional neural network suitable for conversion, where the activation function is replaced by a finite soft step activation function, and the backpropagation algorithm is used to train the network weights; Step 2: Normalize the weights using the training set images. Step 3, construct a soft reset IF neuron model; Step 4: Construct an event-driven maximum pooling layer; Step 5: Use the neuron model and the maximum pooling layer to build a spiking neural network with the same structure as the original ANN, and reuse the weight parameters obtained from the original ANN training; Step 6: Repeat the encoding of the input, calculate the output amplitude through the first convolutional layer, input it into the replaced neuron, output the pulse sequence of the specified time step, and input the pulse sequence into the network to obtain the classification result; Convolutional neural networks include: Several convolutional layers, maximum pooling layers, and fully connected layers; all layer biases are 0; the feature extraction network order is convolution, ReLU, and pooling; the bias gradient is disabled during training; the activation function is replaced by a finite soft step activation function, and the characteristic formula of the activation function is: Among them, a in with a out are the input and output of the activation function respectively; a limit is the output upper limit of the activation function; K is the quantization coefficient; w controls the climbing slope between adjacent steps, and the value of w is adjusted by n. When w is larger, the slope is steeper, the rise is faster, and the quantization accuracy is higher. Without affecting the training, n takes the largest value possible; c is related to the quantization accuracy and is adjusted with a during training. limit Change and change; b i is the input activation value corresponding to the inflection point of the curve. The number of inflection points of the curve is related to K.

2. The low-latency and low-power pulse neural network conversion method according to claim 1, characterized in that: Normalize the weights using the training set images, including: Step 2-1, evenly extract the data of each classification training set Batch input into ANN; Step 2-2, record the maximum activation value corresponding to each layer in layers; Step 2-3: Use the ratio of the maximum activation values ​​of the two layers corresponding to the weight parameter as the scaling factor to scale the weight. The specific method is as follows: Among them, w l represents the weight parameter of the ANN layer l, α l , α l-1 They represent the maximum activation values ​​of the l-1th layer and the lth layer respectively.

3. The low-latency and low-power pulse neural network conversion method according to claim 2, characterized in that: The soft reset IF neuron model is as follows: Among them, V thresh represents the threshold voltage of the neuron, t represents the current moment, T represents the total time step of the network, represents the membrane voltage of the i-th neuron in the l-th layer of the spiking neural network at time t, Indicates whether the i-th neuron in the l-th layer of the spiking neural network at time t emits a pulse. If it does, it is 1, otherwise it is 0.

4. The low-latency and low-power pulse neural network conversion method according to claim 3, characterized in that: Building an event-driven max pooling layer includes: Step 4-1, initialize the event counter and maximum pooling threshold in the pooling area to 0; Step 4-2: Input the event sequence. Determine whether the event counter needs to be updated based on the value of the event sequence. If the input is 0, the counter remains unchanged. If the input is 1, the counter value is updated. In step 4-3, the maximum value of the counter whose current event is 1 is taken as the pre-activation value, and compared with the current threshold. If it exceeds the threshold, a pulse is issued and the threshold is updated to the pre-activation value. Otherwise, no pulse is issued and the threshold remains unchanged.

5. The low-latency and low-power pulse neural network conversion method according to claim 4, characterized in that: The spiking neural network consists only of convolutional layers, pooling layers, and fully connected layers; the bias of each network layer is 0; The neurons in the convolutional layer and the fully connected layer use soft reset IF neurons; the pooling layer uses the maximum pooling layer built based on event-driven.

6. The low-latency and low-power pulse neural network conversion method according to claim 5, characterized in that: The input is repeatedly encoded, and the amplitude of the output is calculated by the first convolutional layer and input into the above-mentioned replaced neurons. The specific characteristics of the pulse sequence output at the specified time step include: The input of T consecutive time steps is the same normalized image. The first convolutional layer encodes the features extracted by the convolution kernel. The calculation results of the convolutional layer are soft-reset to the IF neurons, and the output pulse sequence is considered to be the feature coding sequence of the image.

Images