Maximum pooling layer circuit for SNN target classification

By designing the maximum pooling layer circuit in SNN, using adders and comparators to accumulate the number of pulses of neurons, the maximum pooling operation is achieved, which solves the problem of difficult to achieve maximum pooling in SNN, and improves the accuracy of image classification and The robustness of the model reduces power consumption at the edge.

CN118940807APending Publication Date: 2024-11-12CHINA ACAD OF AEROSPACE SCI & TECH INNOVATION
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411023121.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In SNN, it is difficult to achieve maximum pooling operations on existing brain-like computing hardware platforms, resulting in reduced image classification accuracy, and the traditional average pooling method will cause blurred images and reduce the accuracy of classification tasks.

Method used

A maximum pooling layer circuit is designed, including adder, register, data selector, time step counter, comparator and pulse generator. By accumulating the number of pulses of neurons in each time step, the maximum is selected. The value is output to achieve maximum pooling operation, and reduce signal flip through operand masking design and reduce dynamic power consumption.

Benefits of technology

It improves the accuracy of image target classification, enhances the robustness of the model, and reduces the power consumption of edge-end deployment, solving the high energy consumption problem of traditional DNN networks when deploying at edge-end.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940807A_ABST
    Figure CN118940807A_ABST
Patent Text Reader

Abstract

The invention relates to a maximum pooling layer circuit for SNN target classification. The neuron pulse In1 of the circuit completes accumulation of the number of pulses once in each clock period through the adder and the register, and the output of the register is sent to the data selector; when the time step counter accumulates to a time step, the data selector is controlled to gate the output of the register, and the pulse number Out1 accumulated by neurons in the time step is output to the comparator; when a time step length is not reached, the data selector gates the 0 signal; similarly, the input of the neuron pulses In2, In3,..., InN correspondingly outputs Outt2, Outt3,..., OutN signals, and the Outt2, Outt3,..., OutN signals are input to a comparator for comparison, so that the maximum value of the accumulated pulse number in the time step length of N neurons is obtained and stored in an output register, and finally, according to the accumulated pulse value in the output register, an output pulse is generated through a pulse generator. Therefore, the output of the maximum pooling layer is obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a maximum pooling layer circuit for SNN target classification, and relates to the field of image classification using a spiking neural network (Spiking Neural Network, SNN). Background Art

[0002] At present, the use of deep neural networks (DNN) to classify various types of targets in images can achieve relatively good results. However, when training and reasoning DNN models, due to its complex network model and huge amount of calculation, it requires strong computing power and high energy consumption in hardware implementation, which greatly limits its deployment and application at the edge. SNN based on binary pulse signals is the third generation of neural networks after perceptrons and multi-layer feedforward neural networks (backpropagation, BP neural networks). Its design is inspired by the pulse transmission between neurons in the brain. After receiving enough input, neurons generate pulse signals, which will be transmitted to subsequent neurons to realize information transmission and processing. The principle of SNN is based on the time correlation between neurons. It uses the pulse information and event-driven characteristics of the brain's nervous system. Through discrete binary pulse signals, the high multiplication calculation cost of DNN models in hardware deployment can be avoided, and it has the characteristics of low energy consumption and lightweight.

[0003] Pooling operations are required in the task of using DNN to complete target classification, which can ensure that the key feature information in the image is retained while compressing the image. When an image is input, the main function of the pooling layer is to compress the image, reduce the impact of useless information, improve the training speed and accuracy of the model, and prevent the model from overfitting. The maximum pooling method is usually used in DNN, which can well preserve texture features. However, in SNN, since the output of its network is in the form of pulses, the maximum neuron changes dynamically over time. When implementing the pooling operation on the brain-like computing hardware platform based on SNN, it is difficult to calculate the maximum neuron; therefore, on the existing brain-like computing hardware platform, the average pooling method is generally used. However, using average pooling instead of maximum pooling will cause the image to become blurred, thereby reducing the accuracy of the classification task. Summary of the invention

[0004] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art, to provide a maximum pooling layer circuit for SNN target classification, and to use the maximum pooling method to replace the average pooling method on the existing brain-like computing hardware.

[0005] The technical solution of the present invention is:

[0006] A maximum pooling layer circuit for SNN target classification, comprising: an adder, a register, a data selector, a time step counter, a comparator, an output register and a pulse generator;

[0007] The input of the maximum pooling layer circuit is N neuron pulses In1, In2, In3, ..., InN. Neuron pulse In1 completes the accumulation of the number of pulses once in each clock cycle through the adder and register, and the output of the register is sent to the input of the data selector; the operand shielding design is realized through the data selector. When the time step counter accumulates to a time step, the data selector is controlled to select the register output, and the number of pulses accumulated by the neuron in the time step Out1 is output to the comparator; when a time step is not reached, the data selector selects the 0 signal, thereby reducing the signal flip in the circuit and reducing dynamic power consumption;

[0008] Similarly, the neuron pulses In2, In3, ..., InN inputs correspond to the output Out2, Out3, ..., OutN signals, which are input to the comparator for comparison, so as to obtain the maximum value of the cumulative number of pulses within the N neuron time steps and store them in the output register. Finally, according to the cumulative pulse value in the output register, the output pulse is generated by the pulse generator to obtain the output of the maximum pooling layer.

[0009] Furthermore, the input image is pulse-encoded, the target features are extracted through the convolution layer, and the target feature pulses are input to the next level of pooling layer, that is, the maximum pooling layer circuit.

[0010] Furthermore, the neuron pulses In1, In2, In3, ..., InN are generated in the digital circuit in the form of a high-level signal of one clock cycle.

[0011] Furthermore, the time step is a hyperparameter and is dynamically set according to the size of the input image and the requirement for the size of the output image.

[0012] In a second aspect, the present invention further proposes a SNN image target classification method, comprising:

[0013] (1) Load the trained SNN model and weights onto the brain-inspired computing platform;

[0014] (2) Read remote sensing images that need to be classified;

[0015] (3) Converting remote sensing image data into a Poisson distributed pulse sequence in the pulse generation layer;

[0016] (4) Using convolution in the feature extraction layer to extract target feature information;

[0017] (5) In the maximum pooling layer, the maximum neuron is selected based on the cumulative number of pulses emitted by each neuron before the current moment;

[0018] (6) In the pulse output layer, the pulses are accumulated into the membrane potential, and the targets in the remote sensing image are classified according to the membrane potential;

[0019] (7) Finally output the image classification results.

[0020] Furthermore, the method of selecting the maximum neuron in the maximum pooling layer according to the number of pulses emitted by each neuron before the current moment is implemented in a maximum pooling layer circuit, and the maximum pooling layer circuit includes: an adder, a register, a data selector, a time step counter, a comparator, an output register and a pulse generator;

[0021] The input of the maximum pooling layer circuit is N neuron pulses In1, In2, In3, ..., InN. Neuron pulse In1 completes the accumulation of the number of pulses once in each clock cycle through the adder and register, and the output of the register is sent to the input of the data selector; the operand shielding design is realized through the data selector. When the time step counter accumulates to a time step, the data selector is controlled to select the register output, and the number of pulses accumulated by the neuron in the time step Out1 is output to the comparator; when a time step is not reached, the data selector selects the 0 signal, thereby reducing the signal flip in the circuit and reducing dynamic power consumption;

[0022] Similarly, the neuron pulses In2, In3, ..., InN inputs correspond to the output Out2, Out3, ..., OutN signals, which are input to the comparator for comparison, so as to obtain the maximum value of the cumulative number of pulses within the N neuron time steps and store them in the output register. Finally, according to the cumulative pulse value in the output register, the output pulse is generated by the pulse generator to obtain the output of the maximum pooling layer.

[0023] Furthermore, the neuron pulses In1, In2, In3, ..., InN are generated in the digital circuit in the form of a high-level signal of one clock cycle.

[0024] Furthermore, the time step is a hyperparameter and is dynamically set according to the size of the input image and the requirement for the size of the output image.

[0025] The beneficial effects of the present invention compared with the prior art are:

[0026] (1) The present invention proposes a maximum pooling hardware implementation solution in the pooling layer of the SNN network, which solves the problem that the pooling layer of the model in the existing brain-like computing system cannot be implemented with maximum pooling hardware, and improves the accuracy of target classification in the image;

[0027] (2) The present invention sets the time step in the maximum pooling layer to an adaptive mode, so that the model can adapt to images of different sizes and improve the robustness of the model;

[0028] (3) The present invention adopts a low-energy SNN-based network to implement image classification tasks, solving the problem of power consumption when deploying traditional DNN-based networks at the edge when performing target classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the hardware implementation of the maximum pooling layer;

[0030] Figure 2 This is a flowchart of image target classification based on SNN; DETAILED DESCRIPTION

[0031] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0032] Since the input pulse signal has no size difference in SNN, the pooling layer uses average pooling as the main method in the common SNN image target classification model. The average number of pulses of each neuron in each pooling unit at the current moment is taken as the output of the pooling layer. However, since the average pooling process reduces the maximum pulse frequency of each layer of neurons, the entire network converges slowly and the classification accuracy is low.

[0033] The theoretical basis of the maximum pooling operation is to select the neuron with the maximum pulse firing rate within a time interval as the output, that is, to establish a connection between the neuron output within the maximum pooling window and the neuron with the largest pulse firing rate. However, since the pulse firing rate of neurons changes over time and cannot be predicted, in SNN, the maximum pooling operation needs to update the connection at every moment, resulting in increased network complexity and reduced operating efficiency. A more common simplified solution is to select the maximum value in the pooling window at each moment within the time interval as the output at the current moment, thereby forming a pulse sequence. When the network structure is small, the effect is good, but once the network structure is deepened and the data set is complex, errors such as pulse accumulation will occur, resulting in extremely low accuracy.

[0034] In order to reduce the error caused by maximum pooling in the conversion process, the cumulative number of pulses emitted by each neuron before the current moment is recorded instead of the cumulative membrane voltage of the neuron, so as to measure the maximum pulse rate and select the largest neuron.

[0035] The maximum pooling layer circuit for SNN target classification proposed by the present invention includes: an adder, a register, a data selector, a time step counter, a comparator, an output register and a pulse generator;

[0036] The input of the maximum pooling layer circuit is N neuron pulses In1, In2, In3, ..., InN. Neuron pulse In1 completes the accumulation of the number of pulses once in each clock cycle through the adder and register, and the output of the register is sent to the input of the data selector; the operand shielding design is realized through the data selector. When the time step counter accumulates to a time step, the data selector is controlled to select the register output, and the number of pulses accumulated by the neuron in the time step Out1 is output to the comparator; when a time step is not reached, the data selector selects the 0 signal, thereby reducing the signal flip in the circuit and reducing dynamic power consumption;

[0037] Similarly, the neuron pulses In2, In3, ..., InN inputs correspond to the output Out2, Out3, ..., OutN signals, which are input to the comparator for comparison, so as to obtain the maximum value of the cumulative number of pulses within the N neuron time steps and store them in the output register. Finally, according to the cumulative pulse value in the output register, the output pulse is generated by the pulse generator to obtain the output of the maximum pooling layer.

[0038] Neuron pulses In1, In2, In3, ..., InN are generated in a digital circuit as a high-level signal of one clock cycle. The time step is a hyperparameter and is dynamically set according to the size of the input image and the requirement for the size of the output image.

[0039] Embodiment 1:

[0040] like Figure 1 As shown, the hardware circuit design of the maximum pooling provided in this embodiment is as follows:

[0041] The input image is pulse-encoded, the target features are extracted through the convolution layer, and the target feature pulses are input to the pooling layer of the next level. The maximum pooling window size is selected to be 4 neurons, as shown in the figure above. A total of 4 neuron pulses are input into this window. Taking the first neuron In1 as an example, the pulse input is generated in the digital circuit as a high-level signal of one clock cycle. The number of pulses is accumulated once in each clock cycle through the adder and the register, and the output of the register is sent to the input of the data selector (MUX); the operand masking circuit design is realized through the data selector (MUX). When the time step counter accumulates to a time step, the data selector (MUX) is controlled to select the register output, and the number of pulses accumulated by the neuron in the time step is output to Out1. In addition, when a time step is not reached, the data selector (MUX) selects the 0 signal, thereby reducing the signal flip in the circuit, thereby greatly reducing the dynamic power consumption. The time step here is used as a hyperparameter, which is dynamically set adaptively according to the size of the input image and the requirements for the output image size. Similarly, the corresponding output signals Out2, Out3, Out4 of neurons In2, In3, and In4 are input to the comparator (cmp) for comparison, so as to obtain the maximum value of the cumulative number of pulses within the time step of the four neurons and store it in the output register. Finally, according to the cumulative pulse value in the output register, the output pulse is generated by the pulse generator to obtain the output of the maximum pooling layer.

[0042] Embodiment 2:

[0043] This embodiment provides an SNN image target classification method based on the above-mentioned maximum pooling layer circuit.

[0044] The task design of image target classification based on SNN includes image input layer, pulse generation layer, feature extraction layer, maximum pooling layer and output layer. The image of the input layer is a visible light remote sensing image, and the output is the target category.

[0045] like Figure 2 As shown, the specific target classification process is shown in the figure:

[0046] (1) Load the trained SNN model and weights onto the brain-inspired computing platform;

[0047] (2) Read remote sensing images that need to be classified;

[0048] (3) Converting remote sensing image data into a Poisson distributed pulse sequence in the pulse generation layer;

[0049] (4) Using convolution in the feature extraction layer to extract target feature information;

[0050] (5) In the maximum pooling layer, the maximum neuron is selected according to the cumulative number of pulses emitted by each neuron before the current moment; the maximum pooling layer in this step is implemented using the maximum pooling layer circuit.

[0051] (6) In the pulse output layer, the pulses are accumulated into the membrane potential, and the targets in the remote sensing image are classified according to the membrane potential;

[0052] (7) Finally output the image classification results.

[0053] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.

Claims

1. A maximum pooling layer circuit for SNN target classification, characterized in that include: adders, registers, data selectors, time step counters, comparators, output registers, and pulse generators; The input of the maximum pooling layer circuit is N neuron pulses In1, In2, In3, ..., InN. Neuron pulse In1 completes the accumulation of the number of pulses once in each clock cycle through the adder and register, and the output of the register is sent to the input of the data selector; the operand shielding design is realized through the data selector. When the time step counter accumulates to a time step, the data selector is controlled to select the register output, and the number of pulses Out1 accumulated by the neuron in the time step is output to the comparator; When a time step is not reached, the data selector selects the 0 signal, thereby reducing the signal flip in the circuit and reducing dynamic power consumption; Similarly, the neuron pulses In2, In3, ..., InN inputs correspond to the output Out2, Out3, ..., OutN signals, which are input to the comparator for comparison, so as to obtain the maximum value of the cumulative number of pulses within the N neuron time steps and store them in the output register. Finally, according to the cumulative pulse value in the output register, the output pulse is generated by the pulse generator to obtain the output of the maximum pooling layer.

2. A maximum pooling layer circuit for SNN target classification according to claim 1, characterized in that: The input image is pulse-encoded, the target features are extracted through the convolution layer, and the target feature pulses are input to the next level of pooling layer, that is, the maximum pooling layer circuit.

3. A maximum pooling layer circuit for SNN target classification according to claim 1, characterized in that: Neuron pulses In1, In2, In3, ..., InN are generated in a digital circuit as a high-level signal of one clock cycle.

4. The maximum pooling layer circuit for SNN target classification according to claim 1, characterized in that: The time step is a hyperparameter and is dynamically set according to the size of the input image and the requirement for the output image size.

5. A SNN image target classification method, characterized in that include: (1) Load the trained SNN model and weights onto the brain-inspired computing platform; (2) Read remote sensing images that need to be classified; (3) Converting remote sensing image data into a Poisson distributed pulse sequence in the pulse generation layer; (4) Using convolution in the feature extraction layer to extract target feature information; (5) In the maximum pooling layer, the maximum neuron is selected based on the cumulative number of pulses emitted by each neuron before the current moment; (6) In the pulse output layer, the pulses are accumulated into the membrane potential, and the targets in the remote sensing image are classified according to the membrane potential; (7) Finally output the image classification results.

6. The SNN image target classification method according to claim 5, characterized in that: The method of selecting the maximum neuron in the maximum pooling layer according to the number of pulses emitted by each neuron before the current moment is implemented in the maximum pooling layer circuit, which includes: an adder, a register, a data selector, a time step counter, a comparator, an output register and a pulse generator; The input of the maximum pooling layer circuit is N neuron pulses In1, In2, In3, ..., InN. Neuron pulse In1 completes the accumulation of the number of pulses once in each clock cycle through the adder and register, and the output of the register is sent to the input of the data selector; the operand shielding design is realized through the data selector. When the time step counter accumulates to a time step, the data selector is controlled to select the register output, and the number of pulses accumulated by the neuron in the time step Out1 is output to the comparator; when a time step is not reached, the data selector selects the 0 signal, thereby reducing the signal flip in the circuit and reducing dynamic power consumption; Similarly, the neuron pulses In2, In3, ..., InN inputs correspond to the output Out2, Out3, ..., OutN signals, which are input to the comparator for comparison, so as to obtain the maximum value of the cumulative number of pulses within the N neuron time steps and store them in the output register. Finally, according to the cumulative pulse value in the output register, the output pulse is generated by the pulse generator to obtain the output of the maximum pooling layer.

7. The SNN image target classification method according to claim 6, characterized in that: Neuron pulses In1, In2, In3, ..., InN are generated in a digital circuit as a high-level signal of one clock cycle.

8. The SNN image target classification method according to claim 6, characterized in that: The time step is a hyperparameter and is dynamically set according to the size of the input image and the requirement for the output image size.