Ping-pong architecture based sparse spike neural network accelerator

Abstract

The application provides a sparse pulse neural network accelerator based on a ping-pong architecture, which transmits compressed weight values to a compressed weight calculation module, uses a sparse pulse detection module to extract valid pulse indexes from pulse input signals, avoids each pulse signal from participating in operation, reduces the amount of calculation, and accumulates non-zero values in the compressed weight values to the membrane potential of neurons according to the valid pulse indexes, so as to finally determine whether to fire a pulse or not. Compared with the technical solution in which all synapses are activated and participate in operation in a conventional synaptic cross array, only the synapse weights corresponding to the valid pulse indexes are activated in the application, and other synapses do not participate in operation, so that the amount of calculation is reduced, the running power consumption of the whole chip is reduced, and the running speed, energy efficiency and area efficiency of the pulse neural network are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a sparse spiking neural network accelerator based on a ping-pong architecture. BACKGROUND

[0002] Spiking Neural Network (SNN) can bring the improvement of machine computing power due to its low power consumption and high concurrency, is a very potential computing mode, and is considered to be the future of artificial intelligence research.

[0003] Since the signal conduction mechanism of neurons in the spiking neural network does not match the traditional von Neumann computer architecture, it is urgent to design a suitable hardware accelerator for the spiking neural network to run the spiking neural network. The current neuromorphic accelerator often uses a synaptic cross array with regular structure and fixed size to directly store the synaptic connection matrix of the SNN model. Regardless of the synaptic sparsity, all synapses will participate in the operation, which increases the amount of calculation, which leads to the fact that the spatial sparsity of the SNN model cannot be reflected on such a neuromorphic accelerator. Another accelerator design scheme is to cache the input pulses in the form of a bitmap, which will cause the hardware to judge the pulse validity of each bit in the input pulse vector, increasing the calculation time, and the result is that the time sparsity of the pulse signal cannot be used to improve the operation speed of the hardware.

[0004] It can be seen that the current neural network accelerator cannot fully develop the potential performance advantages of the SNN model, resulting in low power consumption, slow operation speed, and low energy efficiency for running the SNN model. SUMMARY

[0005] The present application provides a sparse spiking neural network accelerator based on a ping-pong architecture to solve the defects that all synapses participate in the operation or each symbol in the input pulse signal participates in the operation in the current spiking neural network accelerator, resulting in large chip power consumption and large amount of calculation, and to realize the low power consumption and low delay operation of the spiking neural network.

[0006] The present application provides a sparse spiking neural network accelerator based on a ping-pong architecture, comprising a pulse input interface, a weight and neuron parameter input interface, a sparse pulse detection module, a compressed weight calculation module, and a leaky integral firing module; wherein,

[0007] The pulse input interface is used to receive a pulse input signal and input the pulse input signal to the sparse pulse detection module;

[0008] The weight and neuron parameter input interface is used to receive a compressed weight value and input the compressed weight value to the compressed weight calculation module;

[0009] The sparse pulse detection module is configured to extract an effective pulse index from the pulse input signal, and the effective pulse index is used to represent the position of a non-zero value in the pulse input signal.

[0010] The compression weight calculation module is configured to decompress the compression weight value according to the effective pulse index to obtain an effective weight value matrix, calculate a weighted sum of the effective weight value matrix and the pulse input signal to obtain a membrane potential increment of each neuron, and update a membrane potential cumulative amount corresponding to each neuron by using the membrane potential increment of each neuron.

[0011] The leaky integrate-and-fire module is configured to determine a size relationship between the updated membrane potential cumulative amount and a preset threshold, and determine an output pulse result corresponding to each neuron according to the size relationship.

[0012] The sparse pulse neural network accelerator based on the ping-pong architecture further includes a pulse cache module group, and the pulse cache module group includes a first pulse cache module and a second pulse cache module.

[0013] The pulse cache module group is configured to control the read-write state of the first pulse cache module and the second pulse cache module in each cache cycle in a ping-pong switching manner, so that one of the pulse cache modules is in a read state and the other pulse cache module is in a write state in each cache cycle.

[0014] The sparse pulse neural network accelerator based on the ping-pong architecture further includes a weight cache module group, and the weight cache module group includes a first weight cache module and a second weight cache module.

[0015] The weight cache module group is configured to control the read-write state of the first weight cache module and the second weight cache module in each cache cycle in a ping-pong switching manner, so that one of the weight cache modules is in a read state and the other weight cache module is in a write state in each cache cycle.

[0016] The sparse pulse neural network accelerator based on the ping-pong architecture further includes a neuron parameter cache module group, and the neuron parameter cache module group includes a first neuron parameter cache module and a second neuron parameter cache module.

[0017] The neuron parameter cache module group is configured to control the read-write state of the first neuron parameter cache module and the second neuron parameter cache module in each cache cycle in a ping-pong switching manner, so that one of the neuron parameter cache modules is in a read state and the other neuron parameter cache module is in a write state in each cache cycle.

[0018] According to the application, a sparse pulse neural network accelerator based on a ping-pong architecture is provided, and the sparse pulse detection module is further configured to divide the pulse input sequence corresponding to the pulse input signal into a plurality of groups of sub-sequences.

[0019] The bit-wise OR operation is sequentially performed on each group of sub-sequences and itself to obtain a bit-wise OR operation result; if the bit-wise OR operation result is all 0, the operation of the current group of sub-sequences is ended;

[0020] If the bit-wise OR operation result is not all 0, the bit-wise OR operation result is taken as a current detection sequence, and a plurality of rounds of detection are performed on the current detection sequence; in each round of detection, a difference value is obtained by subtracting 1 from the current detection sequence; a bit-wise AND operation result is obtained by performing a bit-wise AND operation on the difference value and the current detection sequence; an effective pulse one-hot code is obtained by performing a bit-wise XOR operation on the bit-wise AND operation result and the current detection sequence; an effective pulse index is obtained by performing binary conversion on the effective pulse one-hot code; it is determined whether the bit-wise AND operation result is all 0; if yes, the detection of the current detection sequence is ended, and the step of sequentially performing the bit-wise OR operation on each group of sub-sequences and itself to obtain a bit-wise OR operation result is returned; if no, the bit-wise AND operation result is taken as a current detection sequence, and the step of obtaining a difference value by subtracting 1 from the current detection sequence is returned.

[0021] According to the application, a sparse pulse neural network accelerator based on a ping-pong architecture is provided, and the compression weight calculation module includes a row offset calculation module, a column index weight calculation unit, a column index encoding module, a non-zero weight module, a weight distributor and a processing unit array; wherein,

[0022] The row offset operation unit is configured to obtain a row offset of a current row and a row offset of a next row adjacent to the current row.

[0023] The column index weight operation unit is configured to analyze the row offset of the current row and the row offset of the next row adjacent to the current row to obtain a start address and an end address of the column index encoding module and the non-zero weight module; and obtain a non-zero weight and a column index Delta code from the column index encoding module and the non-zero weight module according to the start address and the end address.

[0024] The weight distributor includes an adder chain composed of a preset number of adders, and the adder chain is configured to take the non-zero weight and the Delta code as inputs of the processing unit array.

[0025] The processing unit array includes a preset number of processing units, each processing unit uses an adder to perform operation, and the membrane potential accumulation amount is updated according to an operation result of the adder.

[0026] According to the application, the column index weight operation unit is further configured to end the processing of the current row when the start address exceeds the end address.

[0027] According to the application, each processing unit further comprises a weight mask generation module, a multiplexer and an adder.

[0028] The weight mask generation module is configured to generate a weight mask according to a weight distribution state.

[0029] The processing unit is further configured to use the weight mask and the non-zero weight as inputs of the multiplexer, so that the multiplexer filters valid non-zero values according to the weight mask, and uses the valid non-zero values and the membrane potential accumulation as inputs of the adder, to obtain the updated membrane potential accumulation output by the adder.

[0030] According to the application, the leaky integrate-and-fire module is further configured to add the updated membrane potential accumulation and a preset leak value to obtain a leak value, and determine that the output pulse result is a firing pulse signal when the leak value is greater than a preset threshold, or determine that the pulse result is no firing pulse when the leak value is less than or equal to the preset threshold.

[0031] According to the application, the sparse spiking neural network accelerator based on the ping-pong architecture further comprises a pulse output interface.

[0032] The leaky integrate-and-fire module is further configured to send the updated membrane potential accumulation to the pulse output interface and reset the membrane potential accumulation when the updated membrane potential accumulation is greater than the preset threshold.

[0033] The sparse spiking neural network accelerator based on the ping-pong architecture provided by the application avoids that each bit of the pulse signal participates in the operation, reduces the calculation amount, and accumulates the non-zero values in the compressed weight value to the membrane potential of the neuron according to the effective pulse index by the compressed weight calculation module, so as to finally determine whether to fire a pulse or not. Compared with the traditional synaptic cross array in which all synapses are activated and participate in the operation, only the synaptic weight corresponding to the effective pulse index is activated in the application, and other synapses do not participate in the operation, thereby reducing the calculation amount, reducing the running power consumption of the entire chip, and improving the running speed, energy efficiency and area efficiency of the pulse neural network. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0035] Figure 1 is a functional module schematic diagram of the sparse pulse neural network accelerator based on the ping-pong architecture provided by the present application;

[0036] Figure 2 is a matrix element schematic diagram of the row compressed matrix provided by the present application;

[0037] Figure 3 is a principle schematic diagram of the LIF neuron provided by the present application;

[0038] Figure 4 is a flow schematic diagram of the ping-pong running method provided by the present application;

[0039] Figure 5 is a hardware circuit schematic diagram of the sparse pulse detection module provided by the present application;

[0040] Figure 6 is a structural principle schematic diagram of the compressed weight calculation module provided by the present application;

[0041] Figure 7 is a functional module schematic diagram of the compressed weight calculation module provided by the present application;

[0042] Figure 8 is a hardware circuit structure schematic diagram of the column index weight operation unit provided by the present application;

[0043] Figure 9 is a decoder schematic diagram of the adder chain provided by the present application;

[0044] Figure 10 is a structural schematic diagram of the PE array provided by the present application. DETAILED DESCRIPTION

[0045] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor also belong to the protection scope of the present application.

[0046] The following will be described in combination with Figures 1-10A specific embodiment of the present application is described:

[0047] The sparse pulse neural network accelerator based on the ping-pong architecture provided by the embodiment of the present application has functional modules as shown in the figure, including a pulse input interface 101, a weight and neuron parameter input interface 102, a sparse pulse detection module 103, a compressed weight calculation module 104, and a leaky integral firing module 105. Figure 1

[0048] The pulse input interface 101 is configured to receive a pulse input signal and input the pulse input signal to the sparse pulse detection module 103.

[0049] Specifically, the pulse input signal is transmitted into the sparse pulse detection module 103 through the pulse input interface 101.

[0050] The weight and neuron parameter input interface 102 is configured to receive a compressed weight value and input the compressed weight value to the compressed weight calculation module 104.

[0051] The compressed weight value refers to a weight value in a sparse matrix storage format. Since the weight parameters in the pulse neural network model are very large, using a sparse data storage format can save a large amount of storage space and speed up the calculation. The sparse matrix representation format used by the present application is the CSR (Compressed Sparse Row) format.

[0052] Specifically, the compressed weight value stored in the CSR format is input to the compressed weight calculation module 104 through the weight and neuron parameter input interface 102.

[0053] The sparse pulse detection module 103 is configured to extract an effective pulse index from the pulse input signal. The effective pulse index is used to represent the position of the non-zero value in the pulse input signal.

[0054] Since the pulse input signal is a sparse vector / tensor (so-called sparse, which means having a large number of zeros), it is necessary to multiply the above sparse vector / tensor with the synaptic weight in the calculation process, which means that most of the synaptic weights are multiplied by "0", and the result is still 0. In order to save computing power, the present application does not need to read each bit of the pulse input signal from the memory, but only needs to read the effective pulse (i.e. the pulse signal with non-zero value). In order to reduce the overall power consumption of the chip, the present application only activates the neurons in the synaptic cross matrix that need to be activated, that is, activates the corresponding neurons in the synaptic cross array according to the position of the non-zero value in the pulse input signal. In order to determine which position of the neuron needs to be activated, the present application uses the sparse pulse detection module 103 to extract the effective pulse index from the pulse input signal. The effective pulse index is used to represent the position of the non-zero value in the pulse input signal. ​

[0055] The compressed weight calculation module 104 is configured to decompress the compressed weight values received from outside the chip according to the effective pulse index to obtain an effective weight value matrix, calculate a weighted sum of the effective weight value matrix and the pulse input signal to obtain a membrane potential increment on each neuron, and update the membrane potential accumulation corresponding to each neuron by using the membrane potential increment on each neuron.

[0056] The effective weight value matrix is shown in the matrix on the left side of the formula (1) and is obtained by restoring the compressed weight values according to the effective pulse index. Figure 2 The compressed weight values are shown in the "non-zero values" in the formula (2), and each element in the compressed weight values is a non-zero value in the effective weight value matrix. Figure 2 Due to limited storage space on the chip and a large number of parameters and weights of the neural network, the CSR (Compressed Row Storage) format is used to store the weights in order to save storage space. The CSR storage format uses three one-dimensional arrays (including row offset, column index and non-zero value) to represent a sparse matrix (i.e., the effective weight value matrix). The row index array is used to store the cumulative count of non-zero values of the current row and the previous row. For example, the mth element in the row offset array represents the number of non-zero values above the mth row in the effective weight value matrix. For example, the 2nd element "4" in the row offset array in the formula (2) represents that the total number of non-zero values in all rows above the 2nd row (the row number is encoded from 0) in the effective weight value matrix is 4 (1, 7, 2 and 8, respectively). Similarly, the number of non-zero values in each row in the effective weight value matrix can be known by subtracting adjacent two elements in the row offset array. For example, for the ith row in the effective weight value matrix, the number of non-zero values in the ith row can be calculated by subtracting the value of the ith element from the value of the ith+1 element in the row offset array. The column index array is used to store the column index of each element in the non-zero value array. For example, the column index corresponding to the non-zero value "7" is 1, indicating that 7 is located in the 1st column (the index is encoded from 0) in the effective weight value matrix. Figure 2

[0057] Specifically, the compressed weight calculation module 104 decompresses the compressed weight values stored in the CSR format to obtain the effective weight value matrix, calculates a weighted sum of the effective weight value matrix and the pulse input signal to obtain a membrane potential increment on each neuron, and accumulates the membrane potential increment on each neuron to the membrane potential accumulation corresponding to each neuron to obtain an updated membrane potential accumulation.

[0058] The leaky integrate-and-fire module 105 is configured to determine a size relationship between the updated membrane potential accumulation and a preset threshold value, and determine an output pulse result corresponding to each neuron according to the size relationship. ​

[0059] Specifically, as Figure 3 shown, Figure 3 The pulse neural network algorithm structure used by the present application. The neuron of the pulse neural network supported by the present application is a linear LIF (Leaky Integrate-and-Fire) model. After accumulating the pulse timing sequence input, a linear leakage operation, threshold comparison and pulse emission are performed. The membrane potential dynamics equation of the LIF neuron model is as follows:

[0060] V j (t+1)=V j (t)+∑x i w ij -λ j (1)

[0061] Where V j (t) is the membrane potential accumulation value of neuron j at time t, w ij is the i-th synaptic weight corresponding to neuron j, x i is the pulse input signal value of the i-th synapse, and λ j is the linear leakage of neuron j.

[0062] The leakage integrate-and-fire module 105 judges the size relationship between the updated membrane potential accumulation Vi(t+1) and the preset threshold value. If it is greater than the preset threshold value, a neural pulse signal is emitted, and the membrane potential accumulation on the neuron is reset.

[0063] In the above embodiment, by transmitting the compressed weight value to the compressed weight calculation module, the sparse pulse detection module is used to extract the effective pulse index from the pulse input signal, avoiding that each bit of the pulse signal participates in the operation, reducing the calculation amount. The compressed weight calculation module accumulates the non-zero values in the above compressed weight value to the membrane potential of the neuron according to the effective pulse index, and finally determines whether to fire a pulse or not. Compared with the traditional synaptic cross array in which all synapses are activated and participate in the operation, only the synaptic weight corresponding to the effective pulse index is activated in the present application, thereby reducing the calculation amount, reducing the running power consumption of the whole chip, improving the running speed, energy efficiency and area efficiency of the pulse neural network.

[0064] In an embodiment, as Figure 1As shown, the sparse spiking neural network accelerator based on the ping-pong architecture further comprises a pulse cache module group 106, which comprises a first pulse cache module and a second pulse cache module; wherein the pulse cache module group 106 is configured to control the read-write state of the first pulse cache module and the second pulse cache module in each cache cycle in a ping-pong switching manner, so that one of the pulse cache modules is in a read state and the other pulse cache module is in a write state in each cache cycle.

[0065] Specifically, the sparse spiking neural network accelerator based on the ping-pong architecture is internally provided with two pulse cache modules RAM (Random Access Memory), which are used to encode the pulse input signals received from outside the chip to obtain pulse input codes suitable for calculation. During the encoding process, the system can update one of the RAMs while the other RAM is completing the calculation. Correspondingly, the output pulse codes also need to be decoded into output pulse signals. During the decoding process, the system also adopts a ping-pong operation method, i.e., one of the RAMs is completing the calculation while the other RAM is being updated.

[0066] The above embodiment uses the pulse cache module group to realize ping-pong caching during the encoding of the pulse input signals or the decoding of the pulse output signals, accelerates the simultaneous existence of I / O operations and data processing operations, and improves the throughput of the accelerator.

[0067] In an embodiment, as shown in Figure 1 As shown, the sparse spiking neural network accelerator based on the ping-pong architecture further comprises a weight cache module group 107, which comprises a first weight cache module and a second weight cache module; wherein the weight cache module group 107 is configured to control the read-write state of the first weight cache module and the second weight cache module in each cache cycle in a ping-pong switching manner, so that one of the weight cache modules is in a read state and the other weight cache module is in a write state in each cache cycle.

[0068] Specifically, the sparse spiking neural network accelerator based on the ping-pong architecture is internally provided with two weight cache modules RAM, which are used to decompress the compressed weight values in the CSR format received from outside the chip to obtain an effective weight matrix. During the decompression process, the system can update one of the RAMs while the other RAM is completing the decompression.

[0069] The above embodiment uses the weight cache module group to realize ping-pong caching during the decompression of the compressed weight values in the CSR format, accelerates the simultaneous existence of I / O operations and data processing operations, and improves the throughput of the accelerator.

[0070] In one embodiment, the accelerator further includes a neuron parameter cache module group 108; the neuron parameter cache module group 108 includes a first neuron parameter cache module and a second neuron parameter cache module; the neuron parameter cache module group 108 is used to control the read and write states of the first neuron parameter cache module and the second neuron parameter cache module in each cache cycle in a ping-pong switching manner, so that one neuron parameter cache module is in a read state and the other neuron parameter cache module is in a write state in each cache cycle.

[0071] Specifically, the sparse spiking neural network accelerator based on the ping-pong architecture has two internal neuron parameter cache modules (RAMs) for decoding neuron parameters received from outside the chip to obtain neuron parameters suitable for computation. During the decoding process, the entire system can complete the decompression of one RAM while updating the other RAM.

[0072] Furthermore, the ping-pong running algorithm used in this invention includes three dimensions of control: how many time slots the input pulse of each time step contains, how many time steps each group of neurons needs to calculate, and how many groups each layer of the network needs to be divided into for batch calculation.

[0073] like Figure 4 As shown, taking a two-layer 1024-512-256 fully connected spiking neural network as an example, the accelerator receives data from external sources after power-on, including spiking RAM#0, weight RAM#0, and neuron parameter RAM#0. Upon receiving the first global synchronization command "Sync_all", the accelerator's weight RAM#1 and neuron parameter RAM#1 receive data from external sources, while the data from spiking RAM#0, weight RAM#0, and neuron parameter RAM#0 are sent to the core computing unit. The calculation results are then sent to spiking RAM#1, at which point the accelerator is calculating the first 256 neurons of the first layer. Upon receiving the second global synchronization command "Sync_all", the accelerator's weight RAM#0 and neuron parameter RAM#0 receive data from external sources, while the data from spiking RAM#0, weight RAM#1, and neuron parameter RAM#1 are sent to the core computing unit. The calculation results are then sent to spiking RAM#1, at which point the accelerator is calculating the last 256 neurons of the first layer. Upon receiving the third global synchronization command "Sync_all", the accelerator's weight RAM#1 and neuron parameter RAM#1 receive data from the outside. The data from pulse RAM#1, weight RAM#0, and neuron parameter RAM#0 are sent to the core computing unit. The calculation results are sent to pulse RAM#0 and the outside of the accelerator. At this time, the accelerator is calculating the 256 neurons in the second layer, thus completing the entire calculation process.

[0074] The above embodiments utilize a neuron parameter caching module group to implement ping-pong caching of neuron parameters during the decoding process, accelerating simultaneous I / O operations and data processing operations, and improving the throughput of the accelerator.

[0075] In one embodiment, the sparse pulse detection module 105 is further configured to divide the pulse input sequence corresponding to the pulse input signal into multiple sub-sequences;

[0076] Perform a bitwise OR operation on each subsequence and itself sequentially to obtain the bitwise OR operation result; if the bitwise OR operation result is all 0s, then end the operation on the current subsequence.

[0077] If the result of the bitwise OR operation is not all zeros, then the result of the bitwise OR operation is used as the current sequence to be detected, and multiple rounds of detection are performed on the current sequence to be detected. In each round of detection, the current sequence to be detected is subtracted by 1 to obtain the difference; the difference is then ANDed with the current sequence to be detected to obtain the bitwise AND operation result; the bitwise AND operation result is then XORed with the current sequence to be detected to obtain the valid pulse one-hot code; the valid pulse one-hot code is then converted to binary to obtain the valid pulse index; it is determined whether the result of the bitwise AND operation is all zeros. If yes, the detection of the current sequence to be detected ends, and the process returns to the step of performing a bitwise OR operation on each subsequence with itself to obtain the bitwise OR operation result; if no, the result of the bitwise AND operation is used as the current sequence to be detected, and the process returns to the step of subtracting 1 from the current sequence to obtain the difference.

[0078] The pulse input sequence is the sequence obtained by encoding the pulse input signal. The effective pulse index refers to the position of the non-zero value in the pulse input sequence.

[0079] Specifically, such as Figure 5 As shown, Figure 5 The internal circuit structure diagram of the sparse pulse detection module 105 is shown, including logic gates such as OR gates, multiplexers, D flip-flops, adders, AND gates, and XOR gates. The sparse pulse detection module 105 is mainly responsible for extracting the effective pulse index of the input pulse sequence. To reduce critical path delay, this invention groups the input pulse sequence. For example, a 1024-bit input pulse sequence is divided into 16 groups of 64-bit subsequences. Each 64-bit subsequence is first bitwise ORed with itself to obtain the bitwise OR operation result. If the bitwise OR operation result is all 0, it means that all 64 bits in this subsequence are 0, and the calculation of this subsequence ends, that is, the calculation of this 64-bit subsequence is skipped.

[0080] If the result of the bitwise OR operation is not all zeros, it is necessary to further detect the position of non-zero values ​​in the current subsequence. The detection method specifically includes: taking the result of the bitwise OR operation as the current sequence to be detected, performing multiple rounds of detection on the current sequence to be detected. In each round of detection, first subtract 1 from the current sequence to be detected (e.g., the 64-bit subsequence mentioned above) to obtain the difference, and then perform a bitwise AND operation between the difference and the current sequence to be detected itself to obtain the bitwise AND operation result; perform a bitwise XOR operation between the bitwise AND operation result and the data before subtracting 1 (i.e., the current sequence to be detected itself) to obtain the valid pulse one-hot code; at the same time, determine whether the result of the bitwise AND operation is all zeros. If yes, end the detection of the current subsequence and proceed to the detection of the next subsequence. In this example, that is, proceed to the detection of the next 64-bit subsequence; if no, take the result of the bitwise AND operation as the current sequence to be detected and return to the step of "subtracting 1 from the current sequence to be detected to obtain the difference". This process is repeated until each subsequence is detected, resulting in several valid pulse one-hot codes. The valid pulse index in binary code form corresponding to the pulse input signal can then be obtained through the decoding unit that converts the one-hot codes to binary codes.

[0081] In the above embodiment, a sparse pulse detection module 105 is obtained by combining multiple logic gate circuits, which can realize effective index detection of sparse pulses.

[0082] In one embodiment, such as Figure 6 As shown, Figure 6 The diagram illustrates the structural principle of the compressed weight calculation module 104. This module includes a row offset module (ROM), a column delta encoding module (CDM), a nonzero weight value module (NWVM), and a PE (Processing Element) array. Each PE unit is used to calculate the membrane potential increment, which is the dot product of the pulse input signal and the synaptic weight, i.e., x in formula (1). i w ij . Figure 6Take the synaptic crossbar array of 1024x256 as an example, 1024 represents 1024 fan-in (i.e. 1024 axons), 256 represents 256 hardware neurons, i.e. each hardware neuron has 1024 fan-in, but not every fan-in has an effective pulse signal. The valid pulse index delivered from the sparse pulse detection module will activate the row corresponding to the valid pulse index, then the row offset module (ROM) calculates the corresponding row offset, the column index delta encoding module (CDM) reads out the column index of the non-zero value of the row according to the row offset, and the non-zero weight value module (NWVM) accumulates the corresponding non-zero value to the membrane potential according to the column index. When all the valid pulses in the input pulse sequence are processed, all the membrane potential cumulative values will be sent to the leaky integrate-and-fire neuron dynamics module for LIF operation.

[0083] Since the effective weight matrix will be represented by three parameters, i.e. row offset, column index and non-zero value, as shown in Figure 2 Accordingly, in order to restore the effective weight matrix according to the above three parameters, as shown in Figure 7 The storage of the above three parameters in the accelerator is set as follows: row offset storage module (Row Offset Module, ROM, which is further divided into Even ROM and Odd ROM), column index delta encoding storage module (Column Delta Module, CDM) and non-zero value storage module (Nonzero Weight Value Module, NWVM). In order to carry out the pipeline, the row offset storage module is further divided into even row offset storage module (Even ROM) and odd row offset storage module (Odd ROM). The pipeline diagram of the compression weight calculation process is as shown in Figure 7As shown, for the input pulse pointing to the i-th row, the CSR decoder reads the row offset of the i-th row and the row offset of the i+1-th row from the Even ROM (even row offset module) and the Odd ROM (odd row offset module) respectively, and subtracts the two to know the number of non-zero values corresponding to the i-th row pulse, thereby obtaining the access address and access round number of the column index Delta encoding storage module (CDM) and the non-zero value storage module (NWVM). Since only one row offset access and operation is needed for the pulse of the i-th row, but more than one column index and weight access and operation may be needed for the i-th row, which will cause the throughput rate of the two types of data structure processing pipelines to be mismatched, therefore, in order to avoid the inefficient uneven pipeline stall, the CSR decoder is decoupled into a row offset operation unit and a column index weight operation unit in the circuit design, and a FIFO (First Input First Output) memory is used for synchronization between the two, as shown in Figure 7 .

[0084] The circuit structure of the column index weight operation unit is shown in Figure 8 , which obtains the row offset RO i of the i-th row and the row offset RO i+1 of the i+1-th row, and after parsing, obtains the access start address Addr i and the end address Addr i+1 of the column index Delta encoding storage module (CDM) and the non-zero value storage module (NWVM) respectively. When the MAE (Masked Autoencoder) output signal is valid, the column index Delta encoding (CI j ) and the non-zero weight (WV) in the Delta encoding format are read from the column index Delta encoding storage module (CDM) and the non-zero value storage module (NWVM) respectively according to Addr i every clock cycle and sent to the subsequent circuit unit (i.e., the corresponding PE unit). When Addr i exceeds Addr i+1 , the column index weight operation unit ends the processing of the i-th row and obtains the row offset of the next row from the row offset storage module.

[0085] As shown in Figure 9 , the Delta encoded column index (i.e., CI j ) read from the column index Delta encoding storage module (CDM) every clock cycle is decoded into the column index (i.e., CI jSince each clock cycle may read ΔCI (i.e., column index Delta encoding) that does not belong to the current row from the column index Delta encoding module (CDM), these ΔCI that do not belong to the current row are filtered out by the MUX (multiplexer) to obtain the filtered Delta encoding ΔCI′. If the current clock cycle is the first cycle of the row operation, ΔCI′0 will be used as CI0 for this cycle; if the current clock cycle is not the first cycle of the row operation, it will be used as CI0 for the previous cycle. 31 Add ΔCI′0 to the base. And the other CI... j =CI j-1 +ΔCI′ j .

[0086] The structure of the PE array is as follows Figure 10 As shown, Figure 10 A PE array containing 32 PE units is shown; each PE unit includes one 16-bit adder and two MUXs (multiplexers), with the accumulated membrane potential values ​​of all dendrites input to the MUX1 (multiple-to-one multiplexer) of each PE. This results in 32 paired CIs. j After (column index) and WV (non-zero weight), the weight allocator will assign each CI pair... j (index of column j) and WV j (the j-th non-zero weight) is the corresponding PE. j Input: where CI j It will serve as the multiplexer signal for MUX1, and will select the corresponding... (The accumulated membrane potential of the neuron corresponding to the j-th column index) is used as an operand of the adder; WV j The input will be fed into one of the two input terminals of the MUX2. After being filtered by a weight mask, the effective WV (weighted output) is obtained. j It will be used as another operand in the adder. The weight assigner updates based on the result of the adder's calculation. The updated As the cumulative value for the next period.

[0087] The purpose of the weight mask is to filter out invalid non-zero weights WV to avoid calculation errors. There are two reasons why invalid non-zero weights WV exist in the calculation: (1) When accessing the NWVM, the weights at the same memory address in the NWVM but not belonging to that row will also be read as non-zero weights WV; (2) In order to implement the fan-in extension function, the 128 columns of the weight matrix are divided into multiple shared fan-in dendrite clusters. When accessing the NWVM, the weights of other dendrites that do not belong to the shared fan-in dendrite cluster that receives the current input pulse vector will also be read as non-zero weights WV.

[0088] The above embodiment further improves the processing level of the pipeline by setting the odd row offset module and the even row offset module.

[0089] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0090] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment or some parts of the embodiment.

[0091] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A sparse spiking neural network accelerator based on a ping-pong architecture, characterized in that, The pulse input interface, the weight and neuron parameter input interface, the sparse pulse detection module, the compressed weight calculation module, and the leakage integral firing module are included. The pulse input interface is configured to receive a pulse input signal and input the pulse input signal to the sparse pulse detection module. The weight and neuron parameter input interface is configured to receive a compressed weight value and input the compressed weight value to the compressed weight calculation module. The sparse pulse detection module is configured to extract an effective pulse index from the pulse input signal, where the effective pulse index is used to represent the position of a non-zero value in the pulse input signal. The compressed weight calculation module is configured to decompress the compressed weight value to obtain an effective weight matrix according to the effective pulse index, calculate the weighted sum of the effective weight matrix and the pulse input signal to obtain a membrane potential increment of each neuron, and update the membrane potential accumulation corresponding to each neuron by using the membrane potential increment of each neuron. The leakage integral firing module is configured to determine the size relationship between the updated membrane potential accumulation and a preset threshold, and determine the output pulse result corresponding to each neuron according to the size relationship.

2. The Ping-Pong architecture based sparse spiking neural network accelerator of claim 1, wherein, The pulse cache module group includes a first pulse cache module and a second pulse cache module. The weight cache module group includes a first weight cache module and a second weight cache module.

3. The Ping-Pong architecture based sparse spike neural network accelerator of claim 2, wherein, ​ The weight cache module group is used to control the read-write state of the first weight cache module and the second weight cache module in each cache cycle in a ping-pong switching manner, so that one of the weight cache modules is in the read state and the other is in the write state in each cache cycle.

4. The Ping-Pong architecture based sparse spike neural network accelerator of claim 3, wherein, The neuron parameter cache module group includes a first neuron parameter cache module and a second neuron parameter cache module. The neuron parameter cache module group is used to control the read-write state of the first neuron parameter cache module and the second neuron parameter cache module in each cache cycle in a ping-pong switching manner, so that one of the neuron parameter cache modules is in the read state and the other is in the write state in each cache cycle.

5. The ping-pong architecture based sparse spike neural network accelerator of claim 1, wherein, The compressed weight calculation module includes a row offset operation unit, a column index weight operation unit, a row offset storage module, a column index Delta encoding storage module, a non-zero weight storage module, a weight distributor, and a processing unit array. The row offset operation unit is used to read the row offset of a current row and the row offset of a next row adjacent to the current row from the row offset storage module. The column index weight operation unit is used to parse the row offset of the current row and the row offset of the next row adjacent to the current row to obtain the start address and the end address of the column index Delta encoding storage module and the non-zero weight storage module; and according to the start address and the end address, column index Delta encodings and non-zero weights are obtained from the column index Delta encoding storage module and the non-zero weight storage module, respectively. The weight distributor includes an adder chain composed of a preset number of adders, and the adder chain is used to take the non-zero weights and the column index Delta encodings as inputs of the processing unit array. The processing unit array includes a preset number of processing units, each of which uses an adder to perform an operation and updates the membrane potential accumulation amount according to the operation result of the adder.

6. The Ping-Pong architecture based sparse spike neural network accelerator of claim 5, wherein, The column index weight operation unit is further used to end the processing of the current row when the start address exceeds the end address.

7. The ping-pong architecture based sparse spike neural network accelerator of claim 5, wherein, Each processing unit further includes a weight mask generation module, a multiplexer, and an adder. The weight mask generation module is used to generate a weight mask according to a weight distribution state. The processing unit is further used to take the weight mask and the non-zero weights as inputs of the multiplexer, so that the multiplexer filters to obtain effective non-zero values according to the weight mask, and takes the effective non-zero values and the membrane potential accumulation amount as inputs of the adder to obtain the updated membrane potential accumulation amount output by the adder.

8. The Ping-Pong architecture based sparse spike neural network accelerator of claim 1, wherein, The leaky integration delivery module is further used to add the updated membrane potential accumulation amount and a preset leak value to obtain a leak integral value; if the leak integral value is greater than the preset threshold value, it is determined that the output pulse result is a pulse signal; and if the leak integral value is less than or equal to the preset threshold value, it is determined that the pulse result is no pulse.

9. The ping-pong architecture based sparse spike neural network accelerator of claim 1, wherein, The pulse output interface is further included. The leakage integral delivery module is further configured to send the updated membrane potential accumulation amount to the pulse output interface and reset the membrane potential accumulation amount if it is determined that the updated membrane potential accumulation amount is greater than the preset threshold.

Images