A general-purpose neuromorphic computing accelerator for flexible convolution and methods of use thereof

Abstract

The application relates to a neuromorphic computing technology, in particular to a general neuromorphic computing accelerator for flexible convolution and a use method thereof, the accelerator comprising a control module, a register group module, a cache module, a preprocessing module, a processing module and an accumulation module, the cache module comprising a weight cache module and an excitation cache module; the application adopts a multi-channel parallel mode for calculation, supports movement of a convolution kernel in a 1D, 2D or 3D mode, further extracts different features, and can realize deployment and inference calculation of various networks.

Description

Technical Field

[0001] This invention relates to neuromorphic computing technology, specifically to a general-purpose neuromorphic computing accelerator with flexible convolution and its usage method. Background Technology

[0002] The application of neural networks in edge devices involves deploying deep learning models to embedded systems or edge devices for inference and decision-making locally, rather than relying on cloud servers. This application principle has several key aspects: Model compression and optimization: Edge devices typically have limited computing resources and memory, necessitating the compression and optimization of deep learning models for efficient operation. This involves techniques such as pruning, quantization, and model simplification to reduce model size and computational complexity. Hardware accelerator support: Some edge devices may integrate dedicated hardware accelerators, such as GPUs, TPUs, or FPGAs, to accelerate the computational process of neural network inference. Utilizing these hardware accelerators can improve the speed and efficiency of model inference. Lightweight model design: Lightweight neural network models specifically designed for edge computing are developed to ensure good performance even on resource-constrained devices. Energy efficiency and low latency: Edge devices typically require consideration of power consumption and latency; therefore, deploying neural networks requires minimizing energy consumption and ensuring low-latency real-time responses. Summary of the Invention

[0003] To address the limitation of existing spiking convolutional neural network architectures in flexibly implementing various convolutional computations, this invention proposes a general-purpose neuromorphic computation accelerator with flexible convolution, comprising a control module, a register group module, a cache module, a preprocessing module, a processing module, and an accumulation module. The cache module includes a weight cache module and an activation cache module, wherein:

[0004] The control module receives the FPGA's start control command, sends the current core status information to the FPGA, reads the configuration data of the control register group module, controls the clearing of the weight cache module and the excitation cache module, controls the start of the preprocessing module to read the cache module, and receives the jump neuron signal transmitted by the accumulation module.

[0005] The register group module receives 11 sets of 16-bit configuration parameters written by the FPGA and parses these 11 sets of 16-bit configuration parameters into 20 configuration parameters, which are used to configure the parameters of the control module and the processing module.

[0006] The weight caching module reads weight data from RAM by address incrementing.

[0007] The excitation buffer module is used to buffer pulse data;

[0008] The preprocessing module controls the reading of weight and pulse data from the weight buffer unit and pulse buffer unit by reading the configuration parameters in the register group unit, performs sparsification processing on the read weight and pulse data, and sends the effective data to the processing unit for calculation.

[0009] The processing module performs multiplication and addition calculations of the pulse and weights to obtain the pre-film potential;

[0010] The accumulation module updates the membrane potential of neurons and fires impulses.

[0011] Furthermore, the 11 sets of 16-bit configuration parameters are as follows:

[0012] The first set of parameters includes the computation core ID (4 bits), the router ID (4 bits), the reset identifier (2 bits), and the width of the input activation data (5 bits).

[0013] The second set of parameters includes the number of core convolution operations on 15-bit data;

[0014] The third set of parameters includes the depth of the 9-bit Weight Cash single-block SRAM and the depth of the 5-bit input activation data;

[0015] The fourth set of parameters includes the mode of the 2-bit data convolution kernel, the activation padding flag of the 2-bit data, the size of the 2-bit data convolution kernel, the stride of the 2-bit data convolution kernel, the storage mode of the 2-bit data weights, and the number of operations for a single 5-bit data convolution kernel.

[0016] The fifth set of parameters includes the depth of the convolution kernel for 5-bit data, the number of channels activated by the input impulse for 5-bit data, and the number of convolution kernels for 5-bit data.

[0017] The sixth and seventh groups of parameters represent the range of the threshold parameters, which is ±32767.

[0018] The eighth and ninth sets of parameters are the reset voltages, i.e., the reset values ​​after the neuron pulses are fired;

[0019] The tenth and eleventh groups of parameters are leakage voltages. When the value is 0, there is no leakage voltage; when the value is greater than 0, the neural network leaks the corresponding voltage value.

[0020] Furthermore, the 2-bit reset flag indicates a direct reset to the reset voltage when it is 00, a linear reset when it is 01, and no reset when it is 10.

[0021] Furthermore, for 2-bit data, the convolution kernel mode is 00 to indicate a one-dimensional convolution kernel, 01 to indicate a two-dimensional convolution kernel, and 10 to indicate a three-dimensional convolution kernel.

[0022] The active padding flag for 2-bit data is 00 for no padding, 01 for 1 bit padding at the edge, and 10 for 2 bits padding at the edge.

[0023] For 2-bit data, a kernel size of 00 indicates a kernel of 2, 01 indicates a kernel of 3, 10 indicates a kernel of 4, and 11 indicates a kernel of 5.

[0024] For 2-bit data, a kernel movement stride of 00 indicates a stride of 1, 01 indicates a stride of 2, 10 indicates a stride of 3, and 11 indicates a stride of 4.

[0025] The storage mode for the weight of 2-bit data is 00, which means using 2-bit weight; 01 means using 4-bit weight; and 10 means using 8-bit weight.

[0026] Furthermore, the caching module includes 32 RAM memory blocks. After reading 32 parallel weights and activations, the 32 parallel pulse activation data and weight data are processed in four pairs in parallel by the preprocessing module to obtain sparse pulse-weight pairs. The preprocessing module sends the non-zero pulse pairs from these four sets of pulse-weight pairs to the processing unit in a serial manner. After receiving the valid pulse-weight pairs, the processing unit performs a multiplication and accumulation calculation of the pulses and weights. After completing the accumulation calculation of a single convolution kernel, the accumulation result, i.e., the pre-membrane potential of the neuron, is sent to the accumulation module. This invention uses a multi-channel parallel approach for computation, supporting the movement of convolution kernels in 1D, 2D, and 3D, thereby extracting different features and enabling the deployment and inference computation of various networks.

[0027] This invention employs a scalable convolution kernel size scheme, supporting configurations of convolution kernel size from 2×2 to 5×5, and supporting convolution kernel movement with strides from 1 to 4.

[0028] This invention adopts a weight configurable scheme, supporting three types of weight precision: 2 bits, 4 bits, and 8 bits, which can realize network deployment with different weight precision.

[0029] This invention employs a 4-way parallel approach for sparsification processing and reduces the computational load of the PE unit through zero-jump pulse operations. Attached Figure Description

[0030] Figure 1 This is a diagram of the convolutional computation architecture in existing technologies;

[0031] Figure 2 This is a diagram of a general-purpose neuromorphic computing accelerator architecture with flexible convolution according to the present invention.

[0032] Figure 3 This is a schematic diagram of the computation mode of a general-purpose neuromorphic computing accelerator with flexible convolution according to the present invention.

[0033] Figure 4 This is a schematic diagram of the data flow of a general-purpose neuromorphic computing accelerator with flexible convolution according to the present invention;

[0034] Figure 5 This is a flowchart of the computation process of a general-purpose neuromorphic computation accelerator with flexible convolution according to the present invention.

[0035] Figure 6 This is a schematic diagram illustrating the register parameters of a general-purpose neuromorphic computing accelerator with flexible convolution in this invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Existing general convolutional computation architectures such as Figure 1 This architecture cannot flexibly implement various convolutional computations, does not achieve a good general-purpose convolutional architecture design, and cannot effectively map spiking convolutional neural networks with different convolutional sizes. Therefore, this invention proposes a general-purpose neuromorphic computation accelerator with flexible convolution, such as... Figure 2 It includes a control module, a register group module, a cache module, a preprocessing module, a processing module, and an accumulation module. The cache module includes a weight cache module and an activation cache module, wherein:

[0038] The control module receives the FPGA's start control command, sends the current core status information to the FPGA, reads the configuration data of the control register group module, controls the clearing of the weight cache module and the excitation cache module, controls the start of the preprocessing module to read the cache module, and receives the jump neuron signal transmitted by the accumulation module.

[0039] The register group module receives 11 sets of 16-bit configuration parameters written by the FPGA and parses these 11 sets of 16-bit configuration parameters into 20 configuration parameters, which are used to configure the parameters of the control module and the processing module.

[0040] The weight caching module reads weight data from RAM by address incrementing.

[0041] The excitation buffer module is used to buffer pulse data;

[0042] The preprocessing module controls the reading of weight and pulse data from the weight buffer unit and pulse buffer unit by reading the configuration parameters in the register group unit, performs sparsification processing on the read weight and pulse data, and sends the effective data to the processing unit for calculation.

[0043] The processing module performs multiplication and addition calculations of the pulse and weights to obtain the pre-film potential;

[0044] The accumulation module updates the membrane potential of neurons and fires impulses.

[0045] This invention presents a general-purpose neuromorphic computing accelerator with flexible convolution, employing a parallel read / write approach across multiple RAM chips. By storing input image data in multiple RAM chips, it directly performs multi-channel convolution calculations, then accumulates the multi-channel calculation results to ultimately achieve multi-channel image convolution. This scheme features a versatility and compact design, with modules that allow for configurable convolution parameters to meet diverse convolution requirements. The invention utilizes register reuse technology, pre-writing parameters into registers to reduce the overhead of large registers in the computing architecture and minimize unnecessary parameter computation. It employs a configurable weight scheme, supporting 2-bit, 4-bit, and 8-bit weight precisions, enabling network deployments with varying weight precisions.

[0046] In this embodiment, a general-purpose neuromorphic computing accelerator with flexible convolution includes a control module, a register group module, a weight cache module, an activation cache module, a preprocessing module, a processing module, and an accumulation module, wherein:

[0047] The control module is the control logic unit of the entire computing architecture. It receives the FPGA's start control command, sends the current core state information to the FPGA, reads the configuration data of the control register group module, controls the clearing of the weight cache module and the activation cache module, controls the start of the sparsification module to read activation and weight data, and receives the jump neuron signal transmitted by the accumulation module.

[0048] The register group module is the parameter storage unit of this architecture. It receives 11 sets of 16-bit configuration parameters written by the FPGA and parses these 11 sets of 16-bit configuration parameters into 20 configuration parameters for use in the control logic of each module. The weight cache module is the on-chip cache unit for weight parameters during inference calculations. It reads weight data from RAM in a progressive address manner and can cache 6400 bits of weight data.

[0049] The excitation cache module is an on-chip cache unit in this architecture used to cache the input pulse excitation, and it can cache 32Kbits of pulse data;

[0050] The preprocessing module is the pulse data preprocessing unit of this architecture. It controls the reading of weight and pulse data from the weight buffer unit and pulse buffer unit by reading the configuration parameters in the register group unit. It performs sparsification processing on the read weight pulse data and sends the effective data to the processing unit for calculation, thereby reducing the computational load of the processing unit.

[0051] The processing module is the convolutional computation processing unit of this architecture. It is built using multiply-accumulators. The PE unit realizes the multiplication-accumulation calculation of pulses and weights to obtain the pre-film potential.

[0052] The accumulation module is the neuronal membrane potential accumulation unit in this architecture, which mainly performs neuronal membrane potential updates and pulse firing;

[0053] The accelerator of this invention performs convolution operations based on input pulse data, and can be connected to the computing core through an external routing module to form an on-chip network, ultimately realizing the hardware deployment and inference computation of the neural network.

[0054] This invention's accelerator supports multi-channel parallel computing. It stores excitation and weights in 32 RAM blocks, reads these 32 channels of excitation and weights in parallel, and then processes these 32 parallel pulse excitation and weight data in four parallel pairs via a sparsification module to obtain sparse pulse-weight pairs. These four pairs are then sent to the sparsification module for further sparsification. A preprocessing module serially sends the non-zero pulse pairs from these four sets of pulse-weight pairs to the processing unit. Upon receiving valid pulse-weight pairs, the processing unit performs a multiplication and accumulation calculation of the pulses and weights. After completing the accumulation calculation for a single convolution kernel, the accumulation result, i.e., the neuron's pre-membrane potential, is sent to the accumulation module. Upon receiving the neuron's pre-membrane potential, the accumulation module reads the corresponding neuron membrane potential from the external RAM, accumulates the pre-membrane potential with the read neuron membrane potential to obtain the latest neuron membrane potential, and then performs operations such as leakage, activation, reset, and pulse emission on the neuron membrane potential. The reset membrane potential is then rewritten into the external RAM, and the pulse data is simultaneously sent to the next computing core for computation via a routing module.

[0055] like Figure 4 The data flow after the operation of a general-purpose neuromorphic computing accelerator with flexible convolution according to the present invention includes the following steps:

[0056] 1. The external control module writes 11 sets of 16-bit configuration parameters to the register group and parses them into 19 configuration parameters for use in the control logic of each module. After receiving all 11 sets of configuration parameters, it sends a configuration completion signal to the control module for state transition.

[0057] 2: The external control module writes weight data and incentive data to the weight cache and incentive cache modules. After the weight data and incentive data are full, it sends a cache full signal to the control module.

[0058] 3: After receiving the idle signal from the controller module and writing the configuration parameters, pulse and weight data to the core, the external control module sends a start command to the controller module.

[0059] 4: After detecting the start command, configuration register full signal, and buffer full signal, the controller module starts the network to perform calculations, begins time step counting, output channel counting, and neuron count, and sends a start signal to the preprocessing module and an accumulation module.

[0060] 5: After receiving the start signal sent by the control module, the processing module reads a set of excitation and weight data from the excitation buffer and weight buffer, rearranges them for subsequent calculations, then extracts 4 sets of pulse weight pairs in sequence for sparsification, and sends the effective pulse weight pairs to the processing module in a serial manner for calculation.

[0061] 6: After receiving the enable signal sent by the preprocessing module, the processing module receives valid pulse weight pairs, accumulates the weights, and waits for the preprocessing module to send a convolution calculation end signal before sending the accumulated result, i.e., the pre-membrane potential, to the accumulation module for calculating the neuron membrane potential.

[0062] 7. After receiving the start signal from the control module, the accumulator reads the membrane potential data from the external RAM according to the membrane potential address provided by the preprocessing module. Then, after receiving the pre-membrane potential sent by the processing module, the accumulator adds the pre-membrane potential to the read membrane potential data to obtain the latest membrane potential data. Finally, it performs leakage and activation operations on this membrane potential to generate a pulse signal.

[0063] 8: The accumulation module sends pulse signals to the external routing module. After the routing module returns a response that it has received the pulse signals, it generates a neuron switching signal and sends it to the control module.

[0064] 9: After receiving the neuron switching signal, the control module increments the neuron count by one until the calculation of the current output channel is completed, then jumps to data stream 4. After that, the control module restarts the preprocessing module to perform calculations until all output channels are calculated, that is, the convolution calculation of one image data is completed. Then it checks whether the time step count has reached the set value. If not, it starts the calculation of the next image.

[0065] like Figure 5 The computational process of a general-purpose neuromorphic computing accelerator with flexible convolution according to the present invention includes the following steps:

[0066] Write configuration parameters, which means writing 11 sets of 16-bit configuration parameters to initialize the computing accelerator;

[0067] Write the pulse excitation and weight parameters, that is, write the weight data and excitation data into the weight cache module and the excitation cache module respectively;

[0068] Start the calculation and perform the calculation;

[0069] After completing one calculation, check if the time step count is complete. If it is complete, rewrite and start the next calculation. If the time step calculation is not complete, check if the output channel count is complete.

[0070] If the channel count has been completed, the time step calculation is incremented by 1 and the process returns to the previous step to re-evaluate whether the time step count has been completed.

[0071] If the channel counting is not completed, the pulse and weight data pairs are read, the four pulse and weight data pairs are sparsified, the PE module (i.e. the processing module in this invention) calculates the premode voltage, the accumulation module accumulates the neuron premode voltage and fires pulses, and it is determined whether the neuron has completed the calculation.

[0072] If the calculation is not completed, read the pulse and weight data pair and continue the calculation; otherwise, update the weight information, increment the channel count by 1, and return to determine whether the channel count is complete.

[0073] This embodiment illustrates the construction of an MNIST handwritten digit recognition CNN neural network with two convolutional layers. The first layer, the input layer, contains only a single-channel input of 32×32-bit image pulses. Six 5×5 convolutional kernels are then used for computation with a stride of 2, resulting in 14×14×6-bit input features for the second layer. Sixteen 5×5 convolutional kernels are used for computation with a stride of 2, ultimately yielding 5×5×16-bit feature data. This data can then be processed and classified using a fully connected computational core. The specific steps include:

[0074] 1. For the first layer, write 11 sets of configuration parameters to the register group. The main configuration parameters are: reset mode 2'd0 (direct reset mode), input stimulus width 5'd31 (width mapped to 32), convolution operation count 15'd1175 (convolution operation count ((32-5+1) / 2)×((32-5+1) / 2)×5=1176 times), weight depth 8'd149 (depth mapped to 5×5×6=150), input stimulus height 5'd31 (height mapped to 32), convolution kernel mode 2'd2 (3D convolution mode), padding number 0, convolution kernel size 2'd3 (mapped to a 5×5 convolution kernel), and convolution kernel stride 2'd1 (stride mapped to...). The parameters are: 2'd2 (weight precision mapped to 8 bits), 5'd24 (single convolution kernel operation count is 5×5=25), 5'd0 (single convolution kernel channel count is 1), 5'd0 (input channel count is 1), 5'd5 (number of convolution kernels mapped to 6), 0 threshold, 0 reset voltage, and 0 leakage voltage. The values ​​are represented in decimal notation, for example, 2'd0 represents 2 bits of data with a decimal value of 0, corresponding to binary 00, indicating a direct reset; another example is 5'd31, which represents 5 bits of data with a maximum decimal value of 31, including 0 to 31, i.e., an actual width of 32. After receiving 11 sets of configuration parameters, a configuration completion signal is sent to the control module for state transition.

[0075] 2: The external control module writes weight data and incentive data to the weight cache and incentive cache modules. After the weight data and incentive data are full, it sends a cache full signal to the control module.

[0076] 3: After receiving the idle signal from the controller module and writing the configuration parameters, pulse and weight data to the core, the external control module sends a start command to the controller module.

[0077] 4: After detecting the start command, configuration register full signal, and buffer full signal, the controller module starts the network to perform calculations, begins time step counting, output channel counting, and neuron count, and sends a start signal to the preprocessing module and an accumulation module.

[0078] 5: After receiving the start signal sent by the control module, the processing module reads a set of excitation and weight data from the excitation buffer and weight buffer, rearranges them for subsequent calculations, then extracts 4 sets of pulse weight pairs in sequence for sparsification, and sends the effective pulse weight pairs to the processing module in a serial manner for calculation.

[0079] 6: After receiving the enable signal sent by the preprocessing module, the processing module receives valid pulse weight pairs, accumulates the weights, and waits for the preprocessing module to send a convolution calculation end signal before sending the accumulated result, i.e., the pre-membrane potential, to the accumulation module for calculating the neuron membrane potential.

[0080] 7. After receiving the start signal from the control module, the accumulator reads the membrane potential data from the external RAM according to the membrane potential address provided by the preprocessing module. Then, after receiving the pre-membrane potential sent by the processing module, the accumulator adds the pre-membrane potential to the read membrane potential data to obtain the latest membrane potential data. Finally, it performs leakage and activation operations on this membrane potential to generate a pulse signal.

[0081] 8: The accumulation module sends pulse signals to the external routing module. After the routing module returns a response that it has received the pulse signals, it generates a neuron switching signal and sends it to the control module.

[0082] 9: After receiving the neuron switching signal, the control module increments the neuron count by one until the calculation of the current output channel is completed, then jumps to 4. After that, the control module restarts the preprocessing module to perform calculations until all output channels are calculated, that is, the convolution calculation of one image data is completed. Then it checks whether the time step count has reached the set value. If not, it starts the calculation of the next image.

[0083] 10: For the second layer, write 11 sets of configuration parameters to the register group. The main configurations are: reset mode 2'd0 direct reset mode, input stimulus width 5'd13, convolution operation count 15'd399, weight depth 8'd399, input stimulus height 5'd13, convolution kernel mode 2'd2, padding 0, convolution kernel size 2'd3, convolution kernel stride 2'd1, weight mode 2'd2, convolution kernel operation count 5'd24, convolution kernel depth 5'd5, input channels 5'd5, number of convolution kernels 5'd15, threshold 0, reset voltage 0, and leakage voltage 0. After receiving the 11 sets of configuration parameters, send a configuration completion signal to the control module for state transition.

[0084] 11: The external control module writes weight data and incentive data to the weight cache and incentive cache modules. After the weight data and incentive data are full, it sends a full cache signal to the control module.

[0085] 12: After receiving the idle signal from the controller module and writing the configuration parameters, pulse and weight data to the core, the external control module sends a start command to the controller module.

[0086] 13: After detecting the start command, configuration register full signal, and buffer full signal, the controller module starts the network to perform calculations, begins time step counting, output channel counting, and neuron count, and sends a start signal to the preprocessing module and an accumulation module.

[0087] 14: After receiving the start signal sent by the control module, the processing module reads a set of excitation and weight data from the excitation buffer and weight buffer, rearranges them for subsequent calculations, then extracts 4 sets of pulse weight pairs in sequence for sparsification, and sends the effective pulse weight pairs to the processing module in a serial manner for calculation.

[0088] 15: After receiving the enable signal sent by the preprocessing module, the processing module receives valid pulse weight pairs, accumulates the weights, and waits for the preprocessing module to send a convolution calculation end signal before sending the accumulated result, i.e., the pre-membrane potential, to the accumulation module for calculating the neuron membrane potential.

[0089] 16: After receiving the start signal from the control module, the accumulation module reads membrane potential data from the external RAM according to the membrane potential address provided by the preprocessing module. Then, after receiving the pre-membrane potential sent by the processing module, the accumulation module adds the pre-membrane potential to the read membrane potential data to obtain the latest membrane potential data. Afterwards, leakage and activation operations are performed on this membrane potential to generate a pulse signal.

[0090] 17: The accumulation module sends a pulse signal to the external routing module. After the routing module returns a response that it has received the pulse signal, it generates a neuron switching signal and sends it to the control module.

[0091] 18: After the control module receives the neuron switching signal, the neuron count is incremented by one until the calculation of the current output channel is completed. Then, it jumps to step 13 and controls the preprocessing module to start the calculation again until all output channels are calculated, that is, the convolution calculation of one image data is completed. Then, it is determined whether the time step count has reached the set value. If not, the calculation of the next image begins.

[0092] Figure 3The computation mode of the accelerator of this invention is as follows: the image input stimulus is stored in the stimulus buffer module, and the weight parameters are cached in the weight buffer module. The number of SRAMs in the weight buffer module is determined according to the depth of the convolution kernel (32 in this embodiment), and the number of SRAMs in the stimulus buffer module is determined according to the number of channels activated by the input pulse (32 in this embodiment). The preprocessing module starts to request pulse stimulus data of up to 32 channels in parallel from the stimulus buffer module according to the mode set by the convolution kernel (the pulse stimulus data is obtained in the order from left to right and from top to bottom), and requests weight data of 32 channels in parallel from the weight SRAM (the weight data is obtained in the order from top to bottom). Then, the obtained pulse stimulus data and weight data are ANDed to obtain pulse weight pairs, that is, the weight data corresponding to the effective pulse stimulus data is stored, and finally 32 pulse weight pairs are obtained. These pulse weight pairs are accumulated to obtain the pre-membrane potential. After the membrane potential of one convolution kernel is accumulated, it is sent to the accumulation module for neuron membrane potential leakage, reset, activation and other operations. In the overall architecture of this invention, the excitation cache module can store pulse excitation data of up to 32×32×32 size. Each of the 32 registers in the weight cache module stores a corresponding channel, and each channel stores a portion of the convolutional kernel information. Each convolutional kernel is 32×8×25 in size, meaning each register stores 1×8×25 of a convolutional kernel. For example, the first channel stores the first row of information for each convolutional kernel. The 1-bit data from the 32 channels of the excitation cache module is ANDed with the 8-bit data from the 32 channels of the weight cache module to obtain 8-bit pre-film potential data. In this embodiment, the weight storage mode is configured as 8-bit weight data, but it can also be configured as 2-bit weight, 4-bit weight, or 8-bit weight. The IT weights, through configuration of the convolution kernel mode and kernel size, can achieve 1D, 2D, or 3D convolution. Specifically, if the kernel mode is 1D and the kernel size is 2 (Kernel_mode value 00, Kernel_size value 00), a 2×1 dimensional convolution is achieved. If the kernel mode is 2D and the kernel size is 3 (Kernel_mode value 01, Kernel_size value 01), a 3×3 dimensional convolution is achieved. If the kernel mode is 3D and the kernel size is 4 (Kernel_mode value 10, Kernel_size value 10), a 4×4×channel-number-dimensional convolution is achieved. In this embodiment, the number of channels is set to 32 (i.e., the number of SRAMs storing activation data), but those skilled in the art can adjust it according to actual conditions.

[0093] Table 1. Parameters of the 11 16-bit registers

[0094]

[0095]

[0096] This embodiment uses 11 sets of 16-bit configuration parameters. For detailed specifications of the configuration parameters, please refer to... Figure 6 Table 1 and the figure record the configuration parameters, their storage locations, and descriptions of the parameters. The maximum values ​​of the reset voltage and leakage voltage can be represented by two 16-bit values, and the threshold parameter ranges from -32767 to 32767.

[0097] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A general-purpose neuromorphic computing accelerator with flexible convolution, characterized in that, It includes a control module, a register group module, a cache module, a preprocessing module, a processing module, and an accumulation module. The cache module includes a weight cache module and an activation cache module, wherein: The control module receives the FPGA's start control command, sends the current core status information to the FPGA, reads the configuration data of the control register group module, controls the clearing of the weight cache module and the excitation cache module, controls the start of the preprocessing module to read the cache module, and receives the jump neuron signal transmitted by the accumulation module. The register group module receives 11 sets of 16-bit configuration parameters written by the FPGA and parses these 11 sets of 16-bit configuration parameters into 20 configuration parameters, which are used to configure the parameters of the control module and the processing module. The 11 sets of 16-bit configuration parameters are as follows: The first set of parameters includes the computation core ID (4 bits), the router ID (4 bits), the reset identifier (2 bits), and the width of the input activation data (5 bits). The second set of parameters includes the number of core convolution operations on 15-bit data; The third set of parameters includes the depth of the 9-bit Weight Cash single-block SRAM and the depth of the 5-bit input activation data; The fourth set of parameters includes the mode of the 2-bit data convolution kernel, the activation padding flag of the 2-bit data, the size of the 2-bit data convolution kernel, the stride of the 2-bit data convolution kernel, the storage mode of the 2-bit data weights, and the number of operations for a single 5-bit data convolution kernel. The fifth set of parameters includes the depth of the convolution kernel for 5-bit data, the number of channels activated by the input impulse for 5-bit data, and the number of convolution kernels for 5-bit data; The sixth and seventh groups of parameters represent the range of the threshold parameters, which is ±32767. The eighth and ninth sets of parameters are the reset voltages, i.e., the reset values ​​after the neuron pulses are fired; The tenth and eleventh sets of parameters represent leakage voltage. When the value is 0, there is no leakage voltage; when the value is greater than 0, the neural network leaks the corresponding voltage value. The weight caching module reads weight data from RAM by address incrementing. The excitation buffer module is used to buffer pulse data; The preprocessing module controls the reading of weight and pulse data from the weight buffer unit and pulse buffer unit by reading the configuration parameters in the register group unit, performs sparsification processing on the read weight and pulse data, and sends the effective data to the processing unit for calculation. The processing module performs multiplication and addition calculations of the pulse and weights to obtain the pre-film potential; The accumulation module updates the membrane potential of neurons and fires impulses.

2. The general-purpose neuromorphic computing accelerator with flexible convolution as described in claim 1, characterized in that, The 2-bit reset flag indicates that the system will reset directly to the reset voltage when it is 00, perform a linear reset when it is 01, and not reset when it is 10.

3. The general-purpose neuromorphic computing accelerator with flexible convolution as described in claim 1, characterized in that, For 2-bit data, the convolution kernel mode is 00 to indicate a one-dimensional convolution kernel, 01 to indicate a two-dimensional convolution kernel, and 10 to indicate a three-dimensional convolution kernel. The active padding flag for 2 bits of data is 00 for no padding, 01 for 1 bit padding at the edge, and 10 for 2 bits padding at the edge. For 2-bit data, a kernel size of 00 indicates a kernel of 2, 01 indicates a kernel of 3, 10 indicates a kernel of 4, and 11 indicates a kernel of 5. For 2-bit data, a kernel stride of 00 indicates a stride of 1, 01 indicates a stride of 2, 10 indicates a stride of 3, and 11 indicates a stride of 4. The storage mode for the weight of 2-bit data is 00, which means using 2-bit weight; 01 means using 4-bit weight; and 10 means using 8-bit weight.

4. A general-purpose neuromorphic computing accelerator with flexible convolution as described in claim 1, characterized in that, The caching module includes 32 RAM memory blocks. After reading 32 parallel weights and activations, the 32 parallel pulse activation data and weight data are processed in four pairs in parallel by the preprocessing module to obtain sparse pulse weight pairs. The preprocessing module sends the non-zero pulse pairs in these four sets of pulse weight pairs to the processing unit in a serial manner. After receiving the valid pulse weight pairs, the processing unit performs a multiplication and accumulation calculation of the pulse and weight. After completing the accumulation calculation of a single convolution kernel, the accumulation result, i.e. the pre-membrane potential of the neuron, is sent to the accumulation module.

5. A method for using a general-purpose neuromorphic computing accelerator with flexible convolution, characterized in that, A general-purpose neuromorphic computing accelerator for flexible convolution as described in claim 1, in a multi-convolutional layer structure, configures parameters for each convolutional layer according to the computational task. The method of using the general-purpose neuromorphic computing accelerator for flexible convolution includes the following steps: Write 11 sets of 16-bit configuration parameters into the register bank module of a flexible convolutional general-purpose neuromorphic computing accelerator; The external control module writes weight data and incentive data to the weight cache and incentive cache modules. After the weight data and incentive data are full, it sends a full cache signal to the control module. After receiving the idle signal from the controller module and writing the configuration parameters, pulse and weight data to the core, the external control module sends a start command to the controller module. After detecting the start command, configuration register full signal, and buffer full signal, the controller module starts the network to perform calculations, begins time step counting, output channel counting, and neuron count, and sends a start signal to the preprocessing module and an accumulator module. After receiving the start signal from the control module, the processing module reads a set of excitation and weight data from the excitation buffer and weight buffer, rearranges them for subsequent calculations, then extracts four sets of pulse weight pairs in sequence for sparsification, and sends the effective pulse weight pairs to the processing module in a serial manner for calculation. After receiving the enable signal from the preprocessing module, the processing module receives valid pulse weight pairs, accumulates the weights, and waits for the preprocessing module to send a convolution calculation completion signal before sending the accumulated result, i.e., the pre-membrane potential, to the accumulation module for calculating the neuron membrane potential. After the accumulator receives the start signal from the control module, it reads the membrane potential data from the external RAM according to the membrane potential address provided by the preprocessing module. Then, after the accumulator receives the pre-membrane potential sent by the processing module, it accumulates the pre-membrane potential with the read membrane potential data to obtain the latest membrane potential data. After that, it performs leakage operation and activation operation on this membrane potential to generate a pulse signal. The accumulator module sends pulse signals to the external routing module. After the routing module returns a response that it has received the pulse signals, it generates a neuron switching signal and sends it to the control module. After receiving the neuron switching signal, the control module increments the neuron count by one until the calculation of the current output channel is completed.

Images