High-parallel pulse convolutional neural network hardware accelerator

Abstract

The invention belongs to the field of integrated circuits, particularly relates to a high-parallel pulse convolutional neural network hardware accelerator, and aims to solve the problem of low calculation parallelism of a hardware accelerator. The device comprises a calculation module which is used for carrying out parallel calculation on pulse data and membrane voltage data of a plurality of time dimensions of a pulse convolutional neural network, and carrying out coding according to the pulse data and the membrane voltage data to obtain output feature map data corresponding to different time dimensions. According to the method, input feature map data of different time dimensions of the pulse convolutional neural network are received through different calculation engines, the calculation engines have dependency relationships of film voltage data of different time dimensions, a calculation module with a higher degree of parallelism is established, parallel calculation of feature map data of multiple time dimensions is supported, and the calculation efficiency is improved. And the operation speed of the hardware accelerator is improved.

Description

Technical Field

[0001] The invention belongs to the field of integrated circuits, and in particular relates to a high-parallel pulse convolutional neural network hardware accelerator. Background Art

[0002] A hardware accelerator is a hardware processor specifically used for training or reasoning AI models. The three main focuses of AI development are data, algorithms, and computing power. With the compounding and improvement of algorithms, data has exploded, the size of models has increased, and the requirements for computing power have become increasingly higher. Relying solely on CPUs and GPUs to support new neural networks requires a large number of CPUs or GPUs, which consumes a lot of energy. For example, Google's AlphaGo requires thousands of CPUs and hundreds of GPUs for each game, which incurs a lot of equipment costs and energy costs. Therefore, it is necessary to design special hardware processors for AI models to replace some CPUs and GPUs to improve energy efficiency and better meet computing power requirements.

[0003] The spiking convolutional neural network combines the neuromorphism of the spiking neural network and the high feature extraction capability of the convolutional neural network. However, the data of the spiking convolutional neural network increases the time dimension, which causes the hardware accelerator to require more computing power to process the data and more storage resources to record the intermediate results, increasing the inference delay and storage requirements of the hardware accelerator. Many institutions at home and abroad have conducted relevant research on the spiking convolutional neural network hardware accelerator. According to the way the data is processed, the hardware accelerator can be divided into event-driven and frame-driven hardware accelerators. IBM's TrueNorth chip and Zhejiang University's "Darwin" chip are based on event-driven accelerator architectures. The pulse data is encoded with address events and then serially input into the accelerator for calculation. Each computing unit in the hardware architecture corresponds to a neuron in the neural network. The FireFly accelerator of the Chinese Academy of Sciences and the SIES accelerator of Nanjing University are based on frame-driven hardware accelerators. The single-bit pulse data in a single frame image is input into the accelerator in parallel for calculation. The computing units in the accelerator can be reused to correspond to multiple neurons in the neural network. Event-driven hardware accelerators do not support neuron reuse. When implementing large-scale neural networks, a large number of computing units are required, resulting in high demand for accelerator hardware resources. Currently, frame-driven hardware accelerator computing units support reuse, but the computing parallelism is low and does not support parallel computing in the time dimension. Parallel computing can be divided into temporal parallelism and spatial parallelism. Parallel computing in the time dimension refers to the processing of multiple computing tasks simultaneously on the time axis, and parallel computing in the spatial dimension refers to the use of multiple processors to perform calculations concurrently. Summary of the invention

[0004] In order to solve the above-mentioned problems in the prior art, that is, to solve the problem of low computational parallelism of pulse convolutional neural network hardware accelerators, the present invention provides a high-parallel pulse convolutional neural network hardware accelerator, including a computing module for parallel computing pulse data and membrane voltage data of multiple time dimensions of a pulse convolutional neural network, encoding output feature map data corresponding to different time dimensions according to the pulse data and membrane voltage data, the computing module includes at least two computing engines, different computing engines receive input feature map data of different time dimensions of the pulse convolutional neural network, calculate and output pulse data and membrane voltage data of the corresponding time dimensions, wherein, among the computing engines corresponding to two adjacent time dimensions, the computing engine corresponding to the latter time dimension determines the pulse data and membrane voltage data of the latter time dimension according to the membrane voltage data output by the computing engine corresponding to the former time dimension and the input feature map data of the latter time dimension.

[0005] Furthermore, when different computing engines perform parallel calculations, each computing engine shares the weight data of the pulse convolutional neural network.

[0006] Furthermore, the computing engine includes a convolution computing structure, a membrane voltage computing structure, a pulse generating structure and a pooling structure. The convolution computing structure is used to perform pulse convolution operations on the input feature map data and weight data of the pulse convolution neural network. The membrane voltage computing structure is used to calculate and output the membrane voltage of the neurons of the pulse convolution neural network. The pulse generating structure is used to determine the pulse data and membrane voltage data output by the computing engine based on the comparison between the membrane voltage and the neuron threshold. The pooling structure is used to perform pooling operations based on the pulse data.

[0007] Furthermore, the convolution calculation structure includes multiple multiplication units, an addition tree and a convolution calculation structure storage unit. The multiplication and addition units are used to perform parallel operations according to the input feature map data corresponding to the multiplication and addition units one by one. The addition tree is used to accumulate the operation output data of each multiplication and addition unit. The convolution calculation structure storage unit is used to store the accumulated data of the addition tree. The convolution calculation structure adds the accumulated data of the addition tree and the data in the convolution calculation structure storage unit to output the convolution data.

[0008] Furthermore, the membrane voltage calculation structure is obtained by MP = MP_past + conv_result - V leak , determine the membrane voltage data of the pulse convolutional neural network in the later time dimension, MP represents the membrane voltage data in the later time dimension, MP_past represents the membrane voltage data in the previous time dimension, conv_result represents the convolution data of the pulse convolutional neural network, V leak Represents the leakage value of the spiking convolutional neural network neuron in each time dimension.

[0009] Furthermore, the pulse generation structure is used to connect MP and the neuron threshold V th In contrast, when MP is not less than V th When the pulse data S output by the computing engine is determined to be 1, when MP is less than V th , it is determined that the pulse data S output by the calculation engine is 0.

[0010] Further, when the pulse data S output by the calculation engine is 1, the membrane voltage data output by the calculation engine is determined to be a mold voltage reset value, and when the pulse data S output by the calculation engine is not 1, the membrane voltage data output by the calculation engine is determined to be MP.

[0011] Furthermore, the high-parallel pulse convolutional neural network hardware accelerator also includes an on-chip storage module, an off-chip storage module and a controller, wherein the on-chip storage module is used to store the weight data, input feature map data and output feature map data required by the computing module, and the off-chip storage module is used to store the weight data, input feature map data and output feature map data of the pulse convolutional neural network; the controller is used to collect the states of the computing module, the on-chip storage module and the off-chip storage module, and control the computing module, the on-chip storage module and the off-chip storage module.

[0012] Furthermore, after the on-chip storage module receives the control signal and instruction information from the controller, the data of the corresponding address is output to the off-chip storage module according to the address contained in the instruction information. After the off-chip storage module receives the control signal and instruction information from the controller, the data of the corresponding address is output to the on-chip storage module according to the address contained in the instruction information.

[0013] Furthermore, the on-chip storage module includes a row cache structure, which is used to cache historical input data and output multiple rows of data in parallel when input data is input in a pipelined single row at a single port.

[0014] Beneficial effects of the present invention:

[0015] (1) The present invention receives input feature graph data of different time dimensions of the pulse convolutional neural network through different computing engines, and the computing engines have dependencies on membrane voltage data of different time dimensions, thereby establishing a computing module with a higher degree of parallelism, supporting parallel calculation of feature graph data of multiple time dimensions, and improving the computing speed of the hardware accelerator.

[0016] (2) The hardware accelerator provided by the present invention can improve the reasoning efficiency of the pulse convolutional neural network, reduce the computing power and energy consumption cost of applying the pulse convolutional neural network, and facilitate the application of the pulse convolutional neural network.

[0017] (3) The hardware accelerator provided by the present invention processes data based on the way the biological brain processes information, and can be implemented as a new type of brain-like chip to better perform large-scale network computing with high speed and low power consumption. Compared with hardware accelerators based on traditional neural networks, it can improve energy efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0019] Figure 1 is a schematic diagram of a computing module provided by an embodiment of the present invention;

[0020] Figure 2 Schematic diagram of a pulse convolution operation process provided by an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of a computing module provided by an embodiment of the present invention;

[0022] Figure 4 It is a schematic diagram of the convolution calculation structure provided by an embodiment of the present invention;

[0023] Figure 5 is a schematic diagram of a membrane voltage calculation structure provided by an embodiment of the present invention;

[0024] Figure 6 It is a schematic diagram of a high-parallel pulse convolutional neural network hardware accelerator architecture provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0026] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] Traditional neural networks in the field of artificial intelligence (convolutional neural networks CNN, recurrent neural networks RNN, generative adversarial networks GAN, autoencoders, etc.) use high-precision floating-point numbers for calculations, while human brain intelligence does not use floating-point numbers for calculations. In the human sensor system and brain, information is transmitted and processed in the form of action voltage or electric pulses. In order to make neural networks closer to the human brain, a spike convolutional neural network is designed, which imitates the way biological brains process information to process data: when a neuron is stimulated, the membrane potential rises rapidly. Once the membrane potential reaches a certain threshold, an action potential is generated. This action potential propagates along the axon to the synapse, and the signal is transmitted to the next neuron by releasing neurotransmitters at the synapse. Correspondingly, in the hardware accelerator, each computing unit in the hardware architecture corresponds to a neuron in the neural network. The current hardware accelerators for spike convolutional neural networks have low computational parallelism and do not support parallel computing in the time dimension, which affects their performance.

[0028] In view of the low computational parallelism of the current hardware accelerator, which does not support parallel computation of the time dimension, an embodiment of the present invention provides a high-parallel pulse convolutional neural network hardware accelerator, including a computing module, which is used to parallelly compute the pulse data and membrane voltage data of multiple time dimensions of the pulse convolutional neural network, and encode the output feature map data corresponding to different time dimensions according to the pulse data and membrane voltage data. Specifically, the computing module includes at least two computing engines, and different computing engines receive the input feature map data of different time dimensions of the pulse convolutional neural network, calculate and output the pulse data and membrane voltage data of the corresponding time dimension, wherein, among the computing engines corresponding to the two adjacent time dimensions, the computing engine corresponding to the latter time dimension determines the pulse data and membrane voltage data of the latter time dimension according to the membrane voltage data output by the computing engine corresponding to the previous time dimension and the input feature map data of the latter time dimension. That is, the computing module is composed of T computing engines. Each computing engine is used to calculate the input feature map data of a single time dimension, and the T computing engines realize the parallel operation of the input feature map data of T time dimensions, wherein T≥2. Since the parallel computation of feature map data of multiple time dimensions is supported, it has a higher computational parallelism to improve the computing speed of the hardware circuit.

[0029] In the above-mentioned computing module, each computing engine corresponds to a neuron in the pulse convolutional neural network. By serially inputting the membrane voltage data output by multiple computing engines in the computing module, the calculation of the membrane voltage of neurons in multiple time dimensions is completed, and the dependency relationship of the membrane voltage data in different time dimensions is established. Therefore, the high-parallelism pulse convolutional neural network hardware accelerator provided by the embodiment of the present invention can better support the pulse convolutional neural network and reduce hardware energy consumption.

[0030] There is a difference between the data processing process of hardware accelerators based on traditional neural networks and the way biological brains process information. Since traditional neural networks use multi-bit width data and weight values ​​for multiplication and addition operations, hardware implementation relies on a large number of multiplication and addition units, resulting in high hardware resource requirements and high energy consumption. When biological brains process information, the learning process relies on the synaptic plasticity generated after the postsynaptic cells are stimulated.

[0031] The hardware accelerator provided by the present invention performs data processing based on the way the biological brain processes information, and can be realized as a new type of brain-like chip, which can better perform large-scale network computing with high speed and low power consumption. Compared with hardware accelerators based on traditional neural networks, it can improve energy efficiency.

[0032] The computing module supports parallel computing in four dimensions, including weight dimension parallel computing, input feature map parallel computing, output feature map parallel computing, and time dimension parallel computing. Figure 1 and Figure 2 Further explanation.

[0033] exist Figure 1 In the illustrated embodiment, the computing module has 8 3×3 weights and 8 input feature maps in 4 time dimensions participating in the calculation in parallel per clock cycle, and can output 8 output feature map data. The computing module includes four computing engines, namely computing engines 01 to 04, computing engines 01 to 04 correspond to time dimensions 0 to 3, respectively, and each computing engine is used to calculate the input feature map data of a single time dimension, that is, each computing engine receives the input feature map data of the same time dimension. Specifically, computing engine 01 receives input feature map data Ifm00 to Ifm07, and input feature map data Ifm00 to Ifm07 all belong to time dimension 0; computing engine 02 receives input feature map data Ifm10 to Ifm17, and input feature map data Ifm10 to Ifm17 all belong to time dimension 1; computing engine 03 receives input feature map data Ifm20 to Ifm27, and input feature map data Ifm20 to Ifm27 all belong to time dimension 2; computing engine 04 receives input feature map data Ifm30 to Ifm37, and input feature map data Ifm30 to Ifm37 all belong to time dimension 3.

[0034] The four computing engines implement parallel operations on the input feature map data of four time dimensions and output pulse data of four time dimensions. For example, computing engine 01 outputs membrane voltage data MP1 to computing engine 02, computing engine 02 outputs membrane voltage data MP2 to computing engine 03, computing engine 03 outputs membrane voltage data MP3 to computing engine 04, and the four computing engines output pulse data respectively. Figure 1In the illustrated embodiment, a total of 32 input feature maps and 8 weights are involved in the calculation. The 32 input feature maps are divided into four groups according to the time dimension, and data connection relationships are established with four computing engines respectively.

[0035] Furthermore, in some embodiments, when different computing engines perform parallel computing, each computing engine shares the weight data of the pulse convolutional neural network. Figure 1 In the illustrated embodiment, the weight data of the pulse convolutional neural network includes weights k00 to k07. The eight weights are shared by four computing engines to achieve weight reuse, reducing the number of weight accesses. Through this setting, the weight data can be reused during the parallel calculation of multi-time dimension data, which improves the reusability of the weight data, reduces the number of data movements between off-chip storage and on-chip storage of the weight data, and can improve the energy efficiency of the hardware.

[0036] The pulse convolution operation process data is as follows Figure 2 As shown in the figure, the input feature map data and the weight data are pulsed convolutionally operated to obtain the output feature map. There are t×n input feature maps in total, and the size of each input feature map is related to the pulse convolutional neural network model, where t represents the time dimension of the input feature map, and the input channel dimension of the input feature map is n. There are m×n weights in total, and the size of each weight is 3×3, where m represents the channel dimension of the output feature map. There are t×m output feature maps, and the size of each output feature map is related to the pulse convolutional neural network model.

[0037] Furthermore, in some embodiments, Figure 3 As shown, the computing engine includes a convolution computing structure, a membrane voltage computing structure, a pulse generating structure and a pooling structure. The convolution computing structure is used to perform a pulse convolution operation on the input feature map data and weight data of the pulse convolution neural network. The membrane voltage computing structure is used to calculate and output the membrane voltage of the neurons of the pulse convolution neural network. The pulse generating structure is used to determine the pulse data and membrane voltage data output by the computing engine according to the comparison between the membrane voltage and the neuron threshold. The pooling structure is used to perform a pooling operation according to the pulse data.

[0038] Further, in some embodiments, the convolution calculation structure includes a plurality of multiplication units, an addition tree, and a convolution calculation structure storage unit, the multiplication and addition unit is used to perform parallel operations according to the input feature map data corresponding to the multiplication and addition unit one by one, the addition tree is used to accumulate the operation output data of each multiplication and addition unit, and the convolution calculation structure storage unit is used to store the accumulated data of the addition tree, and the convolution calculation structure adds the accumulated data of the addition tree and the data in the convolution calculation structure storage unit to output the convolution data. Convolution operation structure, supports N (h k × k ) Input feature map data and N (h k ×k ) The parallel operation of weight data, the N operation results generated are accumulated through the addition tree, and the accumulated results are added to the data in the convolution calculation structure storage unit and then output. Where N ≥ 2, h k and w k The values ​​of are equal to the height and width of the convolution kernel. Figure 4 As shown in the figure, taking the convolution operation structure of the first computing engine as an example, 8 (3×3) input feature map data and 8 (3×3) weight data are respectively calculated in parallel in 8 multiplication and addition calculation unit arrays, and the 8 operation results generated are accumulated through the addition tree, and the accumulated results are added to the data in the convolution calculation structure storage unit and then output.

[0039] Further, in some embodiments, the membrane voltage calculation structure is as follows: MP = MP_past + conv_result - V leak , determine the membrane voltage data of the pulse convolutional neural network in the later time dimension, MP represents the membrane voltage data in the later time dimension, MP_past represents the membrane voltage data in the previous time dimension, conv_result represents the convolution data of the pulse convolutional neural network, V leak Represents the leakage value of the spike convolutional neural network neuron in each time dimension, V leak is a constant. Figure 1 The embodiment shown is used to illustrate that the calculation engine 01 corresponds to the first time dimension of the multiple time dimensions, the input data MP_past of the membrane voltage calculation structure in the calculation engine 01 is equal to 0, and the calculation engine 01 outputs the membrane voltage data MP1, the input data MP_past of the membrane voltage calculation structure in the calculation engine 02 is equal to MP1, and the calculation engine 02 outputs the membrane voltage data MP2, the input data MP_past of the membrane voltage calculation structure in the calculation engine 03 is equal to MP2, and the membrane voltage data MP3 is output, and the input data MP_past of the membrane voltage calculation structure in the calculation engine 04 is equal to MP3, and the calculation engine 04 outputs the membrane voltage data MP4. The process of the calculation engine 01 outputting the membrane voltage data MP1 is as follows: Figure 5 As shown, the convolution data conv_result of the pulse convolutional neural network is added to the input data MP_past of the membrane voltage calculation structure in the calculation engine 01, and then the leakage value is subtracted to obtain the membrane voltage data MP1 output by the calculation engine 01.

[0040] Further, in some embodiments, the pulse generation structure is used to connect MP and the neuron threshold V th In contrast, when MP is not less than V th When the pulse data S output by the calculation engine is determined to be 1, when MP is less than V th When , it is determined that the pulse data S output by the calculation engine is 0.

[0041] Further, in some embodiments, when the pulse data S output by the calculation engine is 1, the membrane voltage data output by the calculation engine is determined to be a mold voltage reset value; when the pulse data S output by the calculation engine is not 1, the membrane voltage data output by the calculation engine is determined to be MP.

[0042] The above process of determining the pulse data S and the membrane voltage data as MP can be expressed as a conditional judgment expression:

[0043] S=((MP≥V th )? 1:0); it means that if MP≥V th True, returns 1, S equals 1, if MP ≥ V th If it is false, it returns 0 and S is equal to 0.

[0044] MP'=((S==1)?V reset :MP), which means that if S is equal to 1, it is true, then V is returned reset , MP' is equal to V reset , if S is equal to 1 is false, then return MP, MP' is equal to MP.

[0045] V reset Represents the reset value of the module voltage. MP' represents the membrane voltage data output by the calculation engine after being determined by the pulse generation structure. The pulse generation structure receives the data MP of the membrane voltage calculation structure and outputs the pulse data S and the membrane voltage data MP' of the current time dimension. The data MP output by the membrane voltage calculation structure is compared with the neuron threshold V th Comparison is performed to generate corresponding control signals and pulse data, wherein the control signal is used to control the membrane voltage data MP' of the current time dimension to be the data MP output by the membrane voltage calculation structure, or the membrane voltage data MP' of the current time dimension to be the membrane voltage reset value.

[0046] Furthermore, in some embodiments, Figure 6As shown, the high parallel pulse convolutional neural network hardware accelerator also includes an on-chip storage module, an off-chip storage module and a controller. The on-chip storage module is used to store the weight data, input feature map data and output feature map data required by the calculation module. The off-chip storage module is used to store the weight data, input feature map data and output feature map data of the pulse convolutional neural network. The off-chip storage module is a synchronous dynamic random access memory SDRAM. The controller is used to collect the states of the calculation module, the on-chip storage module and the off-chip storage module, and control the calculation module, the on-chip storage module and the off-chip storage module. The controller is a central processing unit CPU or a control state machine. Specifically, the controller realizes the state monitoring and instruction control of the calculation module, the on-chip storage module and the off-chip storage module through bus signals. The bus signal can be divided into three instructions: LOAD, SAVE and CALC. The LOAD instruction is used to move the data on the off-chip storage module to the on-chip storage, the SAVE instruction is used to keep the data on the on-chip storage module to the off-chip storage module, and the CALC instruction is used to control the calculation and interruption of the calculation module. The bus signal is a 128-bit AXI (Advanced eXtensibleInterface) bus, an AMBA (Advanced Microcontroller Bus Architecture) bus, an APB (Advanced Peripheral Bus) bus, or an AHB (Advanced High Performance Bus) bus.

[0047] Furthermore, in some embodiments, after the on-chip storage module receives the control signal and instruction information from the controller, the on-chip storage module outputs the data of the corresponding address to the off-chip storage module according to the address contained in the instruction information. After the off-chip storage module receives the control signal and instruction information from the controller, the off-chip storage module outputs the data of the corresponding address to the on-chip storage module according to the address contained in the instruction information.

[0048] Furthermore, in some embodiments, the on-chip storage module includes a row cache structure and a data storage unit. The row cache structure is used to cache historical input data and output multiple rows of data in parallel when the input data is input in a pipelined single row at a single port. The row cache structure can realize the parallel output of multiple rows of data and realize the parallel operation of data. The data storage unit is a DDR (Double Data Rate SDRAM) memory or a static random access memory SRAM (Static Random-Access Memory) or a distributed memory LUTRAM (Look-Up Table Random Access Memory) of an FPGA.

[0049] It should be noted that the high-parallel pulse convolutional neural network hardware accelerator provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiments can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps and are not regarded as improper limitations of the present invention.

[0050] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A highly parallel spike convolutional neural network hardware accelerator, characterized in that: It includes a calculation module for parallelly calculating the pulse data and membrane voltage data of multiple time dimensions of the pulse convolutional neural network, and encoding the output feature map data corresponding to different time dimensions according to the pulse data and membrane voltage data. The computing module includes at least two computing engines, different computing engines receive input feature map data of different time dimensions of the pulse convolutional neural network, calculate and output pulse data and membrane voltage data of the corresponding time dimensions, wherein, among the computing engines corresponding to two adjacent time dimensions, the computing engine corresponding to the latter time dimension determines the pulse data and membrane voltage data of the latter time dimension based on the membrane voltage data output by the computing engine corresponding to the former time dimension and the input feature map data of the latter time dimension.

2. The highly parallel pulse convolutional neural network hardware accelerator according to claim 1, characterized in that: When different computing engines perform parallel calculations, each computing engine shares the weight data of the pulse convolutional neural network.

3. The highly parallel pulse convolutional neural network hardware accelerator according to claim 1, characterized in that: The computing engine includes a convolution computing structure, a membrane voltage computing structure, a pulse generating structure and a pooling structure. The convolution computing structure is used to perform pulse convolution operations on the input feature map data and weight data of the pulse convolution neural network. The membrane voltage computing structure is used to calculate and output the membrane voltage of the neurons of the pulse convolution neural network. The pulse generating structure is used to determine the pulse data and membrane voltage data output by the computing engine based on the comparison between the membrane voltage and the neuron threshold. The pooling structure is used to perform pooling operations based on the pulse data.

4. The highly parallel pulse convolutional neural network hardware accelerator according to claim 3, characterized in that: The convolution calculation structure includes multiple product units, an addition tree and a convolution calculation structure storage unit. The multiplication and addition units are used to perform parallel operations according to the input feature map data corresponding to the multiplication and addition units one by one. The addition tree is used to accumulate the operation output data of each multiplication and addition unit. The convolution calculation structure storage unit is used to store the accumulated data of the addition tree. The convolution calculation structure adds the accumulated data of the addition tree and the data in the convolution calculation structure storage unit to output the convolution data.

5. The highly parallel pulse convolutional neural network hardware accelerator according to claim 3, characterized in that: Membrane voltage calculation structure, through MP = MP_past + conv_result-V leak , determine the membrane voltage data of the pulse convolutional neural network in the later time dimension, MP represents the membrane voltage data in the later time dimension, MP_past represents the membrane voltage data in the previous time dimension, conv_result represents the convolution data of the pulse convolutional neural network, V leak Represents the leakage value of the spiking convolutional neural network neuron in each time dimension.

6. The highly parallel pulse convolutional neural network hardware accelerator according to claim 5, characterized in that: The pulse generation structure is used to connect MP and the neuron threshold V th In contrast, when MP is not less than V th When the pulse data S output by the computing engine is determined to be 1, when MP is less than V th , it is determined that the pulse data S output by the calculation engine is 0.

7. The highly parallel pulse convolutional neural network hardware accelerator according to claim 6, characterized in that: When the pulse data S output by the calculation engine is 1, the membrane voltage data output by the calculation engine is determined to be a mold voltage reset value. When the pulse data S output by the calculation engine is not 1, the membrane voltage data output by the calculation engine is determined to be MP.

8. The highly parallel pulse convolutional neural network hardware accelerator according to claim 1, characterized in that: The high-parallel pulse convolutional neural network hardware accelerator also includes an on-chip storage module, an off-chip storage module and a controller. The on-chip storage module is used to store the weight data, input feature map data and output feature map data required by the computing module. The off-chip storage module is used to store the weight data, input feature map data and output feature map data of the pulse convolutional neural network. The controller is used to collect the states of the computing module, the on-chip storage module and the off-chip storage module, and control the computing module, the on-chip storage module and the off-chip storage module.

9. The highly parallel pulse convolutional neural network hardware accelerator according to claim 8, characterized in that: After the on-chip storage module receives the control signal and instruction information from the controller, it outputs the data of the corresponding address to the off-chip storage module according to the address contained in the instruction information. After the off-chip storage module receives the control signal and instruction information from the controller, it outputs the data of the corresponding address to the on-chip storage module according to the address contained in the instruction information.

10. The highly parallel pulse convolutional neural network hardware accelerator according to claim 8, characterized in that: The on-chip storage module includes a row cache structure, which is used to cache historical input data and output multiple rows of data in parallel when input data is input in a pipelined single row at a single port.

Images