A convolutional neural network inference accelerator and acceleration method
By designing parallel computing unit and multiplication and accumulation unit, combining the sharding and ping-pong operation of SRAM storage module, the problems of slow calculation speed, high power consumption and poor configurability of existing convolutional neural network inference accelerators are solved, and high parallelism is achieved and low power consumption computing efficiency, suitable for a variety of convolutional neural network scenarios.
Patent Information
- Application Number
- CN202010348927.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-04-28
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-04-28
AI Technical Summary
The existing convolutional neural network inference accelerator has slow computing speed, high power consumption, poor configurability and insufficient fixed-point accuracy, and cannot fully utilize parallel computing and storage resources, resulting in insufficient parallelism and additional hardware resource consumption.
A convolutional neural network inference accelerator including the main control module, the address generation module, the SRAM storage module, the data input module, the calculation engine module and the result output module are designed. It adopts parallel computing unit and multiplication and accumulation unit, and supports 16bit/8bit fixed-point Data type and dynamic fixed-point position, high parallelism and configurability are achieved through parallel computing and activation function calculation units, and data sharding and ping-pong operations are performed through SRAM storage module to improve efficiency.
It realizes the performance advantages of high parallelism, reduces power consumption, supports the accuracy requirements of different scenarios, improves calculation speed and scalability, and reduces result errors.
Smart Images

Figure CN111401532B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of hardware implementation of artificial intelligence algorithms, and more specifically, to a convolutional neural network reasoning accelerator and an acceleration method. Background Art
[0002] Convolutional neural network is a deep feedforward artificial neural network and one of the representative algorithms of deep learning. It has been successfully applied to computer vision, natural language processing and other fields. In convolutional neural networks, the convolution layer can account for more than 80% of the total computational workload and computing time of the entire network. Therefore, acceleration of the convolution layer is the key to improving the performance of the entire CNN network. In the calculation of the convolution layer, there are several parameters: convolution kernel size (K), zero padding method (Pa), convolution step size (St), number of input image channels (Ch), number of convolution kernels (Nu), input image size (In), and output image size (Ou). According to the above parameters, the size of the output image is:
[0003]
[0004] According to the analysis of the convolution operator, convolution mainly involves multiplication and accumulation operations. Since the input data is mostly multi-channel two-dimensional data and the number of convolution kernels is multiple, the convolution in deep learning is a typical computationally intensive algorithm that requires strong computing power to support.
[0005] Chinese patent application, application number CN201810068051.6, published on June 19, 2018, discloses a convolutional neural network reasoning accelerator and method, the accelerator includes: an input image buffer module, including N buffers, for loading input image data, each buffer stores data corresponding to a row of the image; N*N operation units, connected to the input image buffer module, for performing convolution operations, the N*N operation units support the pulsation form of image data transmission between adjacent operation units, the operation unit connected to the buffer reads the image data from the buffer, and the remaining operation units read the image data from the adjacent operation units. The invention designs a bidirectional pulsation array for the data reusability brought by the convolutional neural network, improves the data loading efficiency, and thus accelerates the convolutional neural network. Its disadvantage is that the operation unit of the invention is a single calculation, and can only perform convolution of a batch of images each time, and cannot process multiple batches of input images at the same time, resulting in insufficient parallelism. At the same time, the floating-point operation of the operation unit will greatly increase the consumption of hardware resources, and increase additional area and time expenses. Summary of the invention
[0006] 1. Technical problems to be solved
[0007] In view of the problems of slow computing speed, high power consumption, poor configurability and fixed-point accuracy in the prior art, the present invention provides a convolutional neural network inference accelerator and acceleration method, which can improve the computing speed, reduce power consumption, improve the configurability of the device, and ensure the accuracy of the calculation results.
[0008] 2. Technical solution
[0009] The purpose of the present invention is achieved through the following technical solutions.
[0010] A convolutional neural network reasoning accelerator includes a main control module, an address generation module, an SRAM storage module, a data input module, a calculation engine module and a result output module, wherein:
[0011] The main control module receives the start signal and distributes data to the address generation module, the data input module, the calculation engine module and the result output module;
[0012] An address generation module receives data from the main control module and the result output module, and sends a control signal to the SRAM storage module;
[0013] The SRAM storage module receives the control signal of the address generation module and sends the storage data to the data input module;
[0014] A data input module receives data from the main control module and the SRAM storage module and sends the data to the calculation engine module;
[0015] The calculation engine module receives data from the main control module and the data input module, and sends the result data to the result output module;
[0016] The result output module receives the result data of the calculation engine module and sends the result data to the address generation module.
[0017] Furthermore, the computing engine module includes a plurality of parallel computing units, which in turn include a plurality of multiplication-accumulation units and activation function computing units. The multiplication-accumulation units send data to the activation function computing units through a result input bus.
[0018] Furthermore, the multiplication-accumulation unit includes a fixed-point multiplier and a fixed-point adder, and the fixed-point multiplier is connected to the fixed-point adder.
[0019] Furthermore, the activation function unit includes a lookup table memory, a fixed-point multiplier, a fixed-point adder and two registers, the lookup table memory and the first register receive data, the lookup table memory sends the data to the fixed-point multiplier and the second register, and the fixed-point multiplier and the second register send the data to the fixed-point adder.
[0020] Furthermore, the main control module also sends a zero-filling signal to the address generation module, the address generation module generates a zero-filling address and sends it to the result output module, and the result output module sends the result data and the zero-filling address to the SRAM storage module.
[0021] A convolutional neural network reasoning acceleration method, applied to the above-mentioned reasoning accelerator, comprises the following steps:
[0022] Step 1: The main control module receives the start signal and configuration information, sends the configuration information to the calculation engine module and the result output module, and sends a start signal to the data input module and the address generation module;
[0023] Step 2: After receiving the start signal, the address generation module generates a source data address, a weight address and a corresponding SRAM control signal and transmits them to the SRAM storage module. The SRAM storage module reads the source data and weight data according to the source data address, the weight address and the corresponding SRAM control signal and sends them to the data input module;
[0024] Step 3, the data input module generates an input data valid signal, a module shutdown signal and a fixed-point position signal according to the source data and weight data sent by the SRAM storage module, and the source data, weight data, input data valid signal, module shutdown signal and fixed-point position signal;
[0025] Step 4: After receiving the source data, weight data, input data valid signal, fixed-point position signal and module shutdown signal, the computing engine module starts the convolution inference calculation to obtain the result data, generates the result data valid signal and the calculation end signal, and sends the result data, the result data valid signal and the calculation end signal to the result output module;
[0026] Step 5: After receiving the result data, the result data valid signal and the calculation end signal, the result output module sends the result data, the result data valid signal and the calculation end signal to the address generation module;
[0027] Step 6: The address generation module generates a result address and a corresponding SRAM control signal according to the result data, the calculation end signal and the result data valid signal, and sends the result data, the result data address and the SRAM control signal to the SRAM storage module to complete the storage of the result data.
[0028] Furthermore, the parallel computing unit performs parallel computing on the read source data, and the multiplication and accumulation unit of the parallel computing unit performs parallel computing on the convolution kernel. When the computing engine module receives a module shutdown signal, the corresponding unit stops computing, and the remaining units continue to work. The unit is a parallel computing unit, a multiplication and accumulation unit, or an activation function computing unit.
[0029] Furthermore, the fixed-point multiplier of the multiplication-accumulation unit performs fixed-point truncation on the calculation data according to the received fixed-point position signal.
[0030] Furthermore, the result output module performs a zero padding operation on the result data before storing the result data in the SRAM storage module.
[0031] Furthermore, when storing data, the SRAM storage module divides the data into several segments and performs a ping-pong operation on the segments.
[0032] 3. Beneficial effects
[0033] Compared with the prior art, the advantages of the present invention are:
[0034] The present invention makes full use of the concurrency of convolution and designs a parallel computing unit, which achieves the performance advantage of high parallelism based on limited computing resources and storage resources; the parallel computing unit and its internal multiplication-accumulation unit and activation function calculation unit support adjustable parallelism, and power consumption can be reduced by shutting down idle computing units; the multiplication-accumulation unit and the activation function calculation unit support configurable 16bit / 8bit fixed-point data types to meet the accuracy requirements of different scenarios; the multiplication-accumulation unit and the activation function calculation unit support dynamic change of data fixed-point positions for data of different layers, effectively ensuring the accuracy of calculation results; the parallel computing unit supports configurable algorithm parameters, is suitable for various scenarios, and has strong scalability. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a schematic diagram of the overall framework of the convolutional neural network reasoning accelerator of the present invention;
[0036] Figure 2 It is a schematic diagram of the structure of the parallel computing unit of the present invention;
[0037] Figure 3 It is a schematic diagram of the structure of the multiplication and accumulation unit of the present invention;
[0038] Figure 4 It is a schematic diagram of the structure of the activation function calculation unit of the present invention;
[0039] Figure 5 It is a schematic diagram of the structure of the SRAM storage module of the present invention;
[0040] Figure 6 It is a schematic diagram of the sliced pipeline calculation of the parallel computing unit of the present invention;
[0041] Figure 7 It is a schematic diagram of the SRAM storage unit input ping-pong of the present invention. DETAILED DESCRIPTION
[0042] According to the analysis of the convolution operation rules, the main convolution operator multiplication and accumulation operations have a lot of computational parallelism brought by data slicing. For example, multiple convolution kernels are independent and concurrent, and multiple input images are also independent and concurrent. This provides ideas for hardware design.
[0043] Under the constraints of hardware resources and costs, in order to fully utilize the parallelism of convolution operations, we must first meet the requirements of high performance and low power consumption; secondly, we must ensure that the algorithm and parameters are configurable and scalable to meet different application scenarios; finally, we also need to improve calculation accuracy and reduce result errors.
[0044] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0045] like Figure 1 As shown, the present invention provides a convolutional neural network reasoning accelerator, the accelerator includes a main control module, an address generation module, an SRAM storage module, a data input module, a computing engine module and a result output module, wherein the main control module receives a start signal and distributes data to the address generation module, the data input module, the computing engine module and the result output module; the address generation module receives data from the main control module and the result output module, and sends a control signal to the SRAM storage module; the SRAM storage module receives the control signal of the address generation module, and sends storage data to the data input module; the data input module receives data from the main control module and the SRAM storage module, and sends the data to the computing engine module; the computing engine module receives data from the main control module and the data input module, and sends the result data to the result output module; the result output module receives the result data of the computing engine module, and sends the result data to the address generation module. Specifically:
[0046] A main control module is used to receive a start signal, and after receiving the start signal, sends a start signal to the data input module and the address generation module, and sends configuration information to the address generation module, the SRAM module, the data input module, the calculation engine module, and the result output module, wherein the configuration information includes convolution parameters such as image size, number of channels, convolution kernel size, number of convolution kernels, number of image batches, convolution step size, degree of parallelism, and activation function;
[0047] The address generation module is used to receive the start signal of the main control module, generate the source data address, weight address and SRAM control signal after receiving the start signal, and send the source data address, weight address and SRAM control signal to the SRAM module; it is also used to receive the result data, result data valid signal and calculation end signal of the result output module, generate the corresponding result data address and SRAM control signal and send them to the SRAM storage module;
[0048] An SRAM storage module is used to receive the address and SRAM control signal of the address generation module, store or read data according to the address and control signal, and send the read data to the data input module;
[0049] A data input module is used to receive a start signal from the main control module, generate a calculation control signal, receive data from the SRAM storage module, and send the calculation control signal and data to the calculation engine module, wherein the calculation control signal includes an input data valid signal, a module shutdown signal, and a fixed point position signal;
[0050] The calculation engine module is used to receive the calculation control signal and data from the data input module, and at the same time receive the configuration information of the main control module, calculate the data according to the configuration information and the calculation control signal to obtain the result data, and generate a result data valid signal and a calculation end signal, and send the result data, the result data valid signal and the calculation end signal to the result output module;
[0051] The result output module is used to receive the result data, result data valid signal and calculation end signal generated by the calculation engine module, and at the same time receive the configuration information of the main control module, fill the result data with zeros according to the configuration information, and send the zero-filled result data, result data valid signal and calculation end signal to the address generation module.
[0052] Specifically, the computing engine module in this embodiment includes several parallel computing units, one parallel computing unit includes several multiplication and accumulation units (PE) and one activation function calculation unit (ACT). The computing engine module in this embodiment includes 32 parallel computing units, one parallel computing unit includes 32 multiplication and accumulation units (PE) and one activation function calculation unit (ACT). It should be noted that there is no specific number limit for the number of parallel computing units and multiplication and accumulation units and activation function calculation units of the computing engine of the present invention. In the specific implementation, the specific number of units can be set according to the actual calculation requirements. The number of computing units here is only set according to the convolution layer calculation parameters of this embodiment. The parallel computing units of the computing engine module are independent of each other and can be executed concurrently. They realize the source data slice pipeline parallelism of the convolution layer calculation through the algorithm control module. The multiplication and accumulation units in the parallel computing units are also independent of each other and can be executed concurrently. They realize the convolution kernel slice pipeline parallelism of the convolution layer calculation through the algorithm control module. The algorithm control module is the main control module, the address generation module, the data input module and the result output module. The parallel computing unit and its internal multiplication-accumulation unit and activation function computing unit all have a shutdown function and stop working after receiving a valid shutdown signal. The device can send a shutdown signal according to the actual number of computing units required and adjust the number of running units, thereby reducing the power consumption required by the accelerator.
[0053] In such Figure 2 In the internal structure of the parallel computing unit shown, the multiplication and accumulation unit in the parallel computing unit receives the source data and weight data of the data input module through two data input buses. After calculating the data, the calculation result of the multiplication and accumulation unit enters the activation function calculation unit through the result input bus for calculation, and then the calculation result is transmitted to the result output module through the result output bus.
[0054] In such Figure 3 In the internal structure of the multiplication-accumulation unit PE shown, the multiplication-accumulation unit includes a fixed-point multiplier (FM) and a fixed-point adder (FA). The fixed-point multiplier receives source data and weight data, performs multiplication on the data, and performs fixed-point truncation on the calculation result according to the received fixed-point position signal (trunc). The fixed-point truncation is implemented by shifting the data in the fixed-point multiplier to the right. The multiplication-accumulation unit supports two fixed-point data types of 16bit and 8bit, and supports dynamic changes in fixed-point integer bits, which can meet the accuracy requirements of different scenarios. Fixed-point truncation can reduce the floating-point operations of the computing unit, thereby reducing the consumption of hardware resources and improving the computing speed.
[0055] In such Figure 4 In the internal structure of the activation function unit ACT shown, the activation function unit includes a lookup table memory (LUT), a fixed-point multiplier (FM), a fixed-point adder (FA) and two registers (FF). The lookup table (Look-Up-Table) is referred to as LUT. LUT is essentially a RAM in which a truth table is stored. In this embodiment, LUT and a register receive the calculation result data X of the multiplication and accumulation unit, and LUT receives the Trunc fixed-point truncation signal at the same time. LUT performs a logical operation according to the input signal, that is, it performs a table lookup according to the address, finds out the parameters k and b corresponding to the current input x, stores the intermediate result k in another register, sends the intermediate result b and x in one register to the fixed-point multiplier FM, and the fixed-point multiplier shifts according to the fixed-point truncation signal, and finally sends the result data and the intermediate result k in another register to the fixed-point adder, and calculates the result result = b*x+k. The activation function unit supports two fixed-point data types of 16bit and 8bit, and supports dynamic changes in fixed-point integer bits, which can meet the accuracy requirements of different scenarios.
[0056] In such Figure 5In the internal structure of the SRAM storage module shown, the SRAM storage module is divided into three parts, namely the source data area, the weight area and the result area. The storage module in this embodiment uses 128 SRAMs, each of which has a size of 256bit×1k, and can store 16k 16-bit fixed-point data or 32k 8-bit fixed-point data. The number of fixed-point data bits stored in the SRAM storage module is determined by the number of fixed-point data bits of the multiplication and accumulation unit and the activation function calculation unit. The data storage address is in bytes. The 128 SRAMs of the storage module are divided into 128 banks, of which 64 banks are used as source data areas to store source data, 32 banks are used as weight areas to store weight data, and 32 banks are used as result areas to store result data.
[0057] In this embodiment, before the calculation result of the calculation engine module is stored in the result SRAM, a zero padding operation is performed on the convolution result. The zero padding operation is jointly completed by the main control module, the address generation module, and the result output module. The main control module sends a zero padding signal to the address generation module to control the address generation module to generate a zero padding address. The zero padding address corresponds to the address of the result data in the SRAM storage module that needs to be padded with zero. The result output module stores data 0 in the corresponding zero padding address in the SRAM according to the zero padding address. By performing the zero padding operation on the convolution result, the calculation of the next layer of convolution is facilitated.
[0058] In such Figure 6 In the embodiment shown in FIG. 1 , the parallel computing unit realizes the slice pipeline parallelism, and the parallel computing unit receives the source data X abc and weight data W abc Convolution calculation is performed, the source data and weight data are two-dimensional matrices, where a represents the rows of the two-dimensional matrix, b represents the columns of the two-dimensional matrix, c represents the channels of the image, and X abc a, b, c determine the position of the input data in the source data, W abc a, b, and c determine the position of the weight data in the convolution kernel. In the convolution calculation of the computing engine module, all the multiplication and accumulation units in a single parallel computing unit calculate the same set of source data respectively, and different multiplication and accumulation units in a single parallel computing unit correspond to different convolution kernels. The weight data refers to a single weight, and a convolution kernel can contain multiple weight data; different parallel computing units correspond to different source data, and the multiplication and accumulation units at the same position in different parallel computing units correspond to the same convolution kernel. In this embodiment, a parallel computing unit is called a computing group, and each computing group is allocated 32 convolution kernels, corresponding to 32 multiplication and accumulation units PE. kernel0 is allocated to PE0, kernel1 is allocated to PE1, and so on. 31 Assigned to PE 31, and the convolution kernel corresponding to PE0 of computing group 0 is the same as that of PE0 of computing group 1. By allocating a corresponding multiplication-accumulation unit PE to each convolution kernel, the parallel calculation of the convolution kernel in the parallel computing unit is realized. Each multiplication-accumulation unit performs multiplication-accumulation operations on source data and weight data. After the calculation of each multiplication-accumulation unit, a result data of the corresponding channel in the output image can be generated. The result data enters the activation function unit ACT through the result input bus for activation function calculation, and the final result data is output to the result output module by the result output bus.
[0059] The sharded pipeline parallel transmission calculation method can perform parallel calculations on multiple batches of source data and multiple convolution kernels at the same time, which improves the calculation speed of the convolution layer in the convolutional neural network. At the same time, since each parallel computing unit is calculated independently, and each parallel computing unit and its internal PE and ACT have a shutdown function, the accelerator can be adjusted according to the actual number of images calculated, and the corresponding number of parallel computing units can be controlled to stop working by sending an effective shutdown signal to the computing engine module; the accelerator can also be adjusted according to the actual number of convolutions calculated, and a valid shutdown signal can be sent to the computing engine module to control the number of working multiplication and accumulation units PE in the parallel computing unit. By controlling the number of working parallel computing units and multiplication and accumulation units, the parallelism of the calculation is adjustable, and the idle computing units are turned off to reduce the power consumption required by the accelerator.
[0060] The present invention also provides a convolutional neural network reasoning acceleration method, which specifically includes the following steps:
[0061] Step 1: The main control module receives the convolution start signal and algorithm configuration information, transmits the configuration information to the calculation engine module and the result output module, and sends a start signal to the data input module and the address generation module at the same time. The configuration parameters are used by the calculation engine module and the result output module to perform convolution operation on the data;
[0062] Step 2: After receiving the start signal, the address generation module generates a source data address, a weight address and a corresponding SRAM control signal and transmits them to the SRAM storage module. Here, the corresponding SRAM control signal is a data fetch signal.
[0063] Step 3, the data input module generates an input data valid signal, a module shutdown signal and a fixed-point position signal according to the source data and weight data sent by the SRAM storage module, and the source data, weight data, input data valid signal, module shutdown signal and fixed-point position signal, the fixed-point position signal includes a PE fixed-point position signal and an ACT fixed-point position signal;
[0064] Step 4: After receiving the source data, weight data, input data valid signal, fixed-point position signal, and module shutdown signal, the computing engine module starts the convolution inference calculation to obtain the result data, generates the result data valid signal and the calculation end signal, and sends the result data, the result data valid signal, and the calculation end signal to the result output module;
[0065] Step 5, after receiving the result data and the result data valid signal, the result output module performs a corresponding zero-filling operation on the result data according to the configuration information, and sends the zero-filled result data, the result data valid signal and the calculation end signal to the address generation module;
[0066] Step 6, the address generation module generates the result data address and the corresponding SRAM control signal according to the received zero-filled result data, the calculation end signal and the result data valid signal. The corresponding SRAM control signal here is the data storage signal. The result data, the result data address and the corresponding SRAM control signal are sent to the SRAM storage module together to complete the storage of the result data.
[0067] When storing data in the SRAM storage module, when the complete input data cannot be placed in the corresponding SRAM storage module, the source data needs to be moved multiple times. Therefore, the input image, i.e., the source data, needs to be cut, and the input image needs to be divided into several segments and moved into the SRAM in sequence. The ping-pong operation is used to cover up part of the transfer time, thereby improving the computing efficiency. Figure 7 In the data input module input ping-pong embodiment shown, when the input image segment 0 is inferred, the next input image segment 1 that needs to be calculated is simultaneously moved in, thereby achieving partial coverage of data handling time or calculation time and improving the efficiency of the algorithm.
[0068] The method receives external data and information through the main control module of the convolutional neural network inference accelerator, distributes the data to the corresponding modules, calculates the input image data through the computing engine unit, realizes the pipeline parallel calculation of the convolution layer in the convolutional neural network, and stores the calculation results in the SRAM storage module to facilitate the next convolution calculation.
[0069] The above schematically describes the invention and its implementation methods, which is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited to this, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.
Claims
1. A convolutional neural network inference accelerator, characterized in that: It includes a main control module, an address generation module, an SRAM storage module, a data input module, a calculation engine module and a result output module, wherein: The main control module receives the start signal and distributes data to the address generation module, the data input module, the calculation engine module and the result output module; An address generation module receives data from the main control module and the result output module, and sends a control signal to the SRAM storage module; The SRAM storage module receives the control signal of the address generation module and sends the storage data to the data input module; A data input module receives data from the main control module and the SRAM storage module and sends the data to the calculation engine module; The calculation engine module receives data from the main control module and the data input module, and sends the result data to the result output module; A result output module receives result data from the calculation engine module and sends the result data to the address generation module; the calculation engine module includes a plurality of parallel calculation units, which in turn include a plurality of multiplication and accumulation units and an activation function calculation unit, the multiplication and accumulation unit receives data from the data input module through a data input bus, and sends the data to the activation function calculation unit through a result input bus; The result output module performs a corresponding zero-filling operation on the result data according to the configuration information before storing the result data in the SRAM storage module; In the internal structure of the parallel computing unit, the multiplication and accumulation unit in the parallel computing unit receives the source data and weight data of the data input module through two data input buses. After calculating the data, the calculation result of the multiplication and accumulation unit enters the activation function calculation unit through the result input bus for calculation, and then the calculation result is transmitted to the result output module through the result output bus. The multiplication and accumulation unit includes a fixed-point multiplier and a fixed-point adder. The fixed-point multiplier receives source data and weight data, performs multiplication operations on the data, and performs fixed-point truncation on the calculation results according to the received fixed-point position signal.
2. A convolutional neural network inference accelerator according to claim 1, characterized in that: The multiplication-accumulation unit comprises a fixed-point multiplier and a fixed-point adder, and the fixed-point multiplier is connected to the fixed-point adder.
3. A convolutional neural network inference accelerator according to claim 1, characterized in that: The activation function calculation unit includes a lookup table memory, a fixed-point multiplier, a fixed-point adder and two registers. The lookup table memory and the first register receive data, the lookup table memory sends the data to the fixed-point multiplier and the second register, and the fixed-point multiplier and the second register send the data to the fixed-point adder.
4. A convolutional neural network inference accelerator according to claim 1, characterized in that: The main control module also sends a zero-filling signal to the address generation module, the address generation module generates a zero-filling address and sends it to the result output module, and the result output module sends the result data and the zero-filling address to the SRAM storage module.
5. A convolutional neural network reasoning acceleration method, characterized in that: The method is applied to the inference accelerator as claimed in any one of claims 1 to 4, comprising the following steps: Step 1: The main control module receives the start signal and configuration information, sends the configuration information to the calculation engine module and the result output module, and sends a start signal to the data input module and the address generation module; Step 2: After receiving the start signal, the address generation module generates a source data address, a weight address and a corresponding SRAM control signal and transmits them to the SRAM storage module. The SRAM storage module reads the source data and weight data according to the source data address, the weight address and the corresponding SRAM control signal and sends them to the data input module; Step 3, the data input module generates an input data valid signal, a module shutdown signal and a fixed-point position signal according to the source data and weight data sent by the SRAM storage module, and the source data, weight data, input data valid signal, module shutdown signal and fixed-point position signal; Step 4: After receiving the source data, weight data, input data valid signal, fixed-point position signal and module shutdown signal, the computing engine module starts the convolution inference calculation to obtain the result data, generates the result data valid signal and the calculation end signal, and sends the result data, the result data valid signal and the calculation end signal to the result output module; Step 5: After receiving the result data, the result data valid signal and the calculation end signal, the result output module sends the result data, the result data valid signal and the calculation end signal to the address generation module; The result output module performs a corresponding zero-filling operation on the result data according to the configuration information before storing the result data in the SRAM storage module; Step 6: The address generation module generates a result address and a corresponding SRAM control signal according to the result data, the calculation end signal and the result data valid signal, and sends the result data, the result data address and the SRAM control signal to the SRAM storage module to complete the storage of the result data.
6. A convolutional neural network inference accelerator according to claim 5, characterized in that: The parallel computing unit performs parallel computing on the read source data, and the multiplication and accumulation unit of the parallel computing unit performs parallel computing on the convolution kernel. When the computing engine module receives a module shutdown signal, the corresponding unit stops computing, and the remaining units continue to work. The unit is a parallel computing unit, a multiplication and accumulation unit, or an activation function computing unit.
7. A convolutional neural network reasoning acceleration method according to claim 6, characterized in that: The fixed-point multiplier of the multiplication-accumulation unit performs fixed-point truncation on the calculation data according to the received fixed-point position signal.
8. The method for accelerating convolutional neural network reasoning according to claim 5, characterized in that: When storing data, the SRAM storage module divides the data into several segments and performs a ping-pong operation on the segments.
