Efficient hardware cache circuit for convolutional neural networks

Abstract

The application discloses a kind of efficient hardware cache circuits for convolutional neural network, the circuit includes: data cache module, data in DDR is stored to BRAM on chip;Data shift module, the data of BRAM in data cache module is read, and shift operation is carried out;Asynchronous transmission module, the data of data shift module after shifting is carried out asynchronously, and the FIFO therein is used to store data, waiting for calling.Through using the application, the problem that input feature map of FPGA-based convolutional neural network accelerator cannot be efficiently cached can be overcome, while reducing the use of resources, the complexity of circuit design is reduced.The application can be widely applied to chip field.

Description

Technical Field

[0001] This invention relates to the field of chips, and more particularly to a high-efficiency hardware cache circuit for convolutional neural networks. Background Technology

[0002] In recent years, Convolutional Neural Networks (CNNs) have been widely applied in various fields, demonstrating excellent performance. In the field of AI inference, CNNs need to perform forward inference on embedded platforms. However, most embedded platforms have limitations in power consumption and memory, so efficient processing of the forward inference process is required. Therefore, hardware accelerators based on CNNs have become one of the hot research topics in academia. Field Programmable Gate Arrays (FPGAs), as programmable development boards, can serve as research and testing platforms for CNN-based hardware accelerators.

[0003] In recent years, most convolutional neural network accelerators have relied on complex circuit mappings to exhaustively cache various input feature maps on-chip, resulting in significant waste of on-chip cache resources. Furthermore, they fail to efficiently cache input feature maps under different convolutional parameters (kernel size: K, kernel stride: S). Summary of the Invention

[0004] In view of this, in order to solve the problem that the input feature maps of existing convolutional neural network accelerators cannot be efficiently cached, this invention proposes an efficient hardware caching circuit for convolutional neural networks, the circuit comprising:

[0005] The data caching module stores data from DDR into on-chip BRAM;

[0006] The data shifting module reads data from the BRAM in the data buffer module and performs shifting operations.

[0007] The asynchronous transmission module asynchronously transmits the data shifted by the data shifting module, using its FIFO to store the data and wait for a call.

[0008] In some embodiments, the data caching module specifically includes:

[0009] The BRAM module uses source code to generate read / write data;

[0010] The input feature map control module controls the read / write data and read / write address enable signal switching of 25 BRAM lines;

[0011] The top-level module instantiates the BRAM module and the input feature map control module, and connects the corresponding signals to implement the input buffer function.

[0012] In some embodiments, the data shifting module employs a shift register.

[0013] In some embodiments, the asynchronous transmission module employs 16 asynchronous FIFOs to meet the asynchronous computation requirements of 16 PE units; the data obtained after each shift of the shift register group is simultaneously sent to the 16 asynchronous FIFOs, and the data of the 16 asynchronous FIFOs is read out according to the read enable signal given by the PE unit.

[0014] In some embodiments, the BRAM module specifically uses bram_WR64bit_36KB, the input feature map control module specifically uses input_buffer_ctrl, and the top-level module specifically uses bram_25__WR_64bit__36KB.

[0015] In some embodiments, the register group in the data shift module is changed proportionally according to the number of computing units.

[0016] In some embodiments, the input_buffer_ctrl works as follows:

[0017] The input_buffer_ctrl controls the write operation of the BRAM by controlling the value of the output signal wealth through the state transition of the state machine. When the least significant bit of the signal wealth is 1, the write enable signal of BRAM1 is pulled high, and data is written to BRAM1. When the least significant bit and the 21st bit are both 1, it means that the write enable signals of BRAM1 and BRAM21 are both high, and data is written to BRAM1 and BRAM21 at the same time.

[0018] The input_buffer_ctrl controls the write address of the BRAM by controlling the value of the output signal addrout_out through the state transition of the state machine. The bit width of addrout_out is set to 25 * the write address bit width of the BRAM. According to the state transition, the current write address is assigned to different BRAMs. When the state machine jumps to write BRAM3, the current address is assigned to the position of addrout_out corresponding to BRAM3, and all other bits are 0.

[0019] Based on the above scheme, the present invention provides a high-efficiency hardware caching circuit for convolutional neural networks. By combining a data caching module, a data shifting module, and an asynchronous transmission module, the caching of the input feature map is decoupled, reducing resource usage and lowering the complexity of circuit design. At the same time, the high-efficiency hardware caching circuit of the present invention overcomes the problem of overly complex design of other hardware caching circuits for input feature maps. Attached Figure Description

[0020] Figure 1 This is an overall framework diagram of an efficient hardware cache circuit for convolutional neural networks according to the present invention.

[0021] Figure 2 This is a schematic diagram of the data caching module in a specific embodiment of the present invention;

[0022] Figure 3 This is a state transition diagram of the data caching module in a specific embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of a shift register group according to a specific embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of an asynchronous FIFO according to a specific embodiment of the present invention. Detailed Implementation

[0025] There are two main reasons why FPGA-based convolutional neural network accelerators have attracted widespread attention. First, advancements in semiconductor technology have led to continuous increases in the computing power and on-chip storage capacity of FPGAs. Second, advancements in network compression technology have made convolutional computations more irregular, and the flexibility of FPGAs makes them suitable for handling irregular convolutional computations and custom data types. Furthermore, different feature map sizes require efficient reuse of data in on-chip buffers; otherwise, it would lead to more off-chip accesses, thus reducing energy efficiency.

[0026] This patent addresses the problem of inefficient caching of input feature maps in FPGA-based convolutional neural network accelerators by designing a high-efficiency hardware caching circuit that supports different convolution parameters (such as convolution stride S, weight size K, and input feature map size H / L). It not only supports various convolution parameters but also improves the efficiency of resource-constrained FPGA-based convolutional neural network accelerators and reduces resource usage.

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] It should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0029] It should be understood that the terms "system," "apparatus," "unit," and / or "module" used in this application are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0030] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "a," and / or "the" are not specifically singular and may include the plural. Generally, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0031] In the description of the embodiments of this application, "a plurality of" refers to two or more. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0032] Furthermore, flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, the steps can be processed in reverse order or simultaneously. Additionally, other operations can be added to these processes, or one or more steps can be removed from them.

[0033] Reference Figure 1 This is an overall framework diagram of the efficient hardware cache circuit for convolutional neural networks proposed in this invention. The efficient hardware cache circuit includes:

[0034] The data caching module stores data from DDR into on-chip BRAM;

[0035] The data shifting module reads data from the BRAM in the data buffer module and performs shifting operations.

[0036] The asynchronous transmission module asynchronously transmits the data shifted by the data shifting module, using its FIFO to store the data and wait for a call.

[0037] In some feasible embodiments, the data caching module is specifically as follows:

[0038] Reference Figure 2The data buffer module mainly consists of three modules. The first module is the BRAM module, named bram_WR64bit_36KB, which mainly uses the source code to generate a simple dual-port BRAM with a read / write data bit width of 64 bits and a read / write address bit width of 9 bits, i.e., a depth of 512. The second module is the input feature map control module, named input_buffer_ctrl, which controls the jump of the enable signals for reading / writing data and reading / writing addresses of the 25 BRAM lines. The third module is the top-level module of the data buffer module, named bram_25__WR_64bit__36KB, which instantiates the first two smaller modules and connects the corresponding signals to implement the input buffer function.

[0039] The input_buffer_ctrl function of the data caching module performs zero-padding and overlap operations on the input feature maps. These operations are implemented through state machine transitions. Figure 3 The diagram shows the state transitions of `input_buffer_ctrl` when the weight size is 3. Initially, `input_buffer_ctrl` is in the `Idle` state, indicating the state machine is idle. Upon receiving the `start` signal, it jumps to the `init` state to clear the state. Then, it assigns a value to `pad_top` based on the input signals `kernel_size` and `pad_edge`. `pad_top` indicates how many rows of zero values ​​should be added to the upper edge of the input feature map. When the weight size is 3 (i.e., `kernel_size` = 3), `pad_top` is 1, indicating that a row of 0s should be added to the upper edge. Therefore, the first BRAM line is not written to, meaning zeros are written to this line. Entering the state of writing BRAM2, the first row of data from the original input feature map block is written to BRAM2. After writing 20 rows of data, the state machine enters the state of writing BRAM1 and writing BRAM21, writing the 20th row of data from the original input feature map block to both BRAM1 and BRAM21 simultaneously. Then, the state machine enters the state of writing BRAM2 and writing BRAM22, writing the 21st row of data from the original input feature map block to both BRAM2 and BRAM22 simultaneously. Finally, the state machine jumps to the state of writing BRAM3, writing the 22nd row of data from the original input feature map block to BRAM3.

[0040] The `input_buffer_ctrl` function uses state machine transitions to control the value of the output signal `weain`, thereby controlling the write operation of the BRAM. For example, when the least significant bit of the `weain` signal is 1, the write enable signal of BRAM1 is pulled high, and data is written to BRAM1. When both the least significant bit and the 21st bit are 1, it indicates that the write enable signals of both BRAM1 and BRAM21 are high simultaneously, and data is written to both BRAM1 and BRAM21 at the same time.

[0041] The input_buffer_ctrl controls the write address of the BRAM by controlling the value of the output signal addrout_out through the state transition of the state machine. The bit width of addrout_out is set to 25 * the write address bit width of the BRAM. According to the state transition, the current write address is assigned to different BRAMs. For example, when the state machine jumps to write BRAM3, the current address is assigned to the position of addrout_out corresponding to BRAM3, and all other bits are 0.

[0042] In some feasible embodiments, the data shifting module is specifically described as follows:

[0043] The key to the data shifting module is the implementation of the shift register. The shift register supports weights of 1 to 7 and convolution strides of 1 and 2. Figure 4 Examples are given with a weight size of 7 and a convolution stride of 2, and with a weight size of 3 and a convolution stride of 1. The 10 registers in the register group on the right correspond to the 10 computation units of the convolution module, and can be changed proportionally according to the number of computation units.

[0044] When the weight size is 7 and the convolution stride is 2, the shift register group reads 25 data entries from the BRAM (Browser RAM) from the data buffer module. Since the BRAM read width is 64 bits, containing input activation data for 8 channels, each register in the shift register group also has a 64-bit width, storing data from 8 channels. To achieve input activation multiplexing, the registers are shifted upwards according to the convolution stride. For example, in the first cycle, the data in register 0 on the left is assigned to register 0 on the right; in the second cycle, the value in register 2 on the left is assigned to register 0 on the right. The number of upward shifts is equal to the weight size minus 1. Figure 4 It can be seen that when the weight size is 7 and the convolution stride is 2, all 25 registers are used, which is just enough to reuse the input activation data.

[0045] When the weight size is 3 and the convolution stride is 1, the input buffer stores 22 rows of input feature map block data. The shift register group will first read 12 rows of data from the data buffer module and reuse the input activation by assigning values ​​upward through the shift register. For example, in the first cycle, the data of the left register 0 is assigned to the right register 0, and in the second cycle, the value of the left register 1 is assigned to the right register 0. A total of two upward shift assignments are required. After the 12 rows of data are traversed, another 12 rows of data are read from the input buffer and the shift operation is repeated.

[0046] In some feasible embodiments, the asynchronous transmission module is specifically described as follows:

[0047] Since the data transmitted from the input feature map to the computation module needs to be asynchronous, in order to ensure that the multiplication and accumulation calculations of each PE unit can be performed continuously and to improve the computational efficiency of the entire convolutional computation unit, 16 asynchronous FIFOs need to be added to the input buffer to ensure that the data can be transmitted to each PE unit for multiplication and accumulation calculations in a timely manner when needed, thereby improving the transmission efficiency of the input feature map.

[0048] The input buffer data transmission module primarily uses a shift register group to multiplex input activation. The FIFO module meets the asynchronous computation needs of the 16 PE units. After each shift, the data obtained from the shift register group is simultaneously sent to the 16 FIFOs. The data from the 16 FIFOs is read out according to the read enable signal given by the PE unit. The implementation scheme is as follows: Figure 5 As shown.

[0049] In summary, this invention designs an efficient hardware caching circuit based on the input feature map of a convolutional neural network. It caches the input feature map data (DDR, Double Data Rate) into BRAM (Block RAM), then transfers the data from the BRAM to a shift register group, and finally transmits it to the convolutional kernel via asynchronous FIFO (First In, First Out). The key to designing an efficient hardware caching system for input feature maps lies in zero-padding and overlap of the input feature map, which are controlled by different convolutional parameters. Blocking the input feature map can lead to overlap between adjacent blocks. Based on the above, the high-efficiency hardware cache circuit mainly consists of three modules. The first module is the data cache module, which mainly stores data from DDR to on-chip Bram. The focus of this module is handling the overlap caused by different convolution parameters and the zero-padding operation of the input feature map. The second module is the data shift module, which mainly implements a 25-bit data shift operation through a shift register. The focus of this module is also on designing the number of shifts and data mapping changes caused by different convolution parameters. The third module is the asynchronous transmission module, which mainly uses 16 asynchronous FIFOs to temporarily store data for use by the enable signal of the computation module. Based on the design of these three modules, a high-efficiency hardware cache circuit for the input feature map can be realized.

[0050] Furthermore, as CNNs grow larger and contain more parameters, the varying feature map sizes necessitate the efficient reuse of data in the on-chip buffer. Otherwise, more off-chip accesses will occur, reducing energy efficiency. Therefore, designing an efficient on-chip hardware buffer for input feature maps is of paramount importance.

[0051] Therefore, designing an efficient hardware caching circuit for input feature maps is highly significant. Currently, the most common approach is exhaustive enumeration, which, while allowing for individual design of convolution parameters, consumes substantial hardware resources and significantly reduces accelerator flexibility. This invention, through simple circuit design and implementation, uses fewer resources and supports all combinations of convolution parameters. The table below shows the combinations of convolution parameters supported by this system:

[0052] Table 1 Combinations of Convolution Parameters

[0053] 1 1*1 1 support 2 1*1 2 support 3 3*3 1 support 4 3*3 2 support 5 5*5 1 support 6 5*5 2 support 7 7*7 2 support

[0054] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A high-efficiency hardware cache circuit for convolutional neural networks, characterized in that, include: The data caching module stores data from DDR into on-chip BRAM; The data shifting module reads data from the BRAM in the data buffer module and performs shifting operations. The asynchronous transmission module asynchronously transmits the data shifted by the data shifting module, using its FIFO to store the data and wait for it to be called. The data caching module specifically includes: The BRAM module uses source code to generate read / write data; The input feature map control module controls the read / write data and read / write address enable signal switching of 25 BRAM lines; The top-level module instantiates the BRAM module and the input feature map control module, and connects the corresponding signals to realize the input buffer function; The data shifting module uses a shift register; The asynchronous transmission module employs 16 asynchronous FIFOs to meet the asynchronous computation requirements of 16 PE units; the data obtained after each shift of the shift register group is simultaneously sent to the 16 asynchronous FIFOs, and the data in the 16 asynchronous FIFOs is read out according to the read enable signal given by the PE unit; The BRAM module specifically uses bram_WR64bit_36KB, the input feature map control module specifically uses input_buffer_ctrl, and the top-level module specifically uses bram_25__WR_64bit__36KB. The register group in the data shifting module changes proportionally to the number of computation units. When the weight is 7 and the convolution stride is 2, the data cache module reads the input feature map data through 25 BRAMs. The shift register group performs a shift operation on the data output from the 25 BRAMs according to the movement pattern of the convolution window, and completes the reuse of input activation data between corresponding computation units through the register group. When the weight is 3 and the convolution stride is 1, the data cache module stores 22 rows of input feature map block data. The register group first reads 12 rows of input data from the data cache module, and realizes the reuse of input activation data by shifting the register group upward. After processing 12 rows of data, another 12 rows of data are read from the data cache module and the shift operation is repeated. The working process of input_buffer_ctrl is as follows: The input_buffer_ctrl controls the write operation of the BRAM by controlling the value of the first output signal, weain, through the state transition of the state machine. When the least significant bit of weain is 1, the write enable signal of BRAM1 is pulled high, and data is written to BRAM1. When the least significant bit and the 21st bit of weain are both 1, it means that the write enable signals of BRAM1 and BRAM21 are both high, and data is written to BRAM1 and BRAM21 at the same time. The input_buffer_ctrl controls the write address of the BRAM by controlling the value of the second output signal addrout_out through the state machine's state transitions. The bit width of addrout_out is set to... The write address width is determined by the state transition. The current write address is assigned to different BRAMs. When the state machine transitions to write BRAM3, the current address is assigned to the position corresponding to addrout_out in BRAM3, and all other bits are 0.

Images