Design and implementation of efficient access circuit of convolutional neural network accelerator

Abstract

The invention requests to protect an access circuit in a convolutional neural network accelerator (CNNCU) system. Mainly relates to a bidirectional data exchange control circuit (a feature map data exchange microsystem) between an off-chip DDR (Double Data Rate) and a feature map cache module and a unidirectional data reading control circuit (a convolution kernel data reading microsystem) between an off-chip convolution kernel memory and a convolution kernel data cache module. The feature map data exchange microsystem flexibly controls the DDR and the read-write address and the enable signal of the feature map cache module according to the data throughput rate of the tensor calculation unit, and the optimized access circuit can utilize the data bandwidth of the DDR to the maximum extent and improve the operation efficiency of an accelerator; according to the convolution kernel data reading microsystem, a convolution kernel data access scheme suitable for the convolutional neural network accelerator is given by considering the off-chip memory resource condition of the FPGA board card, the data bandwidth of the nonvolatile memory can be utilized to the maximum extent, and the operation efficiency of the accelerator can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of AI hardware accelerators and relates to the design and implementation of an efficient memory access circuit adapted to tensor calculations. Background Art

[0002] In recent years, with the development of computer hardware, the scale of neural networks is no longer strictly limited, and the accuracy of algorithms has been significantly improved and has begun to exceed traditional algorithms. Therefore, deep learning has gradually become a research hotspot in the field of artificial intelligence. As a relatively mature deep learning technology, deep convolutional neural network (DCNN) has been effectively applied in many fields. Because of its more powerful computing power (it can calculate convolution algorithms faster), in some application scenarios, its hardware carrier - convolutional neural network accelerator is gradually replacing traditional chips. In the current neural network hardware acceleration technology, the von Neumann "bottleneck" or "memory wall" problem seriously hinders the speed improvement of accelerators, so the technology of efficiently accessing off-chip memory and internal data caches has been widely studied. Summary of the invention

[0003] In view of this, the object of the present invention is to provide a high-efficiency memory access circuit based on a convolutional neural network accelerator, so as to improve the efficiency of memory access, maximize the transmission bandwidth, and improve the efficiency of tensor calculation.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A convolutional neural network accelerator efficient memory access circuit design and implementation, which includes two memory access microsystems: feature map data exchange control circuit and convolution kernel data reading control circuit.

[0006] The characteristic graph data exchange microsystem consists of DDR read-write control circuit, write FIFO control circuit, read FIFO control circuit, and characteristic graph cache circuit. The write FIFO receives the data output from the data transmitter / CNNCU, and sends the data to the DDR after synchronously caching it. In principle, the bandwidth of the data transmission / CNNCU output data is smaller than the write data bandwidth of the DDR. However, the data port of the DDR is of INOUT type and cannot be read and written at the same time, so the calculation of this part of the data transmission bandwidth cannot be simply compared in this way; after DDR caches a certain amount of data, it sends the data to the read FIFO. In order to efficiently utilize the high data transmission bandwidth of the external memory, the write clock of the internal memory must be the same as the user clock of the DDR, that is, the read and write clocks of the read FIFO are the same as the user clock of the DDR. Ideally, the write data bandwidth of the read FIFO is equal to the bandwidth of the internal cache. However, the data consumption of tensor calculation depends on the data size required by the convolution layer, so in actual situations, this part of the bandwidth calculation cannot be simply evaluated; at the same time, considering the three-dimensional address index space characteristics of the tensor calculation unit consuming and generating data, it is concluded that when performing DDR reading and writing, read-write switching and row switching must be involved. It is known that a single read-write and row switching will make the DDR access efficiency less than 15%, while the normal continuous address read / write DDR efficiency can reach more than 85%. Therefore, consider using the write FIFO and read FIFO to cache data to reduce the read-write switching and row switching during DDR access. Under this scheme, the circuit design must consider the capacity of the write FIFO and read FIFO and the timing of DDR read-write and row switching.

[0007] The original intention of designing the memory access system of the convolutional neural network accelerator is to efficiently access memory - maximize the data transmission bandwidth of the tensor computing unit. Therefore, the external memory access circuit of the feature map data exchange microsystem must first meet the principle of maximizing the read data bandwidth - switch the DDR to write mode under the premise that there is still a certain amount of batch data in the read FIFO, and no read interruption can be generated. The efficient memory access of DDR is explained as follows:

[0008] First, the first frame of the feature image is cached in the DDR through the write FIFO. Then, when there is no data in the write FIFO, the data in the DDR is sent to the read FIFO. At this time, the drive address is incremented according to the frame arrangement of the image data. After that, when the data in the read FIFO is close to full (the write counter is greater than or equal to L RFIFO_WUP , L RFIFO_WUP is the key parameter to be determined), stop reading DDR (DDR is still in the read state, but the read enable signal is turned off), otherwise the read enable is turned on again; when the amount of data in the write FIFO is greater than the level close to full (the write counter is greater than or equal to L WFIFO_WUP , L WFIFO_WUPis the key parameter to be determined); when the amount of data in the read FIFO is less than or equal to the device read delay time (including the DDR read delay time and the FIFO read data delay time), the data burst length that can be read (the read counter is less than or equal to L RDELAY_DDR , L RDELAY_DDR are known parameters). When the tensor calculation unit is implemented in the form of matrix operations, the data demand of the tensor calculation unit is usually much greater than the amount of data generated. If the operating frequency of the convolutional neural network accelerator is set according to the data transmission bandwidth formula, the capacity of the read / write FIFO is reasonably preset, and the capacity of the read FIFO is corrected according to the actual device delay, then the DDR can be set to the read state for a long time, and the number of switching times will be greatly reduced. If the priority order of tensor calculation is set to: first calculate the input channel of the output feature map data, and then calculate the length and width of the output feature map, then there is no need to frequently perform cross-row write operations to store the data in the WFIFO in the DDR. According to the above design method, the DDR reading and writing feature map data can achieve the highest efficiency.

[0009] The memory access circuit inside the feature map data exchange microsystem is a first-in, first-out circuit. This fine-grained dynamic ping-pong caches a portion of the input feature map (the amount of data is L IC ×L ISize ×L KSize , L IC With L ISize is the number and length of input feature map channels, L KSize is the length of the convolution kernel, assuming here that the feature map and the convolution kernel have the same length and width) and then perform a ping-pong access operation: the tensor computing unit completely consumes a portion of the data each time (the amount of data is L BUST_CNNCU , L BUST_CNNCU is a known variable), then the remaining unread data of the input feature graph (the data volume is also L BUST_CNNCU ) is stored in the location where the data is consumed. Because the location of the consumed data is in small blocks and is always sliding, it is called a fine-grained dynamic ping-pong cache method.

[0010] Because it is designed in the form of multi-BANK asynchronous dual-port RAM, its capacity needs to be set to a suitable value (C IMEM =L IC ×L ISize ×L KSize ). Every time the tensor calculation unit outputs a result, there is a block with a volume (capacity) of M×N×L IC (M is the length of the consumed data block, N is the width, M×N=L BUST_CNNCU) slides from the left side of the internal memory to the right side (the length of the internal memory does not really exist, it is a measurement concept, and it is assumed to be equal to the length of the input feature map). When it reaches the rightmost side, it will wrap and continue to slide. The data in the cuboid represents invalid data. If we ignore the discontinuity of writing data, the behavior of writing data can be regarded as filling a ping-pong row of the memory and then wrapping to continue filling (the volume of a ping-pong row is C PPR =L IC ×L ISize ×L Stride ), the filling order inside the ping-pong row is to fill the BANK first, then the column, and then the row. Because M is determined by the hardware resources and the scale of the convolutional neural network, it will not be strictly equal to the length of the internal memory. Therefore, when the internal memory is operated on a first-come-first-out basis, the data will be offset. Consider the following situation: the data is read according to the priority of the image frame line-by-line scanning (input channel> row> column), but is stored in blocks, and the volume of the rectangular block will change. Therefore, only when the read and write pointers of the feature map cache simultaneously use fine-grained dynamic ping-pong pointer forward mapping decoding operations, the system can efficiently complete memory access.

[0011] This forward mapping decoding operation requires the input feature map length to be L Stride n (L Stride is the sliding step size of the current convolutional layer, and the common value is 1 or 2, and n is any natural number. Because the algorithm deployed by the convolutional neural network accelerator is a fully convolutional neural network, and a convolutional layer with a step size of 1 will not change the size of the output image, the unilateral size of the input image is adjusted to L at the inference end. Stride n After L, the input image size of all convolutional layers meets this requirement. From the above analysis, it can be seen that the internal memory access circuit of the feature map data exchange microsystem can be widely used in all fully convolutional neural networks. Stride = 1, memory can be accessed in an address incrementing manner. Stride = 2, since the cuboid (length and width are both 2) representing data consumption slides from left to right until it wraps, this row of data can be used as a ping-pong row. In the preparation stage, the data is first stored in the storage body in an increasing manner according to the address. When writing data to the same ping-pong row for the second time, the address needs to be dynamically mapped: fill the ping-pong row with data according to the priority of BANK> column> row (the width of this column is L Stride, the row length is the same as the internal memory length). When writing data to the same ping-pong row for the third time, the ping-pong row needs to be divided into two blocks (BLOCK0 and BLCOK1) from the row. The dimension priority of the filling address is BANK>BLOCK>column>row. When writing data to the same ping-pong row for the fourth time, the ping-pong row needs to be evenly divided into four blocks (BLOCK0, BLCOK1, BLOCK2, BLOCK3) by row and then filled according to the above address dimension priority. When writing data to the same ping-pong row for the Tth time (the number of BLOCKs is equal to the row length), the address access mode returns to the initial state, that is, the data is accessed in a continuously increasing manner according to the address. Based on this, the dynamic change of address mapping every T times is a cycle, and no more logic is required for control.

[0012] The convolution kernel data reading microsystem uses non-volatile memory to store data files offline. After the controller, the data is directly read into the convolution kernel buffer, or transmitted to the convolution kernel buffer after passing through FIFO. Here, the SD card is used as the convolution kernel data storage. Since the SD card uses sectors as units for data caching, each burst reads 512 bytes of data, and the convolution kernel buffer needs to read the complete convolution kernel data of 16 output channels (in bytes) for each burst. The two data volumes are not integer multiples, so a cache is needed to temporarily store the remaining data. The storage space is 512*8 bits. If the burst size that the current convolution kernel buffer can cache at one time is less than 512 bytes, then there is no need to use another buffer; if the burst size that the current convolution kernel buffer can cache at one time is greater than 512 but less than half of the convolution kernel buffer capacity, in order not to increase the resources occupied by the cache, time-sharing multiplexing of the remaining convolution kernel buffer storage space should be considered; if the burst size that the current convolution kernel buffer can cache at one time is exactly half of the convolution kernel buffer capacity, then at the critical point before half of the convolution kernel buffer capacity is filled, stop reading the data of the next sector, and wait until the data of the other half of the capacity is calculated before burst writing of data.

[0013] The cross-clock domain output signals of the feature map data exchange microsystem and the convolution kernel data reading microsystem mainly consist of two parts. One is the Req_IExtMem_valid (Req_KExtMem_valid) signal used to indicate that the external memory read enable is valid, and the other is the Req_IMem_ready (Req_KMem_ready) signal that informs the tensor computing unit that it can start computing.

[0014] In fact, Req_IExtMem_valid (Req_KExtMem_valid) is generated by the external memory read clock domain signal and the convolutional neural network working clock domain signal. Because the external memory read clock domain is used as the receiving clock domain, the signals under the convolutional neural network working clock domain (receiving instruction success signal and convolutional neural network related decoding data and read address, etc.) need to be synchronized to the receiving clock domain. Since there is synchronization of multi-bit signals and single-bit signals here, if the general asynchronous FIFO synchronization method and handshake synchronization method are used, the final synchronization circuit is complex and large. First, the single-bit signal is transmitted from the slow clock domain to the fast clock domain, and the receiving instruction success signal will not be valid until the current convolution layer is calculated (multi-cycle signal), so a synchronizer composed of two-stage triggers can be used for synchronization (the receiving clock is not ultra-high frequency); and the synchronization of multi-bit signals can be solved by the MCP method: using the enable signal for synchronization, waiting for stability to access the asynchronous data signal will not cause metastable problems. In order to ensure the high bandwidth of the tensor computing unit, the multi-bit signal synchronization cannot simply use a closed-loop MCP circuit, so it is necessary to analyze the actual situation and improve the MCP design method. The actual circuits for multi-bit signal synchronization in this system are divided into two categories: address signal synchronization circuits and decoded data synchronization circuits. For the synchronization of address signals: Since the data access methods of the internal buffer of the feature map / convolution kernel are similar to those of FIFO, their synchronization generally does not require the use of backpressure signals to the sending clock domain for handshake (first address signal, usually a multi-cycle signal, and the feedback circuit is optional). This is a conservative "full" phenomenon, and with the support of the enable signal, there is no need to use Gray code to encode the address (just what the tensor calculation unit needs to read the data address control circuit). For the synchronization of decoded data, since it is itself a multi-cycle signal, there is no need to backpressure the related signals in the sending clock domain when using the MCP method for synchronization. As for the control circuit in which multiple data in the sending clock domain are synchronized to the receiving clock domain and generate the same enable signal, since the triggering time of these multiple data in the control circuit is far apart, there is no asynchrony between multiple data variables.

[0015] In fact, the synchronization of the Req_IMem_ready (Req_KMem_ready) signal is a pure single-bit signal synchronization problem from the fast clock domain to the slow clock domain, which can be solved by using a synchronizer. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0017] Figure 1 This is the overall structure diagram of the efficient memory access circuit of the convolutional neural network accelerator;

[0018] Figure 2 The circuit diagram of the external memory access circuit of the convolutional neural network accelerator;

[0019] Figure 3 A schematic diagram of fine-grained dynamic ping-pong for data exchange in the internal buffer of the microsystem for feature graph data;

[0020] Figure 4 This is the address sequential mapping flow chart of the fine-grained dynamic ping-pong algorithm;

[0021] Reference numerals: Figure 3 The black squares represent ping-pong storage 0, and the other squares represent storage 1; Figure 4 The numbers represent the position encoding of the feature map under line-by-line scanning. DETAILED DESCRIPTION

[0022] The technical scheme in the embodiment of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiment of the present invention. The described embodiment is only a part of the embodiment of the present invention.

[0023] The present invention proposes a convolutional neural network accelerator efficient memory access circuit, which is analyzed by mathematical methods. The circuit structure of the external memory access system of the feature map data exchange microsystem is as follows Figure 2 As shown, the order of data exchange is: at the initial moment, the write data end, write enable end, and write clock end of the write FIFO select the relevant signal of the external sensor as input through data selection, and wait for the write FIFO to cache the first frame of feature image data in sequence to the DDR. The data end, write enable end, and write clock end of the write FIFO reselect the data output end of the CNNCU as input (because the write FIFO does not start writing data immediately after the clock is switched, so only a simple data selector is needed here, and there is no need to use a glitch-free clock switching circuit), and the DDR switches to the read state. From then on, the DDR begins to switch the read and write states regularly: when the data in the read FIFO is close to full (the write counter is greater than or equal to L RFIFO_WUP ) stops reading DDR (DDR is still in the read state, but the read enable signal is turned off), otherwise the read enable is turned on again; when the amount of data in the write FIFO is greater than the level close to full (the write counter is greater than or equal to L WFIFO_WUP ) starts writing DDR; when the amount of data in the read FIFO is less than or equal to the device read delay time (including DDR read delay time and FIFO read data delay time), the data burst length that can be read (the read counter is less than or equal to L RDELAY_DDR ) starts reading DDR.

[0024] To make the problem more specific, it is assumed here that the tensor computing unit can complete 256 multiplication calculations at a time, and is divided into 16 output channels of multipliers. The bit width of the tensor computing unit feature map data input / output port is 128 bits. In the multiplication array of each output channel, the priority is calculated according to the input channel> row> column. The required data transmission time and convolution calculation time of the convolutional neural network accelerator can be used to obtain the following basic calculation formula (not considering the write FIFO and read FIFO):

[0025]

[0026] Among them, T A is the highest clock corresponding cycle of CNNCU, T B L is the average period for transmitting a single byte of data under ideal conditions of DDR. KSize is the convolution kernel area, L IC is the number of input channels, L BUST_CNNCU is the amount of data in the internal cache of the burst access feature map, which is also equal to the amount of data completely consumed by the tensor computing unit last time. B Take the DDR clock cycle, L BUST_CNNCU Take the maximum value L KSize 2 , and usually Much greater than 1, the following ideal conclusion can be obtained:

[0027]

[0028] That is, when the CNNCU clock cycle is equal to 4 times the DDR clock cycle, the internal cache of the feature map will never be empty, and the tensor computing unit can perform calculations continuously. However, due to the delay time of DDR read-write switching, row switching, and device refresh, in actual situations, T B Definitely greater than 4 times the DDR clock period.

[0029] Now we will gradually substitute the actual situation for analysis: if we only consider the consistency of CNCU read data (data buffering function of FIFO), when DDR switches to read state, write FIFO is in a write-only state. When write FIFO is full, data needs to be written to DDR, that is, read FIFO is no longer written, but it still needs to transfer data to CNCU. Therefore, DDR needs to store at least L in read FIFO before entering write state. BUST_CNNCU The amount of data, that is, the read FIFO capacity must be greater than And because DDR has an initial read data delay time. As we know from the above, the capacity of the write FIFO can be:

[0030]

[0031] The capacity of the read FIFO can be:

[0032]

[0033] Then, considering the impact of the actual delay of the device, by taking the minimum value of the write FIFO capacity, taking the maximum value of the read FIFO capacity and adding the correction value (using the read FIFO space in exchange for DDR read and write continuity), we can get the appropriate FIFO capacity and the count value at the switching time, and appropriately reduce the working frequency of CNNCU according to the actual delay situation, that is, we can get the actual working frequency of the tensor computing unit under continuous calculation.

[0034] Now it is known that L BUST_CNNCU The value of has the following cases:

[0035]

[0036] Among them, Ptr0 is the row address where the convolution kernel slides to the lower right corner of the feature map, Ptr1 is its column address, and L1 ISize It is the length or width of the feature map after removing useless data points.

[0037] Therefore, the capacity of the write FIFO can be:

[0038]

[0039] The capacity of the read FIFO can be:

[0040]

[0041] Among them, L SWR and L WSR They are the data burst lengths that can be read within the read-write switching and write FIFO row switching delay times, respectively.

[0042] At the same time, the data width of the write FIFO and read FIFO are both 128 bits.

[0043] In order to improve the operating frequency of the convolutional neural network accelerator, L should be fully utilized when calculating the data transmission bandwidth. BUST_CNNCU The segmented value characteristics of L BUST_CNNCU The average of different values ​​of L Stride 2 , and the value is corrected, then the operating frequency of CNNCU is determined by the following correction formula:

[0044]

[0045] Where T B′ is the average cycle of transmitting single-byte data under the conditions of considering the read FIFO buffering, DDR read / write switching and cross-row read delay, T″ B It is the average cycle for transmitting a single byte of data taking into account the write FIFO buffering, DDR read / write switching, and cross-row write delay.

[0046] in,

[0047]

[0048] At the same time, the relevant parameter values ​​in the DDR controller are as follows:

[0049]

[0050] In the above formula Indicates the average delay time of switching from read to write, Indicates the average delay time of switching from write to read, Indicates the average latency of cross-row writes. Indicates the average latency of cross-row reads. It represents the average delay time of other fixed types of delay of DDR, T SHIFTRW_DDR Indicates the single read-to-write switching delay time, T SHIFTWR_DDR Indicates the delay time of a single write-to-read switch, T SHIFTROW_W_DDR Indicates the delay time of a single cross-row write, T SHIFTROW_R_DDR Indicates the latency of a single cross-row read.

[0051] The memory access circuit inside the feature map data exchange microsystem is a first-in, first-out circuit. This fine-grained dynamic ping-pong caches a portion of the input feature map (the amount of data is L IC ×L ISize ×L KSize , L IC With L ISize is the number and length of input feature map channels, L KSize is the length of the convolution kernel, assuming here that the feature map and the convolution kernel have the same length and width) and then perform a ping-pong access operation: the tensor computing unit completely consumes a portion of the data each time (the amount of data is L BUST_CNNCU , L BUST_CNNCU is a known variable), then the remaining unread data of the input feature graph (the data volume is also L BUST_CNNCU ) is stored in the location where the data is consumed, because the location of the current read data is the light gray area covered by the convolution kernel, the dark gray area is the newly stored data to be read, the white area is the old data to be read, and the black area is the area where data is being written. Because the read area and the write area always slide with the convolution kernel, it is called a fine-grained dynamic ping-pong cache method (ping-pong storage 0 and ping-pong storage 1 perform read-write ping-pong read-write switching).

[0052] Because it is designed in the form of multi-BANK asynchronous dual-port RAM, its capacity needs to be set to a suitable value (C IMEM =L IC ×L ISize ×L KSize ). Every time the tensor calculation unit outputs a result, there is a block with a volume (capacity) of M×N×L IC (M is the length of the consumed data block, N is the width, M×N=L BUST_CNNCU ) slides from the left side of the internal memory to the right side (the length of the internal memory does not really exist, it is a measurement concept, and it is assumed to be equal to the length of the input feature map). When it reaches the rightmost side, it will wrap and continue to slide. The data in the cuboid represents invalid data. If we ignore the discontinuity of writing data, the behavior of writing data can be regarded as filling a ping-pong row of the memory and then wrapping to continue filling (the volume of a ping-pong row is C PPR =L IC ×L ISize ×L Stride ), the filling order inside the ping-pong row is to fill the BANK first, then the column, and then the row. Because M is determined by the hardware resources and the scale of the convolutional neural network, it will not be strictly equal to the length of the internal memory. Therefore, when the internal memory is operated on a first-come-first-out basis, the data will be offset. Consider the following situation: the data is read according to the priority of the image frame line-by-line scanning (input channel> row> column), but is stored in blocks, and the volume of the rectangular block will change. Therefore, only when the read and write pointers of the feature map cache simultaneously use fine-grained dynamic ping-pong pointer forward mapping decoding operations, the system can efficiently complete memory access.

[0053] This forward mapping decoding operation requires the input feature map length to be L Stride n (L Stride is the sliding step size of the current convolutional layer, and the common value is 1 or 2, and n is any natural number. Because the algorithm deployed by the convolutional neural network accelerator is a fully convolutional neural network, and a convolutional layer with a step size of 1 will not change the size of the output image, the unilateral size of the input image is adjusted to L at the inference end. Stride n After L, the input image size of all convolutional layers meets this requirement. From the above analysis, it can be seen that the internal memory access circuit of the feature map data exchange microsystem can be widely used in all fully convolutional neural networks. Stride = 1, memory can be accessed in an address incrementing manner. Stride= 2, since the cuboid (length and width are both 2) representing data consumption slides from left to right until it wraps, this row of data can be used as a ping-pong row. In the preparation stage, the data is first stored in the storage body in an increasing manner according to the address. When writing data to the same ping-pong row for the second time, the address needs to be dynamically mapped: fill the ping-pong row with data according to the priority of BANK> column> row (the width of this column is L Stride , the row length is the same as the internal memory length). When writing data to the same ping-pong row for the third time, the ping-pong row needs to be divided into two blocks (BLOCK0 and BLCOK1) from the row. The dimension priority of the filling address is BANK>BLOCK>column>row. When writing data to the same ping-pong row for the fourth time, the ping-pong row needs to be evenly divided into four blocks (BLOCK0, BLCOK1, BLOCK2, BLOCK3) by row and then filled according to the above address dimension priority. When writing data to the same ping-pong row for the Tth time (the number of BLOCKs is equal to the row length), the address access mode returns to the initial state, that is, the data is accessed in a continuously increasing manner according to the address. Based on this, the dynamic change of address mapping every T times is a cycle, and no more logic is required for control.

[0054] When L ISize =8,L KSize =3,L Stride = 2, the plan view of the first two rows of the internal buffer of the characteristic graph data exchange microsystem is shown in the attached figure. Figure 4 As shown. According to the law of dynamic sliding of the convolution kernel, the write address mapping method of the internal buffer of the feature map will change cyclically in four laws. At the initial moment, the address stores the feature map data in an incremental manner. After the first two rows of data are read by the tensor calculation unit for the first time, they are continuously updated vertically in the order of column>row (two-dimensional address index) (the first ping-pong operation is completed); after the current two rows of data are read by the tensor calculation unit for the second time, they are continuously updated in the two-block memory in the order of block>column>row (the second ping-pong operation is completed); after the current two rows of data are read by the tensor calculation unit for the third time, they are continuously updated in the four-block memory in the order of block>column>row (the third ping-pong operation is completed); after the current two rows of data are read by the tensor calculation unit for the third time, the update method of the write address changes to the rule of row>column priority.

[0055] The convolution kernel data reading microsystem uses non-volatile memory to store data files offline. After the controller, the data is directly read into the convolution kernel buffer, or transmitted to the convolution kernel buffer after passing through FIFO. Here, the SD card is used as the convolution kernel data storage. Since the SD card uses sectors as units for data caching, each burst reads 512 bytes of data, and the convolution kernel buffer needs to read the complete convolution kernel data of 16 output channels (in bytes) for each burst. The two data volumes are not integer multiples, so a cache is needed to temporarily store the remaining data. The storage space is 512*8 bits. If the burst size that the current convolution kernel buffer can cache at one time is less than 512 bytes, then there is no need to use another buffer; if the burst size that the current convolution kernel buffer can cache at one time is greater than 512 but less than half of the convolution kernel buffer capacity, in order not to increase the resources occupied by the cache, time-sharing multiplexing of the remaining convolution kernel buffer storage space should be considered; if the burst size that the current convolution kernel buffer can cache at one time is exactly half of the convolution kernel buffer capacity, then at the critical point before half of the convolution kernel buffer capacity is filled, stop reading the data of the next sector, and wait until the data of the other half of the capacity is calculated before burst writing of data.

[0056] The cross-clock domain output signals of the feature map data exchange microsystem and the convolution kernel data reading microsystem mainly consist of two parts. One is the Req_IExtMem_valid (Req_KExtMem_valid) signal used to indicate that the external memory read enable is valid, and the other is the Req_IMem_ready (Req_KMem_ready) signal that informs the tensor computing unit that it can start computing.

[0057] In fact, Req_IExtMem_valid (Req_KExtMem_valid) is generated by the external memory read clock domain signal and the convolutional neural network working clock domain signal. Because the external memory read clock domain is used as the receiving clock domain, the signals under the convolutional neural network working clock domain (receiving instruction success signal and convolutional neural network related decoding data and read address, etc.) need to be synchronized to the receiving clock domain. Since there is synchronization of multi-bit signals and single-bit signals here, if the general asynchronous FIFO synchronization method and handshake synchronization method are used, the final synchronization circuit is complex and large. First, the single-bit signal is transmitted from the slow clock domain to the fast clock domain, and the receiving instruction success signal will not be valid until the current convolution layer is calculated (multi-cycle signal), so a synchronizer composed of two-stage triggers can be used for synchronization (the receiving clock is not ultra-high frequency); and the synchronization of multi-bit signals can be solved by the MCP method: using the enable signal for synchronization, waiting for stability to access the asynchronous data signal will not cause metastable problems. In order to ensure the high bandwidth of the tensor computing unit, the multi-bit signal synchronization cannot simply use a closed-loop MCP circuit, so it is necessary to analyze the actual situation and improve the MCP design method. The actual circuits for multi-bit signal synchronization in this system are divided into two categories: address signal synchronization circuits and decoded data synchronization circuits. For the synchronization of address signals: Since the data access methods of the internal buffer of the feature map / convolution kernel are similar to those of FIFO, their synchronization generally does not require the use of backpressure signals to the sending clock domain for handshake (first address signal, usually a multi-cycle signal, and the feedback circuit is optional). This is a conservative "full" phenomenon, and with the support of the enable signal, there is no need to use Gray code to encode the address (just what the tensor calculation unit needs to read the data address control circuit). For the synchronization of decoded data, since it is itself a multi-cycle signal, there is no need to backpressure the related signals in the sending clock domain when using the MCP method for synchronization. As for the control circuit in which multiple data in the sending clock domain are synchronized to the receiving clock domain and generate the same enable signal, since the triggering time of these multiple data in the control circuit is far apart, there is no asynchrony between multiple data variables.

[0058] In fact, the synchronization of the Req_IMem_ready (Req_KMem_ready) signal is a pure single-bit signal synchronization problem from the fast clock domain to the slow clock domain, which can be solved by using a synchronizer.

Claims

1. A convolutional neural network accelerator efficient memory access circuit design and implementation, characterized in that: The circuit includes two memory access microsystems: a feature graph data exchange control circuit and a convolution kernel data reading control circuit. The feature map data exchange microsystem includes a DDR read-write control circuit, a write FIFO control circuit, a read FIFO control circuit, and a feature map cache circuit, wherein the first three parts serve as external memory access modules, and the feature map cache circuit is an internal memory access module of the convolutional neural network accelerator. The write FIFO control circuit is the first level (input end) of the external memory access circuit, and is completed using an asynchronous FIFO. The write data input end is switched from the sensor data output and the CNNCU calculation output, and the write data clock is switched by the drive clock of the corresponding data port; The read FIFO control circuit is the third level (output end) of the external memory access circuit. Its data output end is connected to the internal memory access module, and the read data clock is the same as the user clock output by the DDR IP. The DDR read and write control module fully considers the data demand characteristics and memory access bandwidth requirements of the convolutional neural network accelerator to read and write DDR efficiently. In addition, the DDR read and write control circuit customizes the DDR controller for the convolutional neural network accelerator by parameterizing the key data in the read and write control and providing a set of standard interface control methods. Its internal memory access circuit is designed with asynchronous dual-port RAM in the form of multi-BANK, which can be widely adapted to tensor operation units of different computing scales if hardware resources allow. The write data clock frequency of the feature map cache circuit is the same as the user clock output by the DDR-IP, while the read data clock uses the unified working clock of the convolutional neural network accelerator. The synchronization of the cache does not require FIFO synchronization, and can be synchronized in the clock domain through MCP (Multi-Cycle Path, MCP) technology. The key point of the feature map cache circuit is the use of a dynamic pointer mapping circuit, which maps the read and write addresses of the RAM in sequence. With this technical support, no redundant cache circuit is required, and the characteristics of efficient caching can be achieved. At the same time, the pointer mapping circuits on both sides of the RAM read and write have no actual circuit connection relationship, and the decoding operations are performed separately through the same set of standards. This also reduces the coupling of the internal modules of the convolutional neural network accelerator, making it easier to deploy neural networks with different architectures. The convolution kernel data reading microsystem also has two parts: internal and external memory access circuits: its external memory access circuit is mainly used to access non-volatile memories such as SD cards / Flash, and there are fewer factors that affect the data transmission bandwidth. The difficulty in implementation lies in that the SD card performs a continuous read in units of sectors, while the internal memory access requires access in units of single bytes. Therefore, its characteristic point is that a built-in floating cache unit is used to temporarily store data at incompatible time points between continuous reading and single-byte writing.

2. The high-bandwidth feature graph data exchange microsystem according to claim 1, characterized in that: The data failure rate of the feature map buffer and the DDR memory access speed are calculated by standard formulas to minimize the read-write switching and row switching of the DDR memory to improve the DDR memory access speed. At the same time, the standard control method and key parameter values ​​of the DDR controller interface in this scenario (including read-write switching time, write FIFO capacity, read FIFO capacity, read FIFO approximately full flag valid / invalid value, etc.) and the upper limit frequency of the working clock of the convolutional neural network accelerator are given.

3. The internal memory access circuit of the characteristic graph data exchange microsystem according to claim 1, characterized in that: The feature map data is stored in the storage space corresponding to the failed data according to the dynamic fine-grained ping-pong standard, and the positions of the read data block and the write data block change dynamically. Because the tensor calculation described in this patent is a matrix operation based on the Img2col algorithm, when the amount of feature map data consumed by the tensor calculation unit at one time is insufficient to support a whole row of feature map output data, data shift will occur when writing data directly to the feature map, so the read-write pointer cannot be used to directly access the cached data. The internal memory access circuit of the feature map described in this patent uses a fine-grained dynamic ping-pong pointer forward mapping decoding operation on the read-write pointer, so the system can efficiently complete memory access.

4. The convolution kernel data reading microsystem according to claim 1, characterized in that: After reading the data from the low-speed external storage, the internal memory is accessed through a coarse-grained dynamic ping-pong method, and a circuit design method with a built-in floating cache is used to solve the difference in internal and external data access batches. This coarse-grained dynamic ping-pong does not need to refer to the data consumption of the tensor computing unit, and the storage space occupied by the convolution kernel data of the 0-15 and 16-31 output channels is used for ping-pong by default. As for the built-in floating cache circuit, it does not need to occupy excess storage resources, but adopts the idea of ​​time-sharing multiplexing to use the free storage space of the internal data memory at each time node to use the cache to solve the problem of inconsistent data batches.

5. The synchronizer of the feature graph data exchange microsystem and the convolution kernel data reading microsystem according to claim 1, characterized in that: Under the premise of ensuring the maximum bandwidth of the memory access microsystem, the simplest method is used to perform asynchronous signal synchronization processing across clock domains. For the feature map data exchange microsystem and the convolution kernel data reading microsystem, its internal memory uses an asynchronous RAM with different read and write clocks. The write clock is the same as the read clock of the external memory, and the read clock of the internal memory uses the unified working clock in the convolutional neural network accelerator to ensure the maximum data transmission bandwidth during memory access. The synchronizer designed separately outside the internal buffer is used for various cross-clock data interaction scenarios between internal and external memories and between internal memory and tensor accumulation unit. It perfectly solves the defects of asynchronous FIFO as a tensor calculation operand buffer (the same data cannot be accessed multiple times and must be first-in first-out) and the problem that simple asynchronous RAM cannot be synchronized. The most important thing is that its synchronizer structure is simple and saves a lot of circuit resources.

Images