Convolution computing tile architecture based on rram array and neural network acceleration chip

Abstract

The application provides a convolution calculation tile architecture based on an RRAM array and a neural network acceleration chip, which mainly comprises a processing array composed of multiple calculation units, a column condition accumulation module and a row condition accumulation module. The calculation unit is used to realize the multiplication and accumulation operation of the input feature image data to be processed and the weight matrix; the column condition accumulation module is used to accumulate the calculation results of the calculation units in the same column according to the first mark value in the calculation unit; and the row condition accumulation module is used to accumulate the calculation results of the calculation units in the same row according to the second mark value in the calculation unit. The convolution calculation tile supports multiple convolution kernel weight mapping methods and data multiplexing strategies, and multiple calculation units can be combined to realize different scale convolution operations according to the needs, so that the processing efficiency and flexibility of the convolution calculation tile architecture based on the RRAM are improved.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit technology, and in particular to a convolutional computation tile architecture based on an RRAM array and a neural network acceleration chip. Background Technology

[0002] Resistive Random Access Memory (RRAM) is an emerging information device characterized by adjustable resistance, non-volatility, compatibility with CMOS manufacturing processes, and high integration density. By interconnecting RRAMs in a cross-array configuration, adjusting the resistance of each RRAM at the cross-array point to the desired value, clamping the column lines to a reference voltage, and applying a voltage-form signal to each row line, the current generated by the interaction between the input signal and the RRAM, according to Kirchhoff's current law, accumulates on the column lines, which can be considered as completing a multiplication-accumulation operation. This structure, when applied to convolution operations in neural networks, can significantly improve computational efficiency. Currently, several schemes exist for implementing large-scale convolutional neural networks based on the collaborative implementation of multiple RRAM arrays, such as MAX... 2 Examples of existing RRAM-based convolutional computation tile technologies include ISAAC, PRIME, and Pipelayer. However, in these technologies, each RRAM array is relatively independent, and data reuse is not possible within or between arrays. Furthermore, the computation results from each array must be read sequentially, resulting in significant latency and power consumption for both data loading and result retrieval. 2 While it achieves inter-array data reuse during horizontal sliding window convolution, it only supports a single weight mapping method. In summary, existing RRAM-based convolutional computation tile architectures cannot simultaneously achieve both flexibility and efficiency. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a convolutional computation tile architecture and a neural network acceleration chip based on an RRAM array, so as to improve the processing efficiency and flexibility of the RRAM-based convolutional computation tile architecture.

[0004] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0005] In a first aspect, embodiments of the present invention provide a convolutional computation tile architecture based on an RRAM array, comprising: a processing array composed of multiple computation units, a column conditional accumulation module, and a row conditional accumulation module; the computation units are used to perform multiplication and accumulation operations on the input feature image data to be processed and the weight matrix; the column conditional accumulation module is used to accumulate the computation results obtained by the computation units in the same column according to a first marker value in the computation units; the row conditional accumulation module is used to accumulate the computation results obtained by the computation units in the same row according to a second marker value in the computation units.

[0006] In one embodiment, the computing unit includes: a resistive random access memory (RRAM) array, a driving circuit, a data register, a mask register, a column current reading module, an output register, and a computing unit logic control module.

[0007] In one implementation, the MASK register is used to selectively update the data loaded into the data register based on a pre-written mask signal.

[0008] In one implementation, the data register is used to determine the data source of the input feature image data to be processed based on the control signal output by the logic control module of the computing unit; wherein, the data source includes: the input bus, the data register of the computing unit to the right of the same row and / or the data register of the computing unit adjacent below the same column.

[0009] In one embodiment, the calculation unit further includes: a first flag register and a second flag register; the first flag register is used to store a first flag value of the calculation unit; wherein the first flag value is used to mark whether the calculation result of the calculation unit is accumulated with the calculation result of the calculation unit above the calculation unit in the same column; the second flag register is used to store a second flag value of the calculation unit; wherein the second flag value is used to mark whether the calculation result of the calculation unit is accumulated with the calculation result of the calculation unit to the left of the calculation unit in the same row.

[0010] In one implementation, the column condition accumulation module is used to sequentially read the calculation result, first flag value, and second flag value of each calculation unit in the same column from top to bottom, and to determine whether to perform column accumulation of the calculation result of the current calculation unit with the calculation result temporarily stored in the register of the column condition accumulation module based on the first flag value of the current calculation unit, and to update the second flag value temporarily stored in the register of the column condition accumulation module to the read second flag value of the current calculation unit; if the first flag value of the current calculation unit is the first value, then the calculation result of the current calculation unit is accumulated with the calculation result temporarily stored in the register of the column condition accumulation module. The calculation results are accumulated and temporarily stored in the register of the column condition accumulation module. If the first flag value of the current calculation unit is the second value, the calculation result of the current calculation unit is not accumulated with the calculation result temporarily stored in the register of the column condition accumulation module. Instead, the calculation result of the current calculation unit is directly stored in the register of the column condition accumulation module. The calculation results are then determined based on the first flag value of the next calculation unit to see if the accumulation of the column calculation results has been completed. If the first flag value of the next calculation unit is the second value, it indicates that the accumulation of the column calculation results has been completed, and a column accumulation completion flag is generated for the row condition accumulation module to read.

[0011] In one implementation, the row condition accumulation module is used to sequentially read the calculation result, column accumulation completion flag, and second flag value from the register of the column condition accumulation module for each column from left to right, and determine whether to perform row accumulation by adding the calculation result read from the register of the column condition accumulation module to the calculation result temporarily stored in the register of the row condition accumulation module based on the column accumulation completion flag and the second flag value; if the column accumulation completion flag of the column condition accumulation module is not read, the calculation result in the register of the column condition accumulation module is not read; if the column accumulation completion flag of the column condition accumulation module is read and the second flag value is the first value, the calculation result read from the register of the column condition accumulation module is temporarily stored in the register of the row condition accumulation module. The calculation results are accumulated and temporarily stored in the register of the row condition accumulation module. If the column accumulation completion flag of the column condition accumulation module is read and the second flag value is the second value, the calculation results in the register of the column condition accumulation module are not accumulated with the calculation results temporarily stored in the register of the row condition accumulation module. Instead, the calculation result of the currently read column condition accumulation module is directly temporarily stored in the register of the row condition accumulation module. The second flag value of the next column condition accumulation module is used to determine whether the accumulation of the row calculation results has been completed. If the second flag value of the next column condition accumulation module is the second value, it means that the accumulation of the row calculation results has been completed, and the calculation result after the row accumulation is output outside the calculation tile.

[0012] In one implementation, the column condition accumulation module and the row condition accumulation module include multiplexers.

[0013] In a second aspect, embodiments of the present invention provide a neural network acceleration chip, comprising: a plurality of RRAM array-based convolutional computation tiles according to any one of the first aspects; wherein the RRAM array-based convolutional computation tiles are connected to each other via a bus or interconnected via an on-chip network.

[0014] The embodiments of the present invention bring the following beneficial effects:

[0015] The convolutional computation tile architecture and neural network acceleration chip based on an RRAM array provided in this invention include: a processing array composed of multiple computing units, a column conditional accumulation module, and a row conditional accumulation module; the computing units are used for computing the input feature image data to be processed; the column conditional accumulation module is used to accumulate the calculation results obtained by the computing units in the same column according to the first marker value in the computing unit; the row conditional accumulation module is used to accumulate the calculation results obtained by the computing units in the same row according to the second marker value in the computing unit. The above-mentioned convolutional computation tile architecture based on an RRAM array, through the column conditional accumulation module, the row conditional accumulation module, and the two marker values ​​(first marker value and second marker value), can realize the accumulation of calculation results from multiple adjacent computing units, with parallel reading of each column, thus accelerating the reading speed of calculation results; simultaneously, the above-mentioned convolutional computation tile architecture can support multiple convolutional kernel weight mapping methods, and users can determine the number of computing units involved in the calculation and the weight mapping method used as needed, improving the flexibility of the convolutional computation tile in implementing convolution operations. In summary, this invention improves the processing efficiency and flexibility of the RRAM-based convolutional computation tile architecture.

[0016] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1A diagram of a convolutional computation tile architecture based on an RRAM array is provided for an embodiment of the present invention.

[0020] Figure 2 Another convolutional computation tile architecture based on an RRAM array is shown in the embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram illustrating a weight mapping method for convolutional kernels in RRAM, provided by an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of horizontal data multiplexing between PEs provided in an embodiment of the present invention;

[0023] Figure 5 This is a schematic diagram of data reuse provided in an embodiment of the present invention;

[0024] Figure 6 An architecture diagram of a computing unit provided for an embodiment of the present invention;

[0025] Figure 7 A schematic diagram of a data register data loading circuit provided in an embodiment of the present invention;

[0026] Figure 8 This is a schematic diagram illustrating the reading of calculation results and conditional accumulation in an embodiment of the present invention;

[0027] Figure 9 This is a schematic diagram illustrating a data reading method using a pipelined bus, as provided in an embodiment of the present invention.

[0028] Figure 10 This is an example diagram illustrating a multi-PE combination for implementing large-scale convolution operations, provided by an embodiment of the present invention.

[0029] icon:

[0030] 10 - Calculation unit; 20 - Column condition accumulation module; 30 - Row condition accumulation module. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Currently, when implementing large-scale deep neural networks using RRAM arrays, multiple arrays need to work together. The main reasons are: 1) Considering the influence of non-ideal factors such as resistance fluctuations and limited resistance switching ratios of RRAM arrays, it is usually necessary to limit the size of a single array participating in multiply-accumulate operations in order to ensure computational accuracy; 2) To improve computational accuracy, a single network weight usually needs to be represented by multiple RRAMs. When mapping large-scale convolutional kernels, a single RRAM array cannot meet the requirements; 3) When implementing multi-layer convolutional layers, it is usually necessary to map each convolutional layer onto multiple arrays to work in a pipelined manner to improve computational efficiency.

[0033] Currently, there are several implementation schemes based on the collaborative use of multiple RRAM arrays to realize large-scale convolutional neural networks, such as MAX. 2 Examples of RRAM neural network accelerator chips include ISAAC, PRIME, and Pipelayer. A typical RRAM neural network accelerator chip usually consists of multiple interconnected computational tiles. Each computational tile contains one or more RRAM arrays to perform multiplication and accumulation operations, and also includes modules such as a data buffer, a digital-to-analog / analog-to-digital converter (DA / AD) module, a special function unit, and a controller to drive the RRAM array. As can be seen, computational tiles are the basic building blocks of this type of neural network accelerator, and tile performance largely determines chip performance.

[0034] However, the main problems with existing RRAM-based convolutional computation tile technology are as follows:

[0035] (1) Existing RRAM-based computational tile architectures cannot simultaneously achieve flexibility and efficiency. Flexibility refers to the ability of the RRAM arrays within the computational tile to select appropriate weight mapping strategies to meet the needs of convolutions of different scales. Efficiency mainly refers to the ability of the computational tile to support feature image data reuse and rapid accumulation and retrieval of computation results, thereby reducing the time required for data loading and output. In computational architectures represented by ISAAC, Pipelayer, and PRIME, each RRAM array is relatively independent. Although each array can flexibly choose weight mapping methods as needed, data reuse is not possible within or between arrays, and the computation results of each array must be read out sequentially. Therefore, the latency and power consumption required for data loading and result retrieval are relatively large. In contrast, architectures based on MAX... 2The MAX architecture, for example, features a custom design for positional weight mapping, improving data loading and retrieval efficiency through pulsating data propagation and accumulating trees, but it cannot implement other weight mapping strategies. Its hardware architecture limits its support to 3x3 convolutional kernels per channel, while in practice, kernel sizes can be 5x5, 7x7, etc. Although MAX... 2 Increasing the number of PEs in each computational tile to 5*5 or 7*7 can solve this problem, but its input feature map (IFM) data loading and output feature map (OFM) data output patterns make it difficult to flexibly apply to different convolutions and scales as needed.

[0036] (2) Limited data reuse methods. In computing architectures such as ISAAC, Pipelayer, and PRIME, the feature image data of each array needs to be reloaded each time, thus eliminating data reuse. In MAX... 2 In this approach, partial data reuse within adjacent convolutional sliding windows is achieved by horizontally transferring IFM data. However, this data reuse method can only be applied when the convolutional kernel weights are mapped to the nine PEs according to their positions. In some embedded applications, the convolutional kernels used are typically small. To improve computational efficiency, it is necessary to fully expand and map the convolutional kernels within a single processing unit. In this case, the data loaded in each row of the RRAM array needs to be moved vertically to achieve data reuse, but this reuse method cannot be implemented in the current architecture.

[0037] (3) The method for quickly accumulating and reading out multi-array computation results is limited. In computing architectures such as ISAAC, Pipelayer, and PRIME, the computation results of each array must be read out sequentially and individually, therefore there is no fast accumulation and reading out solution. In MAX 2 In this case, the fixed hardware architecture calculates the 9 PEs in the computation tile by accumulating them according to their corresponding columns, which cannot cope with the situation where multiple arrays are flexibly combined to achieve different convolution kernels.

[0038] Based on this, the present invention provides a convolutional computation tile architecture and neural network acceleration chip based on an RRAM array, which can improve the processing efficiency and flexibility of the RRAM-based convolutional computation tile architecture.

[0039] To facilitate understanding of this embodiment, a detailed description of a convolutional computation tile architecture based on an RRAM array disclosed in this embodiment is provided first. This convolutional computation tile supports various convolutional kernel weight mapping methods and data reuse strategies, and can combine multiple computational units to implement convolutional operations of different scales as needed. See [link to documentation]. Figure 1The diagram shows a convolutional computation tile architecture based on an RRAM array. The architecture includes: a processing array composed of multiple computation units 10 (PEs), a column conditional accumulation module 20, and a row conditional accumulation module 30. The computation units 10 perform multiplication and accumulation operations on the input feature image data to be processed and the weight matrix. The column conditional accumulation module 20 accumulates the computation results obtained by the computation units 10 within the same column based on a first marker value in the computation units 10. The row conditional accumulation module 30 accumulates the computation results obtained by the computation units 10 within the same row based on a second marker value in the computation units 10.

[0040] The convolutional computation tile architecture based on an RRAM array provided in this invention, through column conditional accumulation modules, row conditional accumulation modules, and two marker values ​​(first and second marker values), enables the accumulation of computation results from multiple adjacent computation units. Parallel reading of each column accelerates the reading speed of computation results. Simultaneously, this convolutional computation tile architecture supports various convolutional kernel weight mapping methods, allowing users to determine the number of computation units involved and the weight mapping method as needed, thus improving the flexibility of the convolutional computation tile in implementing convolution operations. In summary, this invention improves the processing efficiency and flexibility of the RRAM-based convolutional computation tile architecture.

[0041] Furthermore, the above-mentioned convolutional computation tile architecture in Figure 1 In addition to the above, it also includes: a tile caching module, an output feature image data processing module, and a control logic module; the tile caching module is used to cache the input feature image data to be processed; the output feature image data processing module is used to perform batch normalization, pooling, and / or activation processing on the accumulated calculation results; the control logic module is used for the coordinated control between each module.

[0042] See Figure 2 The diagram shows a convolutional computation tile architecture based on an RRAM array. This convolutional computation tile includes a processing array consisting of N rows and M columns of PEs (Predicted Image Processing Units), as well as a Tile cache (i.e., a tile cache module), control logic (i.e., a control logic module), column condition accumulation (i.e., column condition accumulation module 20), row condition accumulation (i.e., row condition accumulation module 30), and OFM data processing module (i.e., output feature image data processing module). The Tile cache is used to cache the IFM data to be processed. The IFM data may be loaded externally or generated internally within the computation tile. The column condition accumulation module can accumulate the PE calculation results within the same column according to the conditions set in the tile register. The row condition accumulation module can accumulate the PE calculation results within the same row according to the conditions set in the tile register. The OFM data processing module can perform batch normalization, activation, pooling, and other processing on the convolutional results as needed.

[0043] The following describes the convolution kernel weight mapping scheme and data reuse scheme supported by the RRAM array-based convolution computation tiles provided in the embodiments of the present invention.

[0044] like Figure 3 As shown, depending on the size of the RRAM array and the size of the convolutional kernel, the network weights may be mapped using full expansion, positional expansion, or row / column expansion to fully utilize the RRAM units in the RRAM array. Full expansion involves unfolding the convolutional kernel into a vector and mapping it to a column of the RRAM array. N convolutional kernels of size K*K*C will occupy N columns of the RRAM array, with each column having a length of K*K*C. Positional expansion maps the weights in all channels at the same position of the convolutional kernel to a column of the RRAM array. N convolutional kernels of size K*K*C require K*K RRAM arrays, occupying C rows and N columns in each array. Row / column expansion is a compromise, unfolding all weights belonging to the same row or column of the convolutional kernel into a vector and mapping it to a column of the RRAM array. N convolutional kernels of size K*K*C require K RRAM arrays, occupying K*C rows and N columns in each array. When the required number of rows in the RRAM array is large, or when a single convolution kernel is mapped onto multiple arrays, it is necessary to accumulate the calculation results of the corresponding columns of multiple RRAM arrays. The convolution calculation tiles provided in this embodiment of the invention support all of the above-mentioned weight mapping schemes.

[0045] During convolution calculation, the convolution kernel slides horizontally and vertically across the IFM data with a certain stride (usually 1), performing convolution calculations with the IFM data within the sliding window until the entire IFM image has been traversed. Within adjacent sliding windows, some data is repeated. To improve the reuse rate of data in the overlapping parts of the sliding windows and to increase the efficiency of convolution result accumulation, it is necessary to be able to achieve horizontal data shifting between PEs within the same row.

[0046] When using the position-based expansion mapping scheme, as shown in Figure 4, the data within a 3x3 window of the IFM is loaded into nine adjacent PEs. When the convolution window slides, it is only necessary to shift the data in the rightmost PE one column to the left sequentially, and then update the data to be processed in the rightmost PE. It is important to note that... Figure 4 The image only shows the data within one IFM channel; in reality, each PE can load data from multiple channels.

[0047] When using a fully expanded mapping scheme, to facilitate data reuse within adjacent convolutional windows, it is necessary to enable data shifting in the vertical direction within the same PE, as well as data shifting between adjacent PEs within the same column. Figure 5Taking the example shown, the processed IFM image contains only a single channel, requiring 16 convolutional kernels, each with a size of 5*5. The weight mapping method used is to fully expand each convolutional kernel into a vector of length 25 and map it to one column of the RRAM array. All 16 convolutional kernels occupy 16 columns in the RRAM array, requiring an RRAM array size of only 25*16. To improve the utilization of the RRAM array, the following can be used: Figure 5 As shown, the convolution kernels are repeatedly mapped in a staggered manner, allowing for parallel convolution operations across three paths. After the current convolution window calculation is complete, the IFM data needs to be updated. This can be done by moving the data loaded on each row of the RRAM array one row up sequentially, and then updating a data point every four rows (the fifth row) (as shown in Figure 5, 13, 21, 29, 37, 45, 53).

[0048] In one embodiment, the computing unit includes: a resistive random access memory (RRAM) array and a driving circuit; the RRAM array is used for convolution calculation between the input feature image data to be processed and the convolution kernel weights; the driving circuit is used to drive the RRAM array to perform convolution calculation. Specifically, each PE contains an RRAM array and the peripheral circuitry required for the driving array to perform multiplication-accumulation operations.

[0049] See Figure 6 The diagram shows the architecture of the computing unit, which includes: an RRAM array, a data register, a mask register, a column current reading module, an output register, and a computing unit logic control module. The data register stores the input feature image data to be processed. The mask register selectively updates the data loaded into the data register according to pre-written mask signals. The column current reading module reads the current signal of each column of the RRAM array during the calculation process and converts the current signal into a digital signal. The output register stores the calculation results of the RRAM array and outputs the calculation results to the column condition accumulation module. The computing unit logic control module controls the computing unit to perform calculations according to a preset timing sequence.

[0050] In one implementation, the size of the RRAM array can be selected according to requirements; here, it can be assumed to be H rows and W columns. The data register temporarily stores the IFM data to be processed, and the amount of data that can be cached is H (the same as the number of rows in the RRAM array). The column current reading module reads the current on each column of the RRAM array when performing multiplication and accumulation operations and converts it into a digital signal. The multiplication and accumulation calculation result obtained by the PE can be temporarily stored in the output register and output to the column condition accumulation module under external control logic. The PE control logic (i.e., the calculation unit logic control module) can control the above circuit to complete the relevant actions according to the designed timing.

[0051] It's important to note that the PE contains a MASK register, where each bit corresponds to a data register. If the bit is 1, data can be written to the corresponding data register; if the bit is 0, data cannot be written to the corresponding data register. This is because, to improve the loading speed of the data to be processed, the input bus typically has a large bit width (e.g., 32 bits), while the precision of the data to be processed (which is also the data register bit width) is typically low (e.g., 8 bits). Therefore, data can be written to multiple adjacent (e.g., 4) data registers simultaneously. Figure 8 The data reuse requirement shown requires updating only a few pieces of data each time the convolution window slides. The adjacent data are updated by shifting the already loaded data, thus eliminating the need for loading via the input bus. To meet this requirement, the necessary masking signals can be pre-written to the MASK register according to the update requirements to prevent the contents of the relevant data registers from being overwritten by data sent in via the input bus.

[0052] Furthermore, to achieve the aforementioned data reuse scheme, the data register provided in this embodiment of the invention is also used to determine the data source of the input feature image data to be processed based on the control signal output by the logic control module of the computing unit; wherein, the data source includes: the input bus, the data register of the computing unit to the right of the same row and / or the data register of the computing unit adjacent below the same column.

[0053] See Figure 7 As shown, each data register in the processing array can receive data from three directions: the input bus, the data register of the PE in the same row of the right array, and the data register of the PE in the adjacent row below. The specific data source can be controlled by the sel signal (i.e., the control signal output by the logic control module of the computing unit). Among them, writing data through the input bus has the highest priority. That is, when the data source is selected as the data register of the PE in the same row of the right array or the data register of the PE in the adjacent row below, data can still be written through the input bus. This priority setting can avoid frequent modification of the sel signal value during the update of the data to be processed.

[0054] To enable flexible combinations of computational units (PEs) to achieve convolutional kernels of different sizes, the PEs provided in this embodiment of the invention include two flag registers: a first flag register and a second flag register. The first flag register stores a first flag value for the computational unit, indicating whether the computation result of the computational unit is accumulated with the computation result of the computational unit above it in the same column. The second flag register stores a second flag value for the computational unit, indicating whether the computation result of the computational unit is accumulated with the computation result of the computational unit to the left in the same row. That is... Figure 6The C_flag (first flag register) and R_flag (second flag register) shown in the diagram are used to indicate whether the PE calculation result should be added to the adjacent PE calculation result above it in the same column; and whether the R_flag should be added to the adjacent PE calculation result to its left in the same row. When the calculation result is output to the column conditional accumulation module, C_flag and R_flag are output along with the calculation result.

[0055] When the convolution kernel size is large, it needs to be mapped to multiple PEs. Accordingly, the calculation results of these PEs need to be accumulated to obtain the complete convolution calculation result. Based on this, the computational tiles provided in this embodiment of the invention add a column conditional accumulation module to each column of PEs and a row conditional accumulation module before the OFM data processing module. The calculation result of each PE is output with C_flag and R_flag.

[0056] In one implementation, the column condition accumulation module is used to read the calculation result, first flag value and second flag value of each calculation unit in the same column from top to bottom, and to determine whether to perform column accumulation of the calculation result of the current calculation unit with the calculation result temporarily stored in the register of the column condition accumulation module based on the first flag value of the current calculation unit, and to update the second flag value temporarily stored in the register of the column condition accumulation module to the second flag value of the current calculation unit read.

[0057] If the first flag value of the current calculation unit is the first value, then the calculation result of the current calculation unit is added to the calculation result temporarily stored in the register of the column condition accumulation module, and the added calculation result is temporarily stored in the register of the column condition accumulation module. If the first flag value of the current calculation unit is the second value, then the calculation result of the current calculation unit is not added to the calculation result temporarily stored in the register of the column condition accumulation module, and the calculation result of the current calculation unit is directly temporarily stored in the register of the column condition accumulation module. The first flag value of the next calculation unit is used to determine whether the accumulation of the column calculation result has been completed. If the first flag value of the next calculation unit is the second value, it means that the accumulation of the column calculation result has been completed, and a column accumulation completion flag is generated for the row condition accumulation module to read.

[0058] In practical applications, the column condition accumulation module reads the calculation result, first flag value and second flag value of each calculation unit in the same column from top to bottom. Based on the first flag value, it determines whether to accumulate the calculation result of the currently read calculation unit with the calculation result temporarily stored in the register of the column condition accumulation module, and updates the second flag value temporarily stored in the column condition accumulation module to the newly read second flag value.

[0059] If the first flag value is 1 (i.e., the first value), column accumulation of the calculation results is required. The calculation result of the currently read calculation unit is added to the calculation result temporarily stored in the register of the column condition accumulation module, and the accumulated calculation result is temporarily stored in the register of the column condition accumulation module. If the first flag value is 0 (i.e., the second value), column accumulation of the calculation results is not required, and the calculation result of the currently read calculation unit is directly temporarily stored in the register of the column condition accumulation module. At the same time, the column condition accumulation module can determine whether the accumulation of the required column calculation results has been completed based on the first flag value of the next calculation unit. If the first flag value of the next calculation unit is 0, it means that the accumulation has been completed, and a column accumulation completion flag is generated for use by the row condition accumulation module.

[0060] The row condition accumulation module is used to read the calculation result, column accumulation completion flag and second flag value from the register of the column condition accumulation module of each column from left to right, and to determine whether to perform row accumulation by adding the calculation result read from the register of the column condition accumulation module to the calculation result temporarily stored in the register of the row condition accumulation module based on the column accumulation completion flag and the second flag value.

[0061] If the column accumulation completion flag of the column condition accumulation module is not read, the calculation result in the register of the column condition accumulation module is not read. If the column accumulation completion flag of the column condition accumulation module is read, and the second flag value is the first value, the calculation result in the register of the column condition accumulation module is accumulated with the calculation result temporarily stored in the register of the row condition accumulation module, and the accumulated calculation result is temporarily stored in the register of the row condition accumulation module. If the column accumulation completion flag of the column condition accumulation module is read, and the second flag value is the second value, the calculation result in the register of the column condition accumulation module is not accumulated with the calculation result temporarily stored in the register of the row condition accumulation module. The calculation result of the currently read column condition accumulation module is directly temporarily stored in the register of the row condition accumulation module. The second flag value of the next column condition accumulation module is used to determine whether the accumulation of the row calculation result has been completed. If the second flag value of the next column condition accumulation module is the second value, it means that the accumulation of the row calculation result has been completed, and the calculation result after the row accumulation is output outside the calculation tile.

[0062] In practical applications, the row condition accumulation module reads the calculation result, column accumulation completion flag, and second flag value from the register of the column condition accumulation module for each column from left to right. Based on the column accumulation completion flag, it determines whether to start row condition accumulation, and based on the second flag value, it determines whether to accumulate the calculation result read from the register of the column condition accumulation module with the calculation result temporarily stored in the register of the row condition accumulation module.

[0063] If the column accumulation of a certain column is not yet complete (i.e., the column accumulation completion flag of the column condition accumulation module has not been read), the calculation result in the register of that column accumulation module is not read. If the column accumulation of a certain column is complete (i.e., the column accumulation completion flag of the column condition accumulation module is read), and the second flag value of that column condition accumulation module is 1, then the calculation result read from that column accumulation module needs to be accumulated with the result temporarily stored in the row condition accumulation module, and the accumulated calculation result is temporarily stored in the register of the row condition accumulation module. If the column accumulation is complete but the second flag value of that column condition accumulation module is 0, then there is no need to perform row accumulation of the calculation result, and the calculation result of the currently read column condition accumulation module is directly temporarily stored in the register of the row condition accumulation module. At the same time, the row condition accumulation module can determine whether the required row accumulation has been completed based on the second flag value of the next column condition accumulation module. If the second flag value of the next column condition accumulation module is 0, it means that the row accumulation has been completed, and the row accumulation result can be output outside the calculation tile.

[0064] In one implementation, the first flag value and the second flag value are C_flag and R_flag, respectively. When C_flag is 1, it indicates that the calculation result of this PE needs to be added to the calculation result of the PE above it; if it is 0, no addition is needed. When R_flag is 1, it indicates that the calculation result of this PE needs to be added to the calculation result of the PE to its left; if it is 0, no addition is needed. Specifically, in each column, the column condition accumulation module reads the calculation result of each PE from top to bottom through a multiplexer, and determines whether the data needs to be added to the value previously stored in the register of the column condition accumulation module based on C_flag. If addition is needed, the accumulated calculation result is stored in the register of the column condition accumulation module. Figure 8 As shown, PE0's C_flag is 0, and PE1's C_flag is 1. When the column condition accumulation module reads the calculation result and C_flag of PE0, since PE0's C_flag is 0, no accumulation is needed; the calculation result of PE0 is directly stored in the register within the column condition accumulation module. Similarly, when the column condition accumulation module reads the calculation result and C_flag of PE1, since PE1's C_flag is 1, it needs to accumulate the result with PE1's calculation result and store the accumulated result in the register within the column condition accumulation module. Likewise, the calculation results of PE2 and PE3 need to be accumulated, while the results of PE4 and PE5 do not need to be accumulated. It is important to note that the column condition accumulation modules in all columns need to start reading calculation results synchronously. The column condition accumulation module synchronously transmits the C_flag read in the current cycle and the R_flag obtained in the previous cycle to the row condition accumulation module.

[0065] Each time the column condition accumulation module reads a PE result, the row condition accumulation module checks the C_flag value of each column's output to determine if the calculation results of the previous few PEs in that column have been accumulated. If C_flag in a certain column is still 1 (e.g....), Figure 8 If the value is PE1_result and corresponds to C_flag, it indicates that the accumulation within the same column has not yet been completed. If it is 0 (e.g., ...), then the accumulation within the same column has not yet been completed. Figure 8 The `C_flag` corresponding to `PE2_result` indicates that the data in this column does not need to be accumulated with the calculation results of the PEs above it (PE0 and PE1). Then, it checks if the output `R_flag` of each column is 1. If it is 1, it means that the data needs to be accumulated with the calculation results of the PE to its left. The row conditional accumulation module accumulates the calculation results of columns where `R_flag` is consecutively 1. When the row conditional accumulation module encounters a column with `R_flag` of 0, it indicates that the required accumulation has been completed, and the accumulation result can be sent to the tile buffer for output outside the tile or sent again to other PEs for the next layer of convolution operation.

[0066] When accumulating the above PE calculation results, the method of accumulating column conditions first and then row conditions is adopted. In addition to the above method, the method of accumulating row conditions first and then column conditions can also be adopted in the embodiments of the present invention. Specifically, in the PE calculation result reading circuit, the row condition accumulation module is first used to accumulate the calculation results of PE in the same row as needed, and then the column condition accumulation module is used to accumulate the calculation results of PE in the same column as needed.

[0067] Furthermore, in the above embodiment, when the column condition accumulation module reads the calculation result of the PE in its current column, and when the row condition accumulation module reads the calculation result of the PE in its current column, a multiplexer is used to select the target PE and target column to be read. In specific implementation, a pipelined bus approach can also be used for reading to reduce the difficulty of chip placement and routing. The specific implementation method is as follows: Figure 9 As shown in the diagram, the calculation results of each PE are first written to each level of Bus_stage, and then passed to the column condition accumulation module level by level under clock drive. The row condition accumulation module can also adopt a similar approach. Since this implementation only modifies the result reading process, condition accumulation still needs to be performed according to the method designed in this invention.

[0068] In practical applications, different convolutional kernel sizes result in different combinations of PEs (Performance Components) participating in the accumulation; however, the PEs participating in the accumulation must be able to form a rectangle. See also... Figure 10 The examples shown illustrate different combinations of multiple PEs to implement large-scale convolution operations. Based on the column-conditional accumulation and row-conditional accumulation methods proposed in this invention, convolution kernels of various sizes can be flexibly implemented. Figure 10In the case shown in (a), all PEs implement a set of convolutional kernels, and all required accumulation calculations can be completed through 10 accumulations (5 in the vertical direction and 5 in the horizontal direction); Figure 10 In the scenario shown in (b), all PEs implement four sets of convolutional kernels, and all required accumulation calculations can be completed through 12 accumulations (2 accumulations in the vertical direction + 4 accumulations in the horizontal direction + 2 accumulations in the vertical direction + 4 accumulations in the horizontal direction); Figure 10 In the case shown in (c), each row of PE implements a set of convolutional kernels, and six sets of PE accumulation results can be obtained through 30 accumulations (each set of PE is accumulated horizontally 5 times); similarly, in Figure 10 In the case shown in (d), each column of PE implements a set of convolutional kernels, and six sets of PE accumulation results can be obtained through 30 accumulations; while Figure 10 In the scenario shown in (e), all PEs implement three sets of convolutional kernels (one set of 6x3 and two sets of 3x3 convolutional kernels), requiring 10 accumulations (2 accumulations in the vertical direction + 2 accumulations in the horizontal direction + 2 accumulations in the vertical direction + 4 accumulations in the horizontal direction) to complete all the necessary accumulation calculations; Figure 10 In the case shown in (f), all PE calculation results are independent and do not need to be accumulated.

[0069] After the column-conditional and row-conditional summations of the calculation results of each PE are performed, the summation results are sent to the OFM data processing module. As needed, batch normalization, pooling, activation and other processing are performed, and the processed results are output to the tile cache module for output outside the tile or reloaded into the tile for the next round of processing (when adjacent multi-layer convolution is implemented within the tile). The control logic module realizes the coordinated control of each circuit module within the tile.

[0070] The convolution calculation tiles based on RRAM arrays provided in the embodiments of the present invention have the following technical effects:

[0071] (1) Supports flexible combination of multiple PEs to achieve convolution operations of different scales. Based on two flag bits, C_flag and R_flag, and conditional accumulation logic circuit, this invention supports the accumulation of calculation results of multiple adjacent PEs. Multiple small-scale RRAM arrays can be spliced ​​into a large-scale array to meet the needs of convolution calculations of different scales.

[0072] (2) Supports multiple weight mapping methods. Since the present invention does not use a pulsed data loading and result reading circuit with a fixed connection method, and supports result accumulation between any number of PEs (but the area composed of PEs must be rectangular), users can select the appropriate weight mapping mode as needed. Compared with the prior art, the computing tiles provided by the present invention can be applied to more application scenarios.

[0073] (3) Supports horizontal and vertical data reuse, reducing off-chip data loading. The IFM data loading method provided by this invention supports the reuse of data within adjacent convolutional sliding windows. Each data register can receive data from the input bus, the same row of data registers in the PE to its right, and the data registers of the adjacent PE below. Therefore, the data to be processed that has been loaded into the tile can be moved horizontally and vertically to update the data. Only a portion of the data to be processed needs to be read from outside the chip to update the data registers in the rightmost or bottommost PE, thereby greatly reducing the amount of data that needs to be loaded from outside the chip.

[0074] (4) Parallel reading of each column speeds up the reading speed of calculation results. In this embodiment of the invention, when it is necessary to accumulate the calculation results of multiple PEs within a tile according to their corresponding columns, on the one hand, the column condition accumulation module can read the calculation results of each column in parallel, and on the other hand, the column results can be accumulated as needed during the result output process. Therefore, the reading speed is improved. In extreme cases, when all PE results within a tile need to be accumulated, the calculation result reading only requires N+M reads and N+M-2 accumulations. Where N is the number of rows of PEs within the tile; M is the number of columns of PEs within the tile; a single read means that the column condition accumulation module reads one PE calculation result. If all calculation results within a PE are read at once, each read takes one clock cycle. If only a portion of the results are read each time, the number of clock cycles for each read increases proportionally.

[0075] This invention also provides a neural network acceleration chip, comprising: a plurality of RRAM array-based convolutional computation tiles provided in the foregoing embodiments; wherein the RRAM array-based convolutional computation tiles are connected to each other via a bus or interconnected via an on-chip network.

[0076] The neural network acceleration chip provided in this embodiment of the invention, through a column condition accumulation module, a row condition accumulation module, and two marker values ​​(a first marker value and a second marker value), can accumulate the calculation results of multiple adjacent computing units, with parallel reading of each column, thus accelerating the reading speed of the calculation results. Simultaneously, the convolutional computing tile supports multiple convolutional kernel weight mapping methods, allowing users to determine the number of computing units involved and the weight mapping method used as needed, improving the flexibility of the convolutional computing tile in implementing convolution operations. In summary, this invention improves the efficiency and flexibility of the RRAM-based convolutional computing tile architecture.

[0077] It should be noted that the neural network acceleration chip provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned convolutional computation tile embodiment. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the aforementioned convolutional computation tile embodiment.

[0078] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A convolutional computation tile architecture based on an RRAM array, characterized in that, include: A processing array composed of multiple computing units, a column condition accumulation module, and a row condition accumulation module; The computing unit is used to perform multiplication and accumulation operations on the input feature image data to be processed and the weight matrix; The column condition accumulation module is used to accumulate the calculation results obtained by the calculation units in the same column according to the first marker value in the calculation unit; wherein, the first marker value is used to mark whether the calculation result of the calculation unit is accumulated with the calculation result of the calculation unit above the adjacent calculation unit in the same column; The row condition accumulation module is used to accumulate the calculation results obtained by the calculation units in the same row according to the second flag value in the calculation unit; wherein, the second flag value is used to mark whether the calculation result of the calculation unit is accumulated with the calculation result of the calculation unit to the left of the calculation unit in the same row; The column condition accumulation module is used to read the calculation result, the first flag value, and the second flag value of each calculation unit in the same column from top to bottom, and to determine whether to perform column accumulation by adding the calculation result of the current calculation unit to the calculation result temporarily stored in the register of the column condition accumulation module based on the first flag value of the current calculation unit, and to determine whether to generate a column accumulation completion flag based on the first flag value of the next calculation unit, and to update the second flag value temporarily stored in the register of the column condition accumulation module to the second flag value of the current calculation unit read. The row condition accumulation module is used to read the calculation result, the column accumulation completion flag and the second flag value of the register of the column condition accumulation module in each column from left to right, and to determine whether to perform row accumulation by adding the calculation result read from the register of the column condition accumulation module to the calculation result temporarily stored in the register of the row condition accumulation module based on the column accumulation completion flag and the second flag value.

2. The convolutional computation tile architecture according to claim 1, characterized in that, The computing unit includes: a resistive random access memory (RRAM) array, a driving circuit, a data register, a mask register, a column current reading module, an output register, and a computing unit logic control module.

3. The convolution computation tile architecture according to claim 2, characterized in that, The MASK register is used to selectively update the data loaded into the data register according to a pre-written masking signal.

4. The convolution computation tile architecture according to claim 2, characterized in that, The data register is used to determine the data source of the input feature image data to be processed based on the control signal output by the logic control module of the computing unit; wherein, the data source includes: the input bus, the data register of the computing unit on the right side of the same row and / or the data register of the computing unit adjacent below in the same column.

5. The convolution computation tile architecture according to claim 4, characterized in that, The computing unit further includes: a first tag register and a second tag register; The first tag register is used to store the first tag value of the computing unit; The second tag register is used to store the second tag value of the computing unit.

6. The convolution computation tile architecture according to claim 1, characterized in that, The column condition accumulation module and the row condition accumulation module include multiplexers.

7. A neural network acceleration chip, characterized in that, include: The convolutional computation tile architecture based on RRAM arrays as described in any one of claims 1 to 6; wherein the convolutional computation tile architectures based on RRAM arrays are connected by a bus or interconnected by an on-chip network.

Images