A weight mapping method based on a ReRAM neural network accelerator

Abstract

The application discloses a weight mapping method based on a ReRAM neural network accelerator and belongs to the field of neural network accelerators, and the method comprises the following steps: step 1: taking a bank of a ReRAM array as a first mapping priority and using the bank to place a convolution input channel; step 2: taking a bank-row direction of the ReRAM array as a second mapping priority, placing convolution kernels of different channels on different bank-row positions in the output channel dimension of the convolution kernel until the data mapping requirements of all input and output channels are met; and step 3: each weight data is arranged in a cross circuit of a calculation array according to a specific order rule of bits. The application proposes a novel high-parallelism weight mapping algorithm design, which greatly excavates the parallelism of convolution and matrix multiplication calculation and also reduces the hardware complexity.

Description

Technical Field

[0001] This invention relates to the field of neural network accelerators, and more particularly to a weight mapping method based on a ReRAM neural network accelerator. Background Technology

[0002] With the rapid growth of data volume, the demand for hardware computing power and data storage capacity in memory-intensive neural network applications is also increasing. In order to overcome the memory access bandwidth bottleneck caused by the separation of computation and storage in the traditional von Neumann architecture, more and more research is beginning to focus on high-density in-memory computing architectures. The aim is to maximize the energy efficiency of the hardware architecture by tightly coupling computation and storage, reducing the additional memory access energy consumption and bandwidth requirements caused by the frequent movement of data between storage and computation components.

[0003] Emerging non-volatile memory—Resistive Randomized RAM (ReRAM)—uses electrical conductance values ​​for storage and organizes multiplication and accumulation calculations using Ohm's law, enabling in-situ computation capabilities not found in traditional memory. Traditional accelerators, based on the von Neumann architecture, require simultaneously transferring weights and features for computation. ReRAM-based neural network accelerators, however, transform input vectors into voltages applied to the word lines of the ReRAM. Another weight input is pre-mapped to the conductance of each ReRAM cell, and the accumulated current on the bit lines is used to represent the dot product of the two vectors, achieving in-situ processing and significantly reducing data bandwidth pressure to improve performance.

[0004] Currently, mainstream in-memory computing accelerators often employ a hierarchical topology, with circuits divided into functional modules of varying sizes from top to bottom. The smallest granularity computing module in the architecture is a ReRAM cross-connect array and an arithmetic logic unit (ALU). These basic units are combined to form upper-level computation modules, which can then be combined with each other, and so on up to the top layer of the architecture. Based on the scheduling and driving methods of the modules, existing accelerator systems can be divided into two categories: dataflow-driven and instructionflow-driven.

[0005] Dataflow-driven architectures refer to architectures where data access and interaction between functional modules are controlled by state machines during computation, without instruction interference. In neural network inference, once the network structure and weights are determined, the scale of data and execution mode in the entire computation are also determined. Dataflow-driven in-memory computing architectures target this characteristic, mapping the network layer structure, connections, and weight values ​​to corresponding functional arrays before execution. The network inference process can then be achieved through input / output control during execution. Typical accelerators for this type of architecture include ISAAC and PRIME.

[0006] Patent CN202110406885.5 discloses a reduced instruction set in-memory neural network coprocessor based on resistive-mode memristors. It utilizes a ReRAM array to perform fast and accurate convolution operations in memory, accelerating the numerous multiplication and accumulation calculations present in neural networks. This reduces power consumption and time overhead caused by data movement, saving hardware resources. Patent CN202111061410.3 discloses a design for an excess row activation in-memory accelerator based on data sparsity. It constructs a prediction mechanism based on row activation data, models the limitations of peripheral circuit devices and computational parallelism to solve the matching problem between the peripheral circuits and the computational parallelism; it constructs a row activation excess subscription mechanism to adaptively adjust the computational parallelism and resource usage to solve the problems of low utilization and resource redundancy of the computing array and peripheral circuits under sparse data; and it re-plans the control flow and data flow to address the complex circuit design problems introduced by utilizing data sparsity. However, neither of these patents considers the optimization problem of weight mapping in the ReRAM computing array.

[0007] The current mainstream ReRAM weight mapping mode also has problems: 1. The weight mapping mode is relatively fixed, and the mapping algorithm is generally based on the unfolding mode of the weight input channel and output channel, which has poor flexibility; 2. The inherent requirements of the mapping algorithm lead to complex hardware structure, and the ReRAM addressing process needs to traverse all dimensions of convolution calculation. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and propose a weight mapping method based on a ReRAM neural network accelerator.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] A weight mapping method based on a ReRAM neural network accelerator includes the following steps:

[0011] Step 1: Use the banks of the ReRAM array as the first mapping priority to place the convolution input channels;

[0012] Step 2: Using the Bank-Row direction of the ReRAM array as the second mapping priority, place the convolution kernels of different channels in different Bank-Row positions along the output channel dimension of the convolution kernel until the data mapping requirements of all input and output channels are completed.

[0013] Step 3: Each weight data is arranged in the cross circuit of the computing array according to a specific bit order rule.

[0014] Furthermore, the specific order in step 3 includes either a high-to-low or low-to-high order.

[0015] Furthermore, the weighted data in step 3 are evenly distributed in the same Bank-Row across different Arrays.

[0016] Furthermore, the ReRAM array consists of multiple banks and is divided into multiple array groups. Each bank contains multiple bank-rows, and the ReRAM array maps bit data through ReRAM devices.

[0017] Furthermore, each of the ReRAM devices represents 2 bits of data during mapping.

[0018] Furthermore, the output of the ReRAM array is a multi-level adder tree, used to accumulate partial sums of outputs within and between ReRAM arrays.

[0019] The beneficial effects of this invention are:

[0020] (1) A mapping scheme with Bank as the core is adopted, which extends and broadens the computational parallelism in the dimension of the array Bank.

[0021] (2) It solves the problem of complex data stream addressing. ReRAM addressing presents a three-dimensional form, namely Bank group, Bank-Row, and Bank col. The original algorithm needs to traverse each dimension. In this invention, the addressing is reduced to two dimensions, namely Bank group and Bank-Row, and the addressing logic is simpler.

[0022] (3) Simplify the array size, while taking into account the flexibility of the mapping algorithm and reducing the number of array addition tree levels;

[0023] It simplifies the complexity of the control logic and reduces the area and power consumption of the control circuit. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0025] Figure 1 This is a flowchart of the method of the present invention.

[0026] Figure 2 This is a schematic diagram of the mapping process for traditional convolution.

[0027] Figure 3 This is a schematic diagram of the execution process for traditional convolution. Detailed Implementation

[0028] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] This invention addresses the ReRAM accelerator architecture by proposing a neural network weight mapping algorithm with Bank as the first priority. The core principle of a ReRAM neural network accelerator is Kirchhoff's voltage-current principle, utilizing the conductance properties of ReRAM to achieve multiplication and accumulation calculations in the analog domain. The mapping relationship between the weight data of the tensor shape and the conductance in the cross-circuit directly affects the computational efficiency and parallelism, and also alters the ReRAM array layout and adder circuit design, thus impacting the design and implementation of the entire accelerator architecture.

[0031] Currently, common mapping algorithms are relatively primitive. Taking convolution calculations, a common feature in neural networks, as an example, the weight mapping algorithms used in mainstream accelerator structures are mostly brute-force unrolling. That is, the convolutional kernel is unrolled into a one-dimensional column vector according to its width, height, and input channel dimensions, and placed along the column direction of the ReRAM crossover circuit. Convolutional kernels for different output channels are placed in different columns of the crossover circuit. Taking a (O, I, H, W) convolutional kernel as an example, O and I represent the size of the output and input channels of the convolutional kernel, while H and W represent the height and width of the convolutional kernel. When the convolutional kernel is unrolled, the kernels for different output channels are unrolled into vector data of shape I*H*W in a certain order, resulting in an unrolled vector of column O, which is then mapped to different rows and columns of the crossover circuit. In this way, the convolution calculation is converted into a matrix multiplication calculation, and the computation process can be accelerated through in-situ computation arrays.

[0032] This paper describes a ReRAM array comprising multiple arrays, each consisting of multiple banks. Each bank contains multiple bank-rows, and multi-bit data is mapped through the ReRAM device. The output of the ReRAM array is a multi-level adder tree, responsible for accumulating the partial sum data outputs within and between arrays. The "partial sum" data refers to the fact that the array's output data is only a portion of the actual output data; the true data is obtained by accumulating all the partial sum data.

[0033] The primary problem that mapping algorithms need to solve is maximizing computational parallelism, and the following conditions must be met: 1. Limitation on the number of adder tree levels; 2. Limitation on the area of ​​the ReRAM array.

[0034] Therefore, as Figure 1 As shown, this invention proposes a weight mapping method based on a ReRAM neural network accelerator: First, the ReRAM array's banks are prioritized to accommodate more convolutional input channels, fully utilizing the parallelism between banks. Second, the mapping algorithm prioritizes the bank-row direction, placing convolutional kernels of different channels in different bank-row positions along the output channel dimension, until all input and output channels are mapped. Simultaneously, each weight data must be arranged in the cross-circuit of the computation array according to a specific bit order (either high to low or low to high) to ensure the correctness of the computation flow and data accumulation process. For example, a single ReRAM device can represent 2 bits of data; an 8-bit integer requires 4 ReRAMs, arranged from high to low bits, from left to right. Notably, considering the optimal utilization efficiency of the ReRAM array's addition tree to achieve optimal data computation speed, the weight slices are placed as evenly as possible within the same bank-row in different arrays.

[0035] Based on the above principles, a general weight mapping scheme can be proposed. For example, assume that the array has N arrays, each array has M banks, each bank contains bank-rows, and each bank-row has a capacity of Y bits.

[0036] (1) For general matrix multiplication, the mapping algorithm is as follows:

[0037] 1. Calculate the dimensions of the weight matrix, and denote the height and width as H_mat and W_mat, respectively;

[0038] 2. Within the range of Y, take the largest divisor of H_mat as the granularity and divide H_mat into sub-vectors;

[0039] 3. Place N consecutive sub-vectors in the same Bank-Row of different Arrays;

[0040] 4. Repeat step 3, placing the sub-vectors in different rows within the same bank, until placement is complete;

[0041] 5. For data on different W_mat dimensions, repeat steps 2, 3, and 4, placing the data in different banks in the order of W_mat mod M, until all mappings are completed.

[0042] (2) For convolution operations, the mapping algorithm is as follows:

[0043] 1. Calculate the convolution weight data, denoted as O, I, H, and W, which correspond to the output channel, input channel, height, and width of the convolution kernel, respectively;

[0044] 2. Within the Y range, take the largest multiple of H*W as the granularity, and divide the convolution kernel into sub-blocks along the input channel dimension;

[0045] 3. Expand the N consecutive sub-blocks into a one-dimensional vector and place them in the same Bank-Row of the N Arrays;

[0046] 4. Repeat step 3 to map the remaining sub-blocks to the next Bank-Row until all sub-blocks of the convolution kernel have been mapped;

[0047] 5. Switch the convolution kernel on the output channel and repeat steps 3 and 4, placing the convolution kernel on the corresponding bank according to the O mod M rule;

[0048] 6. Repeat step 5 until all convolutional kernels have been mapped.

[0049] like Figure 2 As shown, a general mapping method for traditional convolution computation is illustrated. Figure 1 The left part shows the mapping method for the first 48 input channels of a convolution kernel with a shape of (3,3,128). Figure 1 The right part is a schematic diagram of the first bank of the 4-Array ReRAM array. The mapping correspondence is shown in the figure. The convolution kernel is divided into fine-grained blocks along the input channel direction. Each block is unfolded into a one-dimensional vector in the order of H, W, IC and placed on the Bank-Row in sequence. Adjacent channels are placed on adjacent arrays in sequence to pursue the best parallelism.

[0050] Based on the mapping method of this scheme; after all data mapping is completed, as follows: Figure 3 As shown, when performing convolution calculations in a convolutional layer of a neural network, the following steps can be followed:

[0051] ① Calculate the data during sliding based on the three-dimensional dimensions of length, width, and number of input channels of 3*3*16;

[0052] ② Switch to the calculation row and repeat step ① to complete the calculation of all rows in the convolution sliding window;

[0053] ③ Switch the convolution calculation block and repeat step ② to perform calculations on all channels of the input feature map;

[0054] ④ Repeat step ③ to complete the calculation of all columns within the sliding window. Move the sliding window in the row dimension to complete the calculation of the data in the row direction of the input feature map.

[0055] ⑤ Move the sliding window along the column dimensions of the input feature map and repeat step ④ to complete the data calculation in all column dimensions.

[0056] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0057] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, ROM, RAM, etc.

[0058] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A weight mapping method based on a ReRAM neural network accelerator, characterized in that, Includes the following steps: Step 1: Use the banks of the ReRAM array as the first mapping priority to place the convolution input channels; Step 2: Using the Bank-Row direction of the ReRAM array as the second mapping priority, place the convolution kernels of different channels in different Bank-Row positions along the output channel dimension of the convolution kernel until the data mapping requirements of all input and output channels are completed; Step 3: Each weight data is arranged in the cross circuit of the computation array according to a specific bit order rule, wherein the weight data are evenly placed in the same bank-row in different arrays; The array has N arrays, each array has M banks, each bank contains bank-rows, and each bank-row has a capacity of Y bits. The mapping algorithm for general matrix multiplication is as follows: Step 3.1: Calculate the size of the weight matrix, and denote the height and width of the weight matrix as H_mat and W_mat, respectively; Step 3.2: Within the range of Y, take the largest divisor of H_mat as the granularity and divide H_mat into sub-vectors; Step 3.3: Place N consecutive sub-vectors into the same Bank-Row in different Arrays; Step 3.4: Repeat step 3.3 to place the sub-vectors in different rows of the same bank until placement is complete; Step 3.5: For data on different W_mat dimensions, repeat steps 3.2, 3.3, and 3.4, placing the data in different banks in the order of W_mat mod M, until all mappings are completed; The mapping algorithm for convolution operations is as follows: Step 3.6: Calculate the convolution weight data, denoted as O, I, H, and W, which correspond to the output channel, input channel, height, and width of the convolution kernel, respectively; Step 3.7: Within the Y range, take the largest multiple of H*W as the granularity and divide the convolution kernel into sub-blocks along the input channel dimension; Step 3.8: Expand the N consecutive sub-blocks into a one-dimensional vector and place them in the same Bank-Row of the N Arrays; Step 3.9: Repeat step 3.8 to map the remaining sub-blocks to the next Bank-Row until all sub-blocks of the convolution kernel have been mapped; Step 3.10: Switch the convolution kernel on the output channel, and repeat steps 3.8 and 3.

9. According to the rule of O mod M, place the convolution kernel on the corresponding bank. Step 3.11: Repeat step 3.10 until all convolutional kernels have been mapped.

2. The weight mapping method based on a ReRAM neural network accelerator according to claim 1, characterized in that, The specific order in step 3 includes either a high-to-low or low-to-high order.

3. The weight mapping method based on a ReRAM neural network accelerator according to claim 1, characterized in that, The ReRAM array consists of multiple banks and is divided into multiple array groups. Each bank contains multiple bank-rows. The ReRAM array maps bit data through ReRAM devices.

4. The weight mapping method based on a ReRAM neural network accelerator according to claim 3, characterized in that, Each of the ReRAM devices represents 2 bits of data during mapping.

5. The weight mapping method based on a ReRAM neural network accelerator according to claim 1, characterized in that, The output of the ReRAM array is a multi-level adder tree, which is used to accumulate the partial sums of the outputs within and between ReRAM arrays.

Images