ReRAM-based deep neural network accelerator pruning method and system

Abstract

The application discloses a ReRAM-based deep neural network accelerator pruning method and system, and relates to the technical field of deep neural network accelerator design.The method comprises the following steps: obtaining an activation value matrix of a current convolution layer, setting a flag bit and a threshold value by taking a row and a column as a pruning granularity; according to the flag bit, counting the sparsity of the activation value, and performing sorting according to the sparsity; according to the given threshold value, marking sensitive activation value rows and columns and non-sensitive activation value rows and columns, and pruning the non-sensitive activation value rows and columns; and performing convolution on the high bits of the activation value and weights to obtain high-bit operation results, and pruning the next layer of non-sensitive activation value rows and columns according to the high-bit operation results.The application designs a pruning strategy by using the sparsity of the activation value of the convolutional neural network, adapts to the ReRAM Crossbar structure, improves the parallelism of the system operation, improves the operation speed of the system and reduces the power consumption of the system.

Description

Technical Field

[0001] This invention relates to the field of deep neural network accelerator design technology, and in particular to a ReRAM-based deep neural network accelerator pruning method and system. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, Deep Neural Networks (DNNs) have achieved tremendous success in the field of artificial intelligence, including computer vision, speech recognition, and natural language processing. Increasing the number of neural network layers has become an effective way to improve application accuracy. However, as the number of network layers increases, the number of computational parameters also grows. Traditional processors based on the von Neumann architecture face the memory wall problem when performing neural network operations, with memory performance severely impacting processor performance. Industry and academia have proposed some proprietary acceleration architectures to improve the performance of DNN systems, such as GPUs and TPUs. However, these accelerators inevitably face high energy consumption and long latency due to the large amount of data movement between the processor and memory. With Moore's Law reaching its limits, the potential of acceleration architectures based on traditional technologies is limited.

[0004] The emergence of Processing in Memory (CIM) architecture effectively solves the aforementioned problems. As a novel computing architecture, its basic idea is to combine computation and storage, performing computational operations directly in memory. Compared to traditional CPU and memory-separated computing architectures, CIM technology allows for direct data interaction and computational processing within memory, significantly reducing latency caused by data transmission and potentially breaking down the memory barrier. Based on this, Resistive Random Access Memory (ReRAM), as an emerging non-volatile storage device, has become a popular choice for CIM architecture design in recent years. It achieves data storage by adjusting resistance values, and compared to traditional storage devices, ReRAM offers higher storage density, lower power consumption, faster read / write speeds, and longer lifespan. The Crossbar structure, composed of ReRAM Cells, further enhances this by supporting in-place matrix-vector multiplication (MAC) operations, making ReRAM a promising storage device for accelerating DNNs, especially convolutional neural networks. As a traditional deep neural network model, convolutional layer operations account for over 95% of the overall network's computation, requiring substantial computational resources. Traditional computing architectures use CPUs or GPUs to compute convolution operations. However, because ReRAM Crossbar supports in-place MAC operations, convolution kernel data can be stored directly in ReRAM, and convolution operations can be performed directly in ReRAM. This avoids the transmission and storage of large amounts of data, thereby greatly improving computing speed and energy efficiency.

[0005] In recent years, to further improve the processing performance of ReRAM-based DNN accelerators, some traditional DNN acceleration optimization methods have been continuously offloaded into the accelerator framework design. Among these, pruning techniques have received widespread attention. As a method commonly used for neural network acceleration and compression, it utilizes the non-volatile and high-density storage characteristics of ReRAM to reduce the size of the neural network without sacrificing model accuracy. For the weights and activation values ​​in convolutional layers, two pruning techniques naturally arise: weight pruning and activation value pruning. Existing strategies for activation value pruning are relatively few. Known pruning strategies mainly involve removing zero bits from the quantized activation values ​​to skip unnecessary operations, thereby accelerating computation. However, the acceleration effect of the above methods in practical applications is not ideal. Therefore, how to significantly improve the system's computational speed and reduce system power consumption without affecting system accuracy has become a problem that urgently needs to be solved by existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a deep neural network accelerator pruning method and system based on ReRAM. This method utilizes the sparsity of activation values ​​in convolutional neural networks to design pruning strategies, adapts to the ReRAMCrossbar structure, and reduces unnecessary computations during system operation by pruning rows and columns of insensitive activation values. This maintains system parallelism while improving system speed and reducing system power consumption.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0008] The first aspect of this invention provides a ReRAM-based deep neural network accelerator pruning method, comprising the following steps:

[0009] Obtain the activation value matrix of the current convolutional layer, and set flags and thresholds using rows and columns as the pruning granularity;

[0010] The sparsity of activation values ​​is calculated based on the flag bits, and then sorted according to the sparsity.

[0011] Based on a given threshold, sensitive activation value rows and columns and insensitive activation value rows and columns are labeled, and insensitive activation value rows and columns are pruned.

[0012] The high-order bits of the activation values ​​are convolved with the weights to obtain the high-order operation result. The insensitive rows and columns of the next layer are then pruned based on the high-order operation result. During the pruning process, the sparsity of the rows and columns of the activation value matrix is ​​statistically analyzed according to the generation order of the rows and columns in the next layer activation value matrix, and calculations that do not require low-order convolution operations are marked. The statistical results are sorted, and the insensitive rows and columns of the activation value matrix are pruned according to the sorting results based on a given threshold.

[0013] Furthermore, the sparsity of activation values ​​is determined by counting the number of zeros in rows or columns of the activation value matrix based on the flag bits.

[0014] Furthermore, flags are set based on whether row pruning, column pruning, or mixed row and column pruning is required according to the activation value matrix of the current convolutional layer.

[0015] Furthermore, a threshold is set based on the proportion of the number of rows and columns that need to be pruned in the pruning operation to the total number of rows and columns in the current layer activation matrix.

[0016] Furthermore, the specific process of convolving the high-order bits of the activation values ​​with the weights to obtain the high-order operation result is as follows:

[0017] First, the activation value and weight value are quantized to 8 bits. Then, the high n bits of the activation value and all 8 bits of the weight are multiplied and added to obtain the high bit operation result of the current layer.

[0018] Furthermore, the specific process of pruning the rows and columns of insensitive activation values ​​in the next layer based on the results of the high-order operations is as follows:

[0019] The results of high-order operations are used to perform insensitive activation value pruning in the next layer; at the same time, the calculation process that marks positive values ​​after passing through nonlinear layers is also performed.

[0020] If the result of the high-order operation is still greater than 0 after passing through the nonlinear layer and the row and column where the high-order operation result is located have not been pruned, then the low-order operation result corresponding to the high-order operation result is calculated and added to the high-order operation result, and used as the activation value of the next convolutional layer to continue participating in the operation.

[0021] Furthermore, all the results of the low-order operations are added together with the results of the high-order operations to form the complete activation value of the next layer and then passed down to the next layer.

[0022] A second aspect of the present invention provides a ReRAM-based deep neural network accelerator pruning system, comprising:

[0023] The settings module is configured to obtain the activation value matrix of the current convolutional layer and set flags and thresholds with rows and columns as the pruning granularity.

[0024] The sorting module is configured to count the sparsity of activation values ​​based on flag bits and sort them according to sparsity.

[0025] The pruning module is configured to identify sensitive activation value rows and columns and non-sensitive activation value rows and columns according to a given threshold, and prune the non-sensitive activation value rows and columns.

[0026] The convolution module is configured to perform convolution with the weights using the high-order bits of the activation values ​​to obtain the high-order operation result, and then prune the rows and columns of the next layer of insensitive activation values ​​based on the high-order operation result. Specifically, during the pruning process, the sparsity of the rows and columns of the activation value matrix is ​​statistically analyzed according to the generation order of the rows and columns in the next layer's activation value matrix, and calculations that do not require low-order convolution operations are identified. The statistical results are then sorted, and the insensitive rows and columns of the activation value matrix are pruned according to the sorting results based on a given threshold.

[0027] A third aspect of the present invention provides a medium having a program stored thereon, which, when executed by a processor, implements the steps in the ReRAM-based deep neural network accelerator pruning method described in the first aspect of the present invention.

[0028] A fourth aspect of the present invention provides an apparatus including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the ReRAM-based deep neural network accelerator pruning method as described in the first aspect of the present invention.

[0029] The above one or more technical solutions have the following beneficial effects:

[0030] This invention discloses a deep neural network accelerator pruning method and system based on ReRAM. Based on the in-memory integrated structure of the novel ReRAM storage medium, it proposes an in-memory computation strategy for dynamic real-time activation value pruning for deep neural networks. By performing insensitive activation value row and column pruning, many "operation unit" operations can be skipped, saving computational attempts and reducing the workload of the ReRAM crossbar and various peripheral circuits. This significantly reduces the system's computational load, effectively improving system performance and reducing power consumption. Furthermore, by pruning a large number of sparse activation values, combined with high-bit operation prediction techniques for high-bit activation values ​​and weighted convolution, and forward propagation interruption delay hiding techniques, this invention further improves system performance while reducing delays caused by forward propagation interruptions due to insensitive activation value search and pruning.

[0031] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0032] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0033] Figure 1 This is a diagram of the ReRAM Crossbar architecture in Embodiment 1 of the present invention;

[0034] Figure 2 This is a structural diagram of the activation value pruning strategy applied to non-sensitive activation value rows and columns in Embodiment 1 of the present invention;

[0035] Figure 3 This is a structural diagram of the high-order operation prediction and forward propagation delay concealment combined strategy in Embodiment 1 of the present invention. Detailed Implementation

[0036] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0037] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0038] Example 1:

[0039] Neural network model pruning is a technique used to reduce the number of parameters and complexity of neural network models. It reduces model parameters and computational load by removing unnecessary neurons or activation values, thereby improving model efficiency and inference speed. During DNN operations, activation values ​​exhibit significant sparsity. This invention refers to the highly sparse activation values ​​as insensitive activation values. Pruning these insensitive activation values ​​using activation value pruning strategies can significantly improve system speed and reduce power consumption without affecting system accuracy. However, implementing activation value pruning strategies presents several challenges: how to control the granularity of insensitive activation value pruning and design pruning schemes to adapt to the ReRAM Crossbar parallel computing structure, and how to minimize the additional latency caused by DNN forward propagation interruptions due to activation value pruning. This invention proposes using rows / columns as the basic granularity of activation value pruning and proposes a combined strategy of high-order operation prediction and forward propagation latency hiding to optimize the additional overhead caused by the latency of activation value pruning, thereby solving the above-mentioned technical problems. Details are as follows:

[0040] Embodiment 1 of this invention provides a ReRAM-based deep neural network accelerator pruning method, which simplifies the computation process through pruning strategies, reduces the number of operations of the ReRAM crossbar and various peripheral circuits, and optimizes the deep neural network accelerator. Specifically, as follows... Figure 1As shown, the DNN weights are first encoded as resistance values ​​of the ReRAM cells. Simultaneously, the input activation values ​​are expanded into an activation matrix. A digital-to-analog converter (DAC) converts the digital activation values ​​into analog values, which are then transmitted to the ReRAM crossbar via word lines. Finally, the accumulated current is obtained on each bit line. A sample-and-hold circuit then reads the accumulated current (the sum of product results) from the ReRAM bit lines and feeds it to the DAC. The DAC returns the MAC result. Finally, a shift-and-add circuit shifts and adds the high-order and low-order results to obtain the final calculation result. It is important to note that in actual circuit operation, the cumulative impact of the deviation in the accumulated current of the ReRAM cells on inference accuracy must be considered. Therefore, ReRAM-based matrix-vector multiplication must be performed at a small granularity, i.e., an "operation unit." Figure 1 As shown in the gray area on the right, assuming the operation unit size is set to 4*4, only four rows and four columns of the ReRAM Crossbar can be activated simultaneously at a time. Therefore, by performing Activation-insensitive activation value row and column pruning, many "operation unit" operations can be skipped, saving the number of operations and thus reducing the number of operations of the ReRAM Crossbar and various peripheral circuits, improving operating performance and reducing power consumption.

[0041] More specifically, it includes the following steps:

[0042] Step 1: Obtain the activation value matrix of the current convolutional layer, and set the flags and thresholds using rows and columns as the pruning granularity.

[0043] Step 2: Calculate the sparsity of activation values ​​based on the flag bits, and sort them according to the sparsity.

[0044] Step 3: Define the sensitive activation value rows and columns and the non-sensitive activation value rows and columns according to the given threshold, and prune the non-sensitive activation value rows and columns.

[0045] Step 4: Convolve the high-order bits of the activation values ​​with the weights to obtain the high-order operation results, and prune the rows and columns of the next layer of insensitive activation values ​​based on the high-order operation results.

[0046] In step 1, the selection of activation value pruning granularity and the dynamic, real-time detection of insensitive activation values ​​requiring pruning based on the selected granularity are crucial for maintaining model accuracy, improving computational parallelism, and enhancing computational performance. To address this issue, this embodiment uses the rows and columns of the activation value matrix input to the ReRAM Crossbar as the pruning granularity. Compared to channel-based pruning techniques, row and column pruning allows for finer operations and achieves higher model accuracy. Furthermore, compared to other finer-grained pruning techniques (such as pruning activation value data within a smaller range in the activation value matrix), the pruning strategy in this embodiment is better suited to the parallelism of the ReRAM Crossbar.

[0047] In this embodiment, the activation value pruning strategy uses sparsity to determine whether the current activation value row or column is a non-sensitive activation value row or column. Therefore, during the forward propagation process, each convolutional layer needs to set a flag and a threshold. The flag indicates whether the activation value matrix of the current convolutional layer needs row pruning, column pruning, or a combination of both. The threshold specifies the proportion of rows and columns to be pruned in the pruning operation to the total number of rows and columns in the current layer's activation matrix.

[0048] In steps 2 and 3, such as Figure 2 As shown, the sparsity of activation values ​​is calculated by counting the number of zeros in rows or columns of the activation value matrix based on flag bits. The values ​​are then sorted according to sparsity, and sensitive and insensitive activation value rows and columns are identified based on a given threshold. Insensitive activation value rows and columns are then pruned, significantly reducing computational load while minimizing the impact on system accuracy.

[0049] In step 4, to further improve system computing performance and reduce unnecessary calculations, this embodiment provides a prediction technique based on high-order operation results, such as... Figure 3 As shown: During the forward propagation operation, the activation value and weight value are first quantized to 8 bits, and then the high n bits of the activation value and all 8 bits of the weight are multiplied and added to obtain the high bit operation result of the current layer.

[0050] In one specific implementation, the number of high-order bits of the activation value is obtained experimentally. First, the computational cost of each convolutional layer is calculated to distinguish between layers with high and low computational costs. For layers with high computational costs, n is set smaller; otherwise, n is set larger. Simultaneously, a precision loss threshold is set, with initial n values ​​set relatively large, gradually decreasing until the precision loss approaches the precision loss threshold.

[0051] The result of this operation can replace the full-precision result, mainly due to two factors: first, generally speaking, high-order bits can largely approximate the original data; second, smaller activation values ​​have less impact on the overall operation result. Therefore, even if the value might become 0 when using high-order bits to represent a small value, it will not have a significant impact on the overall operation result. In traditional convolutional neural networks, the result obtained from the convolutional layer usually needs to pass through a non-linear layer, and the more representative non-linear layers are the ReLU layer and the MaxPooling layer. If the result of the high-order operation is less than or equal to 0, it will become 0 after passing through the ReLU layer. At the same time, the MaxPooling layer only retains the largest value in a certain region, so some smaller positive values ​​cannot affect the subsequent operation results. Therefore, this embodiment prunes the rows and columns of insensitive activation values ​​in the next layer based on the result of the high-order operation. The specific process is as follows: use the result of the high-order operation to perform insensitive activation value pruning in the next layer. At the same time, the calculation process that still retains positive values ​​after passing through the non-linear layer is marked.

[0052] If the result of the high-order operation is still greater than 0 after passing through the nonlinear layer and the row and column where the high-order operation result is located have not been pruned, then the low-order operation result corresponding to the high-order operation result is calculated and added to the high-order operation result, and used as the activation value of the next convolutional layer to continue participating in the operation.

[0053] In the first two techniques, this embodiment first calculates the results of the high-order activation values ​​and weighted convolution operations, and marks the computation processes that do not require low-order activation value calculation. However, in the next operation, it is necessary to perform insensitive activation value pruning based on the high-order operation results, perform the necessary low-order convolution operations, add them to the high-order operation results, and pass them to the next convolutional layer. However, this calculation strategy will interrupt the forward propagation process of the DNN, reducing the system execution efficiency. To solve this problem, this embodiment proposes a forward propagation delay hiding technique. During the forward propagation process, the positions of the obtained activation values ​​in the activation value matrix of the next convolutional layer are fixed and arranged according to the calculation order. Therefore, the rows and columns of the activation value matrix of the new layer are generated in an ordered manner, so it is possible to determine the order of generation of the rows and columns in the activation value matrix, such as... Figure 3 As shown, during the process of pruning the rows and columns of insensitive activation values ​​in the next layer based on the results of high-order operations, the sparsity of the rows and columns of the activation value matrix is ​​counted sequentially, and the calculation process that does not require low-order convolution operations is marked. After the sparsity of the activation values ​​in the last row or the last column is obtained, the statistical results are sorted, and the insensitive rows and columns of the activation value matrix are pruned according to the sorting results based on the given threshold.

[0054] Finally, all valid low-order operations are performed, and the result is added to the result of the high-order operations to form the complete activation value for the next layer, which is then passed down to the next layer. This hidden strategy effectively reduces the latency caused by interrupting forward propagation during activation value pruning.

[0055] To better understand the pruning strategy in this embodiment, let's take an activation matrix m, with r rows and c columns, as an example for explanation:

[0056] The number of zero values ​​in each row of the activation value matrix m is z_r, and the number of zero values ​​in each column of the activation value matrix is ​​z_c.

[0057] (1) Row pruning pattern

[0058] The number of zero values ​​in each row of the activation value matrix is ​​counted and assigned to the array z_r. z_r is then sorted in descending order. Based on the number of zero values, sparse rows that meet the row pruning threshold are marked as insensitive rows and pruned to obtain the pruned activation value matrix M.

[0059] (2) Column pruning pattern

[0060] The number of zero values ​​in each column of the activation value matrix is ​​counted and assigned to the array z_c. z_c is then sorted in descending order. Sparse columns that meet the column pruning threshold are marked as insensitive columns and pruned to obtain the pruned activation value matrix M.

[0061] (3) Row-column hybrid pruning mode

[0062] The number of zero values ​​in each row and column of the activation value matrix is ​​counted and assigned to arrays z_r and z_c respectively. z_r and z_c are sorted in descending order. Sparse columns that meet the column pruning threshold are marked as insensitive columns and sparse rows that meet the row pruning threshold are marked as insensitive rows and pruned to obtain the pruned activation value matrix M.

[0063] The non-sensitive activation value row and column pruning technique proposed in this invention can dynamically and in real time detect sparse rows and columns in the activation value matrix and prune them, reducing the computational load of the model while adapting to the parallelism of ReRAM Crossbar.

[0064] To better understand the combined strategy of high-order operation prediction and forward propagation delay concealment in this embodiment, we will use the activation value matrix m, the number of high-order bits of the activation value n, and the weight matrix w of the Nth layer as an example:

[0065] According to the formulas m=m>>(8-n) and m=m<<(8-n), the high n bits of the activation value are obtained. Based on the specific structure of the model, the result C of the Nth layer convolution is obtained. The positions of the data greater than 0 in the convolution result after the nonlinear layer are counted and the operation process is marked. The non-sensitive activation value row and column pruning is performed on the activation value matrix t of the N+1th layer to obtain the pruned activation value matrix T of the N+1th layer.

[0066] Perform the necessary low-order convolution operations and add them to the high-order operation results to obtain the activation value matrix M, which is then passed to the next layer.

[0067] The high-order prediction technique proposed in this invention utilizes the high-order bits of activation values ​​to perform convolution with weights, and executes the activation value pruning strategy for the next layer based on the convolution result. Finally, it performs low-order convolution operations as needed and adds the results to the high-order convolution results. This provides an opportunity to mitigate the latency caused by forward propagation interruptions while further reducing computational load.

[0068] Example 2:

[0069] Embodiment 2 of the present invention provides a ReRAM-based deep neural network accelerator pruning system, comprising:

[0070] The settings module is configured to obtain the activation value matrix of the current convolutional layer and set flags and thresholds with rows and columns as the pruning granularity.

[0071] The sorting module is configured to count the sparsity of activation values ​​based on flag bits and sort them according to sparsity.

[0072] The pruning module is configured to identify sensitive activation value rows and columns and non-sensitive activation value rows and columns according to a given threshold, and prune the non-sensitive activation value rows and columns.

[0073] The convolution module is configured to perform convolution with the weights using the high-order bits of the activation values ​​to obtain the high-order operation result, and then prune the rows and columns of the next layer of insensitive activation values ​​based on the high-order operation result. Specifically, during the pruning process, the sparsity of the rows and columns of the activation value matrix is ​​statistically analyzed according to the generation order of the rows and columns in the next layer's activation value matrix, and calculations that do not require low-order convolution operations are identified. The statistical results are then sorted, and the insensitive rows and columns of the activation value matrix are pruned according to the sorting results based on a given threshold.

[0074] Example 3:

[0075] Embodiment 3 of the present invention provides a medium on which a program is stored. When the program is executed by a processor, it implements the steps in the ReRAM-based deep neural network accelerator pruning method described in Embodiment 1 of the present invention.

[0076] Example 4:

[0077] Embodiment 4 of the present invention provides a device including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the ReRAM-based deep neural network accelerator pruning method described in Embodiment 1 of the present invention.

[0078] The steps and methods involved in Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0079] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0080] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A deep neural network accelerator pruning method based on ReRAM, characterized in that, Includes the following steps: The activation value matrix of the current convolutional layer is obtained. The digital activation values ​​are converted into analog values ​​through the digital-to-analog converter circuit and transmitted to the ReRAM Crossbar through the word line. The accumulated current on the ReRAM bit line is read through the sample-and-hold circuit and fed to the analog-to-digital converter circuit. The analog-to-digital converter circuit returns the MAC result. The flag bits and thresholds are set with rows and columns as the pruning granularity. The sparsity of activation values ​​is calculated based on the flag bits, and then sorted according to the sparsity. By calibrating the sensitive and insensitive activation value rows and columns according to a given threshold, and pruning the insensitive activation value rows and columns, the operation process of many "operation units" can be skipped, thereby reducing the number of operations of the ReRAM Crossbar and various peripheral circuits. The high-order bits of the activation values ​​are convolved with the weights to obtain the high-order operation result. The insensitive rows and columns of the next layer are then pruned based on the high-order operation result. During the pruning process, the sparsity of the rows and columns of the activation value matrix is ​​statistically analyzed according to the generation order of the rows and columns in the next layer activation value matrix, and calculations that do not require low-order convolution operations are marked. The statistical results are sorted, and the insensitive rows and columns of the activation value matrix are pruned according to the sorting results based on a given threshold.

2. The ReRAM-based deep neural network accelerator pruning method as described in claim 1, characterized in that, The sparsity of activation values ​​is determined by counting the number of zeros in rows or columns of the activation value matrix based on flags.

3. The ReRAM-based deep neural network accelerator pruning method as described in claim 1, characterized in that, Set a flag based on whether row pruning, column pruning, or a combination of row and column pruning is required for the current activation value matrix of the convolutional layer.

4. The ReRAM-based deep neural network accelerator pruning method as described in claim 1, characterized in that, A threshold is set based on the proportion of the number of rows and columns to be pruned in the pruning operation to the total number of rows and columns in the current layer activation matrix.

5. The ReRAM-based deep neural network accelerator pruning method as described in claim 1, characterized in that, The specific process of convolving the high-order bits of the activation values ​​with the weights to obtain the high-order bit operation result is as follows: First, the activation value and weight value are quantized to 8 bits. Then, the high n bits of the activation value and all 8 bits of the weight are multiplied and added to obtain the high bit operation result of the current layer.

6. The deep neural network accelerator pruning method based on ReRAM as described in claim 1, characterized in that, The specific process of pruning the rows and columns of insensitive activation values ​​in the next layer based on the high-order operation results is as follows: The results of high-order operations are used to perform insensitive activation value pruning in the next layer; at the same time, the calculation process that marks positive values ​​after passing through nonlinear layers is also performed. If the result of the high-order operation is still greater than 0 after passing through the nonlinear layer and the row and column where the high-order operation result is located have not been pruned, then the low-order operation result corresponding to the high-order operation result is calculated and added to the high-order operation result, and used as the activation value of the next convolutional layer to continue participating in the operation.

7. The ReRAM-based deep neural network accelerator pruning method as described in claim 6, characterized in that, Add all the results of the low-order operations to the results of the high-order operations to form the complete activation value of the next layer and pass it down to the next layer.

8. A ReRAM-based deep neural network accelerator pruning system, characterized in that, include: The configuration module is set to obtain the activation value matrix of the current convolutional layer, convert the digital activation values ​​into analog values ​​through the digital-to-analog converter circuit, transmit them to the ReRAM Crossbar through the word line, read the accumulated current on the ReRAM bit line through the sample-and-hold circuit, feed it to the analog-to-digital converter circuit, and the analog-to-digital converter circuit returns the MAC result. The flag bits and thresholds are set with rows and columns as the pruning granularity. The sorting module is configured to count the sparsity of activation values ​​based on flag bits and sort them according to sparsity. The pruning module is configured to label sensitive activation value rows and columns and non-sensitive activation value rows and columns according to a given threshold, and prune the non-sensitive activation value rows and columns. This can skip the operation process of many "operation units", thereby reducing the number of operations of the ReRAM Crossbar and various peripheral circuits. The convolution module is configured to perform convolution with the weights using the high-order bits of the activation values ​​to obtain the high-order operation result, and then prune the rows and columns of the next layer of insensitive activation values ​​based on the high-order operation result. Specifically, during the pruning process, the sparsity of the rows and columns of the activation value matrix is ​​statistically analyzed according to the generation order of the rows and columns in the next layer's activation value matrix, and calculations that do not require low-order convolution operations are identified. The statistical results are then sorted, and the insensitive rows and columns of the activation value matrix are pruned according to the sorting results based on a given threshold.

9. A computer-readable storage medium, characterized in that, It stores multiple instructions, which are adapted to be loaded and executed by the processor of the terminal device as described in any one of claims 1-7, for the ReRAM-based deep neural network accelerator pruning method.

10. A terminal device, characterized in that, It includes a processor and a computer-readable storage medium, the processor being used to implement various instructions; the computer-readable storage medium being used to store multiple instructions adapted to be loaded by the processor and executed by the processor according to any one of claims 1-7, the ReRAM-based deep neural network accelerator pruning method.

Images