A method and system for accelerating neural network computing based on RRAM

Abstract

This application relates to the field of neural network computing technology and discloses a method and system for neural network accelerated computing based on RRAM. The method includes: obtaining hardware parameters and configuration parameters of multiple RRAMs, the hardware parameters including the number of parallelizable operations, and the configuration parameters including the effective conductance range and effective rows and columns; splitting the computing tasks of any network layer of the neural network to generate a first computing task and a second computing task; determining a first processing path connected to the target RRAM based on the hardware parameters and configuration parameters; processing the first computing task according to the first processing path to obtain a first processing result; determining a second processing path; and processing the second computing task according to the second processing path and the first processing result to obtain an output result of the current network layer. The technical solution provided by this application can ensure the accuracy and efficiency of different RRAMs in neural network accelerated computing.

Description

Technical Field

[0001] The present application relates to the field of neural network computing technology, and in particular to a method and system for accelerating neural network computing based on RRAM. Background Art

[0002] With the surge in demand for computing power in neural network calculations, traditional hardware such as GPUs have bottlenecks in energy efficiency and parallelism. Currently, RRAM is often used in neural network calculations due to its advantages of high speed, low power consumption and high parallelism.

[0003] In practical applications, when using multiple RRAMs for accelerated computing of neural networks, variations in the physical properties of different RRAMs will reduce the accuracy of neural network calculations, and different RRAM hardware configurations will complicate neural network deployment, thereby reducing the computing efficiency of the neural network.

[0004] In view of this, how to ensure the accuracy and computational efficiency of different RRAMs in neural network acceleration calculations is an urgent problem to be solved. Summary of the Invention

[0005] The present application provides a method and system for neural network accelerated computing based on RRAM, which can ensure the accuracy and computing efficiency of different RRAMs in neural network accelerated computing.

[0006] In a first aspect, the present application provides a method for accelerating computing of a neural network based on RRAM, the method comprising: obtaining hardware parameters of one or more RRAMs, the hardware parameters including at least a parallelizable number, and determining configuration parameters of each of the RRAMs based on a preset tolerance, the configuration parameters including at least an effective conductance range and a plurality of valid rows and columns; splitting the computing tasks of any network layer of the neural network, and generating a first computing task and a second computing task based on the splitting result; determining a first processing path for the first computing task based on the parallelizable number and the effective conductance range, the first processing path being connected to a target RRAM, and processing the first computing task according to the valid rows and columns of the target RRAM to obtain a first processing result; determining a second processing path for the second computing task, processing the second computing task according to the second processing path and the first processing result to obtain a second processing result, and using the second processing result as the output result of the current network layer.

[0007] In one possible implementation, after obtaining the second processing result, the first computing task and the second computing task are calculated according to the first processing path to obtain a third processing result; performance parameters of the second processing result and the third processing result are evaluated, and the target RRAM or the first processing path is adjusted based on the evaluation result.

[0008] In one possible implementation, the performance parameters include at least computational accuracy and computational power consumption; evaluating the performance parameters of the second processing result and the third processing result, and adjusting the target RRAM or the first processing path based on the evaluation results includes: if the computational accuracy of the second processing result is lower than that of the third processing result, adjusting hardware parameters of the target RRAM; and if the computational power consumption of the second processing result is greater than that of the third processing result, determining the second processing path as the first processing path.

[0009] In one possible implementation, determining the configuration parameters of each RRAM according to a preset tolerance includes: the RRAM includes a plurality of storage ranks, and the storage ranks that meet the preset tolerance are defined as valid ranks; obtaining an effective conductance value of each valid rank, and determining an effective conductance range of the RRAM based on the effective conductance value.

[0010] In one possible implementation, the computing tasks of the neural network are split, and generating a first computing task and a second computing task based on the splitting results includes: determining the various computing tasks of the neural network according to the type of the neural network, and for any of the computing tasks, determining whether the computing task belongs to a linear computing task; if the computing task belongs to a linear computing task, determining the task as a first computing task; if the computing task does not belong to a linear computing task, determining the task as a second computing task.

[0011] In one possible implementation, determining a first processing path for the first computing task based on the parallelizable number and the effective conductance range, and connecting the first processing path to a target RRAM includes: determining an RRAM that matches the first computing task based on the parallelizable number and the effective conductance range; determining the matching RRAM as the target RRAM, and determining a path connecting the target RRAM to the neural network as the first processing path.

[0012] In one possible implementation, processing the first computing task according to the valid rows and columns of the target RRAM to obtain a first processing result includes: determining a positive weight array and a negative weight array of the target RRAM based on the valid rows and columns; and calculating the first computing task according to the positive weight array and the negative weight array to obtain the first processing result.

[0013] In one possible implementation, determining a positive weight array and a negative weight array of a target RRAM based on the valid rows and columns includes: obtaining effective conductance values ​​of the respective valid rows and columns, determining a reference row and column among the valid rows and columns of the target RRAM, and determining the effective conductance value of the reference row and column as a reference value; determining a plurality of the valid rows and columns whose effective conductance values ​​are greater than or equal to the reference value as a positive weight array, and determining a plurality of the valid rows and columns whose effective conductance values ​​are less than the reference value as a negative weight array.

[0014] In one possible implementation, calculating the first computing task according to the positive weight array and the negative weight array to obtain a first processing result includes: obtaining a positive weight matrix and a negative weight matrix of the neural network, storing the positive weight matrix to the positive weight array, storing the negative weight matrix to the negative weight array, obtaining a first output current of the positive weight array and a second output current of the negative weight array, and taking the difference between the first output current and the second output current as the first processing result.

[0015] In one possible embodiment, the second processing path is connected to the processor; determining the second processing path of the second computing task, and processing the second computing task according to the second processing path and the first processing result to obtain a second processing result includes: determining a target processor connected to the neural network, determining the path connecting the target processor and the neural network as the second processing path, sending the second computing task and the first processing result to the target processor through the first processing path, and the target processor executing the second computing task according to the first processing result to obtain the second processing result.

[0016] According to a second aspect of the present application, a system for accelerating neural network computing based on RRAM is provided. The system includes: a parameter acquisition unit for acquiring hardware parameters of one or more RRAMs, wherein the hardware parameters include at least a parallelizable number, and determining configuration parameters of each RRAM based on a preset tolerance, wherein the configuration parameters include at least an effective conductance range and a plurality of effective rows and columns; a task splitting unit for splitting the computing tasks of any network layer of the neural network, and generating a first computing task and a second computing task based on the splitting result; a task computing unit for determining a first processing path for the first computing task based on the parallelizable number and the effective conductance range, wherein the first processing path is connected to a target RRAM, and the first computing task is processed according to the effective rows and columns of the target RRAM to obtain a first processing result; and determining a second processing path for the second computing task, and processing the second computing task according to the second processing path and the first processing result to obtain a second processing result, and using the second processing result as the output result of the current network layer.

[0017] A third aspect of the present application provides a computer device comprising a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to implement the RRAM-based neural network accelerated computing method described in the first aspect.

[0018] One or more embodiments of the present application provide a technical solution that automatically adapts different RRAMs and processors to jointly process neural networks, thereby improving the computational efficiency and accuracy of neural networks. Specifically, the first processing paths of different RRAMs are automatically adapted based on their hardware parameters and configuration parameters, and the computational tasks of the neural network are split and classified. Each type of computational task is processed through the first and second processing paths to obtain the output results of the neural network. Simultaneously, the adapted first processing path is dynamically adjusted to balance the accuracy and power consumption of computational task processing, thereby improving the efficiency and accuracy of neural network accelerated computation.

[0019] It can be seen that the technical solution provided by this application ensures the computing efficiency and accuracy of the neural network in the multi-configuration RRAM state. At the same time, it automatically adapts to different RRAMs and neural networks to solve the impact of RRAM hardware configuration on the accelerated computing of neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A schematic diagram of the steps of a method for accelerating neural network computing based on RRAM provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of the steps for splitting a neural network provided in one embodiment of the present application;

[0023] Figure 3 A schematic diagram of the steps for determining a positive and negative weight array provided in one embodiment of the present application;

[0024] Figure 4 A schematic diagram of the steps for obtaining a first processing result provided in one embodiment of the present application;

[0025] Figure 5 A schematic diagram of the structure of a system for accelerating neural network computing based on RRAM provided in one embodiment of the present application;

[0026] Figure 6 A schematic structural diagram of a computer device provided in accordance with one embodiment of the present application. DETAILED DESCRIPTION

[0027] To make the purpose, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0028] In addition, the descriptions of "first", "second", etc. in this application are for descriptive purposes only and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, steps, calculations or other actions "based on" or "according to" one or more of the conditions or values ​​can be based on additional conditions or values ​​beyond the described values ​​in practice.

[0029] As neural network applications become more widespread, the demand for computing power is also increasing. Traditional processors like GPUs (Graphics Processing Units) generally have limitations in energy efficiency and parallelism, making them inadequate for neural network applications. Currently, RRAM, due to its high speed, low power consumption, and high parallelism, is often used to accelerate neural network computations in place of GPUs. As a new type of non-volatile memory, RRAM utilizes changes in the material's resistance to store data and performs computations directly within the memory cell, reducing data overhead.

[0030] However, in actual applications, there are hardware differences between different RRAMs, especially the differences in multi-board systems and storage structures, which complicate the deployment of neural networks and reduce the computing efficiency of neural networks. At the same time, RRAM is affected by factors such as temperature, voltage fluctuations, and usage time, and the conductance value will inevitably drift. Hardware instabilities such as conductance value drift will directly interfere with the accuracy of stored data, causing deviations in calculation results and reducing the accuracy of neural network calculations. For example, when performing complex image recognition tasks, slight conductance value drift may cause errors in recognition results. Therefore, there is an urgent need to eliminate the impact of RRAM hardware instability on neural network accelerated calculations and ensure the accuracy and computing efficiency of different RRAMs in neural network accelerated calculations. In view of this, one or more embodiments of the present application provide a method and system for neural network accelerated calculations based on RRAM, which can solve the above problems and improve the accuracy and computing efficiency of different RRAMs in neural network accelerated calculations.

[0031] See also Figure 1 In one embodiment of the present application, a method for accelerating neural network computing based on RRAM is provided. The method may include the following steps:

[0032] S1: Obtain hardware parameters of one or more RRAMs, wherein the hardware parameters include at least a parallelizable number, and determine configuration parameters of each RRAM according to a preset tolerance, wherein the configuration parameters include at least an effective conductance range and a plurality of effective rows and columns.

[0033] The above hardware parameters include the parallel number, and may also include read and write voltages and storage structures. The above parallel number can be understood as the number of computing tasks that RRAM can handle simultaneously. Its high parallelism can significantly accelerate neural network inference. For example, if the parallel number is 128, the system can handle 128 matrix multiplication operations simultaneously. The above read and write voltages are the voltage levels required by RRAM when writing and reading data. Different voltages may affect the speed and accuracy of data conversion. The above storage structure may be a variety of memory structures such as shared memory, local memory, and on-chip memory. Different memory hierarchies may affect data access speed and parallel computing capabilities.

[0034] The above-mentioned preset tolerance can be understood as the allowable range of hardware parameter deviation set by the user to ensure the accuracy and reliability of the calculation. Preferably, the conductance value is allowed to fluctuate by ±10% as the preset tolerance. The above-mentioned configuration parameters include an effective conductance range and effective rows and columns. The above-mentioned effective conductance range can be understood as the conductance range available for the RRAM unit in a stable state. The above-mentioned effective rows and columns can be understood as the actual available rows and columns in the RRAM array. Usually, the conductance value of each row or column of the RRAM unit is the same. By setting the preset tolerance, the above-mentioned effective conductance range and effective rows and columns in the RRAM can be determined. For example, the rows and columns that meet the preset tolerance are used as the effective array. By obtaining the hardware configuration parameters of different RRAMs, cells with unstable conductance values ​​can be filtered out to reduce the impact of errors. At the same time, the parallel number and storage structure can be used to improve the computing throughput, providing key data support for the subsequent division of labor and acceleration of neural network tasks.

[0035] S3: Split the computing task of any network layer of the neural network, and generate a first computing task and a second computing task based on the splitting result.

[0036] The neural network can be a CNN (Convolutional Neural Network), an RNN (Recurrent Neural Network), a GAN (Genrative Adversarial Network), a GNN (Graph Neural Network), or other neural networks. Based on the working types of different neural networks, the computing tasks of the neural network are split to form multiple computing tasks. Furthermore, the multiple computing tasks are divided into a first computing task and a second computing task.

[0037] The first computing task can be a linear computing operation involving linear combinations such as matrix multiplication and convolution. Since linear computing typically has high parallelism and regular data access patterns, it is suitable for performing the first computing task using RRAM. The second computing task is preferably a nonlinear computing operation that cannot be simply completed through linear combinations. Since nonlinear computing typically involves complex mathematical functions and logical reasoning, it is suitable for performing the second computing task using a processor such as a GPU. It should be noted that the second computing task can also be a linear computing operation.

[0038] For example, CNN, as a convolutional neural network, is mainly based on convolutional layers, including a large number of matrix multiplication and pooling operations. The computational tasks of CNN are split, with linear operations such as matrix multiplication of convolution kernels as the first computational task, and nonlinear operations such as activation and pooling as the second computational task. Among them, RNN, as a recurrent neural network, is mainly time-series dependent, including loop calculations and gating mechanisms. It can be used to accelerate the multiplication of loop weight matrices as the first computational task, and process activation functions or gating functions as the second computational task.

[0039] S5: Determine a first processing path for the first computing task based on the parallelizable number and the effective conductance range, wherein the first processing path is connected to a target RRAM and processes the first computing task according to effective rows and columns of the target RRAM to obtain a first processing result.

[0040] In this embodiment, the parallel number and effective conductance range of the current RRAM are obtained to filter out RRAM cells with severe conductance value drift, and RRAM cells that meet the tolerance range are used as target RRAMs. The first computing task is processed through a first processing path connected to the target RRAM to improve the processing efficiency of some computing tasks, while avoiding the impact of hardware instability on neural network accelerated computing.

[0041] In this embodiment, if there are multiple selectable RRAMs, the first computing tasks are allocated to multiple first processing paths connected to the multiple target RRAMs based on the parallelization number and effective conductance range of each RRAM to achieve load balancing. When the target RRAM of a first processing path fails due to conductance drift, the first computing task under the current first processing path is automatically allocated to other first processing paths to obtain the first processing result.

[0042] S7: Determine a second processing path for the second computing task, process the second computing task according to the second processing path and the first processing result to obtain a second processing result, and use the second processing result as the output result of the current network layer.

[0043] In this embodiment, the output of the first computing task of the current network layer is used as the input of the second computing task. The above-mentioned second processing path is a task processing path for processing the second computing task. In a neural network, the first computing task and the second computing task of each network layer have a sequential relationship, and at the same time, there is a sequential relationship between each network layer. The above-mentioned first processing result and the second processing result need to be spliced ​​and transmitted in the order of the neural network layer to ensure the continuity of the data and the accuracy of the calculation. Exemplarily, the neural network may include a convolution layer, an activation layer and a fully connected layer. Taking the convolution layer as an example, the RRAM processes the matrix multiplication of all convolution layers and outputs the intermediate feature map as the first processing result. The intermediate feature map is transmitted to the second processing path for activation operation to obtain the activated feature map as the second processing result, and the above-mentioned second processing result is used as the output result of the convolution layer.

[0044] In light of this, the technical solutions provided in the above-mentioned embodiments of this application improve the computational efficiency and accuracy of neural networks by automatically adapting different RRAMs and processors to jointly process neural networks. Specifically, the first processing paths of different RRAMs are automatically adapted based on their hardware and configuration parameters, and the computational tasks of the neural network are split and classified. Each type of computational task is processed through the first and second processing paths, respectively, to obtain the output results of the neural network. Simultaneously, the adapted first processing path is dynamically adjusted to balance the accuracy and power consumption of computational task processing, thereby improving the efficiency and accuracy of neural network accelerated computation.

[0045] In one possible embodiment, based on the above step S1, since the RRAM includes multiple storage ranks, determining the valid ranks of each RRAM according to a preset tolerance may include taking the storage ranks that meet the preset tolerance as valid ranks, obtaining effective conductance values ​​of the respective valid ranks, and determining the effective conductance range of the RRAM based on the effective conductance values.

[0046] The RRAM is composed of multiple memory cells arranged in a matrix to form a row and column structure. The memory cells in each row or column share a word line or bit line, so each row or column can be set as a memory row. Each memory row has the same conductance value, and by selecting a specific memory row, all memory cells in that row can be accessed.

[0047] In this embodiment, a preset tolerance is set to screen valid rows and columns and determine an effective conductance range. Specifically, the conductance value of each storage row and column is recorded and compared with the preset tolerance to select rows and columns with conductance values ​​within the allowable range as valid rows and columns. Furthermore, the conductance value of each valid row and column is recorded, and a statistical analysis is performed on the conductance values ​​of all valid rows and columns to find the minimum and maximum values ​​to determine the effective conductance range of the RRAM. For example, the effective conductance range is defined as the interval between the minimum and maximum conductance values.

[0048] For example, if an RRAM array includes 100 rows and 100 columns, and the preset tolerance is that the allowable range of conductance value variation does not exceed ±10%, and dynamic data testing finds that the conductance values ​​of 80 rows or 80 columns are within the allowable range, these storage rows and columns are defined as valid rows and columns, and the conductance values ​​of the valid rows and columns are recorded to determine the effective conductance range.

[0049] In one embodiment, see Figure 2 Based on the above step S3, the computing task of the neural network is split, and the first computing task and the second computing task are generated based on the splitting result. The splitting can be performed in the following manner:

[0050] S301: Determine each computing task of the neural network according to the type of the neural network, and determine, for any computing task, whether the computing task is a linear computing task.

[0051] S303: If the computing task is a linear computing task, the task is determined as a first computing task; if the computing task is not a linear computing task, the task is determined as a second computing task.

[0052] These linear computing tasks can be understood as linear computing operations with linear relationships that can be completed through linear combinations, such as matrix multiplication, convolution, loop operations, linear transformations, or other linear operations. Generally speaking, linear operations often involve a large number of independent computing tasks, with high parallelism, low precision, and regular data access patterns, characteristics that are well-suited to RRAM. Assigning these linear computing tasks as the first computing task to the target RRAM in the first computing path can improve the computational efficiency of neural network acceleration.

[0053] The aforementioned nonlinear computing tasks can be understood as nonlinear computing operations where there is no linear relationship between the output and input, such as activation operations, pooling operations, normalization operations, or other linear operations. The aforementioned activation operations can be ReLU (Rectified Linear Unit, a linear rectification function), Sigmoid (a type of activation function), or other commonly used activation functions. The activation function value is quickly calculated by processing multiple data points in parallel or utilizing high-precision floating-point computing capabilities. The aforementioned pooling operations can be maximum pooling or average pooling, which can be quickly completed by processing multiple pooling windows in parallel.

[0054] In this embodiment, the computing tasks of different neural networks are split by judging whether they belong to linear tasks. For example, in the computing tasks of CNN, the first computing task may include convolution operations and calculations of fully connected layers, and the second computing task may include activation operations and pooling operations. In the computing tasks of RNN, the first computing task may include matrix multiplication and loop operations, and the second computing task may include activation operations and gating operations. In the computing tasks of GNN, the first computing task may include calculations of fully connected layers and convolution operations, and the second computing task may include activation operations and normalization operations. In the computing tasks of GNN, the first computing task may include graph convolution operations and linear transformations, and the second computing task may include activation operations and pooling operations.

[0055] In one embodiment, based on the above-mentioned step S5, it is necessary to determine a first processing path for the first computing task based on the parallelizable number and the effective conductance range, specifically including determining an RRAM that matches the first computing task based on the parallelizable number and the effective conductance range; determining the matching RRAM as the target RRAM, and determining the path connecting the target RRAM and the neural network as the first processing path.

[0056] In this embodiment, the target RRAM matching the first computing task is dynamically adjusted based on the parallelization capability and the effective conductance range to determine the first processing path. Specifically, if the first computing task requires high-precision computing, an RRAM with a narrower effective conductance range can be selected as the target RRAM. If the first computing task requires high-parallel processing, an RRAM with a higher parallelization capability can be selected as the target RRAM. The path connecting the target RRAM to the neural network is determined as the first processing path.

[0057] In this embodiment, only valid rows and columns within the RRAM whose conductance values ​​fall within the effective conductance range are used to process the first computation task. The target RRAM can be various RRAM regions of different RRAMs, each of which includes one or more valid rows and columns. Data for the first computation task is processed in blocks based on the number of parallel blocks available, and each block of data is assigned to a different RRAM region for processing. Simultaneously, the conductance drift of the target RRAM is dynamically monitored. If the conductance value of a valid row or column exceeds the tolerance, the valid row or column is automatically skipped, thereby improving the accuracy of RRAM for neural network acceleration computations. For example, if column 5 of the second RRAM fails, the task can be migrated to a spare valid row or column of the third RRAM.

[0058] In one embodiment, based on step S5, the first computing task may be processed according to the valid array of the target RRAM to obtain a first processing result. Specifically, a positive weight array and a negative weight array of the target RRAM are determined based on the valid rows and columns, and the first computing task is calculated according to the positive weight array and the negative weight array to obtain the first processing result.

[0059] The positive weight array is used to store positive weight values, and the negative weight array is used to store negative weight values. Within the active array of the target RRAM, a positive weight array and a negative weight array are separated. By measuring the output current difference between the positive weight array and the negative weight array, a first processing result of the first computing task can be obtained based on the result calculated based on the difference. This reduces errors caused by conductance drift or other hardware noise in a single array, thereby improving the accuracy of RRAM for neural network accelerated computing.

[0060] In this implementation, see Figure 3 , the positive weight array and negative weight array are determined as follows:

[0061] S501: Acquire effective conductance values ​​of the respective active rows and columns, determine a reference row and column among the active rows and columns of the target RRAM, and determine the effective conductance value of the reference row and column as a reference value.

[0062] S503: Determine the plurality of valid rows and columns whose effective conductance values ​​are greater than or equal to a reference value as a positive weight array, and determine the plurality of valid rows and columns whose effective conductance values ​​are less than the reference value as a negative weight array.

[0063] The reference row column can be selected as any active row column of the target RRAM, and the conductance value of this row column is used as a reference value for comparison with the conductance values ​​of other rows and columns. Alternatively, a group of rows and columns at a fixed position can be selected as the reference row column, or the rows with the most stable conductance values ​​can be selected as the reference row column, or the rows with conductance values ​​in the middle range can be selected as the reference row column. Furthermore, the conductance values ​​of each active row column are compared with the reference value, so that rows and columns with conductance values ​​greater than or equal to the reference value are divided into a positive weight array, and rows and columns with conductance values ​​less than the reference value are divided into a negative weight array.

[0064] In this embodiment, by dividing the weights into positive and negative components, a difference calculation can be used to offset the effects of hardware noise and conductance drift. By determining the positive and negative weight arrays based on the effective array of the target RRAM, the first calculation task is processed to obtain a more accurate calculation result.

[0065] In this implementation, see Figure 4 Calculating the first computing task according to the positive weight array and the negative weight array to obtain a first processing result can be performed according to the following steps:

[0066] S511: Acquire a positive weight matrix and a negative weight matrix of the neural network, store the positive weight matrix in the positive weight array, and store the negative weight matrix in the negative weight array.

[0067] S513: Acquire a first output current of the positive weight array and a second output current of the negative weight array, and use a difference between the first output current and the second output current as the first processing result.

[0068] The positive weight matrix and negative weight matrix of the neural network represent the weight values ​​of the current network layer of the neural network, corresponding to positive weights and negative weights, respectively. The weight matrix is ​​used to convert input data into output data of the current network layer. The positive weight matrix can be understood as the positive connection strength between the input data and the output data, and the negative weight matrix can be understood as the negative connection strength between the input data and the output data. Furthermore, the positive weight matrix is ​​written into the positive weight array, and the negative weight matrix is ​​written into the negative weight array.

[0069] In this embodiment, input data is applied to the positive weight array and the negative weight array to obtain a first output current for the positive weight array and a second output current for the negative weight array. The difference between the first and second output currents is used as a first processing result. If temperature changes or conductivity drift affect the positive and negative weight arrays in the same direction, interpolation will offset common-mode errors, resulting in higher accuracy in RRAM calculation results for neural network acceleration.

[0070] For example, get the weight matrix of a layer from the neural network , the weight matrix Decomposed into a positive weight matrix and negative weight matrix , the input data is ,Will Stored in the positive weight matrix, Stored in the negative weight matrix, further, the input data Applied to the positive weight array in the form of voltage to obtain the first output current , input data Applied to the negative weight array in the form of voltage to obtain the first output current , calculate the difference , to convert the difference As the first processing result of the network layer .

[0071] In one embodiment, based on step S7, the second processing path is connected to a processor, and based on the first processing result, the second computing task is processed by the processor connected by the second processing path to obtain a second processing result. Specifically, a target processor connected to the neural network is determined, a path connecting the target processor and the neural network is determined as the second processing path, the second computing task and the first processing result are sent to the target processor via the first processing path, and the target processor executes the second computing task based on the first processing result to obtain the second processing result.

[0072] The second processing path is a task processing path for processing the second computing task through a target processor. The target processor may be a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an NPU (Neural Network Processing Unit), a TPU (Tensor Processing Unit), a DPU (Deep Learning Processing Unit), a microprocessor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of at least two of these processor types.

[0073] For example, the first processing result The second computing task is sent to the target processor through the second processing path. The second computing task takes the activation function as an example, and the target processor takes the GPU as an example. After the GPU receives the first processing result Y, it executes the activation function to obtain the second processing result as the output result of the current network layer.

[0074] In this embodiment, by ensuring that the advance calculation task and the nonlinear calculation task are efficiently executed on different hardware, the high parallelism of RRAM and the high memory bandwidth of the processor are utilized to improve the efficiency of neural network calculation.

[0075] In one embodiment, based on step S7 above, after obtaining the second processing result, the performance parameters of the second processing result are evaluated, and dynamic adjustments are made based on the evaluation results. Specifically, the first computing task and the second computing task are calculated according to the second processing path to obtain a third processing result, the performance parameters of the second processing result and the third processing result are evaluated, and the target RRAM or the first processing path is adjusted based on the evaluation results.

[0076] In this embodiment, by verifying whether the results of accelerating the calculation of the neural network using RRAM meet the performance requirements, it can be used to subsequently optimize the RRAM. Exemplarily, the above-mentioned performance parameters may include one or more parameters of accuracy, calculation speed, calculation power consumption, hardware stability and system scalability. The above-mentioned accuracy can be understood as the calculation error between the RRAM output value and the theoretical value, usually the numerical deviation of the matrix multiplication caused by hardware instability such as the conductance value. The above-mentioned calculation speed can be a single inference delay or throughput, the above-mentioned calculation power consumption can be understood as the total energy consumption of the system, the above-mentioned hardware stability can be the failure rate of the RRAM unit or the drift amplitude of the conductance value, and the above-mentioned system scalability can be understood as the maximum supported neural network scale or the collaborative efficiency of multiple computing tasks.

[0077] In this embodiment, the processor processes both the first and second processing tasks to obtain a third processing result, which is then compared with the second processing result. The third processing result serves as the standard output of the processor's neural network acceleration calculations. By quantifying the difference between the second and third processing results, the RRAM processing results can be evaluated.

[0078] In this embodiment, the performance parameters may include accuracy and computing power consumption. If the computing accuracy of the second processing result is lower than that of the third processing result, the hardware parameters of the target RRAM are adjusted. If the computing power consumption of the second processing result is greater than that of the third processing result, the second processing path is determined to be the first processing path.

[0079] The accuracy can be assessed by quantifying the difference between the second and third processing results. For example, if the accuracy of the third processing result calculated by the GPU is 98%, while the accuracy of the second processing result calculated by the RRAM is 97.2%, the error is 0.8%. It is necessary to analyze whether the error is caused by conductance drift and adjust the RRAM and the first processing path accordingly. Specifically, if the accuracy of the second processing result is lower than that of the third processing result, the hardware parameters of the target RRAM in the first processing path can be adjusted, or an alternative first processing path can be used for another target RRAM.

[0080] The aforementioned computational power consumption can be assessed by recording the real-time power consumption of RRAM and GPU processing using an external instrument. For example, RRAM takes 5ms and consumes 0.1J of power to complete an inference, while the GPU takes 3ms and consumes 0.5J of power. For the same timeframe, RRAM consumes less power. For example, when the computational power consumption of the second computation result is greater than the computational power consumption of the third computation result, the second processing path is determined to be the first processing path, meaning that both the first and second computation tasks are processed by the processor of the first processing path.

[0081] This application provides one or more embodiments of the method for accelerating neural network computing based on RRAM. The embodiment is performed according to the following steps:

[0082] Example 1: Deploy a CNN network for image classification.

[0083] In this embodiment, if there are four RRAM boards, each containing a 128×128 array and supporting parallel computing, the conductance range and number of valid rows and columns of each RRAM board are first determined. If the conductance value of the fourth column of the second RRAM board exceeds the preset tolerance, the storage array of the fourth column of the second RRAM board is skipped, and the remaining storage arrays are used as valid rows and columns. Furthermore, the weight matrix of the CNN network is calculated in parallel in the valid rows and columns using the RRAM, and nonlinear operations such as the ReLU activation function are processed on the GPU to obtain the final CNN processing results. Simultaneously, based on the number of parallel operations and storage results, the data block partitioning and transmission path of the weight matrix are optimized to improve the accuracy and processing efficiency of the CNN network.

[0084] Example 2: Deploy a ResNet-50 (Residual Nerwork 50 layers, a deep residual network) for image classification.

[0085] In this embodiment, if there are four RRAM boards and the conductance of the second column of the first RRAM board exceeds a preset tolerance, the second column of the first board is skipped and the convolution calculation is distributed to other valid rows and columns. For example, the positive and negative weight arrays are stored in the valid rows and columns of the first board and the valid rows and columns of the third board respectively. The difference calculation can offset some hardware errors. The matrix multiplication of all convolutional layers, such as a 3×3 convolution kernel, is processed by the valid rows and columns of the RRAM to obtain a feature map. The GPU then performs normalization and residual connection on the feature map blocks.

[0086] See also Figure 5 The present application also provides a system for accelerating neural network computing based on RRAM. The system includes:

[0087] A parameter acquisition unit 100 is configured to acquire hardware parameters of one or more RRAMs, the hardware parameters including at least a parallelizable number, and determine configuration parameters of each RRAM according to a preset tolerance, the configuration parameters including at least an effective conductance range and a plurality of effective rows and columns;

[0088] A task splitting unit 200 is used to split the computing task of any network layer of the neural network and generate a first computing task and a second computing task based on the splitting result;

[0089] The task computing unit 300 is configured to determine a first processing path for the first computing task based on the parallelizable number and the effective conductance range, the first processing path being connected to a target RRAM, processing the first computing task according to effective rows and columns of the target RRAM to obtain a first processing result, and determining a second processing path for the second computing task, processing the second computing task according to the second processing path and the first processing result to obtain a second processing result, and using the second processing result as the output result of the current network layer.

[0090] In one embodiment, the parameter acquisition unit 100 is specifically configured to take the storage rows and columns that meet a preset tolerance as valid rows and columns, obtain effective conductance values ​​of the respective valid rows and columns, and determine the effective conductance range of the RRAM according to the effective conductance values.

[0091] In one embodiment, the task splitting unit 200 is specifically used to determine the various computing tasks of the neural network according to the type of the neural network, and for any of the computing tasks, determine whether the computing task belongs to a linear computing task; if the computing task belongs to a linear computing task, determine the task as a first computing task; if the computing task does not belong to a linear computing task, determine the task as a second computing task.

[0092] In one embodiment, the task computing unit 300 is specifically used to determine the positive weight array and the negative weight array of the target RRAM based on the valid rows and columns, calculate the first computing task according to the positive weight array and the negative weight array to obtain a first processing result, determine the target processor connected to the neural network, determine the path connecting the target processor and the neural network as the second processing path, send the second computing task and the first processing result to the target processor through the first processing path, and the target processor executes the second computing task according to the first processing result to obtain the second processing result.

[0093] In one embodiment, the system further includes a verification unit, which is configured to calculate the first computing task and the second computing task according to the first processing path to obtain a third processing result, evaluate performance parameters of the second processing result and the third processing result, and adjust the target RRAM or the first processing path based on the evaluation results, wherein the performance parameters include at least calculation accuracy and calculation power consumption.

[0094] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0095] In an embodiment of the present application, a system for accelerating neural network computing based on RRAM is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, or other devices that can provide the above functions.

[0096] In one embodiment, the verification unit may include one or more processors (or processing circuits) configured to implement the above methods, a memory, and a computer program stored in the memory, for implementing any of the above methods. The processor may be selected from: a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an NPU (Neural Network Processing Unit), a TPU (Tensor Processing Unit), a DPU (Deep Learning Processing Unit), a microprocessor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of at least two of these processor types. The memory stores instructions that can be executed by at least one processor, and the at least one processor may be of different types, such as a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc. The computer program may be written in Python, and may also be written in other programming languages, including but not limited to Python, Perl, and TCL.

[0097] In one embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit. The logical relationship of the hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0098] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 6 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0099] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0100] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0101] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0102] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0103] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0104] The systems, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0105] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0106] The present application is described with reference to the flowcharts and / or block diagrams of the methods and systems according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0109] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0110] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0111] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

[0112] Although the embodiments of the present application have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations shall fall within the scope defined by the appended claims.

Claims

1. A method for accelerating neural network computing based on RRAM, characterized in that: The method comprises: Obtaining hardware parameters of one or more RRAMs, the hardware parameters including at least a parallelizable number, and determining configuration parameters of each RRAM according to a preset tolerance, the configuration parameters including at least an effective conductance range and a plurality of effective rows and columns; Splitting the computational task of any network layer of the neural network, and generating a first computational task and a second computational task based on the splitting result; determining a first processing path for the first computing task based on the parallelizable number and the effective conductance range, wherein the first processing path is connected to a target RRAM and processes the first computing task according to effective rows and columns of the target RRAM to obtain a first processing result; determining a second processing path for the second computing task, processing the second computing task according to the second processing path and the first processing result to obtain a second processing result, and using the second processing result as an output result of the current network layer; The step of splitting the computing task of the neural network and generating the first computing task and the second computing task based on the splitting result includes: Determining, according to the type of the neural network, various computing tasks of the neural network, and determining, for any of the computing tasks, whether the computing task is a linear computing task; If the computing task belongs to a linear computing task, the task is determined as a first computing task; if the computing task does not belong to a linear computing task, the task is determined as a second computing task; The second processing path is connected to the processor; determining the second processing path of the second computing task, and processing the second computing task according to the second processing path and the first processing result to obtain a second processing result includes: determining a target processor connected to the neural network, and determining a path connecting the target processor and the neural network as the second processing path; The second computing task and the first processing result are sent to the target processor through the second processing path. The target processor executes the second computing task according to the first processing result to obtain the second processing result.

2. The method according to claim 1, characterized in that After obtaining the second processing result, the method further includes: calculating the first computing task and the second computing task according to the second processing path to obtain a third processing result; Performance parameters of the second processing result and the third processing result are evaluated, and the target RRAM or the first processing path is adjusted based on the evaluation result.

3. The method according to claim 2, characterized in that The performance parameters include at least calculation accuracy and calculation power consumption; evaluating the performance parameters of the second processing result and the third processing result, and adjusting the target RRAM or the first processing path based on the evaluation results includes: If the calculation accuracy of the second processing result is lower than that of the third processing result, adjusting the hardware parameters of the target RRAM; If the computing power consumption of the second processing result is greater than that of the third processing result, the second processing path is determined to be the first processing path.

4. The method according to claim 1, wherein The RRAM includes a plurality of storage rows and columns; determining the configuration parameters of each of the RRAMs according to a preset tolerance includes: Taking the stored rows and columns that meet the preset tolerance as valid rows and columns; The effective conductance values ​​of the effective rows and columns are obtained, and the effective conductance range of the RRAM is determined according to the effective conductance values.

5. The method according to claim 1, wherein Determining a first processing path for the first computing task based on the parallelizable number and the effective conductance range, wherein the first processing path is connected to the target RRAM, includes: Determining an RRAM matching the first computing task according to the parallelizable number and the effective conductance range; The matched RRAM is determined as a target RRAM, and a path connecting the target RRAM and the neural network is determined as a first processing path.

6. The method according to claim 1, characterized in that Processing the first computing task according to valid rows and columns of the target RRAM to obtain a first processing result includes: determining a positive weight array and a negative weight array of a target RRAM based on the valid rows and columns; The first computing task is calculated according to the positive weight array and the negative weight array to obtain a first processing result.

7. The method according to claim 6, characterized in that Determining a positive weight array and a negative weight array of a target RRAM based on the valid rows and columns includes: Obtaining effective conductance values ​​of the respective effective rows and columns, determining a reference row and column among the effective rows and columns of the target RRAM, and determining the effective conductance values ​​of the reference row and column as reference values; The plurality of valid rows and columns whose effective conductance values ​​are greater than or equal to a reference value are determined as a positive weight array, and the plurality of valid rows and columns whose effective conductance values ​​are less than a reference value are determined as a negative weight array.

8. The method according to claim 6, characterized in that Calculating the first computing task according to the positive weight array and the negative weight array to obtain a first processing result includes: Obtaining a positive weight matrix and a negative weight matrix of the neural network, storing the positive weight matrix in the positive weight array, and storing the negative weight matrix in the negative weight array; A first output current of the positive weight array and a second output current of the negative weight array are obtained, and a difference between the first output current and the second output current is used as the first processing result.

9. A system for accelerating neural network computing based on RRAM, characterized in that: The system comprises: a parameter acquisition unit, configured to acquire hardware parameters of one or more RRAMs, the hardware parameters including at least a parallelizable number, and determine configuration parameters of each RRAM according to a preset tolerance, the configuration parameters including at least an effective conductance range and a plurality of effective rows and columns; a task splitting unit, configured to split the computing tasks of any network layer of the neural network, and generate a first computing task and a second computing task based on the splitting result, wherein splitting the computing tasks of the neural network and generating the first computing task and the second computing task based on the splitting result comprises: determining each computing task of the neural network according to the type of the neural network, and for any computing task, determining whether the computing task belongs to a linear computing task, if the computing task belongs to a linear computing task, determining the task as the first computing task, and if the computing task does not belong to a linear computing task, determining the task as the second computing task; A task computing unit is used to determine a first processing path for the first computing task based on the parallelizable number and the effective conductance range, the first processing path being connected to a target RRAM, processing the first computing task according to the effective rows and columns of the target RRAM to obtain a first processing result, and determining a second processing path for the second computing task, processing the second computing task according to the second processing path and the first processing result to obtain a second processing result, and using the second processing result as the output result of the current network layer, wherein the second processing path is connected to a processor; determining the second processing path for the second computing task, processing the second computing task according to the second processing path and the first processing result to obtain a second processing result includes: determining a target processor connected to the neural network, determining the path connecting the target processor and the neural network as the second processing path; sending the second computing task and the first processing result to the target processor through the second processing path, and the target processor executing the second computing task according to the first processing result to obtain the second processing result.

10. The system according to claim 9, characterized in that The system further comprises: a verification unit, configured to calculate the first computing task and the second computing task according to the first processing path to obtain a third processing result, evaluate performance parameters of the second processing result and the third processing result, and adjust the target RRAM or the first processing path based on the evaluation results, wherein the performance parameters include at least calculation accuracy and calculation power consumption.

11. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the RRAM-based neural network accelerated computing method according to any one of claims 1 to 8 by executing the computer instructions.

Images