A weight inference mapping method for optimizing memristor-based neural networks

Abstract

This invention discloses a weight inference mapping method for optimizing memristor-based neural networks. First, the weight mapping parameter C is calculated based on the memristor conductance range and the weight range. t And the signal input transformation parameter θ; the weight matrix is ​​transformed according to the weight mapping parameter C t The signal is amplified proportionally and mapped onto a memristor array via an external circuit board. The voltage signal input to the memristor array is first scaled down proportionally according to the signal input conversion parameter θ, and then input into the mapped memristor array for matrix multiplication and addition. A constant resistor array is used to convert the current value output from the memristor array into a voltage signal. This voltage signal is added to the voltage signal at the next time step to obtain the input voltage signal for the next time step, thus initiating the weight inference mapping process for the next cycle. This method solves the problem of timing information loss during the weight inference mapping process, enabling RNN-type algorithms (such as GRU, LSTM, etc.) to be implemented entirely in hardware.

Description

Technical Field

[0001] This invention relates to the field of memristor-based neural network technology, and in particular to a weight inference mapping method for optimizing memristor-based neural networks. Background Technology

[0002] In memristor-based neural networks, the unique recursive mechanism of recurrent neural networks (RNNs) reduces on-chip area and saves computing power. However, it also causes problems such as difficulty in full hardware inference and low stability of weight mapping. This is one of the main obstacles to memristor-based RNN systems in practical applications. In hardware neural networks, the weight inference and mapping process is an important part that determines the test accuracy.

[0003] Traditional memristor-based RNNs only perform inference and mapping at the last loop iteration. This mapping method introduces significant errors and hinders the full hardware implementation of memristor-based RNNs. Furthermore, in more complex recurrent neural networks (such as Long Short-Term Memory networks and gated recurrent units), this approach leads to uncontrollable return voltage values ​​during training, and this problem becomes more pronounced as the time dimension increases. Summary of the Invention

[0004] The purpose of this invention is to provide a weight inference mapping method for optimizing memristor-based neural networks. This method solves the problem of timing information loss in the weight inference mapping process, enabling RNN-type algorithms (such as GRU, LSTM, etc.) to be implemented entirely in hardware, and providing a new approach for hardware neural network systems.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for optimizing weight inference mapping in memristor-based neural networks, the method comprising:

[0007] Step 1: Calculate the weight mapping parameter C based on the memristor conductance range and weight range. t and signal input conversion parameter θ;

[0008] Step 2: Adjust the weight matrix according to the weight mapping parameter C. t The circuit is amplified proportionally and then mapped onto the memristor array via an external circuit board.

[0009] Step 3: First, scale down the voltage signal input to the memristor array according to the signal input conversion parameter θ, and then input it into the memristor array mapped in Step 2 for matrix multiplication and addition operations.

[0010] Step 4: Use a constant resistor array to convert the current value output by the memristor array in Step 3 into a voltage signal. Add this voltage signal to the voltage signal at the next time step to obtain the input voltage signal at the next time step, so as to perform the weight reasoning mapping process for the next cycle.

[0011] As can be seen from the technical solution provided by the present invention, the above method solves the problem of loss of temporal information in the weight inference mapping process, enabling RNN-type algorithms (such as GRU, LSTM, etc.) to be implemented entirely in hardware, and providing a new approach for hardware neural network systems. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of the weight inference mapping method for optimizing memristor-based neural networks provided in an embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, and do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0015] like Figure 1 The diagram shown is a flowchart of a weight inference mapping method for optimizing a memristor-based neural network provided by an embodiment of the present invention. The method includes:

[0016] Step 1: Calculate the weight mapping parameter C based on the memristor conductance range and weight range. t and signal input conversion parameter θ;

[0017] In this step, the weight mapping parameter C is specifically calculated according to the following formulas (1) and (2). t And signal input conversion parameter θ:

[0018]

[0019] in, The maximum value minus the minimum value of the weight matrix after inference is completed at time t represents the range of values ​​for the weight matrix. It is the maximum resistance that the memristor array can achieve at time t minus the minimum resistance (in kΩ).

[0020] Step 2: Adjust the weight matrix according to the weight mapping parameter C. t The circuit is amplified proportionally and then mapped onto the memristor array via an external circuit board.

[0021] In this step, the weight matrix is ​​specifically processed using the following formula (3):

[0022]

[0023] in, Let be the resistance value of the i-th row and j-th column in the memristor array at time t; Let C be the weight in the i-th row and j-th column of the weight matrix after inference is completed at time t; t These are the weight mapping parameters.

[0024] Step 3: First, scale down the voltage signal input to the memristor array according to the signal input conversion parameter θ, and then input it into the memristor array mapped in Step 2 for matrix multiplication and addition operations.

[0025] In this step, the voltage signal of the input memristor array is specifically processed using the following formula (4):

[0026]

[0027] Among them, X t θ represents the voltage signal input to the memristor array; θ is the signal input conversion parameter. X after proportional scaling t .

[0028] Step 4: Use a constant resistor array to convert the current value output by the memristor array in Step 3 into a voltage signal. Add this voltage signal to the voltage signal at the next time step to obtain the input voltage signal at the next time step, so as to perform the weight reasoning mapping process for the next cycle.

[0029] In this step, assuming that the number of memristor array columns mapped in step 2 is n, then the size of the constant resistor array used is 1×n.

[0030] The key to memristor-based RNN weight inference and mapping lies in the loss of information during inference and the low robustness during mapping, making it unable to handle complex timing information and thus unable to solve complex timing problems. To address this issue, this invention utilizes the geometric topology theory of memristors from cells to arrays, scaling the input electrical signal and weight matrix proportionally according to different parameters and performing cyclic normalization, thereby solving the problem of timing information loss during the inference and mapping process.

[0031] It is worth noting that the contents not described in detail in the embodiments of the present invention belong to the prior art known to those skilled in the art.

[0032] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A method for optimizing weight inference mapping in memristor-based neural networks, characterized in that, The method includes: Step 1: Calculate the weight mapping parameter C based on the memristor conductance range and weight range. t and signal input conversion parameter θ; in step 1, Specifically, the weight mapping parameter C is calculated according to the following formulas (1) and (2). t And signal input conversion parameter θ: in, The maximum value minus the minimum value of the weight matrix after inference is completed at time t represents the range of values ​​for the weight matrix. The maximum resistance that the memristor array can achieve at time t is the minimum resistance value minus the maximum resistance value. Step 2: Adjust the weight matrix according to the weight mapping parameter C. t The circuit is amplified proportionally and then mapped onto the memristor array via an external circuit board. Step 3: First, scale down the voltage signal input to the memristor array proportionally according to the signal input conversion parameter θ, and then input it into the memristor array mapped in Step 2 for matrix multiplication and addition operations; specifically, the voltage signal input to the memristor array is processed using the following formula (4): Among them, X t θ represents the voltage signal input to the memristor array; θ is the signal input conversion parameter. X after proportional scaling t ; Step 4: Use a constant resistor array to convert the current value output by the memristor array in Step 3 into a voltage signal. Add this voltage signal to the voltage signal at the next time step to obtain the input voltage signal at the next time step, so as to perform the weight reasoning mapping process for the next cycle.

2. The weight inference mapping method for optimizing memristor-based neural networks according to claim 1, characterized in that, In step 2, the weight matrix is ​​processed using the following formula (3): in, Let be the resistance value of the i-th row and j-th column in the memristor array at time t; Let C be the weight in the i-th row and j-th column of the weight matrix after inference is completed at time t; t These are the weight mapping parameters.

3. The weight inference mapping method for optimizing memristor-based neural networks according to claim 1, characterized in that, In step 4, Assuming that the number of memristor array columns completed in step 2 is n, then the size of the constant resistor array used is 1×n.

Images