A reRAM random write error-oriented typical deep learning network precision compensation method

Abstract

This invention relates to a typical deep learning network accuracy compensation method for random write errors in ReRAM. It pertains to the fields of artificial intelligence hardware acceleration and neural network model optimization. The invention involves multiple writes of different target conductance values ​​onto a ReRAM device, establishing a probability distribution of conductance error using kernel density estimation. This distribution is then fitted to a normal distribution, and a functional relationship is established between the standard deviation and the ideal conductance value. Based on this function, a prior distribution of the weights in a Bayesian neural network is constructed, and an L2 regularization penalty term is introduced into the posterior distribution mean during training to suppress error amplification sensitivity. This invention achieves model-level compensation for ReRAM hardware errors, effectively improving network inference accuracy and robustness without requiring redundant mapping or online calibration.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence hardware acceleration and neural network model optimization technology, and is a typical deep learning network accuracy compensation method for ReRAM random write errors. Background Technology

[0002] As the energy efficiency requirements for artificial intelligence inference tasks continue to increase, neuromorphic computing architectures based on ReRAM devices have gradually become a research hotspot. ReRAM devices have advantages such as non-volatility and high density, and are widely used for storing neural network weights and accelerating matrix multiplication and addition operations.

[0003] ReRAM (Resistive Random Access Memory) is a non-volatile memory developed by Fujitsu. Based on the memristor principle, it stores data by utilizing the resistance changes in a metal-dielectric-metal (MIM) three-layer structure. It combines the read / write speed of dynamic random access memory (DRAM) with the non-volatility of solid-state drives (SSDs), retaining data even after power is off. It also features low power consumption and compatibility with CMOS technology.

[0004] The development of this technology began in 2013 with Panasonic's global debut of the ReRAM product. Fujitsu and Panasonic jointly developed a 4MB chip in 2016, and launched the 12Mbit capacity product MB85AS12MT in 2022. Major international companies, including Heycom, Elpida, Sony, Micron, and SK Hynix, have all deployed ReRAM for commercial applications. ReRAM offers read and write speeds 1000 times faster than NAND Flash and consumes 15 times less power, making it applicable to IoT, autonomous driving, PCs, and in-memory computing architectures. It enables instant response without loading time, breaking through the von Neumann bottleneck. Fujitsu is expanding capacity through stacking technology and plans to develop a new generation of products.

[0005] However, due to the non-ideal nature of the ReRAM physical mechanism, even when the same pulse sequence is applied during actual writing, the final conductance value fluctuates significantly. This random write error (variation) will significantly affect the inference accuracy of the neural network, especially in low-power, low-precision mapping environments.

[0006] In existing technologies, the impact of ReRAM errors is mainly mitigated in the following ways: using multiple ReRAM cells to redundantly store the same weights improves accuracy, but significantly increases area and power consumption; repeatedly reading and writing to the ReRAM array to train the network can improve accuracy, but the hardware overhead is large; some literature attempts to use Bayesian neural networks (BNN) instead of CNN to improve tolerance to weight perturbations, but their prior distribution design is often empirical and fails to be customized and optimized for ReRAM error models.

[0007] Therefore, there is an urgent need for a method that can fully utilize the statistical laws of ReRAM errors, embed them into the neural network training process, and improve the robustness of hardware inference from the algorithm level. Summary of the Invention

[0008] This invention provides a typical deep learning network accuracy compensation method for ReRAM random write errors. By combining modeling the actual write error distribution of ReRAM devices with a Bayesian neural network training mechanism, this invention achieves algorithm-level compensation for hardware errors, solving the problem of decreased inference accuracy in existing deployments. The invention provides the following technical solutions: A typical deep learning network accuracy compensation method for ReRAM random write errors, the method comprising the following steps: The method includes the following steps: Step 1: In the ReRAM device, for the target conductance value G ideal The same number of pulses were applied repeatedly, the conductivity value after writing was recorded, and the probability distribution curve of the KDE was obtained by kernel density estimation. Step 2: Fit the obtained conductivity distribution to a normal distribution, extract the mean, and repeat the above steps for multiple target conductivity values ​​to establish the standard deviation. With the target conductivity value The functional relationship; Step 3: Construct a prior distribution based on the functional relationship obtained in Step 2, which serves as the prior probability distribution of the weight parameters of each layer of the Bayesian neural network. Step 4: During the variational inference training of BNN, modify the loss function and introduce an L2 regularization term of the posterior mean to penalize the sum of squares of the mean terms in all weighted distributions; Step 5: Train the Bayesian neural network based on the modified loss function to obtain the posterior mean parameter of the weight distribution, and map the mean parameter as the weight value into the ReRAM array for accuracy compensation.

[0009] Preferably, standard deviation With the target conductivity value The functional relationship between them is in the form of a linear function:

[0010] Where k is a scaling factor obtained by fitting multiple sets of ReRAM writing experiments.

[0011] Preferably, the kernel density estimation KDE used in step 1 is a Gaussian kernel function to smooth the probability density distribution of the ReRAM writing conductivity value.

[0012] Preferably, step 3 specifically comprises: Construct a prior distribution based on the functional relationship obtained in step 2, which serves as the prior probability distribution for the weight parameters of each layer in the Bayesian neural network. This distribution is then applied to each weight in the trained traditional CNN network. Let its prior distribution be:

[0013] Preferably, step 4 specifically comprises: The BNN is trained using a variational Bayesian method, introducing a posterior mean penalty term based on the standard variational inference loss function ELBO.

[0014] This regularization term constrains the mean of the posterior distribution, compresses the weight magnitude, and suppresses the ReRAM error amplification effect.

[0015] Preferably, the posterior mean penalty term is:

[0016] in, Let represent the mean of the i-th weight distribution. The regularization coefficient is used to adjust the degree of influence of this factor.

[0017] Preferably, step 5 specifically comprises: Extract the posterior mean of each weight in the trained Bayesian neural network. As the conductance value mapped to the ReRAM array, the required number of pulses for writing is determined by looking up a table or fitting a relationship, and a conformance is introduced into the writing conductance. The perturbation is used to simulate the inference accuracy under the error of actual devices.

[0018] A typical deep learning network accuracy compensation system for ReRAM random write errors, the system comprising: The data recording module is for the target conductivity value G. ideal The same number of pulses were applied repeatedly, the conductivity value after writing was recorded, and the probability distribution curve of the KDE was obtained by kernel density estimation. The fitting module fits the obtained conductance distribution to a normal distribution, extracts the mean, and repeats the above steps for multiple target conductance values ​​to establish the standard deviation. With the target conductivity value The functional relationship; The model building module constructs a prior distribution based on the obtained functional relationship, which serves as the prior probability distribution of the weight parameters of each layer of the Bayesian neural network. The model training module modifies the loss function during the variational inference training of the BNN by introducing an L2 regularization term of the posterior mean to penalize the sum of squares of the mean terms in all weighted distributions. The inference compensation module trains the Bayesian neural network based on the modified loss function to obtain the posterior mean parameter of the weight distribution, and maps the mean parameter as a weight value into the ReRAM array for accuracy compensation.

[0019] A readable storage medium that stores data in the form of electrical conductivity is characterized by being able to perform computations locally on the storage device, achieving in-memory computation, for use in implementing a typical deep learning network accuracy compensation method for ReRAM random write errors.

[0020] The present invention has the following beneficial effects: Compared with existing technologies, this invention establishes an error statistical model based on the characteristics of actual ReRAM devices and constructs a prior model that can be embedded with a Bayesian neural network. –G mapping relationship.

[0021] This invention introduces a posterior mean penalty to further compress high-error weights and improve the robustness of system inference; compared with existing solutions, it can achieve accuracy improvement through a single training session without hardware redundancy or complex online training. Attached Figure Description

[0022] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 The flowchart shown is the overall process flow of the method of the present invention. Figure 2 The diagram shows the kernel density estimation results and normal distribution fitting curves under multiple conductivity target values ​​of the present invention. Figure 3 The standard deviation of this invention is shown. With target conductivity Fitting curves between; Figure 4 The diagram shows the structure of the improved BNN training loss function of this invention. Detailed Implementation

[0024] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] The present invention will be described in detail below with reference to specific embodiments. Specific Implementation Example 1: according to Figures 1 to 4 As shown, the specific optimization technical solution adopted by the present invention to solve the above-mentioned technical problems is: The present invention relates to a typical deep learning network accuracy compensation method for ReRAM random write errors.

[0027] This invention provides a typical deep learning network accuracy compensation method for ReRAM random write errors, the method comprising the following steps: Step 1: In the ReRAM device, for the target conductance value G ideal The same number of pulses were applied repeatedly, the conductivity value after writing was recorded, and the probability distribution curve of the KDE was obtained by kernel density estimation. Step 2: Fit the obtained conductivity distribution to a normal distribution, extract the mean, and repeat the above steps for multiple target conductivity values ​​to establish the standard deviation. With the target conductivity value The functional relationship; Step 3: Construct a prior distribution based on the functional relationship obtained in Step 2, which serves as the prior probability distribution of the weight parameters of each layer of the Bayesian neural network. Step 4: During the variational inference training of BNN, modify the loss function and introduce an L2 regularization term of the posterior mean to penalize the sum of squares of the mean terms in all weighted distributions; Step 5: Train the Bayesian neural network based on the modified loss function to obtain the posterior mean parameter of the weight distribution, and map the mean parameter as the weight value into the ReRAM array for accuracy compensation.

[0028] This invention relates to a method for compensating for random write errors in ReRAM in deep learning networks, belonging to the field of neural network model hardware mapping and inference optimization technology. The method includes the following steps: writing different target conductance values ​​multiple times to the ReRAM device, establishing a probability distribution of conductance error using kernel density estimation; fitting the obtained distribution to a normal distribution and establishing a functional relationship between the standard deviation and the ideal conductance value; constructing a prior distribution of weights in a Bayesian neural network (BNN) based on this function, and introducing an L2 regularization penalty term to the posterior distribution mean during training to suppress error amplification sensitivity; obtaining a robust set of weight mean values ​​through training, and mapping them to the ReRAM array for inference. This method achieves model-level compensation for ReRAM hardware errors, effectively improving network inference accuracy and robustness without redundant mapping or online calibration, and has good versatility and engineering practical value. Specific Implementation Example 2: The only difference between Embodiment 2 and Embodiment 1 of this application is that: Standard deviation With the target conductivity value The functional relationship between them is in the form of a linear function:

[0030] Where k is a scaling factor obtained by fitting multiple sets of ReRAM writing experiments. Specific Implementation Example 3: The only difference between Embodiment 3 and Embodiment 2 of this application is that: The kernel density estimation KDE used in step 1 is a Gaussian kernel function to smooth the probability density distribution of the ReRAM write conductivity value. Specific Implementation Example 4: The only difference between Embodiment 4 and Embodiment 3 of this application is that: Step 3 specifically involves: Construct a prior distribution based on the functional relationship obtained in step 2, which serves as the prior probability distribution for the weight parameters of each layer in the Bayesian neural network. This distribution is then applied to each weight in the trained traditional CNN network. Let its prior distribution be: Specific Implementation Example 5: The difference between Embodiment 5 and Embodiment 4 of the present invention lies only in: Step 4 specifically involves: The BNN is trained using a variational Bayesian method, introducing a posterior mean penalty term based on the standard variational inference loss function ELBO.

[0034] This regularization term constrains the mean of the posterior distribution, compresses the weight magnitude, and suppresses the ReRAM error amplification effect. Specific Implementation Example Six: The difference between Embodiment Six and Embodiment Five of the present invention lies only in: The posterior mean penalty term is:

[0036] in, Let represent the mean of the i-th weight distribution. The regularization coefficient is used to adjust the degree of influence of this factor. Specific Implementation Example 7: The difference between Embodiment Seven and Embodiment Six of the present invention lies only in: Step 5 specifically involves: Extract the posterior mean of each weight in the trained Bayesian neural network. As the conductance value mapped to the ReRAM array, the required number of pulses for writing is determined by looking up a table or fitting a relationship, and a conformance is introduced into the writing conductance. The perturbation is used to simulate the inference accuracy under the error of actual devices. Specific Implementation Example 8: The difference between Embodiment 8 and Embodiment 7 of the present invention lies only in: This invention provides a typical deep learning network accuracy compensation system for ReRAM random write errors, the system comprising: The data recording module is for the target conductivity value G. ideal The same number of pulses were applied repeatedly, the conductivity value after writing was recorded, and the probability distribution curve of the KDE was obtained by kernel density estimation. The fitting module fits the obtained conductance distribution to a normal distribution, extracts the mean, and repeats the above steps for multiple target conductance values ​​to establish the standard deviation. With the target conductivity value The functional relationship; The model building module constructs a prior distribution based on the obtained functional relationship, which serves as the prior probability distribution of the weight parameters of each layer of the Bayesian neural network. The model training module modifies the loss function during the variational inference training of the BNN by introducing an L2 regularization term of the posterior mean to penalize the sum of squares of the mean terms in all weighted distributions. The inference compensation module trains the Bayesian neural network based on the modified loss function to obtain the posterior mean parameter of the weight distribution, and maps the mean parameter as a weight value into the ReRAM array for accuracy compensation. Specific Implementation Example Nine: The difference between Embodiment Nine and Embodiment Eight of the present invention lies only in: This invention provides a readable storage medium that stores data in conductive form and can perform operations locally on the storage device, achieving in-memory computing integration, for use in implementing a typical deep learning network accuracy compensation method for ReRAM random write errors.

[0040] The method includes the following steps: Step 1: In the ReRAM device, for the target conductance value G ideal The same number of pulses were applied repeatedly, the conductivity value after writing was recorded, and the probability distribution curve of the KDE was obtained by kernel density estimation. Step 2: Fit the obtained conductivity distribution to a normal distribution, extract the mean, and repeat the above steps for multiple target conductivity values ​​to establish the standard deviation. With the target conductivity value The functional relationship; Step 3: Construct a prior distribution based on the functional relationship obtained in Step 2, which serves as the prior probability distribution of the weight parameters of each layer of the Bayesian neural network. Step 4: During the variational inference training of BNN, modify the loss function and introduce an L2 regularization term of the posterior mean to penalize the sum of squares of the mean terms in all weighted distributions; Step 5: Train the Bayesian neural network based on the modified loss function to obtain the posterior mean parameter of the weight distribution, and map the mean parameter as the weight value into the ReRAM array for accuracy compensation. Specific Implementation Example 10: The only difference between Embodiment 10 and Embodiment 9 of the present invention is that: A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement a typical deep learning network accuracy compensation method for ReRAM random write errors.

[0042] The method includes the following steps: Step 1: In the ReRAM device, for the target conductance value G ideal The same number of pulses were applied repeatedly, the conductivity value after writing was recorded, and the probability distribution curve of the KDE was obtained by kernel density estimation. Step 2: Fit the obtained conductivity distribution to a normal distribution, extract the mean, and repeat the above steps for multiple target conductivity values ​​to establish the standard deviation. With the target conductivity value The functional relationship; Step 3: Construct a prior distribution based on the functional relationship obtained in Step 2, which serves as the prior probability distribution of the weight parameters of each layer of the Bayesian neural network. Step 4: During the variational inference training of BNN, modify the loss function and introduce an L2 regularization term of the posterior mean to penalize the sum of squares of the mean terms in all weighted distributions; Step 5: Train the Bayesian neural network based on the modified loss function to obtain the posterior mean parameter of the weight distribution, and map the mean parameter as the weight value into the ReRAM array for accuracy compensation. Specific Implementation Example Eleven: The only difference between Embodiment Eleven and Embodiment Ten of this invention is that: This invention belongs to the field of artificial intelligence hardware acceleration and neural network model optimization, specifically involving a typical deep learning network accuracy compensation method for ReRAM random write errors, aiming to improve the inference robustness and accuracy when deploying neural networks in ReRAM arrays.

[0044] like Figure 1 As shown, the implementation process of this invention includes five steps: Step 1: In the ReRAM device, for the target conductance value G ideal The same number of pulses were applied repeatedly, the conductivity value after writing was recorded, and its probability distribution curve was obtained using kernel density estimation (KDE). In this embodiment, a series of target conductance values ​​are selected, specifically 6. 9 12 15 18 、 twenty one ,twenty four 27 A fixed number of pulses were applied to the ReRAM device for writing, and this process was repeated 1000 times, recording the final conductance value obtained from each write operation. Gaussian kernel density estimation (KDE) was used to model the probability density of each target conductance value, such as... Figure 2 As shown, the resulting distribution approximates a normal distribution; In step one, the kernel density estimation used is a Gaussian kernel function to smooth the probability density distribution of the ReRAM writing conductivity value; Step 2: Fit the obtained conductivity distribution to a normal distribution and extract the mean. The above steps were repeated at multiple target conductance values ​​to establish the standard deviation. With the target conductivity value The functional relationship, ; like Figure 3As shown, the two exhibit a strong linear correlation, leading to the establishment of the following error function model:

[0045] Where k is the proportionality coefficient obtained from the fitting, indicating that the higher the conductivity value, the greater the error.

[0046] Step 3: Construct a prior distribution based on the functional relationship obtained in Step 2, which serves as the prior probability distribution of the weight parameters of each layer of the Bayesian Neural Network (BNN). In this example, the above error model is used to construct the prior distribution of each weight in a Bayesian neural network (BNN). For each weight in a trained traditional CNN network... Let its prior distribution be:

[0047] in From the above formula The decision is to differentiate the model based on the conductivity value it maps to.

[0048] Step 4: During the variational inference training of BNN, modify the loss function. In addition to including the expected negative log-likelihood term and the KL divergence between the prior and posterior, further introduce the L2 regularization term of the posterior mean to penalize the sum of squares of the mean terms in all weighted distributions. In this example, the variational Bayesian method is used to train the BNN. For example... Figure 4 As shown, based on the standard variational inference loss function (ELBO), a posterior mean penalty term is further introduced:

[0049] This regularization term constrains the mean of the posterior distribution, which helps to compress the weight magnitude, suppress the ReRAM error amplification effect, and improve the robustness of the final deployment.

[0050] In step four, the posterior mean penalty term is: ;in, Let represent the mean of the i-th weight distribution. The regularization coefficient is used to adjust the degree of influence of this factor.

[0051] In step four, during the inference phase, a write conductance in each ReRAM array is introduced that conforms to the... The normal perturbation of the relationship is used to simulate the actual impact of ReRAM write errors on network performance.

[0052] Step 5: Train the BNN based on the modified loss function to obtain the posterior mean parameter of the weight distribution, and map the mean parameter as the weight value into the ReRAM array for subsequent inference execution.

[0053] Extract the posterior mean of each weight in the trained BNN. This is used as the conductance value mapped to the ReRAM array. The required number of pulses for writing is determined by looking up a table or fitting a relation. During the inference phase, to more realistically evaluate the impact of ReRAM errors, a parameter that conforms to the write conductance can be introduced. The perturbation is used to simulate the inference accuracy under the error of actual devices.

[0054] In step five, the mean weights after training are directly mapped to the target conductance values ​​of the devices in the ReRAM array. Combined with the conductance-pulse relationship, the number of pulses is determined for writing.

[0055] Experimental results show that this method can achieve significant robustness improvements on typical neural networks (such as LeNet, AlexNet, VGG, etc.). At the same ReRAM write error level, compared with the original CNN or error-free BNN scheme, the inference accuracy is significantly improved, demonstrating good generalizability and practical deployment value.

[0056] This invention relates to a method for compensating for random write errors in ReRAM in deep learning networks, belonging to the field of neural network model hardware mapping and inference optimization technology. The method includes the following steps: writing different target conductance values ​​multiple times to the ReRAM device, establishing a probability distribution of conductance error using kernel density estimation; fitting the obtained distribution to a normal distribution and establishing a functional relationship between the standard deviation and the ideal conductance value; constructing a prior distribution of weights in a Bayesian neural network (BNN) based on this function, and introducing an L2 regularization penalty term to the posterior distribution mean during training to suppress error amplification sensitivity; obtaining a robust set of weight mean values ​​through training, and mapping them to the ReRAM array for inference. This method achieves model-level compensation for ReRAM hardware errors, effectively improving network inference accuracy and robustness without redundant mapping or online calibration, and has good versatility and engineering practical value. Detailed Implementation Method Twelve: The only difference between Embodiment Twelve and Embodiment Eleven of the present invention is that: This invention provides a typical deep learning network precision compensation method for ReRAM random write errors, comprising the following steps: Step 1: In the ReRAM device, for the target conductance value G idealThe same number of pulses were applied repeatedly, the conductivity value after writing was recorded, and its probability distribution curve was obtained using kernel density estimation (KDE). Step 2: Fit the obtained conductivity distribution to a normal distribution and extract the mean. The above steps were repeated at multiple target conductance values ​​to establish the standard deviation. With the target conductivity value G ideal The functional relationship, ; Step 3: Construct a prior distribution based on the functional relationship obtained in Step 2, which serves as the prior probability distribution of the weight parameters of each layer of the Bayesian Neural Network (BNN). Step 4: During the variational inference training of BNN, modify the loss function. In addition to including the expected negative log-likelihood term and the KL divergence between the prior and posterior, further introduce the L2 regularization term of the posterior mean to penalize the sum of squares of the mean terms in all weighted distributions. Step 5: Train the BNN based on the modified loss function to obtain the posterior mean parameter of the weight distribution, and map the mean parameter as the weight value into the ReRAM array for subsequent inference execution.

[0058] In step one, the standard deviation With the target conductivity value G ideal The functional relationship between them is in the form of a linear function: Where k is a scaling factor obtained by fitting multiple sets of ReRAM writing experiments.

[0059] Furthermore, in step one, the kernel density estimation used is a Gaussian kernel function to smooth the probability density distribution of the ReRAM write conductivity value.

[0060] In step four, the posterior mean penalty term is: ;in, Let represent the mean of the i-th weight distribution. The regularization coefficient is used to adjust the degree of influence of this factor.

[0061] In step four, during the inference phase, a write conductance in each ReRAM array is introduced that conforms to the... The normal perturbation of the relationship is used to simulate the actual impact of ReRAM write errors on network performance.

[0062] In step five, the mean weights after training are directly mapped to the target conductance values ​​of the devices in the ReRAM array. Combined with the conductance-pulse relationship, the number of pulses is determined for writing.

[0063] Compared with existing technologies, this invention establishes an error statistical model based on the characteristics of actual ReRAM devices and constructs a prior model that can be embedded with a Bayesian neural network. –G mapping relationship.

[0064] This invention introduces a posterior mean penalty to further compress high-error weights and improve the robustness of system inference; compared with existing solutions, it can achieve accuracy improvement through a single training session without hardware redundancy or complex online training.

[0065] The above description is merely a preferred embodiment of a typical deep learning network precision compensation method for ReRAM random write errors. The scope of protection for such a method is not limited to the above embodiments; all technical solutions falling within this framework are within the scope of protection of this invention. It should be noted that for those skilled in the art, any improvements and variations made without departing from the principles of this invention should also be considered within the scope of protection of this invention.

Claims

1. A typical deep learning network precision compensation method for ReRAM random write errors, characterized by: The method includes the following steps: Step 1: In the ReRAM device, for the target conductance value G ideal The same number of pulses were applied repeatedly, the conductivity value after writing was recorded, and the probability distribution curve of the KDE was obtained by kernel density estimation. Step 2: Fit the obtained conductivity distribution to a normal distribution, extract the mean, and repeat steps 1 and 2 above for multiple target conductivity values ​​to establish the standard deviation. With the target conductivity value The functional relationship; Step 3: Construct a prior distribution based on the functional relationship obtained in Step 2, which serves as the prior probability distribution of the weight parameters of each layer of the Bayesian neural network. Step 3 specifically involves: Construct a prior distribution based on the functional relationship obtained in step 2, which serves as the prior probability distribution for the weight parameters of each layer in the Bayesian neural network. This distribution is then applied to each weight in the trained traditional CNN network. Let its prior distribution be: ； Step 4: During the variational inference training of BNN, modify the loss function and introduce an L2 regularization term of the posterior mean to penalize the sum of squares of the mean terms in all weighted distributions; Step 4 specifically involves: The BNN is trained using a variational Bayesian method, introducing a posterior mean penalty term based on the standard variational inference loss function ELBO. This regularization term constrains the mean of the posterior distribution, compresses the weight magnitude, and suppresses the ReRAM error amplification effect; The posterior mean penalty term is: in, Let represent the mean of the i-th weight distribution. The regularization coefficient used to adjust the influence of the posterior mean penalty term; Step 5: Train the Bayesian neural network based on the modified loss function to obtain the posterior mean parameter of the weight distribution, and map the mean parameter as the weight value into the ReRAM array for accuracy compensation.

2. The method according to claim 1, characterized in that: Standard deviation With the target conductivity value The functional relationship between them is in the form of a linear function: Where k is a scaling factor obtained by fitting multiple sets of ReRAM writing experiments.

3. The method according to claim 1 or 2, characterized in that: The kernel density estimation KDE used in step 1 is a Gaussian kernel function to smooth the probability density distribution of the ReRAM write conductivity value.

4. The method according to claim 3, characterized in that: Step 5 specifically involves: Extract the posterior mean of each weight in the trained Bayesian neural network. As the conductance value mapped to the ReRAM array, the required number of pulses for writing is determined by looking up a table or fitting a relationship, and a conformance is introduced into the writing conductance. The perturbation is used to simulate the inference accuracy under the error of actual devices.

5. A typical deep learning network precision compensation system for ReRAM random write errors, the system operating based on the typical deep learning network precision compensation method for ReRAM random write errors according to claim 1, characterized in that: The system includes: The data logging module, located in the ReRAM device, is configured to record data for the target conductivity value G. ideal The same number of pulses were applied repeatedly, the conductivity value after writing was recorded, and the probability distribution curve of the KDE was obtained by kernel density estimation. The fitting module fits the obtained conductance distribution to a normal distribution, extracts the mean, and repeats the corresponding steps for multiple target conductance values ​​to establish the standard deviation. With the target conductivity value The functional relationship; The model building module constructs a prior distribution based on the obtained functional relationship, which serves as the prior probability distribution of the weight parameters of each layer of the Bayesian neural network. The model training module modifies the loss function during the variational inference training of the BNN by introducing an L2 regularization term of the posterior mean to penalize the sum of squares of the mean terms in all weighted distributions. The compensation module trains the Bayesian neural network based on the modified loss function to obtain the posterior mean parameter of the weight distribution, and maps the mean parameter as a weight value into the ReRAM array for accuracy compensation.

6. A readable storage medium that stores data in the form of electrical conductivity, characterized in that, It can perform computation locally on the storage device, realizing in-memory computing, and can be used to implement the typical deep learning network accuracy compensation method for ReRAM random write error as described in any one of claims 1-4.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements a typical deep learning network accuracy compensation method for ReRAM random write errors as described in any one of claims 1-4.

Images