Memristive Bayesian deep neural network online learning method and device

Abstract

The invention relates to a memristor Bayesian deep neural network online learning method and device, and the method comprises the steps: carrying out the offline training on a digital computer through employing a training data set, obtaining an initial BDNN model, writing a trained network weight into a memristor, and obtaining an actual BDNN model; predicting a data category of each data sample in the unmarked data set by using an actual BDNN model, and calculating a prediction uncertainty value of each data sample; and determining a target data sample in the unmarked data set based on the prediction uncertainty value, adding the target data sample into the training data set to obtain a new training data set, and performing online training by using the new training data set to update the conductance value of the memristor until a preset stop condition is met. And obtaining a memristor BDNN model meeting a preset performance condition. Therefore, the technical problems that a deterministic computing platform has huge delay and energy consumption in the computing process and is difficult to meet the requirements of high-speed and efficient computing of Bayesian deep neural network learning in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the technical field of electrical digital data processing, and in particular to a memristor Bayesian deep neural network online learning method and device. Background Art

[0002] As society pays more and more attention to the security of artificial intelligence systems, BDNN (Bayesian Deep Neural Networks), as a far-reaching probabilistic machine learning algorithm, has been widely used in the field of artificial intelligence systems where security is critical. The main difference between it and traditional deep neural networks is that traditional deep neural networks obtain deterministic weights through learning, while Bayesian deep neural networks complete the learning of uncertainty by integrating Bayesian methods and deep neural networks to update the probability weights of Gaussian distribution during the learning process. Therefore, Bayesian deep neural networks are able to express the uncertainty of results in predictions (that is, give the degree of confidence in the results), thereby solving the problem of artificial intelligence system security. In addition, such as Figure 1 As shown in the figure, compared with traditional deep neural networks, Bayesian deep neural networks can also implement DBAL (Deep Bayesian active learning), so it is a very important probability-based machine learning algorithm.

[0003] However, CMOS-based deterministic computing platforms (such as CPUs and GPUs) can no longer meet the urgent needs of high-speed and efficient computing for Bayesian deep neural network learning. First, due to the "von Neumann bottleneck" caused by the separation of storage and computing, extremely dense matrix-vector multiplication operations cause large amounts of data to be frequently moved between the processor and the memory during the entire learning process, resulting in huge delays and energy consumption. Second, the uncertainty learning ability of Bayesian deep neural networks stems from the additional Gaussian noise added to the gradient of weight parameter updates. Therefore, the network requires a large amount of Gaussian random number generation during the learning process. Random number generation with complex calculation steps brings greater additional computing delays and energy consumption. As the size of CMOS is difficult to further reduce, the space for performance improvement of CMOS-based deterministic computing platforms has gradually shrunk. These have seriously hindered the future application of Bayesian deep neural networks in scenarios that require real-time learning and high-reliability decision-making, and it is even more impossible to build advanced large models based on Bayesian deep neural networks and study their intelligence emergence phenomena, which is of great significance for the realization of true general artificial intelligence.

[0004] In summary, the deterministic computing platforms in related technologies have huge delays and energy consumption during the calculation process, which makes it difficult to meet the needs of high-speed and efficient computing for Bayesian deep neural network learning, and urgently needs to be improved. Summary of the invention

[0005] The present application provides a memristive Bayesian deep neural network online learning method and device to solve the technical problem in the related art that the deterministic computing platform has huge delays and energy consumption during the calculation process, and it is difficult to meet the high-speed and efficient computing requirements of Bayesian deep neural network learning.

[0006] The first aspect of the present application provides a memristor Bayesian deep neural network online learning method, comprising the following steps: on a digital computer, using a training data set to perform offline training to obtain an initial BDNN model, and writing the trained network weights into a memristor to obtain an actual BDNN model; using the actual BDNN model to predict the data category of each data sample in an unlabeled data set, and calculating a prediction uncertainty value for each data sample; determining a target data sample in the unlabeled data set based on the prediction uncertainty value, and adding the target data sample to the training data set to obtain a new training data set, and using the new training data set to perform online training to update the conductance value of the memristor until a preset stop condition is met, thereby obtaining a memristor BDNN model that meets preset performance conditions.

[0007] Optionally, in one embodiment of the present application, offline training is performed on a digital computer using a training data set to obtain an initial BDNN model, and the trained network weights are written into a memristor to obtain an actual BDNN model, including: simulating a memristor model on the digital computer; using the memristor conductance value of the memristor model as a weight, calculating the gradient of the memristor conductance value of the memristor model to obtain the sign of the gradient, and determining the operation direction corresponding to each device of the memristor model based on the sign; updating the memristor conductance value of the memristor model based on the operation direction, and performing conductance modulation in combination with a random gradient learning algorithm and the training data set; obtaining the initial BDNN model in combination with the generation of Gaussian noise during the conductance modulation process, and writing the network weights of the initial BDNN into the memristor.

[0008] Optionally, in one embodiment of the present application, the combining the stochastic gradient learning algorithm and the training data set to perform conductivity modulation includes: optimizing the stochastic gradient learning algorithm using a pre-constructed loss function, so as to perform conductivity modulation using the optimized stochastic gradient learning algorithm,

[0009] Among them, the expression of the loss function is:

[0010]

[0011] Among them, q(w I) represents the memristor weight distribution of the neural network, P(w) represents the prior distribution of weights, and P(y|x,w I ) represents the predicted distribution of the neural network when the input sample is x, KL[·] represents the KL divergence, represents the calculation expectation, and β represents the weight of the two items before and after the balance;

[0012] The expression for conductivity modulation using the optimized stochastic gradient learning algorithm is:

[0013]

[0014] Where ΔI represents the current change, represents the gradient, η m represents the Gaussian noise.

[0015] Optionally, in one embodiment of the present application, the online training is performed using the new training data set to update the conductance value of the memristor until a preset stop condition is met to obtain a memristor BDNN model that meets preset performance conditions, including: recording the weight corresponding to each device of the memristor during each online training process to obtain the weight change ratio of each device; based on the weight change ratio and the conductance gradient corresponding to each device, the device that meets the first preset change condition is screened out, and the weight of the device is updated so that the weight change ratio of the device in the next round of iteration meets the second preset change condition.

[0016] Optionally, in one embodiment of the present application, during the online training process, based on the increase in the number of iterations, the number of devices that update weights in the memristor is reduced so that the Gaussian noise of the total current of the memristor dominates until the preset stop condition is met.

[0017] The second aspect of the present application provides a memristor Bayesian deep neural network online learning device, including: a training module, which is used to perform offline training on a digital computer using a training data set to obtain an initial BDNN model, and write the trained network weights into a memristor to obtain an actual BDNN model; a calculation module, which is used to use the actual BDNN model to predict the data category of each data sample in an unlabeled data set, and calculate the prediction uncertainty value of each data sample; a learning module, which is used to determine a target data sample in the unlabeled data set based on the prediction uncertainty value, and add the target data sample to the training data set to obtain a new training data set, and use the new training data set to perform online training to update the conductance value of the memristor until a preset stop condition is met, thereby obtaining a memristor BDNN model that meets preset performance conditions.

[0018] Optionally, in one embodiment of the present application, the training module includes: a simulation unit, used to simulate the memristor model on the digital computer; a calculation unit, used to use the memristor conductance value of the memristor model as a weight, calculate the gradient of the memristor conductance value of the memristor model to obtain the sign of the gradient, and determine the operation direction corresponding to each device of the memristor model based on the sign; a modulation unit, used to update the memristor conductance value of the memristor model based on the operation direction, and perform conductance modulation in combination with the stochastic gradient learning algorithm and the training data set; a writing unit, used to obtain the initial BDNN model in combination with the Gaussian noise generated during the conductance modulation process, and write the network weights of the initial BDNN into the memristor.

[0019] Optionally, in one embodiment of the present application, the training module further includes: an optimization unit, configured to optimize the stochastic gradient learning algorithm using a pre-constructed loss function, so as to perform conductivity modulation using the optimized stochastic gradient learning algorithm,

[0020] Among them, the expression of the loss function is:

[0021]

[0022] Among them, q(w I ) represents the memristor weight distribution of the neural network, P(w) represents the prior distribution of weights, and P(y|x,w I ) represents the predicted distribution of the neural network when the input sample is x, KL[·] represents the KL divergence, represents the calculation expectation, and β represents the weight of the two items before and after the balance;

[0023] The expression for conductivity modulation using the optimized stochastic gradient learning algorithm is:

[0024]

[0025] Where ΔI represents the current change, represents the gradient, η m represents the Gaussian noise.

[0026] Optionally, in one embodiment of the present application, the learning module includes: a recording unit, used to record the weight corresponding to each device of the memristor during each online training process to obtain the weight change ratio of each device; an updating unit, used to screen out devices that meet a first preset change condition based on the weight change ratio and the conductance gradient corresponding to each device, and update the weight of the device so that the weight change ratio of the device in the next round of iteration meets the second preset change condition.

[0027] Optionally, in one embodiment of the present application, during the online training process, based on the increase in the number of iterations, the number of devices that update weights in the memristor is reduced so that the Gaussian noise of the total current of the memristor dominates until the preset stop condition is met.

[0028] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the online learning method of the memristive Bayesian deep neural network as described in the above embodiment.

[0029] The fourth aspect of the present application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the memristive Bayesian deep neural network online learning method as described in the above embodiments.

[0030] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned memristive Bayesian deep neural network online learning method.

[0031] The embodiment of the present application can use a memristor as a hardware platform, and the random characteristics of the memristor can be used to achieve fast, low-energy weight updates and sampling, thereby improving the efficiency of BDNN online learning. On a digital computer, offline training is performed using a training data set to obtain an initial BDNN model, and the trained network weights are written into the memristor to obtain an actual BDNN model, and then the data category of each data sample in the unlabeled data set is predicted, and the prediction uncertainty value of each data sample is calculated to determine the target data sample in the unlabeled data set, and the target data sample is added to the training data set to obtain a new training data set, and the new training data set is used for online training to update the conductance value of the memristor until the preset stop condition is met, and a memristor BDNN model that meets the preset performance condition is obtained. By first deploying in the process of increasing sample training, there is no need to train the neural network from scratch on the memristor hardware, reducing the negative impact of the nonlinear conductance modulation of the memristor, and improving network performance. Thus, the technical problem that the deterministic computing platform has huge delays and energy consumption in the calculation process in the related art and is difficult to meet the requirements of high-speed and efficient computing of Bayesian deep neural network learning is solved.

[0032] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0034] Figure 1 It is a schematic diagram of the application of deep Bayesian active learning based on Bayesian deep neural network in the background technology;

[0035] Figure 2 A flowchart of a memristor Bayesian deep neural network online learning method provided according to an embodiment of the present application;

[0036] Figure 3 A schematic diagram of smooth transition between two stages of a BDNN learning process according to an embodiment of the present application;

[0037] Figure 4 A schematic diagram of a back propagation process for calculating a gradient of memristor conductance according to an embodiment of the present application;

[0038] Figure 5 It is a schematic diagram of an improved mSGLD according to an embodiment of the present application;

[0039] Figure 6 A schematic diagram of an iterative training process according to an embodiment of the present application;

[0040] Figure 7 A flowchart of deep Bayesian active learning based on memristor storage and computing integrated hardware according to one embodiment of the present application;

[0041] Figure 8 A schematic diagram of the structure of a memristor Bayesian deep neural network online learning device provided according to an embodiment of the present application;

[0042] Fig. 9 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0044] The following describes the memristor Bayesian deep neural network online learning method and device of the embodiment of the present application with reference to the accompanying drawings. In view of the technical problem that in the related technology mentioned in the above background technology, the deterministic computing platform has huge delays and energy consumption in the calculation process, and it is difficult to meet the high-speed and efficient computing requirements of Bayesian deep neural network learning, the present application provides a memristor Bayesian deep neural network online learning method. In this method, a memristor can be used as a hardware platform, and the random characteristics of the memristor can be used to achieve fast and low-energy weight updates and sampling, thereby improving the efficiency of BDNN online learning. On a digital computer, offline training is performed using a training data set to obtain an initial BDNN model, and the trained network weights are written into the memristor to obtain an actual The BDNN model is used to predict the data category of each data sample in the unlabeled data set, and the prediction uncertainty value of each data sample is calculated to determine the target data sample in the unlabeled data set, and the target data sample is added to the training data set to obtain a new training data set, and the new training data set is used for online training to update the conductance value of the memristor until the preset stop condition is met, and a memristor BDNN model that meets the preset performance condition is obtained. By first deploying and then adding sample training, there is no need to train the neural network from scratch on the memristor hardware, reducing the negative impact of the nonlinear conductance modulation of the memristor and improving the network performance. In this way, the technical problem in the related technology that the deterministic computing platform has huge delays and energy consumption in the calculation process and is difficult to meet the high-speed and efficient computing requirements of Bayesian deep neural network learning is solved.

[0045] Understandably, memristors are widely considered to be one of the best hardware platforms for efficiently implementing deep neural network learning due to their non-volatility, high density, and adjustable conductance.

[0046] Memristor-based probabilistic computing can not only eliminate this extensive data movement, but also utilize the intrinsic random properties of memristors to efficiently generate random numbers. First, by storing weights in the conductance of the memristor array, based on Ohm's law and Kirchhoff's current law, only one parallel read operation is needed to achieve in-situ calculation of matrix-vector multiplication operations. Second, due to the physical random motion of internal ions, the conductance of the memristor exhibits random fluctuation behavior, so that read operations or conductance modulation operations can efficiently simulate the random number generation process.

[0047] Therefore, the memristor Bayesian deep neural network online learning method of the embodiment of the present application can use the memristor hardware platform to meet the high-speed and efficient computing requirements of the two core computing tasks of the BDNN learning process: matrix-vector multiplication and Gaussian random number generation.

[0048] Specifically, Figure 2A flowchart of an online learning method for a memristive Bayesian deep neural network provided in an embodiment of the present application.

[0049] like Figure 2 As shown, the memristor Bayesian deep neural network online learning method includes the following steps:

[0050] In step S201, offline training is performed on a digital computer using a training data set to obtain an initial BDNN model, and the trained network weights are written into a memristor to obtain an actual BDNN model.

[0051] During the actual execution process, the embodiment of the present application can be trained offline on a digital computer using a small-scale training data set to obtain an initial BDNN model, and then the trained network weights are written into the memristor using a write verification method to obtain an actual BDNN model based on the memristor.

[0052] Optionally, in one embodiment of the present application, offline training is performed on a digital computer using a training data set to obtain an initial BDNN model, and the trained network weights are written into a memristor to obtain an actual BDNN model, including: simulating a memristor model on a digital computer; using the memristor conductance value of the memristor model as a weight, calculating the gradient of the memristor conductance value of the memristor model to obtain the sign of the gradient, and determining the operation direction corresponding to each device of the memristor model based on the sign; updating the memristor conductance value of the memristor model based on the operation direction, and performing conductance modulation in combination with a random gradient learning algorithm and the training data set; obtaining an initial BDNN model in combination with the generation of Gaussian noise during the conductance modulation process, and writing the network weights of the initial BDNN into the memristor.

[0053] During the online training process, the read noise model and the conductivity modulation model are used in the network prediction and weight update process to make the network weights more adaptable to the characteristics of the memristor array. The conductivity modulation model is established based on the measured memristor conductivity state transition data. The model describes the average conductivity state value μ after applying a single SET / RESET voltage pulse in the current conductivity state. set , μ reset and standard deviation σ set , σ reset The modulated conductivity state obeys a Gaussian distribution with this as the parameter:

[0054]

[0055] Specifically, the embodiment of the present application pre-simulates a memristor model, and uses the memristor conductance value of the memristor model as a weight to calculate the gradient of the memristor conductance value of the memristor model. The conductance of the memristor is then updated according to the sign of the gradient, mimicking an efficient stochastic gradient learning algorithm.

[0056] Since the transfer probability of conductance during conductance modulation is Gaussian, the Gaussian noise η added to the formula is m This can be achieved by the inherent random fluctuations of the device, where the actual update operation of the device on the memristor array is as follows:

[0057]

[0058] That is to say, when the gradient of the device is positive, a SET operation is performed on the device; conversely, when the gradient is negative, a RESET operation is performed.

[0059] Furthermore, the embodiments of the present application can update the memristor conductance value of the memristor model in combination with the operation direction, and perform conductance modulation in combination with the random gradient learning algorithm and the training data set, and obtain the initial BDNN model in combination with the generated Gaussian noise during the conductance modulation process, so as to write the network weights of the initial BDNN into the memristor.

[0060] Optionally, in one embodiment of the present application, combining a stochastic gradient learning algorithm and a training data set for conductance modulation includes: optimizing the stochastic gradient learning algorithm using a pre-constructed loss function, so as to perform conductance modulation using the optimized stochastic gradient learning algorithm,

[0061] Among them, the expression of the loss function is

[0062]

[0063] Among them, q(w I ) represents the memristor weight distribution of the neural network, P(w) represents the prior distribution of weights, and P(y|x,w I ) represents the predicted distribution of the neural network when the input sample is x, KL[·] represents the KL divergence, represents the calculation expectation, and β represents the weight of the two items before and after the balance;

[0064] The expression for conductance modulation using the optimized stochastic gradient learning algorithm is:

[0065]

[0066] Where ΔI represents the current change, represents the gradient, η m represents Gaussian noise.

[0067] In some embodiments, offline training can be performed based on an improvement to the Stochastic gradient Langevin dynamics (SGLD) algorithm to achieve conductivity modulation by combining a stochastic gradient learning algorithm with a training data set.

[0068] According to the random characteristics of memristors, the embodiment of the present application can improve SGLD and propose memristor SGLD (mSGLD). In the proposed mSGLD training method, the loss function can be set as:

[0069]

[0070] Where q(w I ) represents the memristor weight distribution of the neural network, P(w) represents the prior distribution of weights, and P(y|x,w I ) represents the predicted distribution of the neural network when the input sample is x, KL[·] represents the Kullback-Leibler (KL) divergence, It represents the expected value, and β is the weight to balance the two terms.

[0071] In order to calculate the gradient of the memristor conductance The embodiment of the present application can use the Backpropagation (BP) algorithm to backpropagate the gradient to each device according to the relationship between the device and the weight, that is, to obtain the conductivity gradient of each device.

[0072] And in the initial stage, the gradient of the memristor conductance is calculated The conductance of the memristor is then updated according to the sign of the gradient, mimicking an effective stochastic gradient learning algorithm:

[0073]

[0074] In step S202, the actual BDNN model is used to predict the data category of each data sample in the unlabeled data set, and the prediction uncertainty value of each data sample is calculated.

[0075] The embodiment of the present application can use the actual BDNN model to predict the data category in the unlabeled data set and calculate the prediction uncertainty value. During the network prediction process, the transistors in the cross array are fully turned on, the read voltage is applied to the SL of the device row by row, and the fluctuating read current flows through the virtual ground BL and is measured by the ADC. Every time the actual BDNN model makes a prediction, the output will be affected by the fluctuation of the read current. Therefore, the embodiment of the present application can use multiple prediction outputs of the initial BDNN model to derive the prediction distribution, and use quantitative criteria such as entropy and variance to calculate the uncertainty of the prediction.

[0076] In step S203, the target data sample in the unlabeled data set is determined based on the predicted uncertainty value, and the target data sample is added to the training data set to obtain a new training data set. The new training data set is used for online training to update the conductance value of the memristor until the preset stop condition is met, thereby obtaining a memristor BDNN model that meets the preset performance condition.

[0077] Furthermore, according to the prediction uncertainty in the unlabeled data set, a sample with the largest uncertainty is selected to query its label, and the sample and the queried label are added to the training data set. The sample with the largest uncertainty often means that the sample has greater information, and knowing its label is often the most useful for improving the classification performance of the network.

[0078] Based on the training dataset with new samples added, the actual BDNN model written into the memristor is trained online. Taking advantage of the inherent analog conductance plasticity of the memristor, the neural network can be trained in situ using the training data.

[0079] After BDNN finishes online training, it will proceed to the next cycle - it will continue to select samples with the largest uncertainty from the unlabeled data set and add them to the training data set, and then conduct online training. This active learning cycle will not stop until the network performance reaches the expected level or the number of query labels is exhausted.

[0080] Optionally, in one embodiment of the present application, online training is performed using a new training data set to update the conductance value of the memristor until a preset stop condition is met, thereby obtaining a memristor BDNN model that meets preset performance conditions, including: recording the weight corresponding to each device of the memristor during each online training process to obtain the weight change ratio of each device; screening out devices that meet the first preset change condition based on the weight change ratio and the conductance gradient corresponding to each device, and updating the weight of the device so that the weight change ratio of the device in the next round of iteration meets the second preset change condition.

[0081] During repeated network training, the embodiment of the present application may continue to use the previous offline training process, and during the iterative training process, select a gradually smaller weight ratio to update the key weight.

[0082] First, the embodiment of the present application can be based on the conductivity gradient The absolute values ​​are sorted from large to small, and then the top weights are selected according to the current weight update ratio p. In the next iteration, the weight update ratio p gradually decreases. Then one of the devices with the selected weight is randomly selected to update the device conductance.

[0083] Optionally, in one embodiment of the present application, during online training, based on the increase in the number of iterations, the number of devices that update weights in the memristor is reduced so that the Gaussian noise of the total current of the memristor dominates until a preset stop condition is met.

[0084] It should be noted that in order to achieve a smooth transition in the training phase, in the embodiment of the present application, as the number of training iterations increases, the classification performance of the memristor network continues to improve, the amplitude of the gradient gradually decreases, and the injected Gaussian noise will dominate. In order to smoothly transition to the second stage of training, as the number of training iterations increases, the embodiment of the present application can reduce the number of device updates, and finally the Gaussian noise of the total current dominates, thereby simulating the Langevin dynamic MH process.

[0085] Combination Figures 3 to 7 As shown, the working principle of the memristive Bayesian deep neural network online learning method of the embodiment of the present application is described in detail by taking an embodiment as an example.

[0086] The actual implementation process of the embodiments of the present application involves two aspects: utilizing the random characteristics of memristors to realize online learning of memristor Bayesian deep neural networks and deep Bayesian active learning based on memristors.

[0087] It can be understood that the embodiment of the present application performs offline training by simulating a memristor model and a training data set to perform initial network deployment, and after the initial network is deployed to the memristor, performs online training using the actual BDNN model, thereby achieving deep Bayesian active learning based on the memristor after multiple iterations.

[0088] First, the embodiments of the present application can explain the part of online learning of memristor Bayesian deep neural network using the random characteristics of memristors, that is, the part of offline training and online training.

[0089] A uses the random characteristics of memristors to realize online learning of memristor Bayesian deep neural networks.

[0090] In the process of active learning, the online learning ability of the memristor network is very important. The random gradient training method relies on the plasticity of the memristor weights, that is, the modulation process of the conductance. The conductance update amount of the device needs to be modulated according to the gradient. However, based on the physical response analysis of nanodevices, the conductance modulation process has random characteristics. This random characteristic reduces the learning ability of the network, thereby affecting the classification ability of the network. On the other hand, this random fluctuation under the equal-amplitude pulse can also be regarded as random number generation.

[0091] Therefore, the embodiment of the present application can be improved based on the Stochastic gradient Langevin dynamics (SGLD) algorithm to realize online learning of the memristor Bayesian deep neural network by utilizing the random characteristics of the memristor.

[0092] The SGLD algorithm is very straightforward: it takes the gradient steps of the traditional training algorithm and also adds Gaussian noise to the weight updates. Figure 3 As shown, in the initial stage ( Figure 3 a), the gradient will dominate and the algorithm will mimic an efficient stochastic gradient algorithm. In the later stage ( Figure 3 b), the injected Gaussian noise will dominate, so the algorithm will mimic the Langevin dynamic MH algorithm. And as the number of training iterations increases, the SGLD algorithm will smoothly transition between the two stages. The SGLD algorithm can capture parameter uncertainty by preventing the weight parameters from collapsing to the maximum a posteriori solution.

[0093] According to the random characteristics of memristor, the embodiment of the present application improves SGLD and proposes memristor SGLD (mSGLD). In the proposed mSGLD training method, the loss function can be set as:

[0094]

[0095] Where q(w I ) represents the memristor weight distribution of the neural network, P(w) represents the prior distribution of weights, and P(y|x,w I ) represents the predicted distribution of the neural network when the input sample is x, KL[·] represents the Kullback-Leibler (KL) divergence, It represents the expected value, and β is the weight to balance the two terms.

[0096] In order to calculate the gradient of the memristor conductance The embodiment of the present application can be based on the relationship between the device and the weight ( Figure 4 ), use the Backpropagation (BP) algorithm to backpropagate the gradient to each device, that is, to obtain the conductivity gradient of each device

[0097] In the initial stage, the embodiment of the present application can calculate the gradient of the memristor conductance The conductance of the memristor is then updated according to the sign of the gradient, mimicking an effective stochastic gradient learning algorithm:

[0098]

[0099] Since the transfer probability of conductance during conductance modulation is Gaussian, the Gaussian noise η added to the formula is m can be achieved by the inherent random fluctuations of the device ( Figure 5 a). Therefore, on the memristor array, the actual update operation of the device is as follows:

[0100]

[0101] That is to say, when the gradient of the device is positive, a SET operation can be performed on the device; conversely, when the gradient is negative, a RESET operation is performed.

[0102] As the number of training iterations increases, the classification performance of the memristor network continues to improve, the magnitude of the gradient gradually decreases, and the injected Gaussian noise will dominate ( Figure 5 b) In order to smoothly transition to the second stage of training, as the number of training iterations increases, the embodiment of the present application can reduce the number of device updates, and eventually the Gaussian noise of the total current dominates, thereby simulating the Langevin dynamic MH process.

[0103] Specifically, if Figure 6 As shown, the embodiment of the present application can select gradually smaller weight ratios to update key weights during iterative training.

[0104] First, the embodiment of the present application can be based on the conductivity gradient The absolute values ​​are sorted from large to small, and then the top weights are selected according to the current weight update ratio p. In the next iteration, the weight update ratio p gradually decreases. Then one of the devices with the selected weight is randomly selected to update the device conductance.

[0105] Through the above online learning algorithm, the embodiments of the present application can utilize the random characteristics exhibited in the reading process and the modulation process to respectively realize the generation of Gaussian random numbers in the network prediction and learning processes. On the one hand, in the prediction process of the Bayesian deep neural network, the Gaussian weights need to be repeatedly sampled and perform matrix-vector multiplication calculations with the input vector. The method of implementing memristive Gaussian weights in a cross array promises to efficiently implement weight sampling and matrix-vector multiplication using a single read operation. On the other hand, in the learning process of the Bayesian deep neural network, the weight update amount is the gradient value plus Gaussian noise, so that parameter uncertainty can be captured in a Bayesian manner.

[0106] B Deep Bayesian active learning based on memristor.

[0107] like Figure 7 As shown, this part may include the following steps:

[0108] Step S1: Initial network deployment: On a digital computer, the embodiment of the present application can use a small-scale training data set for offline training to obtain an initial BDNN model, and then use the write verification method to write the trained network weights into the memristor. During the online training process, the read noise model and the conductivity modulation model are used for the network prediction and weight update process to make the network weights more adaptable to the characteristics of the memristor array. The conductivity modulation model is established based on the measured memristor conductivity state transfer data. The model describes the average conductivity state value μ in the current conductivity state after applying a single SET / RESET voltage pulse. set , μ reset and standard deviation σ set , σ reset The modulated conductivity state obeys a Gaussian distribution with this as the parameter:

[0109]

[0110] Step S2: Network prediction and uncertainty estimation: The embodiment of the present application can use the actual BDNN to predict the data category in the unlabeled data set and calculate the prediction uncertainty value. During the network prediction process, the transistors in the cross array are fully turned on, the read voltage is applied to the SL of the device row by row, and the fluctuating read current flows through the virtual ground BL and is measured by the ADC. Every time the actual BDNN makes a prediction, the output will be affected by the fluctuation of the read current. Therefore, the embodiment of the present application can use multiple prediction outputs of the actual BDNN to derive the prediction distribution, and use quantitative criteria such as entropy and variance to calculate the uncertainty of the prediction.

[0111] Step S3: Sample selection based on uncertainty: The embodiment of the present application can select a sample with the largest uncertainty to query its label based on the predicted uncertainty of the sample in the unlabeled data set, and add the sample and the queried label to the training data set. The sample with the largest uncertainty often means that the sample has greater information, and knowing its label is often the most useful for improving the classification performance of the network.

[0112] Step S4: Based on the training data set with the new sample added, the actual BDNN is trained online. By utilizing the inherent analog conductance plasticity of the memristor, the neural network can be trained in situ using the training data.

[0113] Step S5: Continue to the next cycle: After the actual BDNN completes the online training, it will proceed to the next cycle - it will continue to select the samples with the largest uncertainty from the unlabeled data set and add them to the training data set, and then conduct online training. This active learning cycle will not stop until the network performance reaches the expected level or the number of query labels is exhausted.

[0114] In summary, the embodiments of the present application utilize the random characteristics of memristors to achieve fast, low-energy weight updates and sampling, thereby improving the efficiency of BDNN online learning. Compared with traditional hardware, it can significantly reduce computing delays and energy consumption, and improve real-time performance; by first deploying the initial BDNN and then continuously increasing the sample training process, it is possible to train the neural network from scratch on the memristor hardware without having to reduce the negative impact of the nonlinear conductivity modulation of the memristor; deep Bayesian active learning is achieved through memristor storage and computing integrated hardware, that is, utilizing the natural advantages of the memristor array, which has low power consumption and fast speed; during the iterative learning process, new data samples will be added to the training data set, and the performance of the network will also be adjusted and improved.

[0115] According to the memristor Bayesian deep neural network online learning method proposed in the embodiment of the present application, the memristor can be used as a hardware platform, and the random characteristics of the memristor can be used to achieve fast and low-energy weight updates and sampling, thereby improving the efficiency of BDNN online learning. On a digital computer, offline training is performed using a training data set to obtain an initial BDNN model, and the trained network weights are written into the memristor to obtain an actual BDNN model, and then the data category of each data sample in the unlabeled data set is predicted, and the prediction uncertainty value of each data sample is calculated to determine the target data sample in the unlabeled data set, and the target data sample is added to the training data set to obtain a new training data set, and the new training data set is used for online training to update the conductance value of the memristor until the preset stop condition is met, so as to obtain a memristor BDNN model that meets the preset performance condition. By first deploying the process of increasing sample training, there is no need to train the neural network from scratch on the memristor hardware, thereby reducing the negative impact of the nonlinear conductance modulation of the memristor and improving network performance. This solves the technical problem in related technologies that deterministic computing platforms have huge delays and energy consumption during the calculation process, making it difficult to meet the needs of high-speed and efficient computing for Bayesian deep neural network learning.

[0116] Next, the memristive Bayesian deep neural network online learning device proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0117] Figure 8 It is a block diagram of a memristive Bayesian deep neural network online learning device according to an embodiment of the present application.

[0118] like Figure 8 As shown, the memristive Bayesian deep neural network online learning device 10 includes: a training module 100, a calculation module 200 and a learning module 300.

[0119] Specifically, the training module 100 is used to perform offline training on a digital computer using a training data set to obtain an initial BDNN model, and write the trained network weights into a memristor to obtain an actual BDNN model.

[0120] The calculation module 200 is used to predict the data category of each data sample in the unlabeled data set using the actual BDNN model, and calculate the prediction uncertainty value of each data sample.

[0121] The learning module 300 is used to determine the target data sample in the unlabeled data set based on the predicted uncertainty value, and add the target data sample to the training data set to obtain a new training data set, and use the new training data set to perform online training to update the conductance value of the memristor until a preset stop condition is met, thereby obtaining a memristor BDNN model that meets the preset performance condition.

[0122] Optionally, in one embodiment of the present application, the training module 100 includes: a simulation unit, a calculation unit, a modulation unit and a writing unit.

[0123] Among them, the simulation unit is used to simulate the memristor model on a digital computer.

[0124] The calculation unit is used to calculate the gradient of the memristor conductance value of the memristor model by taking the memristor conductance value of the memristor model as a weight to obtain the sign of the gradient, and determine the operation direction corresponding to each device of the memristor model based on the sign.

[0125] The modulation unit is used to update the memristor conductance value of the memristor model based on the operation direction, and perform conductance modulation in combination with the stochastic gradient learning algorithm and the training data set.

[0126] The writing unit is used to obtain the initial BDNN model by combining the Gaussian noise generated in the conductance modulation process, and write the network weights of the initial BDNN into the memristor.

[0127] Optionally, in one embodiment of the present application, the training module 100 further includes: an optimization unit.

[0128] The optimization unit is used to optimize the stochastic gradient learning algorithm using a pre-built loss function, so as to perform conductance modulation using the optimized stochastic gradient learning algorithm.

[0129] Among them, the expression of the loss function is

[0130]

[0131] Among them, q(w I ) represents the memristor weight distribution of the neural network, P(w) represents the prior distribution of weights, and P(y|x,w I) represents the predicted distribution of the neural network when the input sample is x, KL[·] represents the KL divergence, represents the expected value, and β represents the weight of the two items before and after the balance.

[0132] The expression for conductance modulation using the optimized stochastic gradient learning algorithm is:

[0133]

[0134] Where ΔI represents the current change, represents the gradient, η m represents Gaussian noise.

[0135] Optionally, in one embodiment of the present application, the learning module 300 includes: a recording unit and an updating unit.

[0136] The recording unit is used to record the weight corresponding to each device of the memristor during each online training process to obtain the weight change ratio of each device.

[0137] An updating unit is used to screen out devices that meet a first preset change condition based on a weight change ratio and a conductivity gradient corresponding to each device, and to update the weight of the device so that the weight change ratio of the device in the next round of iterations meets a second preset change condition.

[0138] Optionally, in one embodiment of the present application, during online training, based on the increase in the number of iterations, the number of devices that update weights in the memristor is reduced so that the Gaussian noise of the total current of the memristor dominates until a preset stop condition is met.

[0139] It should be noted that the aforementioned explanation of the embodiment of the memristive Bayesian deep neural network online learning method is also applicable to the memristive Bayesian deep neural network online learning device of this embodiment, and will not be repeated here.

[0140] According to the memristor Bayesian deep neural network online learning device proposed in the embodiment of the present application, a memristor can be used as a hardware platform, and the random characteristics of the memristor can be used to achieve fast, low-energy weight updates and sampling, thereby improving the efficiency of BDNN online learning. On a digital computer, offline training is performed using a training data set to obtain an initial BDNN model, and the trained network weights are written into the memristor to obtain an actual BDNN model, and then the data category of each data sample in the unlabeled data set is predicted, and the prediction uncertainty value of each data sample is calculated to determine the target data sample in the unlabeled data set, and the target data sample is added to the training data set to obtain a new training data set, and the new training data set is used for online training to update the conductance value of the memristor until the preset stop condition is met, so as to obtain a memristor BDNN model that meets the preset performance condition. By first deploying the process of increasing sample training, there is no need to train the neural network from scratch on the memristor hardware, thereby reducing the negative impact of nonlinear conductance modulation of the memristor and improving network performance. This solves the technical problem in related technologies that deterministic computing platforms have huge delays and energy consumption during the calculation process, making it difficult to meet the needs of high-speed and efficient computing for Bayesian deep neural network learning.

[0141] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0142] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .

[0143] When the processor 902 executes the program, the memristor Bayesian deep neural network online learning method provided in the above embodiment is implemented.

[0144] Furthermore, the electronic device further comprises:

[0145] The communication interface 903 is used for communication between the memory 901 and the processor 902 .

[0146] The memory 901 is used to store computer programs that can be executed on the processor 902 .

[0147] The memory 901 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0148] If the memory 901, the processor 902 and the communication interface 903 are implemented independently, the communication interface 903, the memory 901 and the processor 902 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0149] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.

[0150] The processor 902 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0151] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned memristive Bayesian deep neural network online learning method is implemented.

[0152] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the memristive Bayesian deep neural network online learning method provided by an embodiment of the present invention.

[0153] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0154] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0155] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0156] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0157] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0158] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0159] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0160] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A memristive Bayesian deep neural network online learning method, characterized in that: The following steps are involved: On a digital computer, offline training is performed using the training data set to obtain an initial BDNN model, and the trained network weights are written into the memristor to obtain the actual BDNN model; Predicting the data category of each data sample in the unlabeled data set using the actual BDNN model, and calculating the prediction uncertainty value of each data sample; Based on the prediction uncertainty value, a target data sample in the unlabeled data set is determined, and the target data sample is added to the training data set to obtain a new training data set. The new training data set is used for online training to update the conductance value of the memristor until a preset stop condition is met, thereby obtaining a memristor BDNN model that meets the preset performance condition.

2. The method according to claim 1, characterized in that The method of performing offline training on a digital computer using a training data set to obtain an initial BDNN model, and writing the trained network weights into a memristor to obtain an actual BDNN model includes: simulating a memristor model on the digital computer; Using the memristor conductance value of the memristor model as a weight, calculating the gradient of the memristor conductance value of the memristor model to obtain the sign of the gradient, and determining the operation direction corresponding to each device of the memristor model based on the sign; updating the memristor conductance value of the memristor model based on the operation direction, and performing conductance modulation in combination with a stochastic gradient learning algorithm and the training data set; The initial BDNN model is obtained by combining the Gaussian noise generated during the conductivity modulation process, and the network weights of the initial BDNN are written into the memristor.

3. The method according to claim 2, characterized in that The step of combining the stochastic gradient learning algorithm with the training data set to perform conductivity modulation includes: optimizing the stochastic gradient learning algorithm using a pre-constructed loss function to perform conductance modulation using the optimized stochastic gradient learning algorithm, Among them, the expression of the loss function is Among them, q(w I ) represents the memristor weight distribution of the neural network, P(w) represents the prior distribution of weights, and P(y|x,w I ) represents the predicted distribution of the neural network when the input sample is x, KL[·] represents the KL divergence, represents the calculation expectation, and β represents the weight of the two items before and after the balance; The expression for conductivity modulation using the optimized stochastic gradient learning algorithm is: Where ΔI represents the current change, represents the gradient, η m represents the Gaussian noise.

4. The method according to claim 2, characterized in that: The online training using the new training data set to update the conductance value of the memristor until a preset stop condition is met to obtain a memristor BDNN model that meets the preset performance condition includes: Recording the weight corresponding to each device of the memristor during each online training process to obtain the weight change ratio of each device; Based on the weight change ratio and the conductivity gradient corresponding to each device, the devices that meet the first preset change condition are screened out, and the weights of the devices are updated so that the weight change ratio of the device in the next round of iterations meets the second preset change condition.

5. The method according to claim 4, characterized in that in, During the online training process, based on the increase in the number of iterations, the number of devices in the memristor that update weights is reduced so that the Gaussian noise of the total current of the memristor dominates until the preset stop condition is met.

6. A memristive Bayesian deep neural network online learning device, characterized in that: include: A training module is used to perform offline training on a digital computer using a training data set to obtain an initial BDNN model, and write the trained network weights into a memristor to obtain an actual BDNN model; A calculation module, used to predict the data category of each data sample in the unlabeled data set using the actual BDNN model, and calculate the prediction uncertainty value of each data sample; A learning module is used to determine a target data sample in the unlabeled data set based on the prediction uncertainty value, and add the target data sample to the training data set to obtain a new training data set, and use the new training data set to perform online training to update the conductance value of the memristor until a preset stop condition is met, thereby obtaining a memristor BDNN model that meets the preset performance condition.

7. The device according to claim 6, characterized in that The training module includes: A simulation unit for simulating a memristor model on the digital computer; a calculation unit, configured to calculate the gradient of the memristor conductance value of the memristor model by taking the memristor conductance value of the memristor model as a weight to obtain the sign of the gradient, and determine the operation direction corresponding to each device of the memristor model based on the sign; A modulation unit, configured to update the memristor conductance value of the memristor model based on the operation direction, and perform conductance modulation in combination with a stochastic gradient learning algorithm and the training data set; A writing unit is used to obtain the initial BDNN model in combination with the Gaussian noise generated during the conductivity modulation process, and write the network weights of the initial BDNN into the memristor.

8. The device according to claim 7, characterized in that The training module also includes: an optimization unit, configured to optimize the stochastic gradient learning algorithm using a pre-constructed loss function, so as to perform conductivity modulation using the optimized stochastic gradient learning algorithm, Among them, the expression of the loss function is: Among them, q(w I ) represents the memristor weight distribution of the neural network, P(w) represents the prior distribution of weights, and P(y|x,w I ) represents the predicted distribution of the neural network when the input sample is x, KL[·] represents the KL divergence, represents the calculation expectation, and β represents the weight of the two items before and after the balance; The expression for conductivity modulation using the optimized stochastic gradient learning algorithm is: Where ΔI represents the current change, represents the gradient, η m represents the Gaussian noise.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the memristive Bayesian deep neural network online learning method as described in any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the memristive Bayesian deep neural network online learning method as described in any one of claims 1 to 5.

Images