Training method and device, electronic equipment and computer storage medium

Abstract

A training method, apparatus, electronic device, and storage medium for a Bayesian neural network based on a memristor array are disclosed. The conductance values ​​of the memristors in the memristor array are used to map the weights of the Bayesian neural network. The training method includes: acquiring first and second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristors; calculating the total loss function of the Bayesian neural network based on the first prior knowledge; performing backpropagation on the total loss function to update the current parameters in the Bayesian neural network to obtain object parameters; and constraining the object parameters based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network. This training method can improve the robustness of the network to fluctuations in the memristor conductance values.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a training method and apparatus, electronic device, and computer storage medium for a Bayesian neural network based on a memristor array. Background Technology

[0002] A memristor is a non-volatile device whose conductance can be adjusted by applying an external stimulus. According to Kirchhoff's current law and Ohm's law, an array of memristors can perform multiplication-accumulation calculations in parallel, with both storage and computation occurring within the individual devices in the array. Based on this computing architecture, in-memory computing can be implemented without requiring extensive data movement. Furthermore, multiplication-accumulation is a core computational task required for running neural networks. Therefore, using the conductance of the memristors in the array to represent weight values, highly energy-efficient neural network operations can be achieved based on this in-memory computing approach. In neural network operations, the conductance values ​​of the memristors on the memristor array represent the synaptic weights in the neural network. These conductance values ​​should be written into the memristors before the memristor array is used for computational acceleration. Some widely used offline training methods can achieve this by writing the conductance values ​​into the memristors.

[0003] A Bayesian neural network (BNN) is a parameterized model that places the flexibility of neural networks within a Bayesian framework. Unlike fixed values ​​in traditional neural networks, all weights in a BNN are represented by probability distributions. Summary of the Invention

[0004] At least one embodiment of this disclosure provides a training method for a Bayesian neural network based on a memristor array, wherein the conductance values ​​of the memristors in the memristor array are used to map the weights of the Bayesian neural network. The training method includes: acquiring first prior knowledge and second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristors; calculating the total loss function of the Bayesian neural network based on the first prior knowledge; performing backpropagation on the total loss function to update the current parameters in the Bayesian neural network to obtain object parameters; and constraining the object parameters based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network.

[0005] For example, at least one embodiment of the present disclosure provides a training method in which the first prior knowledge includes the standard deviation of weight fluctuation. Based on the first prior knowledge, the method calculates the total loss function of the Bayesian neural network, including: calculating the total loss function using variational learning based on the standard deviation of weight fluctuation, wherein the variational learning includes a complexity cost term, and the standard deviation of weight fluctuation is used as the prior standard deviation in the complexity cost term.

[0006] For example, in the training method provided in at least one embodiment of this disclosure, the Bayesian neural network is initialized with initialization parameters before the first training, the initial parameterization following a Gaussian distribution, wherein the initialization parameters serve as the current parameters for the first training of the Bayesian neural network.

[0007] For example, in at least one embodiment of this disclosure, the object parameters include the mean of the Gaussian distribution, and the second prior knowledge includes the weight window range of the memristor cells in the memristor array; constraining the object parameters based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network includes: constraining the mean of the Gaussian distribution in the object parameters to the weight window range.

[0008] For example, in at least one embodiment of the training method provided in this disclosure, the object parameters include the standard deviation of the Gaussian distribution, and the second prior knowledge includes the read fluctuation standard deviation, which is the error caused by reading the conductance value of the memristor in the memristor array; constraining the object parameters based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network includes: constraining the standard deviation of the Gaussian distribution in the object parameters such that the standard deviation of the Gaussian distribution is greater than or equal to the read fluctuation standard deviation.

[0009] For example, in at least one embodiment of the training method provided in this disclosure, obtaining first prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristors includes: performing multiple sets of electrical tests on the memristors in the memristor array, obtaining multiple test results of the multiple sets of electrical tests, wherein each test result includes the weighted standard deviation of the memristors in the memristor array; and using the maximum value of the weighted standard deviation as the first prior knowledge.

[0010] For example, in the training method provided in at least one embodiment of this disclosure, the memristor array includes multiple rows and columns of memristor cells, and each memristor cell includes a pair of first and second memristors.

[0011] The second prior knowledge includes the weight window range of the memristors in the memristor array. The weights of the Bayesian neural network are represented by the difference between the conductance values ​​of the first and second memristors included in each memristor unit of the memristor array. Obtaining the second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristors includes: calculating the range of values ​​of the difference between the conductance values ​​of the first and second memristors included in each memristor unit of the memristor array; and using the range of values ​​as the weight window range.

[0012] For example, in at least one embodiment of the training method provided in this disclosure, the training result includes the training mean of the weights in the Bayesian neural network. The training method further includes: determining whether the training of the Bayesian neural network has reached a preset number of training iterations; if the preset number of training iterations has been reached, mapping the training mean onto a memristor in the memristor array to perform calculations using the memristor array; or if the preset number of training iterations has not been reached, using the training result as the current parameter, and training the Bayesian neural network again until the preset number of training iterations is reached.

[0013] At least one embodiment of this disclosure also provides a training apparatus for a Bayesian neural network based on a memristor array, wherein the conductance values ​​of the memristors in the memristor array are used to map the weights of the Bayesian neural network. The training apparatus includes: an acquisition unit configured to acquire first prior knowledge and second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristors; a calculation unit configured to calculate the total loss function of the Bayesian neural network based on the first prior knowledge; an update unit configured to backpropagate the total loss function to update the current parameters in the Bayesian neural network to obtain object parameters; and a constraint unit configured to constrain the object parameters based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network.

[0014] At least one embodiment of this disclosure also provides an electronic device, including: a processor; and a memory storing one or more computer program instructions; wherein the one or more computer program instructions, when executed by the processor, are used to implement the training method provided in at least one embodiment of this disclosure.

[0015] At least one embodiment of this disclosure also provides a computer-readable storage medium that non-temporarily stores computer-readable instructions, which, when executed by a processor, are used to implement the training method provided in at least one embodiment of this disclosure. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0017] Figure 1A A schematic structure of a memristor array is shown;

[0018] Figure 1B This is a schematic diagram of a memristor device;

[0019] Figure 1CThis is a schematic diagram of another memristor device;

[0020] Figure 1D A schematic diagram showing the mapping of the weight matrix of a Bayesian neural network to a memristor array is shown.

[0021] Figure 2 A schematic flowchart of a training method for a Bayesian neural network based on a memristor array, provided in at least one embodiment of this disclosure, is shown.

[0022] Figure 3 A schematic flowchart illustrating a method for obtaining first prior knowledge provided in at least one embodiment of this disclosure is shown.

[0023] Figure 4A A schematic flowchart illustrating a method for obtaining second prior knowledge provided in at least one embodiment of this disclosure is shown;

[0024] Figure 4B A schematic diagram of another memristor array provided for at least one embodiment of this disclosure;

[0025] Figure 4C A schematic diagram of another memristor array provided for at least one embodiment of this disclosure;

[0026] Figure 5 A schematic flowchart of another training method provided by at least one embodiment of the present disclosure is shown;

[0027] Figure 6 A schematic block diagram of a training apparatus for a Bayesian neural network based on a memristor array, provided in at least one embodiment of the present disclosure, is shown.

[0028] Figure 7 A schematic block diagram of an electronic device provided for some embodiments of this disclosure;

[0029] Figure 8 A schematic block diagram of another electronic device provided for some embodiments of this disclosure;

[0030] Figure 9 This is a schematic diagram of a storage medium provided for some embodiments of this disclosure. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0032] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0033] In the offline training process of neural networks, the neural network is first trained using a relevant software platform, and then the trained target weights are transferred to a memristor array. However, due to the diffusion or recombination of oxygen vacancies in multiple weakly conductive filament regions, the conductivity units of the memristor may fluctuate. Even with the same programming conditions, the conductivity changes of the memristor may differ. Due to this non-ideal characteristic of conductivity variation, the deviation between the weights written to the memristor and the trained target weights is unavoidable, leading to a significant decrease in network performance. Currently, training algorithms for neural networks based on memristor arrays cannot be coordinated with the inherent non-ideal characteristics of memristors (e.g., inter-device fluctuations, device conductivity hysteresis, conductivity state drift, etc.). The trained networks are often based on models with predetermined weights. In this method, each parameter is described by a single value, without considering the uncertainty of the weights. This type of training method lacks means to handle the random characteristics of memristors, so perturbations in the memristor conductivity values ​​can have a significant impact on network performance. In most artificial intelligence systems, capturing the uncertainty in parameters is crucial. Probabilistic models offer a way to address uncertainty by enabling people to make informed decisions based on the model’s predictions while remaining cautious about the uncertainty of those predictions.

[0034] A Bayesian Neural Network (BNN) is a probabilistic model that places a neural network within a Bayesian framework, capable of describing complex stochastic patterns. To account for parameter uncertainty, it is best to construct a Bayesian model. In a Bayesian model, parameters are not represented by single values, but rather by probability distributions. Given observed data, the distribution of all parameters in a Bayesian model is called the posterior distribution. As an analogy to deriving an optimal deterministic model through gradient-based updates, the goal of Bayesian machine learning is to learn an approximation of the posterior distribution.

[0035] At least one embodiment of this disclosure provides a training method for a Bayesian neural network based on a memristor array, wherein the conductance values ​​of the memristors in the memristor array are used to map the weights of the Bayesian neural network. The training method includes: acquiring first and second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristors; calculating the total loss function of the Bayesian neural network based on the first prior knowledge; performing backpropagation on the total loss function to update the current parameters in the Bayesian neural network to obtain object parameters; and constraining the object parameters based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network.

[0036] The training method provided by the above embodiments of this disclosure can ensure that the output of the memristor neural network is robust and reliable even under the perturbation of the memristor weights, thereby improving the robustness of the neural network to fluctuations in the memristor conductance.

[0037] At least one embodiment of this disclosure also provides a training apparatus corresponding to the above-described training apparatus for a Bayesian neural network based on a memristor array.

[0038] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0039] First, let's explain the training method for Bayesian neural networks.

[0040] Given a dataset D, the training objective of a Bayesian neural network is to optimize the posterior distribution p(w|D) of the weights using Bayes' theorem:

[0041]

[0042] Here, p(w) is the prior weight distribution, p(D|w) = p(y|x,w) is the likelihood corresponding to the Bayesian neural network output, and p(D) is the marginal likelihood, i.e., the evidence. Since the true posterior distribution p(w|D) is difficult to realize, it is not directly computed but is usually approximated using inference methods. For example, variational learning methods are used to approximate the posterior distribution of the Bayesian neural network weights.

[0043] Variational learning seeks parameters θ of the weight distribution q(w|θ) of a Bayesian neural network that minimize the Kullback-Leibler (KL) divergence between this distribution and the true posterior distribution. KL divergence measures how closely the distribution q(w|θ) approximates the true posterior distribution. Also known as relative entropy or information divergence, KL divergence is an asymmetric measure of the difference between two probability distributions. Through mathematical transformation, the objective of minimizing the KL divergence between q(w|θ) and p(w|D) can be expressed as:

[0044]

[0045] During optimization, the backpropagation algorithm can be used to optimize the first two terms: KL(q(w|θ)|p(w)) and E. q(w|θ) [logp(D|w). Where KL(q(w|θ)|p(w))( is called the complexity cost term, E q(w|θ) [logp(D|w) is called the likelihood cost term. For the weights of a Gaussian distribution, θ corresponds to the mean μ and standard deviation σ, then the posterior (w|θ) can be expressed as:]

[0046]

[0047] BNNs derive the posterior weight distribution using prior p(w) and likelihood probability p(D|w). This key feature introduces weight uncertainty into the learning process. Therefore, the learned weight parameters and computation are robust to weight perturbations.

[0048] Figure 1A A schematic structure of a memristor array is shown, which, for example, consists of multiple memristor cells forming an M-row, N-column array, where M and N are both positive integers. Each memristor cell includes a switching element and one or more memristors. Figure 1A In the middle, WL <1> WL <2> ...WL <m>These represent the word lines for the first, second, ..., Mth rows, respectively. The control electrode (e.g., the gate of a transistor) of the switching element in the memristor cell circuit of each row is connected to the corresponding word line of that row; BL <1> BL <2> BL <n>These represent the bit lines of the first, second, ..., Nth columns, respectively. The memristor in the memristor cell circuit of each column is connected to the corresponding bit line of that column; SL <1> SL <2> ……SL <m>source lines representing the first row, the second row, …, the Mth row, respectively, the source of the transistor in the memristor cell circuit of each row is connected to the source line corresponding to the row. According to Kirchhoff's law, the above-mentioned memristor array can complete the multiply-accumulate calculation in parallel by setting the state (for example, the resistance value) of the memristor unit and applying the corresponding word line signal and bit line signal on the word line and bit line.

[0049] Figure 1B It is a schematic diagram of a memristor device, which includes a memristor array and its peripheral driving circuit. For example, as shown in Figure 1B The memristor device includes a signal acquisition device, a word line driving circuit, a bit line driving circuit, a source line driving circuit, a memristor array, and a data output circuit.

[0050] For example, the signal acquisition device is configured to convert a digital signal into a plurality of analog signals through a digital to analog converter (DAC) to input to a plurality of column signal input terminals of the memristor array.

[0051] For example, the memristor array includes M source lines, M word lines, and N bit lines, and the array is arranged as a plurality of memristor units in M rows and N columns.

[0052] For example, the operation of the memristor array is realized through the word line driving circuit, the bit line driving circuit, and the source line driving circuit.

[0053] For example, the word line driving circuit includes a plurality of multiplexers (Mux) for switching the word line input voltage; the bit line driving circuit includes a plurality of multiplexers for switching the bit line input voltage; and the source line driving circuit also includes a plurality of multiplexers (Mux) for switching the source line input voltage. For example, the source line driving circuit further includes a plurality of ADCs for converting analog signals into digital signals. In addition, a trans-impedance amplifier (TIA) (not shown in the figure) can be further provided between the Mux and the ADC in the source line driving circuit to complete the conversion from current to voltage, so as to facilitate the processing of the ADC.

[0054] For example, the memristor array includes an operation mode and a calculation mode. When the memristor array is in the operation mode, the memristor unit is in an initialization state, and the numerical values of the parameter elements in the parameter matrix can be written into the memristor array. For example, the source line input voltage, the bit line input voltage, and the word line input voltage of the memristor are switched to the corresponding preset voltage interval through the multiplexer.

[0055] For example, by Figure 1B The control signal WL_sw[1:M] of the multiplexer in the word line driver circuit switches the word line input voltage to the corresponding voltage range. For example, when setting the memristor, the word line input voltage is set to 2V (volts); when resetting the memristor, the word line input voltage is set to 5V. The word line input voltage can be adjusted by... Figure 1B The voltage signal V_WL[1:M] in the middle is obtained.

[0056] For example, through Figure 1B The control signal SL_sw[1:M] of the multiplexer in the source line drive circuit switches the source line input voltage to the corresponding voltage range. For example, when setting the memristor, the source line input voltage is set to 0V; when resetting the memristor, the source line input voltage is set to 2V. The source line input voltage can be adjusted by... Figure 1B The voltage signal V_SL[1:M] is obtained.

[0057] For example, through Figure 1B The control signal BL_sw[1:N] of the multiplexer in the bit line drive circuit switches the bit line input voltage to the corresponding voltage range. For example, when setting the memristor, the bit line input voltage is set to 2V; when resetting the memristor, the bit line input voltage is set to 0V. The bit line input voltage can be... Figure 1B The DAC is obtained.

[0058] For example, when the memristor array is in compute mode, the memristors in the array are in a conductive state that can be used for computation. The bit line input voltage at the column signal input terminal does not change the conductance of the memristors. For example, calculations can be performed by multiplying and adding operations using the memristor array. Figure 1B The control signal WL_sw[1:M] of the multiplexer in the word line driver circuit switches the word line input voltage to the corresponding voltage range. For example, when an enable signal is applied, the word line input voltage of the corresponding row is set to 5V; when no enable signal is applied, the word line input voltage of the corresponding row is set to 0V. For example, the GND signal is turned on. Figure 1B The control signal SL_sw[1:M] of the multiplexer in the source line drive circuit switches the source line input voltage to the corresponding voltage range, for example, setting the source line input voltage to 0V, so that the current signals of multiple row signal output terminals can flow into the data output circuit. Figure 1B The control signal BL_sw[1:N] of the multiplexer in the bit line drive circuit switches the bit line input voltage to the corresponding voltage range, for example, setting the bit line input voltage to 0.1V-0.3V, thereby using the memristor array to perform multiplication and addition operations.

[0059] For example, the data output circuit may include multiple transimpedance amplifiers (TIAs) and ADCs, which can convert the current signals from multiple row signal outputs into voltage signals, and then into digital signals for subsequent processing.

[0060] Figure 1C This is a schematic diagram of another type of memristor device. Figure 1C The memristor device shown is Figure 1B The memristor devices shown have essentially the same structure, also including a memristor array and its peripheral drive circuitry. For example, as... Figure 1C As shown, the memristor device includes a signal acquisition device, a word line driving circuit, a bit line driving circuit, a source line driving circuit, a memristor array, and a data output circuit.

[0061] For example, a memristor array may include M source lines, 2M word lines, and 2N bit lines, as well as multiple memristor cells arranged in an M x N array. For instance, each memristor cell may be a 2T2R structure, which allows for mapping between positive and negative values. The operation of mapping the parameter matrix used for transformation processing to different memristor cells in the memristor array will not be elaborated here. It should be noted that a memristor array may also include M source lines, M word lines, and 2N bit lines, as well as multiple memristor cells arranged in an M x N array.

[0062] The descriptions of the signal acquisition device, control drive circuit, and data output circuit can be found in the previous descriptions and will not be repeated here.

[0063] For example, the process of mapping the weight matrix of a Bayesian neural network to a memristor array can be found in [reference needed]. Figure 1D .

[0064] Figure 1D This illustrates the process of mapping the weight matrix of a Bayesian neural network to a memristor array. The weight matrix between layers in the Bayesian neural network is implemented using a memristor array. For each weight, N memristors are used to represent the distribution corresponding to that weight, where N is an integer greater than or equal to 2. For the random probability distribution corresponding to that weight, N conductance values ​​are calculated, and these N conductance values ​​are mapped to the N memristors. In this way, the weight matrix in the Bayesian neural network is transformed into target conductance values ​​mapped to the crossover sequence of the memristor array.

[0065] like Figure 1D As shown, the left side of the diagram represents a three-layer Bayesian neural network, comprising three interconnected layers of neurons. For example, the input layer consists of a first layer of neurons, the hidden layer consists of a second layer of neurons, and the output layer consists of a third layer of neurons. The input layer receives input data and passes it to the hidden layer, which then performs calculations and transformations on the input data before sending it to the output layer. The output layer then outputs the structure of the Bayesian neural network.

[0066] like Figure 1D As shown, the input layer, hidden layer, and output layer each contain multiple neuron nodes, and the number of neuron nodes in each layer can be set according to different application scenarios. For example, the input layer has 2 neurons (including N1 and N2), the hidden layer has 3 neurons (including N3, N4, and N5), and the output layer has 1 neuron (including N6).

[0067] like Figure 1D As shown, adjacent layers of neurons in a Bayesian neural network are connected by a weight matrix. For example, the weight matrix is ​​composed of... Figure 1D The memristor array on the right side is implemented.

[0068] Figure 1D The structure of the memristor array on the right side of the image is as follows: Figure 1A As shown, the memristor array may include multiple memristors arranged in an array. For example... Figure 1D In the example shown, weights are mapped to the conductance of the memristor array according to a certain rule. The weights connecting the input N1 and the output N3 are composed of three memristors (G... 11 G 12 G 13 This is implemented in the same way, and the other weights in the weight matrix can be implemented similarly. More specifically, source line SL1 corresponds to neuron N3, source line SL2 corresponds to neuron N4, source line SL5 corresponds to neuron N5, bit lines BL1, BL2, and BL3 correspond to neuron N1, and a weight between the input layer and the hidden layer (the weight between neuron N1 and neuron N3) is converted into three target conductance values ​​according to a distribution and mapped to the cross sequence of the memristor array, where the target conductance values ​​are G... 11 G 12 and G 13 It is outlined with a dashed box in the memristor array.

[0069] In other embodiments of this disclosure, the weights in the weight matrix are directly programmed as the conductance of the memristor array, meaning that there is a one-to-one correspondence between the weights in the weight matrix and the memristors in the memristor array. Each weight is implemented using one memristor.

[0070] In other embodiments of this disclosure, the difference in conductance between two memristors can also be used to represent a weight. For example, the difference in conductance between two memristors in the same column and adjacent rows represents a weight. That is, each weight is implemented using two memristors to correspond to that weight.

[0071] Figure 2 A schematic flowchart of a training method for a Bayesian neural network based on a memristor array, provided in at least one embodiment of the present disclosure, is shown.

[0072] For example, a memristor array comprises multiple memristors arranged in an array with multiple rows and columns; for example, the weight matrix obtained by training a Bayesian neural network is mapped into the memristor array.

[0073] For example, the structure of a Bayesian neural network includes fully connected structures or convolutional neural network structures. Each weight in this Bayesian neural network is a random variable. For example, after the Bayesian neural network is trained, each weight follows a distribution, such as a Gaussian distribution or a Laplace distribution.

[0074] For example, the weight matrix can be obtained by offline training of a Bayesian neural network. The training method for a Bayesian neural network can refer to conventional methods, such as using a central processing unit (CPU), image processing unit (GPU), neural network processing unit (NPU), neural network accelerator, etc., which will not be elaborated here.

[0075] like Figure 2 As shown, the training method includes the following steps S101 to S104.

[0076] Step S101: Obtain the first and second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristor.

[0077] Step S102: Based on the first prior knowledge, calculate the total loss function of the Bayesian neural network.

[0078] Step S103: Backpropagate the total loss function to update the current parameters in the Bayesian neural network to obtain the object parameters.

[0079] Step S104: Constrain the object parameters based on the second prior knowledge to obtain the training results of the weights of the Bayesian neural network.

[0080] The training method proposed in this disclosure integrates the influence of memristor conductance fluctuations into the training of the Bayesian neural network, which ensures that the output of the memristor neural network is robust and reliable even under perturbations of the memristor weights.

[0081] In some embodiments of this disclosure, the weight values ​​written to the memristor array can be regarded as samples generated from a certain distribution. For example, the weights represented by the memristor can be regarded as following a Gaussian distribution N(μ,σ) in a BNN. 2 The weights are uncertain. The average value μ is the target weight value that will be transferred to the memristor array. Secondly, when optimizing the complexity cost term in the BNN objective function, the weight distribution N(μ,σ) is... 2 It will be as similar as possible to the prior.

[0082] For step S101, the inherent non-ideal characteristics of the memristor include, for example, inter-device fluctuations, device conductance hysteresis, and conductance state drift. These inherent non-ideal characteristics of the memristor can cause the conductance value written to the memristor to drift, thereby affecting the calculation accuracy of the memristor array.

[0083] The first prior knowledge may include, for example, the fluctuation parameter of the conductance value of the memristor cell. The fluctuation parameter may include, for example, the weighted standard deviation of the fluctuation (i.e., the standard deviation of the conductance value fluctuation), the weighted mean of the fluctuation (i.e., the mean of the conductance value fluctuation), etc. The fluctuation parameter is not limited to the weighted standard deviation of the fluctuation and the weighted mean of the fluctuation; it may also be other parameters, and this disclosure does not specifically limit the fluctuation parameter. For example, the fluctuation parameter may also include the fluctuation range of the conductance value. In embodiments of this disclosure, the memristor cell may include a combination of one memristor and one transistor (1T1R), a combination of two memristors and two transistors (2T2R), etc.

[0084] The second prior knowledge may include, for example, the reading ripple parameter. The reading ripple parameter refers to the error introduced when reading the conductance value of a memristor. For example, this error may arise from the inaccuracy of the voltmeter, ammeter, or multimeter used to read the conductance value of the memristor. The reading ripple parameter may also include, for example, the reading ripple standard deviation, which can be calculated by reading the conductance value of the memristor multiple times.

[0085] The second prior knowledge may also include, for example, a weighted window range. The weighted window range refers, for example, to the range of conductance values ​​for a memristor cell corresponding to a given conductance value. A memristor cell may include one or more memristors.

[0086] The following text Figure 3 Figure 4 illustrates an implementation method for obtaining the first and second prior knowledge, as described below.

[0087] For step S102, for example, the first prior knowledge includes the weighted fluctuation standard deviation σ. proir Step S102 includes: calculating the total loss function using variational learning based on the standard deviation of weight fluctuations. Variational learning includes a complexity cost term, with the standard deviation of weight fluctuations serving as the prior standard deviation within the complexity cost term.

[0088] For example, in some embodiments of this disclosure, as described above, the total loss function obtained using variational learning includes a KL loss term and a likelihood loss term. For instance, the total loss function is expressed as follows:

[0089] F(D,θ)=KL[q(w|θ)||P(w)]-E q(w|θ) [logP(D|w)],

[0090] Where KL[q(w|θ)||P(w)] is the KL loss term, E q(w|θ) [logP(D|w)] is the likelihood loss term.

[0091] In this example, the standard deviation of the weighted fluctuation σ proir As the prior standard deviation P(w), the weighted standard deviation of fluctuation σ proir Substituting these values ​​into the expression of the loss function above, we can calculate the total loss function.

[0092] Given that the first prior knowledge is other parameters, a person skilled in the art can calculate the total loss function for other parameters based on the expression of the total loss function.

[0093] For step S103, for example, when training the parameters such as the weights of the neural network using gradient descent, it is necessary to use backpropagation to calculate the partial derivatives of the loss function with respect to the weights, thereby obtaining the gradient of the memristor conductance state of each weight in the weight matrix used for the Bayesian neural network. Geometrically, the direction of the gradient is where the function increases the fastest, and the opposite direction of the gradient is where the function decreases the fastest, making it easier to find the minimum value.

[0094] For example, for each parameter μ of a Bayesian neural network i ,σ i Each weight w i Follows a Gaussian distribution Backpropagation is performed using the total loss function calculated in step S102 to calculate each current parameter μ. i ,σ i The update amount Δ is used to update each parameter. For example, parameter μ i Updated to μ i +Δ, parameter σ i Updated to σ i +Δ. The object parameter is the parameter μ of the current parameter. i ,σ i Updated parameter μ i +Δ and σ i +Δ.

[0095] In some embodiments of this disclosure, before the first training, the Bayesian neural network is initialized with initialization parameters that follow a Gaussian distribution, and these initialization parameters serve as the current parameters for the first training of the Bayesian neural network.

[0096] In some embodiments of this disclosure, the object parameters include the mean of a Gaussian distribution, and the second prior knowledge includes the weighted window range of the memristor cells in the memristor array.

[0097] Step S104 may include constraining the mean of the Gaussian distribution in the object parameters to the weight window range.

[0098] For example, the weight window range is [-w max ,w max ], to take μ from the object parameter i +Δ constraint to [-w max ,w max [Inside. For example, μ] i +Δ<-w max Then μ i +Δ constraint is -w max .

[0099] In this embodiment, since the conductivity window of the memristor is limited, the weights will be truncated within a symmetrical range, namely the aforementioned weight window range [-w]. max ,w max Within this framework, we ensure greater compatibility between the Bayesian neural network and the memristor array, avoiding situations where the mean of the Gaussian distribution is outside the weight window range, making it difficult to map to the memristor cells.

[0100] The weight window range is the range of conductance values ​​of the memristor unit. For example, a memristor unit includes a first memristor and a second memristor. The difference between the conductance values ​​of the first memristor and the second memristor forms the conductance value of the memristor unit, which is used as the weight of the Bayesian neural network.

[0101] In other embodiments of this disclosure, the object parameters include the standard deviation of a Gaussian distribution, and the second prior knowledge includes the standard deviation of the read fluctuation, which is the error caused by reading the conductance value of the memristors in the memristor array.

[0102] Step S104 may include constraining the standard deviation of the Gaussian distribution in the object parameters so that the standard deviation of the Gaussian distribution is greater than or equal to the standard deviation of the fluctuation.

[0103] For example, the standard deviation of a Gaussian distribution is σ. i Equal to 0.1, the standard deviation of the fluctuation is σ. read Equals 0.15, σ i <σ read Then the standard deviation σ of the Gaussian distribution is... i The constraint is 0.15, i.e., σ i The value is updated to 0.15.

[0104] Constraining the standard deviation of the Gaussian distribution to be greater than or equal to the standard deviation of the read fluctuation allows the Bayesian neural network to accommodate errors caused by read fluctuations, resulting in better robustness. Since read fluctuations always exist throughout the memristor's lifespan, the conductivity deviation will not be zero. Therefore, the minimum value of the posterior standard deviation is further limited during the learning process (i.e., the standard deviation of the read fluctuation is σ). read ).

[0105] Figure 3 A schematic flowchart of a method for obtaining first prior knowledge provided by at least one embodiment of the present disclosure is shown.

[0106] like Figure 3 As shown, the method may include steps S111 and S121.

[0107] Step S111: Perform multiple sets of electrical tests on the memristors in the memristor array to obtain multiple test results. Each test result includes the weighted standard deviation of the memristors in the memristor array.

[0108] Step S121: Take the maximum value of the standard deviation of weighted fluctuations as the first prior knowledge.

[0109] This embodiment can use the worst-case weight fluctuations of the memristor as prior knowledge to obtain a large posterior standard deviation during training. After training, the learned weight distribution has a high tolerance for the worst case while ensuring network performance.

[0110] For step S111, for example, each group of electrical tests includes multiple electrical tests. The multiple electrical tests may be performed on one memristor in the memristor array or on multiple memristors in the memristor array.

[0111] For example, each electrical test includes a write operation, a delay operation, and a read operation on the memristor. The delay operation is performed after a preset time length. The delay operation is used to wait for the conductance value written to the memristor to drift due to the inherent non-ideal characteristics of the memristor. The preset time length is, for example, 1 second, 10 seconds, etc., and this disclosure does not limit the specific length of the preset time. For example, for each set of electrical tests, an initial conductance value is first written to the memristor. After the preset time length, the current conductance value of the memristor is read, and the absolute value of the difference between the current conductance value and the initial conductance value is used as the weighted fluctuation value of this electrical test. Multiple electrical tests are performed according to the above electrical test method to obtain multiple weighted fluctuation values, and the standard deviation of the multiple weighted fluctuation values ​​is the weighted fluctuation standard deviation obtained for this set of electrical tests.

[0112] For step S121, for example, the weighted fluctuation standard deviations obtained from multiple sets of electrical tests are compared to obtain the maximum value of the weighted fluctuation standard deviation, and this maximum value is used as the first prior knowledge, i.e., the weighted fluctuation standard deviation σ. proir .

[0113] Figure 4A A schematic flowchart of a method for obtaining second prior knowledge provided by at least one embodiment of the present disclosure is shown.

[0114] like Figure 4A As shown, the method may include steps S131 and S141.

[0115] Step S131: Calculate the range of values ​​for the difference between the conductance values ​​of the first memristor and the second memristor in each memristor cell of the memristor array.

[0116] Step S141: Use the range of values ​​as the weight window range.

[0117] In this embodiment, the memristor array includes multiple rows and columns of memristor cells, each memristor cell including a first memristor and a second memristor provided in pairs. The second prior knowledge includes a weight window range for the memristors in the memristor array, and the weights of the Bayesian neural network are represented based on the difference between the conductance values ​​of the first and second memristors included in each memristor cell of the memristor array.

[0118] For step S131, for example, the conductance of the first memristor is a, the conductance of the second memristor is b, and the difference between the conductance of the first memristor and the conductance of the second memristor is ab. a>0, and b>0.

[0119] Figure 4B This is a schematic structural diagram of a memristor array provided for at least one embodiment of the present disclosure.

[0120] like Figure 4B As shown, memristors 401 and 402 can form a memristor pair, and the conductance of memristor 401 is expressed as G. 11 The conductance of memristor 402 is expressed as G. 12 Since memristor 402 is connected to an inverter, when memristor 401 receives a positive input voltage signal, the inverter can flip the polarity of the input voltage signal, thus allowing memristor 402 to receive a negative input voltage signal. For example, the input voltage signal received by memristor 401 is denoted by v(t), and the input voltage signal received by memristor 402 is denoted by -v(t). Memristors 401 and 402 are connected to two different SLs, and the input voltage signal generates an output current through the memristors. At the end of the SL, the output current through memristor 401 and the output current through memristor 402 are superimposed. Therefore, the result of multiplying and summing the memristors 401 and 402 is v(t)G. 11 +(-v(t))G 12 That is, v(t)(G 11 -G 12 Therefore, the memristor pair consisting of memristor 401 and memristor 402 can correspond to a weight, and the weight value is G. 11 -G 12 By configuring G 11 -G 12 The numerical relationships can realize positive, zero, and negative elements.

[0121] Figure 4C This is a schematic diagram of another memristor array provided for at least one embodiment of the present disclosure.

[0122] like Figure 4C As shown, for example, memristors 401 and 402 can form a memristor pair, and the conductance of memristor 401 is expressed as G. 11 The conductance of memristor 402 is expressed as G. 12 .and Figure 4A The difference is that memristor 402 is not connected to an inverter. Therefore, when memristor 401 receives a positive input voltage signal, memristor 402 also receives a positive input voltage signal. For example, the input voltage signal received by memristor 401 is denoted by v(t), and the input voltage signal received by memristor 402 is also denoted by v(t). Memristors 401 and 402 are connected to two different SLs. At the end of the SLs, the output current through memristor 401 is subtracted from the output current through memristor 402. Therefore, the result of multiplying and accumulating the values ​​of memristors 401 and 402 is v(t)G. 11 -v(t)G 12 That is, v0(t)(G 11 -G 12 Therefore, the memristor pair consisting of memristor 401 and memristor 402 can have a weight, and the weight value is G. 11 -G 12 By configuring G 11 -G 12 The numerical relationships can realize positive, zero, and negative elements.

[0123] For example, to the first memristor G 11 The written conductance value is within the first range, and is sent to the second memristor G. 12 The written conductance value is within the second value range, and the range of the difference between the conductance values ​​of the first memristor and the second memristor is calculated based on the first value range and the second value range.

[0124] For step S141, for example, the difference between the conductance values ​​of the first memristor and the second memristor ranges from [-w max ,w max If ], then the weight window range is [-w max ,w max ].

[0125] Figure 5 A schematic flowchart of another training method provided by at least one embodiment of the present disclosure is shown.

[0126] like Figure 5 As shown, this training method includes Figure 2 Based on steps S101 to S104, steps S501 to S503 are also included. Steps S501 to S503 can be performed, for example, after step S104. The training result includes the training mean of the weights in the Bayesian neural network.

[0127] Step S501: Determine whether the training of the Bayesian neural network has reached the preset number of training iterations.

[0128] Step S502: If the preset number of training iterations is reached, the training mean is mapped onto the memristors in the memristor array for calculation.

[0129] Step S503: If the preset number of training iterations has not been reached, use the training result as the current parameter and train the Bayesian neural network again until the preset number of training iterations is reached.

[0130] For step S501, the preset number of training iterations is set by those skilled in the art based on experience. For example, during the training of the Bayesian neural network, the number of training iterations is counted. For instance, each time object parameters are obtained, it indicates that the Bayesian neural network has completed one training iteration, and this is counted once. It is then determined whether the current count has reached the preset number of training iterations.

[0131] For step S502, if the current count value reaches the preset number of training times, the mean matrix formed by the training mean of the Bayesian neural network is mapped to the memristor array, and the elements in the mean matrix correspond one-to-one with the memristor units in the memristor array.

[0132] For step S503, for example, if the preset number of training iterations has not been reached, the training result is used as the current parameter, and steps S102 to S103 and step S501 are executed again until the Bayesian neural network is trained to the preset number of training iterations.

[0133] Figure 6 A schematic block diagram of a training device 600 for a Bayesian neural network based on a memristor array, provided in at least one embodiment of this disclosure, is shown. This training device 400 can be used to perform… Figure 2 The training method is illustrated. For example, a memristor array consists of multiple memristors arranged in an array, and the weight matrix obtained by training a Bayesian neural network is mapped onto the memristor array.

[0134] like Figure 6 As shown, the training device 600 includes an acquisition unit 601, a calculation unit 602, an update unit 603, and a constraint unit 604.

[0135] The acquisition unit 601 is configured to acquire first and second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristor.

[0136] The calculation unit 602 is configured to calculate the total loss function of the Bayesian neural network based on the first prior knowledge.

[0137] The update unit 603 is configured to backpropagate the total loss function to update the current parameters in the Bayesian neural network to obtain the object parameters.

[0138] The constraint unit 604 is configured to constrain the object parameters based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network.

[0139] The technical effects of the above training device and Figure 2 The training methods shown have the same technical effect, and will not be described in detail here.

[0140] For example, the acquisition unit 601, calculation unit 602, update unit 603, and constraint unit 604 can be hardware, software, firmware, or any feasible combination thereof. For example, the acquisition unit 601, calculation unit 602, update unit 603, and constraint unit 604 can be dedicated or general-purpose circuits, chips, or devices, or a combination of a processor and memory. The embodiments of this disclosure do not limit the specific implementation of the above units.

[0141] It should be noted that in the embodiments of this disclosure, each unit of the training device 600 corresponds to each step of the aforementioned training method. For the specific functions of the training device 600, please refer to the relevant description of the training method, which will not be repeated here. Figure 6 The components and structure of the training device 600 shown are merely exemplary and not limiting; the training device 600 may also include other components and structures as needed.

[0142] At least one embodiment of this disclosure also provides an electronic device including a processor and a memory storing one or more computer program instructions. When executed by the processor, the one or more computer program instructions are used to implement the training method described above. This electronic device can improve the robustness of the network to fluctuations in memristor conductance.

[0143] Figure 7 This is a schematic block diagram of an electronic device provided for some embodiments of this disclosure. For example... Figure 7 As shown, the electronic device 700 includes a processor 710 and a memory 720. The memory 720 stores non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 710 executes the non-transitory computer-readable instructions, which, when executed by the processor 710, can perform one or more steps in the training method described above. The memory 720 and the processor 710 can be interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0144] For example, processor 710 may be a central processing unit (CPU), a graphics processing unit (GPU), or other form of processing unit with data processing and / or program execution capabilities. For example, the central processing unit (CPU) may be an x76 or ARM architecture. Processor 710 may be a general-purpose processor or a special-purpose processor, capable of controlling other components in electronic device 700 to perform desired functions.

[0145] For example, memory 720 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and processor 710 may run one or more computer program modules to implement various functions of electronic device 700. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0146] It should be noted that, in the embodiments of this disclosure, the specific functions and technical effects of the electronic device 700 can be referred to the description of the training method above, and will not be repeated here.

[0147] Figure 8 This is a schematic block diagram of another electronic device provided in some embodiments of this disclosure. The electronic device 800 is, for example, suitable for implementing the training methods provided in the embodiments of this disclosure. For example, the electronic device 800 may be a terminal device, etc. It should be noted that... Figure 8 The illustrated electronic device 800 is merely an example and does not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0148] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 810, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 820 or a program loaded from a storage device 880 into a random access memory (RAM) 830. The RAM 830 also stores various programs and data required for the operation of the electronic device 800. The processing device 810, ROM 820, and RAM 830 are interconnected via a bus 840. An input / output (I / O) interface 850 is also connected to the bus 840.

[0149] Typically, the following devices can be connected to I / O interface 850: input devices 860 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 870 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 880 including, for example, magnetic tapes, hard disks, etc.; and communication devices 890. Communication device 890 allows electronic device 800 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 8 An electronic device 800 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 800 may alternatively implement or have more or fewer devices.

[0150] For example, according to embodiments of this disclosure, the training method described above can be implemented as a computer software program. For instance, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the training method described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 890, or installed from a storage device 880, or installed from a ROM 820. When the computer program is executed by the processing device 810, it can implement the functions defined in the training method provided by embodiments of this disclosure.

[0151] At least one embodiment of this disclosure also provides a computer-readable storage medium for storing non-transitory computer-readable instructions that, when executed by a computer, enable the training method described above. Using this computer-readable storage medium, the robustness of the network to fluctuations in memristor conductance can be improved.

[0152] Figure 9 This is a schematic diagram of a storage medium provided for some embodiments of this disclosure. For example... Figure 9 As shown, storage medium 900 is used to store non-transitory computer-readable instructions 910. For example, when the non-transitory computer-readable instructions 910 are executed by a computer, one or more steps in the training method described above can be performed.

[0153] For example, the storage medium 900 can be used in the aforementioned electronic device 700. For example, the storage medium 900 can be... Figure 7 The memory 720 in the illustrated electronic device 700. For example, a description of the storage medium 900 can be found here. Figure 7 The corresponding description of the memory 720 in the illustrated electronic device 700 will not be repeated here.

[0154] The following points need to be explained:

[0155] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0156] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0157] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.< / m> < / n> < / m>

Claims

1. A training method for a Bayesian neural network based on a memristor array, wherein the conductance values ​​of the memristors in the memristor array are used to map the weights of the Bayesian neural network, the training method comprising: The first prior knowledge and the second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristor are obtained. The first prior knowledge includes the fluctuation parameter of the memristor conductance value, and the second prior knowledge includes either the standard deviation of the fluctuation and the weighted window range. The standard deviation of the fluctuation is the error caused by reading the conductance value of the memristor in the memristor array, and the weighted window range is the range of conductance values ​​of the memristor cell corresponding to the conductance value. Based on the first prior knowledge, calculate the total loss function of the Bayesian neural network; Backpropagation is performed on the total loss function to update the current parameters in the Bayesian neural network to obtain the object parameters; and The object parameters are constrained based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network. The step of calculating the total loss function of the Bayesian neural network based on the first prior knowledge includes: Based on the fluctuation parameters, the total loss function is calculated using variational learning, wherein the variational learning includes a complexity cost term, and the fluctuation parameters serve as prior parameters in the complexity cost term.

2. The training method according to claim 1, wherein, Before the initial training, the Bayesian neural network is initialized with initialization parameters that follow a Gaussian distribution. The initialization parameters are used as the current parameters during the first training of the Bayesian neural network.

3. The training method according to claim 2, wherein, The object parameters include the mean of the Gaussian distribution, and the second prior knowledge includes the weight window range; Constraining the object parameters based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network includes: The mean of the Gaussian distribution in the object parameters is constrained to the range of the weight window.

4. The training method according to claim 2, wherein, The object parameters include the standard deviation of the Gaussian distribution, and the second prior knowledge includes the standard deviation of the reading fluctuation. Constraining the object parameters based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network includes: The standard deviation of the Gaussian distribution in the object parameters is constrained such that the standard deviation of the Gaussian distribution is greater than or equal to the standard deviation of the reading fluctuation.

5. The training method according to claim 1, wherein, Obtaining first prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristors includes: Multiple sets of electrical tests are performed on the memristors in the memristor array, and multiple test results are obtained from these tests. Each test result includes the weighted standard deviation of the memristors in the memristor array; and The maximum value of the standard deviation of the weighted fluctuation is taken as the first prior knowledge.

6. The training method according to claim 1, wherein, The memristor array comprises multiple rows and columns of memristor cells, each memristor cell comprising a pair of first and second memristors. The second prior knowledge includes the weight window range of the memristors in the memristor array, and the weights of the Bayesian neural network are represented based on the difference between the conductance values ​​of the first memristor and the second memristor included in each memristor cell of the memristor array. Obtaining second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristors includes: Calculate the range of values ​​for the difference between the conductance values ​​of the first and second memristors in each memristor cell of the memristor array; and The range of values ​​is used as the weight window range.

7. The training method according to claim 1, wherein, The training result includes the training mean of the weights in the Bayesian neural network, and the training method further includes: Determine whether the training of the Bayesian neural network has reached the preset number of training iterations; If the preset number of training iterations is reached, the training mean is mapped onto the memristors in the memristor array for calculation using the memristor array, or If the preset number of training iterations is not reached, the training result is used as the current parameter, and the Bayesian neural network is trained again until the preset number of training iterations is reached.

8. A training apparatus for a Bayesian neural network based on a memristor array, wherein the conductance values ​​of the memristors in the memristor array are used to map the weights of the Bayesian neural network, the training apparatus comprising: The acquisition unit is configured to acquire first prior knowledge and second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristor, wherein the first prior knowledge includes the fluctuation parameter of the memristor conductance value, and the second prior knowledge includes either the standard deviation of the reading fluctuation or the weighted window range, wherein the standard deviation of the reading fluctuation is the error caused by reading the conductance value of the memristor in the memristor array, and the weighted window range is the range of conductance values ​​of the memristor cell corresponding to the conductance value; The computing unit is configured to calculate the total loss function of the Bayesian neural network based on the first prior knowledge; The update unit is configured to backpropagate the total loss function to update the current parameters in the Bayesian neural network to obtain the object parameters; and The constraint unit is configured to constrain the object parameters based on the second prior knowledge to obtain the training result of the weights of the Bayesian neural network. The step of calculating the total loss function of the Bayesian neural network based on the first prior knowledge includes: Based on the fluctuation parameters, the total loss function is calculated using variational learning, wherein the variational learning includes a complexity cost term, and the fluctuation parameters serve as prior parameters in the complexity cost term.

9. An electronic device, comprising: processor; Memory, which stores one or more computer program instructions; Wherein, the one or more computer program instructions, when executed by the processor, are used to implement the training method according to any one of claims 1-7.

10. A computer-readable storage medium that non-transitoryly stores computer-readable instructions, wherein, When the computer-readable instructions are executed by a processor, they are used to implement the training method according to any one of claims 1-7.

Images