Weight writing method and apparatus, electronic device, and storage medium

Abstract

A method, apparatus, electronic device, and storage medium for writing weights to a neural network applied to a memristor array are disclosed. The weight writing method includes: acquiring a target weight value to be written to a target memristor cell in the memristor array and an error range for writing the target weight value, the error range being determined based on a probability distribution corresponding to the target weight value; acquiring the current weight value of the target memristor cell; determining whether the absolute value of the difference between the current weight value and the target weight value is within the error range; and determining that the target weight value has been written to the target memristor cell in response to the absolute value of the difference between the current weight value and the target weight value being within the error range. This weight writing method can improve the efficiency of writing weights to a memristor array in a neural network.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a method and apparatus for writing weights to a neural network applied to a memristor array, an electronic device, and a storage medium. Background Technology

[0002] A memristor is a non-volatile device whose conductance can be adjusted by applying an external stimulus. According to Kirchhoff's current law and Ohm's law, an array of memristors can perform multiplication-accumulation calculations in parallel, with both storage and computation occurring within the individual devices in the array. Based on this computing architecture, in-memory computing can be implemented without requiring extensive data movement. Furthermore, multiplication-accumulation is a core computational task required for running neural networks. Therefore, using the conductance of the memristors in the array to represent weight values, highly energy-efficient neural network operations can be achieved based on this in-memory computing approach. In neural network operations, the conductance values ​​of the memristors in the memristor array represent the synaptic weights in the neural network. Summary of the Invention

[0003] At least one embodiment of this disclosure provides a weight writing method for a neural network applied to a memristor array, comprising: obtaining a target weight value to be written into a target memristor cell in the memristor array and an error range for writing the target weight value, the error range being determined based on a probability distribution corresponding to the target weight value; obtaining a current weight value of the target memristor cell; determining whether the absolute value of the difference between the current weight value and the target weight value is within the error range; and determining that the target weight value has been written into the target memristor cell in response to the absolute value of the difference between the current weight value and the target weight value being within the error range.

[0004] For example, in the weight writing method provided in at least one embodiment of this disclosure, the probability distribution is a Gaussian distribution, the target weight value is the mean of the Gaussian distribution, and the error magnitude is determined based on the standard deviation of the Gaussian distribution.

[0005] For example, in the weight writing method provided in at least one embodiment of this disclosure, the error magnitude is positively correlated with the standard deviation.

[0006] For example, in the weight writing method provided in at least one embodiment of this disclosure, the error magnitude is the product of the standard deviation and the scaling factor, where the scaling factor is greater than 1.

[0007] For example, in the weight writing method provided in at least one embodiment of this disclosure, the target memristor unit includes a first memristor and a second memristor. Obtaining the current weight value of the target memristor unit includes: obtaining the conductance value of the first memristor and the conductance value of the second memristor; calculating the difference between the conductance value of the first memristor and the conductance value of the second memristor, and using the difference as the current weight value.

[0008] For example, in at least one embodiment of the weight writing method provided in this disclosure, the method further includes: in response to the absolute value of the difference between the current weight value and the target weight value not being within the error range, programming the target memristor cell until the absolute value of the difference between the current weight value and the target weight value is within the error range.

[0009] For example, in the weight writing method provided in at least one embodiment of this disclosure, the neural network is a Bayesian neural network, and the weight writing method further includes: training the Bayesian neural network to obtain a training result, the training result including multiple weights in the Bayesian neural network, the multiple weights being mapped to the conductance values ​​of multiple memristor units included in the memristor array, the multiple memristor units including the target memristor unit, and obtaining the target weight value to be written into the target memristor unit in the memristor array, including: determining the target weight value to be written into the target memristor unit from the multiple weights.

[0010] For example, in the weight writing method provided in at least one embodiment of this disclosure, training the Bayesian neural network to obtain a training result includes: obtaining a prior standard deviation applied to the Bayesian neural network; and training the Bayesian neural network based on the prior standard deviation to obtain the training result.

[0011] For example, in the weighted writing method provided in at least one embodiment of this disclosure, the prior standard deviation includes the weighted fluctuation standard deviation of the memristor array based on the conductance value of the memristor.

[0012] For example, in the weight writing method provided in at least one embodiment of this disclosure, the training result is obtained by training the Bayesian neural network based on the prior standard deviation, including: calculating the total loss function of the Bayesian neural network based on the weight fluctuation standard deviation; performing backpropagation on the total loss function to update the current weight values ​​in the Bayesian neural network to obtain object weight values; obtaining the constraint conditions of the object weight values; and constraining the object weight values ​​based on the constraint conditions to obtain the training result of the plurality of weights of the Bayesian neural network.

[0013] For example, in the weight writing method provided in at least one embodiment of this disclosure, the prior standard deviation is determined based on prior knowledge of the learning task targeted by the Bayesian neural network.

[0014] For example, in at least one embodiment of the weight writing method provided in this disclosure, the method further includes: obtaining a deep neural network that performs operations using the memristor array; and, in response to the deep neural network not being the Bayesian neural network, converting the deep neural network into the Bayesian neural network, wherein the network structure of the Bayesian neural network is the same as that of the deep neural network.

[0015] At least one embodiment of this disclosure also provides a weight writing device for a neural network applied to a memristor array, comprising: a first acquisition unit configured to acquire a target weight value to be written into a target memristor cell in the memristor array and an error range for writing the target weight value, the error range being determined based on a probability distribution corresponding to the target weight value; a second acquisition unit configured to acquire a current weight value of the target memristor cell; a judgment unit configured to judge whether the absolute value of the difference between the current weight value and the target weight value is within the error range; and a determination unit configured to determine that the target weight value has been written into the target memristor cell in response to the absolute value of the difference between the current weight value and the target weight value being within the error range.

[0016] At least one embodiment of this disclosure also provides an electronic device including: a processor; a memory storing one or more computer program instructions; wherein the one or more computer program instructions, when executed by the processor, are used to implement the weight writing method provided in any embodiment of this disclosure.

[0017] At least one embodiment of this disclosure also provides a computer-readable storage medium that non-temporarily stores computer-readable instructions, which, when executed by a processor, are used to implement the weight writing method provided in any embodiment of this disclosure. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0019] Figure 1A A schematic structure of a memristor array is shown;

[0020] Figure 1B This is a schematic diagram of a memristor device;

[0021] Figure 1C This is a schematic diagram of another memristor device;

[0022] Figure 1D A schematic diagram showing the mapping of the weight matrix of a Bayesian neural network to a memristor array is shown.

[0023] Figure 2 A schematic flowchart of a weight writing method for a neural network applied to a memristor array, provided in at least one embodiment of the present disclosure, is shown.

[0024] Figure 3A Provided for at least one embodiment of this disclosure Figure 2 Flowchart of the method for step S20;

[0025] Figure 3B A schematic structural diagram of a memristor array provided in at least one embodiment of this disclosure;

[0026] Figure 3C A schematic diagram of another memristor array provided for at least one embodiment of this disclosure;

[0027] Figure 4 A flowchart illustrating a method for training a Bayesian neural network to obtain training results, provided in at least one embodiment of this disclosure, is shown.

[0028] Figure 5 At least one embodiment of the present disclosure is shown. Figure 4 Flowchart of step S402;

[0029] Figure 6 A schematic diagram of another weight writing method provided by at least one embodiment of the present disclosure is shown;

[0030] Figure 7 A flowchart of another weight writing method provided by at least one embodiment of this disclosure is shown;

[0031] Figure 8 A schematic block diagram of a weight writing device for a neural network applied to a memristor array, provided in at least one embodiment of the present disclosure, is shown.

[0032] Figure 9 A schematic block diagram of an electronic device provided for some embodiments of this disclosure;

[0033] Figure 10 A schematic block diagram of another electronic device provided for some embodiments of this disclosure;

[0034] Figure 11 This is a schematic diagram of a storage medium provided for some embodiments of this disclosure. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0036] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0037] To write an externally trained offline neural network model onto a memristor array, a write-verification operation is used to program the memristor cells to the target conductance state within an acceptable error range. Write-verification significantly reduces the deviation between the weights in the neural network and the conductance values ​​written into the memristor, while maintaining almost no change in network performance. For example, a write-verification operation involves programming the memristor (i.e., writing the desired conductance value), reading the memristor's conductance value, comparing the absolute value of the difference between the read memristor conductance value and the weight value to see if it is within the error range. If the absolute value of the difference is within the error range, the write verification passes; if it is not, the write verification operation is repeated until the absolute value of the difference is within the error range. Therefore, the write-verification process is very energy- and time-consuming, requiring numerous on-chip read and write operations. In large-scale memristor array applications that require reprogramming for different tasks, the enormous time and energy consumption is unacceptable. Applications such as mobile edge computing, for example, have high demands for high-speed and efficient write verification schemes. However, a high-speed and efficient solution for writing weights to memristor arrays is still lacking.

[0038] Therefore, at least one embodiment of this disclosure provides a weight writing method for a neural network applied to a memristor array. The weight writing method includes: obtaining a target weight value to be written into a target memristor cell in the memristor array and an error range for writing the target weight value, wherein the error range is determined based on a probability distribution corresponding to the target weight value; obtaining a current weight value of the target memristor cell; determining whether the absolute value of the difference between the current weight value and the target weight value is within the error range; and determining that the target weight value has been written into the target memristor cell in response to the absolute value of the difference between the current weight value and the target weight value being within the error range.

[0039] This weight writing method can set an appropriate error range according to the probability distribution corresponding to the target weight value, thereby appropriately widening the error range of the target weight value, reducing read and programming operations, and thus improving the efficiency of weight writing to the memristor array.

[0040] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0041] Figure 1A A schematic structure of a memristor array is shown, which, for example, consists of multiple memristor cells forming an M-row, N-column array, where M and N are both positive integers. Each memristor cell includes one or more switching elements and one or more memristors. In different examples, the memristor cell can be a 1T1R structure (i.e., one transistor and one memristor) or a 2T2R structure (i.e., two transistors and two memristors). Figure 1A In the middle, WL<1>, WL <2> ......WL <m>These represent the word lines for the first, second, ..., Mth rows, respectively. The control electrode (e.g., the gate of a transistor) of the switching element in the memristor cell circuit of each row is connected to the corresponding word line for that row; BL <1> BL <2> ......BL <n>These represent the bit lines of the first, second, ..., Nth columns, respectively. The memristor in the memristor cell circuit of each column is connected to the corresponding bit line of that column; SL <1> SL <2> ......SL <m>These represent the source lines of the first row, second row, ..., Mth row, respectively. The source of the transistor in each row's memristor cell circuit is connected to the corresponding source line of that row. According to Kirchhoff's laws, by setting the state of the memristor cells (e.g., resistance value) and applying corresponding word line signals and bit line signals to the word line and bit line, the above memristor array can perform multiply-accumulate calculations in parallel.

[0042] Figure 1B This is a schematic diagram of a memristor device, which includes a memristor array and its peripheral driving circuitry. For example, such as... Figure 1B As shown, the memristor device includes a signal acquisition device, a word line driving circuit, a bit line driving circuit, a source line driving circuit, a memristor array, and a data output circuit.

[0043] For example, the signal acquisition device is configured to convert a digital signal into multiple analog signals via a digital-to-analog converter (DAC) for input to multiple column signal inputs of a memristor array.

[0044] For example, a memristor array includes M source lines, M word lines, and N bit lines, as well as multiple memristor cells arranged in M ​​rows and N columns.

[0045] For example, the operation of the memristor array can be achieved through word line driving circuits, bit line driving circuits, and source line driving circuits.

[0046] For example, the word line driver circuit includes multiplexers (Muxes) for switching word line input voltages; the bit line driver circuit includes multiplexers for switching bit line input voltages; and the source line driver circuit also includes multiplexers (Muxes) for switching source line input voltages. For example, the source line driver circuit also includes multiple ADCs for converting analog signals to digital signals. Furthermore, a trans-impedance amplifier (TIA) (not shown in the figure) can be further placed between the Mux and ADC in the source line driver circuit to perform current-to-voltage conversion for ADC processing.

[0047] For example, a memristor array includes an operating mode and a computation mode. When the memristor array is in operating mode, the memristor cells are in an initialization state, and the values ​​of the parameter elements in the parameter matrix can be written into the memristor array. For example, the source line input voltage, bit line input voltage, and word line input voltage of the memristor can be switched to the corresponding preset voltage range through a multiplexer.

[0048] For example, through Figure 1B The control signal WL_sw[1:M] of the multiplexer in the word line driver circuit switches the word line input voltage to the corresponding voltage range. For example, when setting the memristor, the word line input voltage is set to 2V (volts); when resetting the memristor, the word line input voltage is set to 5V. The word line input voltage can be adjusted by... Figure 1B The voltage signal V_WL[1:M] in the middle is obtained.

[0049] For example, through Figure 1B The control signal SL_sw[1:M] of the multiplexer in the source line drive circuit switches the source line input voltage to the corresponding voltage range. For example, when setting the memristor, the source line input voltage is set to 0V; when resetting the memristor, the source line input voltage is set to 2V. The source line input voltage can be adjusted by... Figure 1B The voltage signal V_SL[1:M] is obtained.

[0050] For example, through Figure 1B The control signal BL_sw[1:N] of the multiplexer in the bit line drive circuit switches the bit line input voltage to the corresponding voltage range. For example, when setting the memristor, the bit line input voltage is set to 2V; when resetting the memristor, the bit line input voltage is set to 0V. The bit line input voltage can be... Figure 1B The DAC is obtained.

[0051] For example, when the memristor array is in compute mode, the memristors in the array are in a conductive state that can be used for computation. The bit line input voltage at the column signal input terminal does not change the conductance of the memristors. For example, calculations can be performed by multiplying and adding operations using the memristor array. Figure 1B The control signal WL_sw[1:M] of the multiplexer in the word line driver circuit switches the word line input voltage to the corresponding voltage range. For example, when an enable signal is applied, the word line input voltage of the corresponding row is set to 5V; when no enable signal is applied, the word line input voltage of the corresponding row is set to 0V. For example, the GND signal is turned on. Figure 1B The control signal SL_sw[1:M] of the multiplexer in the source line drive circuit switches the source line input voltage to the corresponding voltage range, for example, setting the source line input voltage to 0V, so that the current signals of multiple row signal output terminals can flow into the data output circuit. Figure 1B The control signal BL_sw[1:N] of the multiplexer in the bit line drive circuit switches the bit line input voltage to the corresponding voltage range, for example, setting the bit line input voltage to 0.1V-0.3V, thereby using the memristor array to perform multiplication and addition operations.

[0052] For example, the data output circuit may include multiple transimpedance amplifiers (TIAs) and ADCs, which can convert the current signals from multiple row signal outputs into voltage signals, and then into digital signals for subsequent processing.

[0053] Figure 1C This is a schematic diagram of another type of memristor device. Figure 1C The memristor device shown is Figure 1B The memristor devices shown have essentially the same structure, also including a memristor array and its peripheral drive circuitry. For example, as... Figure 1C As shown, the memristor device includes a signal acquisition device, a word line driving circuit, a bit line driving circuit, a source line driving circuit, a memristor array, and a data output circuit.

[0054] For example, a memristor array may include M source lines, 2M word lines, and 2N bit lines, as well as multiple memristor cells arranged in an M x N array. For instance, each memristor cell may be a 2T2R structure, which allows for mapping between positive and negative values. The operation of mapping the parameter matrix used for transformation processing to different memristor cells in the memristor array will not be elaborated here. It should be noted that a memristor array may also include M source lines, M word lines, and 2N bit lines, as well as multiple memristor cells arranged in an M x N array.

[0055] The descriptions of the signal acquisition device, control drive circuit, and data output circuit can be found in the previous descriptions and will not be repeated here.

[0056] For example, in some embodiments of this disclosure, memristor arrays are used for weight writing to a Bayesian neural network. A Bayesian neural network is a probabilistic model that places a neural network within a Bayesian framework and can describe complex stochastic patterns. To account for the uncertainty of the weights, it is best to construct a Bayesian model. In a Bayesian model, the weights are not represented by a single value, but by a probability distribution. Given observed data, the distribution of the weights in the Bayesian model is called the posterior distribution. As an analogue to deriving an optimal deterministic model through gradient-based updates, the goal of Bayesian machine learning is to learn an approximation of the posterior distribution.

[0057] Given a dataset D, the training objective of a Bayesian neural network is to optimize the posterior distribution p(w|D) of the weights using Bayes' theorem:

[0058]

[0059] Here, p(w) is the prior weight distribution, p(D|w) = p(y|x,w) is the likelihood corresponding to the Bayesian neural network output, and p(D) is the marginal likelihood, i.e., the evidence. Since the true posterior distribution p(w|D) is difficult to realize, it is not directly computed but is usually approximated using inference methods. For example, variational learning methods are used to approximate the posterior distribution of the Bayesian neural network weights.

[0060] Variational learning seeks parameters θ of the weight distribution q(w|θ) of a Bayesian neural network that minimize the Kullback-Leibler (KL) divergence between this distribution and the true posterior distribution. KL divergence measures how closely the distribution q(w|θ) approximates the true posterior distribution. Also known as relative entropy or information divergence, KL divergence is an asymmetric measure of the difference between two probability distributions. Through mathematical transformation, the objective of minimizing the KL divergence between q(w|θ) and p(w|D) can be expressed as:

[0061]

[0062] During optimization, the backpropagation algorithm can be used to optimize the first two terms: KL(q(w|θ)|p(w)) and E. q(w|θ) [logp(D|w). Where KL(q(w|θ)|p(w)) is called the complexity cost term, E q(e|θ) [logp(D|w) is called the likelihood cost term. For the weights of a Gaussian distribution, θ corresponds to the mean μ and standard deviation σ, then the posterior (w|θ) can be expressed as:]

[0063]

[0064] BNNs derive the posterior weight distribution using prior p(w) and likelihood probability p(D|w). This key feature introduces weight uncertainty into the learning process. Therefore, the learned weight parameters and computation are robust to weight perturbations.

[0065] Figure 1D This illustrates the process of mapping the weight matrix of a neural network to a memristor array. The weight matrix between layers in a Bayesian neural network is implemented using a memristor array. For each weight, N memristors are used to represent the distribution corresponding to that weight, where N is an integer greater than or equal to 1. Here, N memristors can be considered as a single memristor unit. For the random probability distribution corresponding to that weight, N conductance values ​​are calculated, and these N conductance values ​​are mapped to the N memristors. In this way, the weight matrix in the Bayesian neural network is transformed into target conductance values ​​mapped to the crossover sequence of the memristor array.

[0066] like Figure 1D As shown, the left side of the diagram represents a three-layer Bayesian neural network, comprising three interconnected layers of neurons. For example, the input layer consists of a first layer of neurons, the hidden layer consists of a second layer of neurons, and the output layer consists of a third layer of neurons. The input layer receives input data and passes it to the hidden layer, which then performs calculations and transformations on the input data before sending it to the output layer. The output layer then outputs the structure of the Bayesian neural network.

[0067] like Figure 1D As shown, the input layer, hidden layer, and output layer each contain multiple neuron nodes, and the number of neuron nodes in each layer can be set according to different application scenarios. For example, the input layer has 2 neurons (including N1 and N2), the hidden layer has 3 neurons (including N3, N4, and N5), and the output layer has 1 neuron (including N6).

[0068] like Figure 1D As shown, adjacent layers of neurons in a Bayesian neural network are connected by a weight matrix. For example, the weight matrix is ​​composed of... Figure 1D The memristor array on the right side is implemented.

[0069] Figure 1D The structure of the memristor array on the right side of the image is as follows: Figure 1A As shown, the memristor array may include multiple memristors arranged in an array. For example... Figure 1D In the example shown, weights are mapped to the conductance of the memristor array according to a certain rule. The weights connecting the input N1 and the output N3 are composed of three memristors (G... 11 G 12 G 13 This is implemented in the same way, and the other weights in the weight matrix can be implemented similarly. More specifically, source line SL1 corresponds to neuron N3, source line SL2 corresponds to neuron N4, source line SL5 corresponds to neuron N5, bit lines BL1, BL2, and BL3 correspond to neuron N1, and a weight between the input layer and the hidden layer (the weight between neuron N1 and neuron N3) is converted into three target conductance values ​​according to a distribution and mapped to the cross sequence of the memristor array, where the target conductance values ​​are G... 11 G 12 and G 13 It is outlined with a dashed box in the memristor array.

[0070] In other embodiments of this disclosure, the weights in the weight matrix are directly programmed as the conductance of the memristor array, meaning that there is a one-to-one correspondence between the weights in the weight matrix and the memristors in the memristor array. Each weight is implemented using one memristor.

[0071] In other embodiments of this disclosure, the difference in conductance between two memristors can also be used to represent a weight. For example, the difference in conductance between two memristors in the same column and adjacent rows represents a weight. That is, each weight is implemented using two memristors to correspond to that weight.

[0072] Figure 1D An implementation of mapping the weight matrix of a Bayesian neural network to a memristor array has been described, but it is merely exemplary and not intended to limit the scope of this disclosure.

[0073] Figure 2 A schematic flowchart of a weight writing method for a neural network applied to a memristor array, provided in at least one embodiment of the present disclosure, is shown.

[0074] like Figure 2 As shown, the weight writing method includes the following steps S10 to S40.

[0075] Step S10: Obtain the target weight value to be written into the target memristor cell in the memristor array and the error magnitude used to write the target weight value. The error magnitude is determined based on the probability distribution corresponding to the target weight value.

[0076] Step S20: Obtain the current weight value of the target memristor cell.

[0077] Step S30: Determine whether the absolute value of the difference between the current weight value and the target weight value is within the error range.

[0078] Step S40: In response to the absolute value of the difference between the current weight value and the target weight value being within the error range, determine that the target weight value has been written into the target memristor cell.

[0079] The weight writing method proposed in this embodiment sets the error range according to the probability distribution of the target weight value. Compared with the scheme where the error range of all target weight values ​​is the same, this embodiment can appropriately relax the error range of some weights, thereby reducing the time and energy consumed by the write verification operation and improving the writing efficiency of the neural network weights to the memristor array.

[0080] It should be noted that although the above example uses the mapping of weights from a Bayesian neural network to a memristor array, this does not mean that the weight writing method provided in this disclosure is only applicable to Bayesian neural networks. In fact, the weight writing method provided in this disclosure is applicable to any neural network that can utilize a memristor array for neural network computation, such as convolutional neural networks, recurrent neural networks, generative adversarial neural networks, etc.

[0081] For example, a neural network includes a weight matrix, and the weights in the weight matrix need to be deployed in a memristor array to enable fast multiplication and addition operations using the memristor array. For instance, the weights in the weight matrix correspond one-to-one with the memristor cells in the memristor array, with each memristor cell containing N memristors, where N is greater than or equal to 1. The target weight can be any parameter value in the weight matrix.

[0082] For step S10, the target memristor cell can be the memristor cell that corresponds to the target weight value among the multiple memristor cells contained in the memristor array.

[0083] In some embodiments of this disclosure, the target weight values ​​to be written to the target memristor cell can be considered as samples generated from a certain distribution. For example, the weights represented by the memristor cell can be regarded as following a Gaussian distribution N(μ,σ) in a BNN. 2 The uncertain weights are: The average value μ is the target weight value to be written into the memristor cells on the memristor array.

[0084] For example, the target weight value can be determined based on the probability density of the probability distribution at 0. If the probability density of the probability distribution at 0 is large, the target weight value will have a smaller impact on the performance of the neural network, and a larger error margin can be set.

[0085] For example, in this embodiment, the probability distribution is a Gaussian distribution, meaning the target weight value follows a Gaussian distribution, the target weight value is the mean of the Gaussian distribution, and the error magnitude is determined based on the standard deviation of the Gaussian distribution. The larger the standard deviation σ, the larger the acceptable deviation of the weight, and the larger the error magnitude of the write verification can be. This appropriately relaxes the error magnitude of weights with larger acceptable deviations, reducing the read and write operations of weights with larger acceptable deviations, and improving the efficiency of writing weights to the memristor while maintaining the performance of the neural network.

[0086] For example, the error magnitude is positively correlated with the standard deviation. For example, the error magnitude EM i,j The standard deviation σ i,j The product of EM and the proportionality constant k, where k is greater than 1, is EM. i,j =k·σ i,j For example, the scaling factor k is the same for each weight. Those skilled in the art can determine the value of k based on the neural network and the learning task. For instance, for a neural network with high recognition accuracy, the scaling factor k can be larger, and for a neural network with high recognition accuracy, the scaling factor k can be smaller. The performance of the neural network is not sensitive to some degree of deviation in the memristor weights (related to σ), which is guaranteed by the training method of the neural network. In other words, it is not necessary to use a small, uniform error range for verification during the write verification process; therefore, the scaling factor k is greater than 1, thereby appropriately widening the error range and improving write efficiency.

[0087] In embodiments of this disclosure, the probability distribution is not limited to a Gaussian distribution, but may also be a Weibull distribution, etc.

[0088] For step S20, for example, the current weight value of the target memristor cell is read. For example, a voltage signal is applied to the target memristor cell, and the current signal output by the target memristor cell is read, thereby obtaining the current conductance value of the target memristor cell, i.e., the current weight value, using Ohm's law.

[0089] For steps S30 and S40, for example, calculating the current weight value W i,j and target weight value The absolute value of the difference judge Is it smaller than EM? i,j .like Smaller than EM i,j Determine the target weight value It has been written to the target memristor cell.

[0090] like Figure 2 As shown, the weight writing method includes step S50 in addition to steps S10 to S40.

[0091] Step S50: In response to the absolute value of the difference between the current weight value and the target weight value being outside the error range, program the target memristor cell until the absolute value of the difference between the current weight value and the target weight value is within the error range.

[0092] For example, Greater than or equal to EM i,j According to With EM i,j The difference is used to program the target memristor cell until the absolute value of the difference between the current weight and the target weight is within the error range. For example, if the current weight value W... i,j Less than the target weight value A voltage is then applied to the target memristor cell to increase its conductance, thereby increasing the current weight value and reducing the absolute value of the difference between the current weight value and the target weight value.

[0093] Figure 3A Provided for at least one embodiment of this disclosure Figure 2 The method flowchart for step S20 is shown below. In this embodiment, the target memristor unit includes a first memristor and a second memristor.

[0094] Figure 3B This is a schematic structural diagram of a memristor array provided for at least one embodiment of the present disclosure.

[0095] like Figure 3B As shown, memristors 301 and 302 can form a memristor pair, and the conductance of memristor 301 is expressed as G. 11 The conductance of memristor 302 is expressed as G. 12 Since memristor 302 is connected to an inverter, when memristor 301 receives a positive input voltage signal, the inverter can flip the polarity of the input voltage signal, thus allowing memristor 302 to receive a negative input voltage signal. For example, the input voltage signal received by memristor 301 is denoted by v(t), and the input voltage signal received by memristor 302 is denoted by -v(t). Memristors 301 and 302 are connected to two different SLs, and the input voltage signal generates an output current through the memristors. At the end of the SL, the output current through memristor 301 and the output current through memristor 302 are superimposed. Therefore, the result of multiplying and summing the memristors 301 and 302 is v(t)G. 11 +(-v(t))G 12 That is, v(t)(G 11 -G 12 Therefore, the memristor pair consisting of memristor 301 and memristor 302 can correspond to a weight, and the weight value is G. 11 -G 12 By configuring G 11 -G 12 The numerical relationships can realize positive, zero, and negative elements.

[0096] Figure 3C This is a schematic diagram of another memristor array provided for at least one embodiment of the present disclosure.

[0097] like Figure 3C As shown, for example, memristors 301 and 302 can form a memristor pair, and the conductance of memristor 301 is expressed as G. 11 The conductance of memristor 302 is expressed as G. 12 .and Figure 3A The difference is that memristor 302 is not connected to an inverter. Therefore, when memristor 301 receives a positive input voltage signal, memristor 302 also receives a positive input voltage signal. For example, the input voltage signal received by memristor 301 is denoted by v(t), and the input voltage signal received by memristor 302 is also denoted by v(t). Memristors 301 and 302 are connected to two different SLs. At the end of the SLs, the output current through memristor 301 is subtracted from the output current through memristor 302. Therefore, the result of multiplying and accumulating the values ​​of memristors 301 and 302 is v(t)G. 11 -v(t)G 12 That is, v0(t)(G 11 -G 12 Therefore, the memristor pair consisting of memristor 301 and memristor 302 can have a weight, and the weight value is G. 11 -G 12 By configuring G 11 -G 12 The numerical relationships can realize positive, zero, and negative elements.

[0098] like Figure 3A As shown, step S20 includes steps S21 and S22.

[0099] Step S21: Obtain the conductance values ​​of the first memristor and the second memristor.

[0100] Step S22: Calculate the difference between the conductance values ​​of the first memristor and the second memristor, and set the difference as the current weight value.

[0101] For example, the memristor unit corresponding to the target weight value includes memristor 601 and memristor 602 as the first and second memristors, respectively. For example, the conductance values ​​of memristor 601 and memristor 602 are read as G. 11 and G 12 The current weight value is G. 11 -G 12 .

[0102] In some embodiments of this disclosure, the neural network is a Bayesian neural network. Figure 2 The weight writing method shown can also include training a Bayesian neural network to obtain training results, which include multiple weights in the Bayesian neural network. These multiple weights can, for example, be the weight matrix that forms the Bayesian neural network.

[0103] Multiple weights are mapped to the conductance values ​​of multiple memristor cells included in the memristor array, where the multiple memristor cells include a target memristor cell. In this embodiment, Figure 2 Step S10 includes determining the target weight value to be written to the target memristor cell from a plurality of weights. For example, the plurality of weights are sequentially used as target weight values, thereby sequentially writing the plurality of weights into the target memristor cells corresponding one-to-one with the plurality of weights.

[0104] For example, the structure of a Bayesian neural network includes fully connected structures or convolutional neural network structures. Each weight in this Bayesian neural network is a random variable. For example, after the Bayesian neural network is trained, each weight follows a distribution, such as a Gaussian distribution or a Laplace distribution.

[0105] For example, the weight matrix can be obtained by offline training of a Bayesian neural network. The training method for a Bayesian neural network can refer to conventional methods, such as using a central processing unit (CPU), image processing unit (GPU), neural network processing unit (NPU), neural network accelerator, etc., which will not be elaborated here.

[0106] For example, the weights in a Bayesian neural network follow a probability distribution, and the target weight is the mean of that probability distribution. Alternatively, the weights in a Bayesian neural network may follow a Gaussian or normal distribution, and the target weight is the mean of the Gaussian distribution.

[0107] Figure 4 A flowchart illustrating a method for training a Bayesian neural network to obtain training results, provided in at least one embodiment of this disclosure, is shown.

[0108] like Figure 4 As shown, the method may include steps S401 to S402.

[0109] Step S401: Obtain the prior standard deviation applied to the Bayesian neural network.

[0110] Step S402: Based on the prior standard deviation, train the Bayesian neural network to obtain the training result.

[0111] For step S401, the prior standard deviation includes the standard deviation of the weighted fluctuation of the memristor array based on the memristor conductance values. Memristors possess inherent non-ideal characteristics, such as inter-device fluctuations, device conductance hysteresis, and conductance state drift. These inherent non-ideal characteristics can cause drift in the conductance values ​​written to the memristors. The training method proposed in this disclosure integrates the influence of memristor conductance fluctuations into the training of the Bayesian neural network, ensuring that the output of the memristor neural network is robust and reliable even under perturbations of the memristor weights.

[0112] For example, multiple electrical tests are performed on the memristors in the memristor array to obtain multiple test results. Each test result includes the weighted fluctuation value of the memristor. The standard deviation of the multiple test results is the weighted fluctuation standard deviation.

[0113] Figure 5 At least one embodiment of the present disclosure is shown. Figure 4 The flowchart of step S402.

[0114] like Figure 5 As shown, step S402 includes steps S412 to S442.

[0115] Step S412: Calculate the total loss function of the Bayesian neural network based on the standard deviation of weight fluctuations.

[0116] Step S422: Backpropagate the total loss function to update the current weight values ​​in the Bayesian neural network to obtain the object weight values.

[0117] Step S432: Obtain the constraints for the object weight values.

[0118] Step S442: Constrain the object weight values ​​based on the constraints to obtain the training results of multiple weights of the Bayesian neural network.

[0119] For step S412, for example, as described above, the total loss function obtained using variational learning includes a KL loss term and a likelihood loss term. For example, the total loss function is expressed as follows:

[0120] F(D,θ)=KL[q(w|θ)||P(w)]-E q(w|θ) [logP(D|w)],

[0121] Where KL[q(w|θ)||P(w)] is the KL loss term, E q(w|θ) [logP(D|w)] is the likelihood loss term.

[0122] In this example, the weighted standard deviation is used as the prior standard deviation P(w). By substituting the weighted standard deviation into the expression of the loss function above, the total loss function is calculated.

[0123] For step S422, for example, when training the parameters such as the weights of a neural network using gradient descent, backpropagation is needed to calculate the partial derivatives of the loss function with respect to the weights, thereby obtaining the gradient of the memristor conductance state of each weight in the weight matrix used in the Bayesian neural network. Geometrically, the direction of the gradient is where the function increases the fastest, and the opposite direction of the gradient is where the function decreases the fastest, making it easier to find the minimum value.

[0124] For example, for each parameter μ of a Bayesian neural network i ,σ i Each weight w i N(μ) follows a Gaussian distribution i ,σ i 2 Using the total loss function calculated in step S412, backpropagation is performed to calculate each current parameter μ. i , σ i The update amount Δ is used to update each parameter. For example, parameter μ i Updated to μ i +Δ, which is the current weight value μ i Update to object weight value μ i +Δ, parameter σ i Updated to σ i +Δ.

[0125] For steps S432 and S442, the constraints on the object weight values ​​include, for example, the weight window range of the memristor cells. For instance, the weight window range is [-w max ,w max ], to transfer μ from the object parameter i +Δ constraint to [-w max ,w max [Inside. For example, μ] i +Δ<-w max Then μ i +Δ constraint is -w max .

[0126] In this embodiment, since the conductivity window of the memristor is limited, the weights will be truncated within a symmetrical range, namely the aforementioned weight window range [-w]. max ,w max Within this framework, we ensure greater compatibility between the Bayesian neural network and the memristor array, avoiding situations where the mean of the Gaussian distribution is outside the weight window range, making it difficult to map to the memristor cells.

[0127] Constraints on object weight values ​​may include, for example, the standard deviation of a Gaussian distribution. For instance, the standard deviation of a Gaussian distribution in object parameters may be constrained to be greater than or equal to the standard deviation of the fluctuation. For example, the standard deviation of the Gaussian distribution might be σ. i Equal to 0.1, the standard deviation of the fluctuation is σ. read Equals 0.15, σ i <σ read Then the standard deviation σ of the Gaussian distribution is... i The constraint is 0.15, i.e., σ i The value is updated to 0.15.

[0128] return Figure 4 For step S401, the prior standard deviation is determined based on the prior knowledge of the learning task targeted by the Bayesian neural network.

[0129] For example, prior knowledge may include a probability distribution that the learning task follows, or some data information. For instance, by conducting multiple trials on the learning task to obtain the probability distribution, the prior standard deviation of the learning task can be determined. This disclosure does not limit the learning task; for example, the learning task may include speech recognition, text recognition, image recognition, etc. Those skilled in the art can determine the prior standard deviation of the learning task based on experience or related technologies.

[0130] Figure 6 A schematic diagram of another weight writing method provided by at least one embodiment of the present disclosure is shown.

[0131] like Figure 6 As shown, this weight writing method is applied to map the weight matrix of a conventional deep neural network 601 onto a memristor array 603, so that the memristor array 603 can be used to perform operations on the deep neural network 601. First, the deep neural network 601 that performs operations using the memristor array 603 is obtained. After obtaining the deep neural network 601, steps S61 to S63 are executed. The conventional deep neural network 601 can be any neural network other than a Bayesian neural network, such as a convolutional neural network (CNN), a recurrent neural network (RNN), a deep belief network (DBN), a deep autoencoder, and a generative adversarial network (GAN).

[0132] Step S61: In response that the deep neural network 601 is not a Bayesian neural network, the deep neural network 601 is converted into a Bayesian neural network 603, and the network structure of the Bayesian neural network 603 is the same as that of the deep neural network 601.

[0133] For example, if a deep neural network 601 is a 3-layer convolutional neural network, then this 3-layer convolutional neural network can be converted into a 3-layer Bayesian neural network 603. The network structure of the Bayesian neural network 603 is the same as that of the deep neural network 601; for example, it can mean that the neural network contains the same number of layers, and each layer has the same function. For instance, if a 3-layer convolutional neural network includes an input layer, hidden layers, and an output layer, then a 3-layer Bayesian neural network also includes an input layer, hidden layers, and an output layer.

[0134] Traditional neural networks can be converted into Bayesian neural networks using calculation methods based on relevant data (e.g., probability weighting), which will not be elaborated here.

[0135] Step S62: Obtain the error magnitude of each weight in the Bayesian neural network 603.

[0136] For example, it can be used Figure 2 The method in step S10 obtains the error magnitude of each weight in the Bayesian neural network.

[0137] Step S63: Map each weight to the memristor array 803 according to the error magnitude of each weight.

[0138] For example, a write check operation is performed for each weight until the difference between the conductance value read from the memristor corresponding to that weight and the weight itself is within the error range of that weight.

[0139] Figure 7 A flowchart of another weight writing method provided by at least one embodiment of this disclosure is shown.

[0140] like Figure 7 As shown, the weight writing method includes steps S701 to S705.

[0141] Step S701: Obtain the target weights in the neural network and the target weight Error range EM i,j .

[0142] Step S702: Obtain the conductance value of each memristor in the memristor unit. For example, if the memristor unit includes a first memristor and a second memristor, then obtain the conductance value G of the first memristor. + ij The conductance G of the second memristor - ij .

[0143] Step S703: Calculate the current weight value W of the memristor cell. i,j For example, calculating the difference G between the conductance values ​​of the first and second memristors. + ij -G - ij As the current weight value W i,j .

[0144] Step S704: Is it less than the preset threshold EM? i,j .like Then, to obtain the next target weight, the next target weight is written to the memristor array according to the weight writing method described above. If Then return to step S705.

[0145] Step S705: Continue programming the memristor cell corresponding to the current target weight. After performing the programming operation, return to step S702.

[0146] Figure 8 A schematic block diagram of a weight writing device 800 for a neural network applied to a memristor array, provided in at least one embodiment of the present disclosure, is shown. This weight writing device 800 can be used to perform... Figure 2 The weight writing method is shown.

[0147] like Figure 8 As shown, the weight writing device 800 includes a first acquisition unit 801, a second acquisition unit 802, a judgment unit 803, and a determination unit 803.

[0148] The first acquisition unit 801 is configured to acquire the target weight value to be written into the target memristor cell in the memristor array and the error magnitude used to write the target weight value, wherein the error magnitude is determined based on the probability distribution corresponding to the target weight value. The first acquisition unit 801, for example, performs... Figure 2 Step S10.

[0149] The second acquisition unit 802 is configured to acquire the current weight value of the target memristor cell. The second acquisition unit 802, for example, performs... Figure 2 Step S20.

[0150] The judgment unit 803 is configured to determine whether the absolute value of the difference between the current weight value and the target weight value is within the error range. For example, the judgment unit 803 executes... Figure 2 Step S30.

[0151] Determination unit 804 is configured to determine that the target weight value has been written to the target memristor cell in response to the absolute value of the difference between the current weight value and the target weight value being within the error range. Determination unit 804, for example, performs... Figure 2 Step S40.

[0152] The technical effects of the aforementioned weight writing device and Figure 2 The weight writing method shown achieves the same technical effect, so it will not be described in detail here.

[0153] For example, the first acquisition unit 801, the second acquisition unit 802, the judgment unit 803, and the determination unit 803 can be hardware, software, firmware, or any feasible combination thereof. For example, the first acquisition unit 801, the second acquisition unit 802, the judgment unit 803, and the determination unit 803 can be dedicated or general-purpose circuits, chips, or devices, or they can be a combination of a processor and a memory. The embodiments of this disclosure do not limit the specific implementation of the above-mentioned units.

[0154] It should be noted that in the embodiments of this disclosure, each unit of the weight writing device 800 corresponds to each step of the aforementioned weight writing method. For the specific functions of the weight writing device 800, please refer to the relevant description of the weight writing method, which will not be repeated here. Figure 8 The components and structure of the weight writing device 800 shown are merely exemplary and not limiting. The weight writing device 800 may also include other components and structures as needed.

[0155] At least one embodiment of this disclosure also provides an electronic device including a processor and a memory, the memory storing one or more computer program instructions. When executed by the processor, the one or more computer program instructions are used to implement the weight writing method described above. This electronic device can improve the efficiency of weight writing.

[0156] Figure 9 This is a schematic block diagram of an electronic device provided for some embodiments of this disclosure. For example... Figure 9 As shown, the electronic device 900 includes a processor 910 and a memory 920. The memory 920 stores non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 910 executes the non-transitory computer-readable instructions, which, when executed by the processor 910, can perform one or more steps in the weighted writing method described above. The memory 920 and the processor 910 can be interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0157] For example, processor 910 may be a central processing unit (CPU), a graphics processing unit (GPU), or other form of processing unit with data processing and / or program execution capabilities. For example, the central processing unit (CPU) may be an X106 or ARM architecture. Processor 910 may be a general-purpose processor or a special-purpose processor, capable of controlling other components in electronic device 900 to perform desired functions.

[0158] For example, memory 920 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and processor 910 may run one or more computer program modules to implement various functions of electronic device 900. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0159] It should be noted that, in the embodiments of this disclosure, the specific functions and technical effects of the electronic device 900 can be referred to the description of the weight writing method above, and will not be repeated here.

[0160] Figure 10 This is a schematic block diagram of another electronic device provided in some embodiments of this disclosure. The electronic device 1000 is, for example, suitable for implementing the weight writing method provided in the embodiments of this disclosure. For example, the electronic device 1000 may be a terminal device, etc. It should be noted that... Figure 10 The illustrated electronic device 1000 is merely an example and does not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0161] like Figure 10 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 1010, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1020 or a program loaded from a storage device 1080 into a random access memory (RAM) 1030. The RAM 1030 also stores various programs and data required for the operation of the electronic device 1000. The processing device 1010, ROM 1020, and RAM 1030 are interconnected via a bus 1040. An input / output (I / O) interface 1050 is also connected to the bus 1040.

[0162] Typically, the following devices can be connected to the I / O interface 1050: input devices 1060 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1070 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1080 including, for example, magnetic tape, hard disk, etc.; and communication devices 1090. Communication device 1090 allows electronic device 1000 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 10 An electronic device 1000 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 1000 may alternatively implement or have more or fewer devices.

[0163] For example, according to embodiments of this disclosure, the weight writing method described above can be implemented as a computer software program. For instance, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the weight writing method described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1090, or installed from a storage device 1080, or installed from a ROM 1020. When the computer program is executed by the processing device 1010, the functions defined in the weight writing method provided by embodiments of this disclosure can be implemented.

[0164] At least one embodiment of this disclosure also provides a computer-readable storage medium for storing non-transitory computer-readable instructions that, when executed by a computer, can implement the weight writing method described above. Using this computer-readable storage medium can improve the efficiency of weight writing.

[0165] Figure 11 This is a schematic diagram of a storage medium provided for some embodiments of this disclosure. For example... Figure 11 As shown, storage medium 1100 is used to store non-transitory computer-readable instructions 1110. For example, when the non-transitory computer-readable instructions 1110 are executed by a computer, one or more steps in the weighted writing method described above can be performed.

[0166] For example, the storage medium 1100 can be used in the aforementioned electronic device 1000. For example, the storage medium 1100 can be... Figure 9 The memory 920 in the illustrated electronic device 900. For example, a description of the storage medium 1100 can be found here. Figure 9 The corresponding description of the memory 920 in the illustrated electronic device 900 will not be repeated here.

[0167] The following points need to be explained:

[0168] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0169] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0170] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.< / m> < / n> < / m>

Claims

1. A method for writing weights to a neural network applied to a memristor array, comprising: Obtain the target weight value to be written to the target memristor cell in the memristor array and the error magnitude for writing the target weight value, wherein the error magnitude is determined based on the probability distribution corresponding to the target weight value; Obtain the current weight value of the target memristor cell; Determine whether the absolute value of the difference between the current weight value and the target weight value is within the error range; In response to the absolute value of the difference between the current weight value and the target weight value being within the error range, it is determined that the target weight value has been written into the target memristor cell; The target memristor unit includes a first memristor and a second memristor. The process of obtaining the current weight value of the target memristor cell includes: Obtain the conductance values ​​of the first memristor and the second memristor; Calculate the difference between the conductance values ​​of the first memristor and the second memristor, and use the difference as the current weight value; The neural network in question is a Bayesian neural network. The weight writing method further includes: The Bayesian neural network is trained to obtain a training result, wherein the training result includes multiple weights in the Bayesian neural network. Wherein, the plurality of weights are mapped to the conductance values ​​of the plurality of memristor cells included in the memristor array, and the plurality of memristor cells include the target memristor cell. The process of obtaining the target weight value written into the target memristor cell in the memristor array includes: The target weight value to be written into the target memristor cell is determined from the plurality of weights.

2. The weight writing method according to claim 1, wherein, The probability distribution is a Gaussian distribution, the target weight value is the mean of the Gaussian distribution, and the error magnitude is determined based on the standard deviation of the Gaussian distribution.

3. The weight writing method according to claim 2, wherein, The error magnitude is positively correlated with the standard deviation.

4. The weight writing method according to claim 3, wherein, The error magnitude is the product of the standard deviation and the proportionality coefficient, where the proportionality coefficient is greater than 1.

5. The weight writing method according to claim 1 further includes: In response to the absolute value of the difference between the current weight value and the target weight value being outside the error range, the target memristor cell is programmed until the absolute value of the difference between the current weight value and the target weight value is within the error range.

6. The weight writing method according to claim 1, wherein, The training results obtained by training the Bayesian neural network include: Obtain the prior standard deviation applied to the Bayesian neural network; and The training result is obtained by training the Bayesian neural network based on the prior standard deviation.

7. The weight writing method according to claim 6, wherein, The prior standard deviation includes the weighted fluctuation standard deviation of the memristor array based on the conductance value of the memristor.

8. The weight writing method according to claim 7, wherein, The training result is obtained by training the Bayesian neural network based on the prior standard deviation, including: The total loss function of the Bayesian neural network is calculated based on the standard deviation of the weight fluctuation. The total loss function is backpropagated to update the current weight values ​​in the Bayesian neural network to obtain the object weight values; The constraints for obtaining the weight values ​​of the object; and The object weight values ​​are constrained based on the constraints to obtain the training results of the multiple weights of the Bayesian neural network.

9. The weight writing method according to claim 6, wherein, The prior standard deviation is determined based on prior knowledge of the learning task for which the Bayesian neural network is targeting.

10. The weight writing method according to claim 1, further comprising: Obtain a deep neural network that performs operations using the memristor array; In response to the fact that the deep neural network is not the Bayesian neural network, the deep neural network is converted into the Bayesian neural network, wherein the network structure of the Bayesian neural network is the same as that of the deep neural network.

11. A weight writing device for a neural network applied to a memristor array, comprising: The first acquisition unit is configured to acquire a target weight value to be written into a target memristor cell in the memristor array and an error magnitude for writing the target weight value, wherein the error magnitude is determined based on the probability distribution corresponding to the target weight value; The second acquisition unit is configured to acquire the current weight value of the target memristor unit; The judgment unit is configured to determine whether the absolute value of the difference between the current weight value and the target weight value is within the error range; The determining unit is configured to determine that the target weight value has been written to the target memristor unit in response to the absolute value of the difference between the current weight value and the target weight value being within the error range; The target memristor unit includes a first memristor and a second memristor. The second acquisition unit is further configured as follows: Obtain the conductance values ​​of the first memristor and the second memristor; Calculate the difference between the conductance values ​​of the first memristor and the second memristor, and use the difference as the current weight value; The neural network in question is a Bayesian neural network. The weight writing device is further configured as follows: The Bayesian neural network is trained to obtain a training result, wherein the training result includes multiple weights in the Bayesian neural network. Wherein, the plurality of weights are mapped to the conductance values ​​of the plurality of memristor cells included in the memristor array, and the plurality of memristor cells include the target memristor cell. The first acquisition unit is further configured as follows: The target weight value to be written into the target memristor cell is determined from the plurality of weights.

12. An electronic device, comprising: processor; Memory, which includes one or more computer program instructions; The one or more computer program instructions are stored in the memory and, when executed by the processor, implement the weight writing method according to any one of claims 1-10.

13. A computer-readable storage medium that non-temporarily stores computer-readable instructions, wherein, The weight writing method according to any one of claims 1-10 is implemented when the computer-readable instructions are executed by a processor.

Images