Weight deployment method and apparatus, electronic device, and storage medium

Abstract

A method, apparatus, electronic device, and storage medium for weight deployment in a neural network based on a memristor array are disclosed. The weight deployment method includes: obtaining target weights in the neural network to be deployed to the memristor array; determining whether the target weights are first-type weights or second-type weights, where first-type weights are more important to the neural network than second-type weights; performing a weight mapping operation on the memristor array: if the target weights are first-type weights, mapping the target weights to the corresponding memristor cells in the memristor array; or, if the target weights are second-type weights, writing the conductance value of the memristor cells corresponding to the target weights in the memristor array as the selected target value. This weight deployment method can improve the efficiency of deploying neural network weights to the memristor array.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a weight deployment method and apparatus for a neural network based on a memristor array, an electronic device, and a storage medium. Background Technology

[0002] A memristor is a non-volatile device whose conductance can be adjusted by applying an external stimulus. According to Kirchhoff's current law and Ohm's law, an array of memristors can perform multiplication-accumulation calculations in parallel, with both storage and computation occurring within the individual devices in the array. Based on this computing architecture, in-memory computing can be implemented without requiring extensive data movement. Furthermore, multiplication-accumulation is a core computational task required for running neural networks. Therefore, using the conductance of the memristors in the array to represent weight values, highly energy-efficient neural network operations can be achieved based on this in-memory computing approach. In neural network operations, the conductance values ​​of the memristors in the memristor array represent the synaptic weights in the neural network. Summary of the Invention

[0003] At least one embodiment of this disclosure provides a weight deployment method for a neural network based on a memristor array, comprising: obtaining target weights in the neural network to be deployed to the memristor array; determining whether the target weights are first-type weights or second-type weights, wherein the first-type weights are more important to the neural network than the second-type weights; performing a weight mapping operation on the memristor array, wherein if the target weights are first-type weights, the target weights are mapped to memristor cells in the memristor array corresponding to the target weights, or, if the target weights are second-type weights, the conductance value of the memristor cells in the memristor array corresponding to the target weights is written as a selected target value.

[0004] For example, in the weight deployment method provided in at least one embodiment of this disclosure, determining whether the target weight is a first type weight or a second type weight includes: determining whether the probability density of the target weight at a selected reference value is greater than a probability density threshold; in response to the probability density of the target weight at the selected reference value being greater than the probability density threshold, the target weight is a second type weight; or in response to the probability density of the target weight at the selected reference value being less than or equal to the probability density threshold, the target weight is a first type weight.

[0005] For example, in the weight deployment method provided in at least one embodiment of this disclosure, the neural network is a Bayesian neural network, the weights in the Bayesian neural network follow a probability distribution, the target weight is the mean of the probability distribution, and determining whether the target weight is a first type weight or a second type weight includes: determining the signal-to-noise ratio (SNR) of the probability distribution at a selected reference value based on the mean and standard deviation of the probability distribution; and determining whether the SNR is greater than a SNR threshold; in response to the SNR being greater than the SNR threshold, the target weight is a first type weight; or in response to the SNR being less than or equal to the SNR threshold, the target weight is a second type weight.

[0006] For example, in the weight deployment method provided in at least one embodiment of this disclosure, the probability distribution is a Gaussian distribution.

[0007] For example, in at least one embodiment of the weight deployment method provided in this disclosure, the method further includes: performing an initialization operation on the memristor array to initialize the conductance values ​​of the memristors in the memristor array.

[0008] For example, in the weight deployment method provided in at least one embodiment of this disclosure, the neural network includes multiple weights, the multiple weights including the target weight, and the weight mapping operation performed on the memristor array includes: if the target weight is a second type weight, then directly obtain the next weight among the multiple weights and update the target weight to the next weight; if the target weight is a first type weight, then map the target weight to the memristor cell in the memristor array corresponding to the target weight, then obtain the next weight among the multiple weights and update the target weight to the next weight.

[0009] For example, in a weight deployment method provided in at least one embodiment of this disclosure, a memristor array includes multiple rows and columns of memristor units, each memristor unit including a pair of first and second memristors, the difference in conductance values ​​of the first and second memristors representing the target weight, and if the target weight is a first type of weight, mapping the target weight to the memristor unit in the memristor array corresponding to the target weight includes: if the target weight is the first type of weight, obtaining a current weight value formed by the first and second memristors included in the memristor unit corresponding to the target weight; determining whether the deviation between the current weight value and the target weight is within the error range; and in response to the deviation between the current weight value and the target weight not being within the error range, programming the conductance values ​​of the first and second memristors to reduce the deviation to within the error range.

[0010] For example, in the weight deployment method provided in at least one embodiment of this disclosure, mapping the target weight to the memristor cell in the memristor array corresponding to the target weight further includes: in response to the deviation between the current weight value and the target weight being within the error range, skipping the target weight, obtaining the next weight among the plurality of weights, and updating the target weight to the next weight.

[0011] For example, in the weight deployment method provided in at least one embodiment of this disclosure, the selected target value is 0.

[0012] At least one embodiment of this disclosure also provides a weight deployment apparatus for a neural network based on a memristor array, comprising: an acquisition unit configured to acquire target weights in the neural network to be deployed to the memristor array; a determination unit configured to determine whether the target weights are first-type weights or second-type weights, wherein the first-type weights are more important to the neural network than the second-type weights; and a mapping unit configured to perform a weight mapping operation on the memristor array, wherein if the target weights are first-type weights, the target weights are mapped to memristor cells in the memristor array corresponding to the target weights, or if the target weights are second-type weights, the conductance value of the memristor cells in the memristor array corresponding to the target weights is written as a selected target value.

[0013] At least one embodiment of this disclosure also provides an electronic device including: a processor; a memory storing one or more computer program instructions; wherein the one or more computer program instructions, when executed by the processor, are used to implement the weight deployment method provided in any embodiment of this disclosure.

[0014] At least one embodiment of this disclosure also provides a computer-readable storage medium that non-temporarily stores computer-readable instructions, which, when executed by a processor, are used to implement the weight deployment method provided in any embodiment of this disclosure. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0016] Figure 1A A schematic structure of a memristor array is shown;

[0017] Figure 1B This is a schematic diagram of a memristor device;

[0018] Figure 1CThis is a schematic diagram of another type of memristor device;

[0019] Figure 1D A schematic diagram showing the mapping of the weight matrix of a Bayesian neural network to a memristor array is shown.

[0020] Figure 2 A schematic flowchart of a weight deployment method for a neural network based on a memristor array, provided in at least one embodiment of this disclosure, is shown.

[0021] Figure 3 A flowchart of the method for step S20 provided in at least one embodiment of this disclosure is shown;

[0022] Figure 4 A flowchart of another step S20 provided in at least one embodiment of this disclosure is shown;

[0023] Figure 5 A flowchart of another step S30 provided in at least one embodiment of this disclosure is shown;

[0024] Figure 6A A schematic structural diagram of a memristor array provided in at least one embodiment of this disclosure;

[0025] Figure 6B A schematic diagram of another memristor array provided for at least one embodiment of this disclosure;

[0026] Figure 7 A flowchart of another weight deployment method provided by at least one embodiment of this disclosure is shown;

[0027] Figure 8 A schematic diagram of another weight deployment method provided by at least one embodiment of the present disclosure is shown;

[0028] Figure 9 A schematic block diagram of a weight deployment device for a neural network based on a memristor array, provided in at least one embodiment of the present disclosure, is shown.

[0029] Figure 10 A schematic block diagram of an electronic device provided for some embodiments of this disclosure;

[0030] Figure 11 A schematic block diagram of another electronic device provided for some embodiments of this disclosure;

[0031] Figure 12 This is a schematic diagram of a storage medium provided for some embodiments of this disclosure. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0033] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0034] To write an externally trained offline neural network model onto a memristor array, a write-verification operation is used to program the memristor cells to the target conductance state within an acceptable error range. Write-verification significantly reduces the deviation between the weights in the neural network and the conductance values ​​written into the memristor, while maintaining almost no change in network performance. For example, a write-verification operation involves programming the memristor (i.e., writing the desired conductance value), reading the memristor's conductance value, comparing the difference between the read conductance value and the weight value to a preset range. If the difference is within the preset range, the write-verification passes; otherwise, it continues until the difference is within the preset range. Therefore, the write-verification process is very energy- and time-consuming, requiring numerous on-chip read and write operations. In large-scale memristor array applications that require reprogramming for different tasks, this significant time and energy consumption is unacceptable. Applications such as mobile edge computing have high requirements for high-speed and efficient write verification schemes. However, a high-speed and efficient solution for writing weights to memristor arrays is still lacking.

[0035] Therefore, at least one embodiment of this disclosure provides a weight deployment method for a neural network based on a memristor array. The weight deployment method includes: obtaining target weights in the neural network to be deployed to the memristor array; determining whether the target weights are first-type weights or second-type weights, wherein the first-type weights are more important to the neural network than the second-type weights; and performing a weight mapping operation on the memristor array. If the target weight is a first-type weight, the target weight is mapped to a memristor cell in the memristor array corresponding to the target weight; or, if the target weight is a second-type weight, the conductance value of the memristor cell in the memristor array corresponding to the target weight is written as the selected target value.

[0036] This weight deployment method can select and write only the target weights with higher importance, thereby reducing the number of weights written to the memristor array while ensuring the performance of the neural network. This reduces the time and energy consumed by the write verification method, thus improving the efficiency of weight deployment.

[0037] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0038] Figure 1A A schematic structure of a memristor array is shown, which, for example, consists of multiple memristor cells forming an M-row, N-column array, where M and N are both positive integers. Each memristor cell includes one or more switching elements and one or more memristors. In different examples, the memristor cell can be a 1T1R structure (i.e., one transistor and one memristor) or a 2T2R structure (i.e., two transistors and two memristors). Figure 1A In the middle, WL <1> WL <2> ...WL <m>These represent the word lines for the first, second, ..., Mth rows, respectively. The control electrode (e.g., the gate of a transistor) of the switching element in the memristor cell circuit of each row is connected to the corresponding word line for that row; BL <1> BL <2> BL <n>These represent the bit lines of the first, second, ..., Nth columns, respectively. The memristor in the memristor cell circuit of each column is connected to the corresponding bit line of that column; SL <1> SL <2> ……SL <m>These represent the source lines of the first row, the second row, ..., the Mth row, respectively. The source of the transistor in the memristor unit circuit of each row is connected to the corresponding source line of that row.

[0039] According to Kirchhoff's laws, by setting the state of the memristor cells (e.g., resistance value) and applying corresponding word line signals and bit line signals to the word line and bit line, the memristor array described above can perform multiply-accumulate calculations in parallel.

[0040] Figure 1B This is a schematic diagram of a memristor device, which includes a memristor array and its peripheral driving circuitry. For example, such as... Figure 1B As shown, the memristor device includes a signal acquisition device, a word line driving circuit, a bit line driving circuit, a source line driving circuit, a memristor array, and a data output circuit.

[0041] For example, the signal acquisition device is configured to convert a digital signal into multiple analog signals via a digital-to-analog converter (DAC) for input to multiple column signal inputs of a memristor array.

[0042] For example, a memristor array includes M source lines, M word lines, and N bit lines, as well as multiple memristor cells arranged in M ​​rows and N columns.

[0043] For example, the operation of the memristor array can be achieved through word line driving circuits, bit line driving circuits, and source line driving circuits.

[0044] For example, the word line driver circuit includes multiplexers (Muxes) for switching word line input voltages; the bit line driver circuit includes multiplexers for switching bit line input voltages; and the source line driver circuit also includes multiplexers (Muxes) for switching source line input voltages. For example, the source line driver circuit also includes multiple ADCs for converting analog signals to digital signals. Furthermore, a trans-impedance amplifier (TIA) (not shown in the figure) can be further placed between the Mux and ADC in the source line driver circuit to perform current-to-voltage conversion for ADC processing.

[0045] For example, a memristor array includes an operating mode and a computation mode. When the memristor array is in operating mode, the memristor cells are in an initialization state, and the values ​​of the parameter elements in the parameter matrix can be written into the memristor array. For example, the source line input voltage, bit line input voltage, and word line input voltage of the memristor can be switched to the corresponding preset voltage range through a multiplexer.

[0046] For example, through Figure 1B The control signal WL_sw[1:M] of the multiplexer in the word line driver circuit switches the word line input voltage to the corresponding voltage range. For example, when setting the memristor, the word line input voltage is set to 2V (volts); when resetting the memristor, the word line input voltage is set to 5V. The word line input voltage can be adjusted by... Figure 1B The voltage signal V_WL[1:M] in the middle is obtained.

[0047] For example, through Figure 1B The control signal SL_sw[1:M] of the multiplexer in the source line drive circuit switches the source line input voltage to the corresponding voltage range. For example, when setting the memristor, the source line input voltage is set to 0V; when resetting the memristor, the source line input voltage is set to 2V. The source line input voltage can be adjusted by... Figure 1B The voltage signal V_SL[1:M] is obtained.

[0048] For example, through Figure 1B The control signal BL_sw[1:N] of the multiplexer in the bit line drive circuit switches the bit line input voltage to the corresponding voltage range. For example, when setting the memristor, the bit line input voltage is set to 2V; when resetting the memristor, the bit line input voltage is set to 0V. The bit line input voltage can be... Figure 1B The DAC is obtained.

[0049] For example, when the memristor array is in compute mode, the memristors in the array are in a conductive state that can be used for computation. The bit line input voltage at the column signal input terminal does not change the conductance of the memristors. For example, calculations can be performed by multiplying and adding operations using the memristor array. Figure 1B The control signal WL_sw[1:M] of the multiplexer in the word line driver circuit switches the word line input voltage to the corresponding voltage range. For example, when an enable signal is applied, the word line input voltage of the corresponding row is set to 5V; when no enable signal is applied, the word line input voltage of the corresponding row is set to 0V. For example, the GND signal is turned on. Figure 1B The control signal SL_sw[1:M] of the multiplexer in the source line drive circuit switches the source line input voltage to the corresponding voltage range, for example, setting the source line input voltage to 0V, so that the current signals of multiple row signal output terminals can flow into the data output circuit. Figure 1B The control signal BL_sw[1:N] of the multiplexer in the bit line drive circuit switches the bit line input voltage to the corresponding voltage range, for example, setting the bit line input voltage to 0.1V-0.3V, thereby using the memristor array to perform multiplication and addition operations.

[0050] For example, the data output circuit may include multiple transimpedance amplifiers (TIAs) and ADCs, which can convert the current signals from multiple row signal outputs into voltage signals, and then into digital signals for subsequent processing.

[0051] Figure 1C This is a schematic diagram of another type of memristor device. Figure 1C The memristor device shown is Figure 1B The memristor devices shown have essentially the same structure, also including a memristor array and its peripheral drive circuitry. For example, as... Figure 1C As shown, the memristor device includes a signal acquisition device, a word line driving circuit, a bit line driving circuit, a source line driving circuit, a memristor array, and a data output circuit.

[0052] For example, a memristor array may include M source lines, 2M word lines, and 2N bit lines, as well as multiple memristor cells arranged in an M x N array. For instance, each memristor cell may be a 2T2R structure, which allows for mapping between positive and negative values. The operation of mapping the parameter matrix used for transformation processing to different memristor cells in the memristor array will not be elaborated here. It should be noted that a memristor array may also include M source lines, M word lines, and 2N bit lines, as well as multiple memristor cells arranged in an M x N array.

[0053] The descriptions of the signal acquisition device, control drive circuit, and data output circuit can be found in the previous descriptions and will not be repeated here.

[0054] For example, in some embodiments of this disclosure, memristor arrays are used for weight deployment in a Bayesian neural network. A Bayesian neural network is a probabilistic model that places a neural network within a Bayesian framework and can describe complex stochastic patterns. To account for the uncertainty of the weights, it is best to construct a Bayesian model. In a Bayesian model, the weights are not represented by a single value, but by a probability distribution. Given observed data, the distribution of the weights in the Bayesian model is called the posterior distribution. As an analogue to deriving an optimal deterministic model through gradient-based updates, the goal of Bayesian machine learning is to learn an approximation of the posterior distribution.

[0055] Given a dataset D, the training objective of a Bayesian neural network is to optimize the posterior distribution p(w|D) of the weights using Bayes' theorem:

[0056]

[0057] Here, p(w) is the prior weight distribution, p(D|w) = p(y|x,w) is the likelihood corresponding to the Bayesian neural network output, and p(D) is the marginal likelihood, i.e., the evidence. Since the true posterior distribution p(w|D) is difficult to realize, it is not directly computed but is usually approximated using inference methods. For example, variational learning methods are used to approximate the posterior distribution of the Bayesian neural network weights.

[0058] Variational learning seeks parameters θ of the distribution q(w|θ) of the weights of a Bayesian neural network that minimize the Kullback-Leibler (KL) divergence between this distribution and the true posterior distribution. KL divergence measures how closely the distribution q(w|θ) approximates the true posterior distribution. Also known as relative entropy or information divergence, KL divergence is an asymmetric measure of the difference between two probability distributions. Through mathematical transformation, the objective of minimizing the KL divergence between q(w|θ) and p(w|D) can be expressed as:

[0059]

[0060] During optimization, the backpropagation algorithm can be used to optimize the first two terms: KL(q(w|θ)|p(w)) and E. q(w|θ) [logp(D|w). Where KL(q(w|θ)|p(w)) is called the complexity cost term, E q(w|θ) [logp(D|w) is called the likelihood cost term. For the weights of a Gaussian distribution, θ corresponds to the mean μ and standard deviation σ, then the posterior (w|θ) can be expressed as:]

[0061]

[0062] BNNs derive the posterior weight distribution using prior p(w) and likelihood probability p(D|w). This key feature introduces weight uncertainty into the learning process. Therefore, the learned weight parameters and computation are robust to weight perturbations.

[0063] Figure 1D This illustrates the process of mapping the weight matrix of a neural network to a memristor array. The weight matrix between layers in a Bayesian neural network is implemented using a memristor array. For each weight, N memristors are used to represent the distribution corresponding to that weight, where N is an integer greater than or equal to 1. For example, here, N memristors are used as a single memristor unit. For the random probability distribution corresponding to that weight, N conductance values ​​are calculated, and these N conductance values ​​are mapped to the N memristors. In this way, the weight matrix in the Bayesian neural network is transformed into target conductance values ​​mapped to the crossover sequence of the memristor array.

[0064] like Figure 1D As shown, the left side of the diagram represents a three-layer Bayesian neural network, comprising three interconnected layers of neurons. For example, the input layer consists of a first layer of neurons, the hidden layer consists of a second layer of neurons, and the output layer consists of a third layer of neurons. The input layer receives input data and passes it to the hidden layer, which then performs calculations and transformations on the input data before sending it to the output layer. The output layer then outputs the structure of the Bayesian neural network.

[0065] like Figure 1D As shown, the input layer, hidden layer, and output layer each contain multiple neuron nodes, and the number of neuron nodes in each layer can be set according to different application scenarios. For example, the input layer has 2 neurons (including N1 and N2), the hidden layer has 3 neurons (including N3, N4, and N5), and the output layer has 1 neuron (including N6).

[0066] like Figure 1D As shown, adjacent layers of neurons in a Bayesian neural network are connected by a weight matrix. For example, the weight matrix is ​​composed of... Figure 1D The memristor array on the right side is implemented.

[0067] Figure 1D The structure of the memristor array on the right side of the image is as follows: Figure 1A As shown, the memristor array may include multiple memristors arranged in an array. For example... Figure 1D In the example shown, weights are mapped to the conductance of the memristor array according to a certain rule. The weights connecting the input N1 and the output N3 are composed of three memristors (G... 11 G 12 G 13 This is implemented in the same way, and the other weights in the weight matrix can be implemented similarly. More specifically, source line SL1 corresponds to neuron N3, source line SL2 corresponds to neuron N4, source line SL5 corresponds to neuron N5, bit lines BL1, BL2, and BL3 correspond to neuron N1, and a weight between the input layer and the hidden layer (the weight between neuron N1 and neuron N3) is converted into three target conductance values ​​according to a distribution and mapped to the cross sequence of the memristor array, where the target conductance values ​​are G... 11 G 12 and G 13 It is outlined with a dashed box in the memristor array.

[0068] In other embodiments of this disclosure, the weights in the weight matrix are directly programmed as the conductances of the memristor array, meaning that there is a one-to-one correspondence between the weights in the weight matrix and the memristors in the memristor array. Each weight is implemented using a memristor to correspond to that weight.

[0069] In other embodiments of this disclosure, the difference in conductance between two memristors can also be used to represent a weight. For example, the difference in conductance between two memristors in the same column and adjacent rows represents a weight. That is, each weight is implemented using two memristors to correspond to that weight.

[0070] Figure 1D An implementation of mapping the weight matrix of a Bayesian neural network to a memristor array has been described, but it is merely exemplary and not intended to limit the scope of this disclosure.

[0071] Figure 2 A schematic flowchart of a weight deployment method for a neural network based on a memristor array, provided in at least one embodiment of this disclosure, is shown.

[0072] like Figure 2 As shown, the weight deployment method includes the following steps S10 to S30.

[0073] Step S10: Obtain the target weights in the neural network to be deployed into the memristor array.

[0074] Step S20: Determine whether the target weight is a first-type weight or a second-type weight. First-type weights are more important to the neural network than second-type weights.

[0075] Step S30: Perform a weight mapping operation on the memristor array. If the target weight is a first type of weight, map the target weight to the memristor cell in the memristor array that corresponds to the target weight. Alternatively, if the target weight is a second type of weight, write the conductance value of the memristor cell in the memristor array that corresponds to the target weight as the selected target value.

[0076] The weighting method proposed in the above embodiments of this disclosure maps target weights that have a significant impact on the performance of the neural network to a memristor array, and writes the selected target value to the memristor corresponding to the target weights that have a smaller impact on the performance of the neural network. This reduces the time and energy consumed by write verification operations, thereby improving the deployment efficiency of the neural network weights while ensuring the performance of the neural network.

[0077] For step S10, the neural network can be, for example, a Bayesian neural network, or a convolutional neural network, a recurrent neural network, a generative adversarial neural network, etc. This disclosure uses a Bayesian neural network as an example to illustrate embodiments of the present disclosure; however, this is not intended to limit the embodiments of the present disclosure. The embodiments provided in this disclosure are applicable to any neural network that performs, for example, matrix multiplication and addition operations (matrix-vector multiplication).

[0078] For example, a neural network includes a weight matrix, and the weights in the weight matrix need to be deployed in a memristor array to enable fast matrix multiplication and addition operations using the memristor array. For instance, the weights in the weight matrix correspond one-to-one with the memristor cells in the memristor array, with each memristor cell containing N memristors, where N is greater than or equal to 1. The target weight can be any parameter value in the weight matrix.

[0079] In some embodiments of this disclosure, the target weights to be deployed to the memristor array can be considered as samples generated from a certain distribution. For example, the weights represented by the memristor can be viewed as following a Gaussian distribution N(μ,σ) in a BNN. 2 The uncertain weights are: The average value μ is the target weight that will be transferred to the memristor array.

[0080] For step S20, the performance of the neural network is sensitive to different weights differently. In embodiments of this disclosure, weights that have a significant impact on the performance of the neural network are called critical weights, and weights that have a minor impact on the performance of the neural network are called non-critical weights. For example, in some embodiments of this disclosure, weights that have no or almost no impact on the performance of the neural network even if they are reset to the selected target value (e.g., 0) are called non-critical weights; and weights other than non-critical weights in the neural network are called critical weights.

[0081] Key weights are more important to neural networks than non-key weights. For example, the impact (importance) of a weight on a neural network can be obtained through theoretical calculations or computer simulations and verification. Weights with importance greater than a certain threshold can be called "key weights," while those with less importance are called "non-key weights." For instance, the first type of weight is key weight, and the second type is non-key weight. Importance can also be represented by the signal-to-noise ratio (SNR); if the SNR is greater than a threshold, it is a key weight, and vice versa.

[0082] In some embodiments of this disclosure, for example, those skilled in the art can pre-label the weights in the neural network to distinguish between first-type weights and second-type weights, thereby determining whether the target weight to be deployed in the memristor array is a first-type weight or a second-type weight based on the labeling. Figure 3 Another embodiment of step S20 is shown, please refer to the description below.

[0083] For step S30, the weight mapping operation performed on the memristor array includes, for example, the write verification operation described above. For instance, if the target weight is a first-type weight, the target weight is mapped to the memristor cell in the memristor array corresponding to that target weight according to the write verification operation described above. For instance, if the target weight is a second-type weight, the memristor cell in the memristor array corresponding to the target weight can be directly programmed to a low-conductivity state, i.e., the conductance value of the memristor cell is set to 0. 0 is an example of a selected target value.

[0084] In embodiments of this disclosure, the target value is not limited to "0"; the target value can be any other suitable value that does not affect the performance of the neural network. Setting the target value to 0 is not only easy to implement and reduces write verification operations, but also reduces the impact of memristor conductance offset.

[0085] Figure 3 A flowchart of a method for step S20 provided in at least one embodiment of the present disclosure is shown.

[0086] like Figure 3 As shown, step S20 includes steps S21 to S23.

[0087] Step S21: Determine whether the probability density of the target weight at the selected reference value is greater than the probability density threshold.

[0088] Step S22: In response to the probability density of the target weight at the selected reference value being greater than the probability density threshold, the target weight is a second-type weight.

[0089] Step S23: In response to the probability density of the target weight at the selected reference value being less than or equal to the probability density threshold, the target weight is a first-type weight.

[0090] For step S21, for example, if the target weight follows a certain distribution, the probability density of the target weight at a selected reference value is calculated based on this distribution. The selected reference value is, for example, chosen by someone skilled in the art based on the range of values ​​for the target weight or experience. The selected reference value is the minimum or a smaller value within the range of values ​​for the target weight.

[0091] For example, if the target weight ranges from [0,1], then the selected reference value could be 0. Or, if the target weight ranges from [0,10], then the selected reference value could include both 0 and 1.

[0092] For example, if the selected reference value is 0, calculate the probability density of the target weight at 0, and determine whether the probability density of the target weight at 0 is greater than the threshold.

[0093] In some embodiments of this disclosure, the probability density threshold can be determined based on the actual neural network model and dataset. For example, the probability density threshold can be 0.9.

[0094] For step S22, for example, if the probability density of the target weight at 0 is 0.95, and 0.95 is greater than the probability density threshold of 0.9, then the target weight is a second type weight.

[0095] For step S23, for example, if the probability density of the target weight at 0 is 0.1, and 0.1 is less than the probability density threshold of 0.9, then the target weight is a first type weight.

[0096] In this embodiment, the probability density of the target weight at the selected reference value is used to determine whether the target weight is a first-type weight, which can accurately determine the impact of the target weight on the performance of the neural network.

[0097] In some embodiments of this disclosure, the neural network is a Bayesian neural network. For example, the weight matrix obtained after training the Bayesian neural network is mapped into a memristor array.

[0098] For example, the structure of a Bayesian neural network includes fully connected structures or convolutional neural network structures. Each weight in this Bayesian neural network is a random variable. For example, after the Bayesian neural network is trained, each weight follows a distribution, such as a Gaussian distribution or a Laplace distribution.

[0099] For example, the weight matrix can be obtained by offline training of a Bayesian neural network. The training method for a Bayesian neural network can refer to conventional methods, such as using a central processing unit (CPU), image processing unit (GPU), neural network processing unit (NPU), neural network accelerator, etc., which will not be elaborated here.

[0100] For example, the weights in a Bayesian neural network follow a probability distribution, and the target weight is the mean of that probability distribution. Alternatively, the weights in a Bayesian neural network may follow a Gaussian or normal distribution, and the target weight is the mean of the Gaussian distribution.

[0101] It should be understood that although this disclosure uses a Gaussian distribution of weights in a Bayesian neural network as an example, this is not intended to limit the embodiments of this disclosure. For example, the probability distribution could also be a Weibull distribution, etc.

[0102] Figure 4 A flowchart of another step S20 provided in at least one embodiment of the present disclosure is shown.

[0103] like Figure 4 As shown, step S20 may include steps S201 to S204. In this embodiment, the weights in the Bayesian neural network follow a probability distribution.

[0104] Step S201: Determine the signal-to-noise ratio of the probability distribution at the selected reference value based on the mean and standard deviation of the probability distribution.

[0105] Step S202: Determine whether the signal-to-noise ratio is greater than the signal-to-noise ratio threshold.

[0106] Step S203: In response to the signal-to-noise ratio being greater than the signal-to-noise ratio threshold, the target weight is the first type of weight.

[0107] Step S204: In response to a signal-to-noise ratio less than or equal to a signal-to-noise ratio threshold, the target weight is a second-type weight.

[0108] For example, if the probability distribution is Gaussian and the chosen reference value is 0, then the probability density of the target weight at 0 is equivalent to the signal-to-noise ratio (SNR) at 0. For instance, if the SNR = μ / σ > the SNR threshold... th That is, the signal-to-noise ratio (SNR) of the target weight is greater than the signal-to-noise ratio threshold (SNR). th If the signal-to-noise ratio (SNR) is large, then the probability density of the target weight at 0 is considered sufficiently low, and the target weight is more significant to the neural network; that is, the weight is a type I weight. This method directly uses the SNR to represent the probability density, simplifying the calculation and further improving the efficiency of weight deployment.

[0109] In some embodiments of this disclosure, the signal-to-noise ratio threshold (SNR) th The signal-to-noise ratio (SNR) threshold can be determined based on the actual neural network model and dataset. For example, the SNR threshold could be 0.6.

[0110] In other embodiments of this disclosure, for example, all weights in the neural network are sorted in descending order of signal-to-noise ratio, and the weights ranked before a preset rank are classified as first-type weights. The preset rank can be determined based on the neural network model; for example, if the neural network includes 100 weights, the preset rank could be 90.

[0111] like Figure 2 As shown, in some other embodiments of this disclosure, the weight deployment method may include step S40 in addition to steps S10 to S30. Step S40 is performed, for example, before step S10.

[0112] Step S40: Perform an initialization operation on the memristor array to initialize the conductance values ​​of the memristors in the memristor array. For example, the initialization operation initializes the conductance values ​​of the memristors in the memristor array to selected target values.

[0113] Before performing weight mapping operations on the memristor array, this embodiment first performs an initial operation on the memristor array, so that the conductance values ​​in the memristor array are initialized to the selected target values. This allows the second type of weights to be skipped directly when performing weight mapping operations on the memristor array, further improving the weight deployment efficiency.

[0114] For example, a low conductivity value can be directly written to the memristor array to initialize the memristor array, setting the conductivity value of the memristors in the array to 0.

[0115] In some embodiments of this disclosure, step S30 includes, if the target weight is a second type weight, directly obtaining the next weight among multiple weights and updating the target weight to the next weight. In this embodiment, the second type weight has already been initialized to the selected target value during the initialization operation. Therefore, if the target weight is a second type weight, during the weight mapping operation on the memristor array, it is not necessary to perform a write verification operation on the target weight again; the target weight can be skipped directly, and mapping can be performed on the next target weight to be deployed in the neural network. That is, in this embodiment, it is not necessary to perform a write verification operation on the second type weight again; the second type weight can be skipped directly, and only the first type weight is subjected to a write verification operation, thereby saving weight deployment time.

[0116] If the target weight is a first-type weight, then after mapping the target weight to the corresponding memristor cell in the memristor array, the next weight among multiple weights is obtained, and the target weight is updated to the next weight. For example, the target weight is mapped to the corresponding memristor cell in the memristor array through the write verification operation described above.

[0117] Figure 5 A flowchart of another step S30 provided in at least one embodiment of the present disclosure is shown.

[0118] like Figure 5 As shown, step S30 may include steps S31 to S33.

[0119] Step S31: If the target weight is a first type of weight, obtain the current weight value formed by the first memristor and the second memristor in the memristor unit corresponding to the target weight.

[0120] Step S32: Determine whether the deviation between the current weight value and the target weight is within the error range.

[0121] Step S33: In response to the deviation between the current weight value and the target weight being outside the error range, the conductance values ​​of the first memristor and the second memristor are programmed to reduce the deviation to within the error range.

[0122] In this embodiment, each memristor unit includes a first memristor and a second memristor provided in pairs, and the difference in conductance between the first memristor and the second memristor represents the target weight. A negative target weight can be constructed by using the difference in conductance between the two memristors to represent the target weight.

[0123] The following is through Figure 6A , Figure 6B Let's take a specific example of a memristor array that can achieve negative values.

[0124] Figure 6A This is a schematic structural diagram of a memristor array provided for at least one embodiment of the present disclosure.

[0125] like Figure 6A As shown, memristors 601 and 602 can form a memristor pair, and the conductance of memristor 601 is expressed as G. 11 The conductance of memristor 602 is expressed as G. 12 Since memristor 602 is connected to an inverter, when memristor 601 receives a positive input voltage signal, the inverter can flip the polarity of the input voltage signal, thus allowing memristor 602 to receive a negative input voltage signal. For example, the input voltage signal received by memristor 601 is denoted by v(t), and the input voltage signal received by memristor 602 is denoted by -v(t). Memristors 601 and 602 are connected to two different SLs, and the input voltage signal generates an output current through the memristors. At the end of the SL, the output current through memristor 601 and the output current through memristor 602 are superimposed. Therefore, the result of multiplying and summing the memristors 601 and 602 is v(t)G. 11 +(-v(t))G 12 That is, v(t)(G 11 -G 12 Therefore, the memristor pair consisting of memristor 601 and memristor 602 can correspond to a weight, and the weight value is G. 11 -G 12 By configuring G 11 -G 12 The numerical relationships can realize positive, zero, and negative elements.

[0126] Figure 6B This is a schematic diagram of another memristor array provided for at least one embodiment of the present disclosure.

[0127] like Figure 6B As shown, for example, memristors 601 and 602 can form a memristor pair, and the conductance of memristor 601 is expressed as G. 11 The conductance of memristor 602 is expressed as G. 12 .and Figure 6A The difference is that memristor 602 is not connected to an inverter. Therefore, when memristor 601 receives a positive input voltage signal, memristor 602 also receives a positive input voltage signal. For example, the input voltage signal received by memristor 601 is denoted by v(t), and the input voltage signal received by memristor 602 is also denoted by v(t). Memristors 601 and 602 are connected to two different SLs. At the end of the SLs, the output current through memristor 601 is subtracted from the output current through memristor 602. Therefore, the result of multiplying and summing the values ​​of memristors 601 and 602 is v(t)G. 11 -v(t)G 12 That is, v0(t)(G 11 -G 12 Therefore, the memristor pair consisting of memristor 601 and memristor 602 can have a weight, and the weight value is G. 11 -G 12 By configuring G 11 -G 12 The numerical relationships can realize positive, zero, and negative elements.

[0128] For step S31, for example, the memristor unit corresponding to the target weight includes memristor 601 and memristor 602 as the first and second memristors, respectively. For example, the conductance values ​​of memristor 601 and memristor 602 are read as G. 11 and G 12 The current weight value is G. 11 -G 12 .

[0129] For step S32, the error range can be set in advance by those skilled in the art. For example, determining whether G... 11 -G 12 Whether the absolute value is less than or equal to the preset threshold EM, where EM>0.

[0130] For step S33, for example if G 11 -G 12 If the absolute value of the deviation is greater than a preset threshold EM, the conductance values ​​of the first memristor 601 and the second memristor 602 are programmed to reduce the deviation to within the error range. For example, the conductance value of memristor 602 can be increased, or the conductance value of memristor 601 can be decreased.

[0131] like Figure 5 As shown, in some other embodiments of this disclosure, step S30 may include step S34 in addition to steps S31 to S33.

[0132] Step S34: In response to the deviation between the current weight value and the target weight being within the error range, skip the target weight, obtain the next weight among the multiple weights, and update the target weight to the next weight.

[0133] If the deviation between the current weight value and the target weight is within the error range, it indicates that the target weight has been written into the memristor cell. Directly skip the target weight, obtain the next weight among the multiple weights, and update the target weight to the next weight to map the next weight into the memristor array.

[0134] Figure 7 The flowchart shows another weight deployment method provided by at least one embodiment of the present disclosure.

[0135] As Figure 7 shown, the weight deployment method includes steps S701 to S706.

[0136] Step S701: Obtain the next weight to be deployed in the neural network as the target weight W'.

[0137] Step S702: Determine whether the target weight W' is a first type of weight. If the target weight W' is a first type of weight, execute step S703; if the target weight is not a first type of weight, execute step S701, that is, skip the current target weight W' and directly obtain the next target weight.

[0138] Step S703: Obtain the conductance value of the memristor cell. The memristor cell includes, for example, a first memristor and a second memristor.

[0139] Step S704: Calculate the current weight value W of the memristor cell. For example, calculate the difference between the conductance values of the first memristor and the second memristor as the current weight value.

[0140] Step S705: Is |W - W'| less than the preset threshold EM? That is, determine whether the absolute value of the difference between the current weight value W and the target weight W' is less than the preset threshold EM. If |W - W'| < EM, return to execute step S701 to map the next weight to be deployed. If |W - W'| >= EM, return to execute step S706.

[0141] Step S706: Continue to perform a programming operation on the memristor cell corresponding to the current target weight. After executing the programming operation, then return to execute step S703.

[0142] Figure 8 The schematic diagram shows another weight deployment method provided by at least one embodiment of the present disclosure.

[0143] As Figure 8 As shown, this weight deployment method is applied to map the weight matrix of the Bayesian neural network 802 to the memristor array 801.

[0144] like Figure 8 As shown, the weight deployment method includes steps S81 to S83.

[0145] Step S81: The memristor array provides prior knowledge of the memristor weights as a prior for the Bayesian neural network. The Bayesian neural network is trained offline based on the prior knowledge of the memristor weights to obtain the posterior distribution of the weights of the Bayesian neural network 802.

[0146] For example, first and second prior knowledge of the memristor array based on the inherent non-ideal characteristics of the memristor is obtained; based on the first prior knowledge, the total loss function of the Bayesian neural network is calculated; the total loss function is backpropagated to update the current parameters in the Bayesian neural network to obtain the object parameters; and the object parameters are constrained based on the second prior knowledge to obtain the training results of offline training of the weights of the Bayesian neural network.

[0147] For example, the first prior knowledge includes the standard deviation of weight fluctuations. Based on this first prior knowledge, the total loss function of the Bayesian neural network is calculated, including: calculating the total loss function using variational learning based on the standard deviation of weight fluctuations. Variational learning includes a complexity cost term, with the standard deviation of weight fluctuations serving as the prior standard deviation within that complexity cost term.

[0148] As shown above, the total loss function obtained using variational learning includes a KL loss term and a likelihood loss term. For example, the total loss function can be expressed as follows:

[0149] F(D,θ)=KL[q(w|θ)||P(w)]-E q(w|θ) [log P(D|w)],

[0150] Where KL[q(w|θ)||P(w)] is the KL loss term, E q(w|θ) [log P(D|w)] is the likelihood loss term.

[0151] In this example, the standard deviation of the weighted fluctuation σ proir As the prior standard deviation P(w), the weighted standard deviation of fluctuation σ proir Substituting these values ​​into the expression of the loss function above, we can calculate the total loss function.

[0152] For example, when training parameters such as weights in a neural network using gradient descent, backpropagation is needed to calculate the partial derivatives of the loss function with respect to the weights, thereby obtaining the gradient of the memristor conductance state of each weight in the weight matrix used in the Bayesian neural network. Geometrically, the direction of the gradient is where the function increases the fastest, and the opposite direction of the gradient is where the function decreases the fastest, making it easier to find the minimum value.

[0153] For example, for each parameter μ of a Bayesian neural network i ,σ i Each weight w i Follows a Gaussian distribution Backpropagation is performed using the total loss function calculated in step S102 to calculate each current parameter μ. i ,σ i The update amount Δ is used to update each parameter. For example, parameter μ i Updated to μ i +Δ, parameter σ i Updated to σ i +Δ. The object parameter is the parameter μ of the current parameter. i ,σ i Updated parameter μ i +Δ and σ i +Δ.

[0154] For example, object parameters include the mean of a Gaussian distribution, and the second prior knowledge includes the weight window range of memristor cells in the memristor array.

[0155] Constraining object parameters based on second prior knowledge can include constraining the mean of the Gaussian distribution of the object parameters to a weighted window range. The weighted window range is the range of conductance values ​​of the memristor cells.

[0156] For example, the weight window range is [-w max ,w max ], to take μ from the object parameter i +Δ constraint to [-w max ,w max [Inside. For example, μ] i +Δ<-w max Then μ i +Δ constraint is -w max .

[0157] In this embodiment, since the conductivity window of the memristor is limited, the weights will be truncated within a symmetrical range, namely the aforementioned weight window range [-w]. max ,w max Within this framework, we ensure greater compatibility between the Bayesian neural network and the memristor array, avoiding situations where the mean of the Gaussian distribution is outside the weight window range, making it difficult to map to the memristor cells.

[0158] The object parameters include the standard deviation of the Gaussian distribution, and the second prior knowledge includes the standard deviation of the read fluctuation, which is the error introduced by reading the conductance value of the memristors in the memristor array. Constraining the object parameters based on the second prior knowledge can include constraining the standard deviation of the Gaussian distribution in the object parameters, such that the standard deviation of the Gaussian distribution is greater than or equal to the standard deviation of the read fluctuation.

[0159] Step S82: Select and retain key weights in the offline-trained Bayesian neural network 802. For example, using... Figure 3 or Figure 4 The described method extracts key weights from a Bayesian neural network 802.

[0160] Step S83: Initialize the memristor array, then map the key weights to the memristor array. Similar to... Figure 2 Steps S30 and S40 will not be described in detail.

[0161] Figure 9 A schematic block diagram of a weight deployment device 900 for a neural network based on a memristor array, provided in at least one embodiment of this disclosure, is shown. This weight deployment device 900 can be used to perform... Figure 2 The weight deployment method is shown.

[0162] like Figure 9 As shown, the weight deployment device 900 includes an acquisition unit 901, a determination unit 902, and a mapping unit 903.

[0163] The acquisition unit 901 is configured to acquire the target weights in the neural network to be deployed into the memristor array.

[0164] The determination unit 902 is configured to determine whether the target weight is a first type weight or a second type weight, where the first type weight is more important to the neural network than the second type weight.

[0165] The mapping unit 903 is configured to perform a weight mapping operation on the memristor array. If the target weight is a first type weight, the target weight is mapped to the memristor cell in the memristor array corresponding to the target weight. Alternatively, if the weight to be deployed is a second type weight, the conductance value of the memristor cell in the memristor array corresponding to the target weight is written as the selected target value.

[0166] The technical effects of the above-mentioned weight deployment device and Figure 2 The weight deployment methods shown have the same technical effect, so they will not be described in detail here.

[0167] For example, the acquisition unit 901, the determination unit 902, and the mapping unit 903 can be hardware, software, firmware, or any feasible combination thereof. For example, the acquisition unit 901, the determination unit 902, and the mapping unit 903 can be dedicated or general-purpose circuits, chips, or devices, or they can be a combination of a processor and memory. The embodiments of this disclosure do not limit the specific implementation of the above-mentioned units.

[0168] It should be noted that in the embodiments of this disclosure, each unit of the weight deployment device 900 corresponds to each step of the aforementioned weight deployment method. For the specific functions of the weight deployment device 900, please refer to the relevant description of the weight deployment method, which will not be repeated here. Figure 9 The components and structure of the weight deployment device 900 shown are merely exemplary and not limiting. The weight deployment device 900 may also include other components and structures as needed.

[0169] At least one embodiment of this disclosure also provides an electronic device including a processor and a memory, the memory storing one or more computer program instructions. When executed by the processor, the one or more computer program instructions are used to implement the weight deployment method described above. This electronic device can improve the efficiency of weight deployment.

[0170] Figure 10 This is a schematic block diagram of an electronic device provided for some embodiments of this disclosure. For example... Figure 10 As shown, the electronic device 1000 includes a processor 1010 and a memory 1020. The memory 1020 stores non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 1010 executes the non-transitory computer-readable instructions, which, when executed by the processor 1010, can perform one or more steps in the weight deployment method described above. The memory 1020 and the processor 1010 can be interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0171] For example, processor 1010 may be a central processing unit (CPU), a graphics processing unit (GPU), or other form of processing unit with data processing and / or program execution capabilities. For example, the central processing unit (CPU) may be an X106 or ARM architecture, etc. Processor 1010 may be a general-purpose processor or a special-purpose processor, which can control other components in electronic device 1000 to perform desired functions.

[0172] For example, memory 1020 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and processor 1010 may run one or more computer program modules to implement various functions of electronic device 1000. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0173] It should be noted that, in the embodiments of this disclosure, the specific functions and technical effects of the electronic device 1000 can be referred to the description of the weight deployment method above, and will not be repeated here.

[0174] Figure 11 This is a schematic block diagram of another electronic device provided in some embodiments of this disclosure. The electronic device 1100 is, for example, suitable for implementing the weight deployment method provided in embodiments of this disclosure. For example, the electronic device 1100 may be a terminal device, etc. It should be noted that... Figure 11 The illustrated electronic device 1100 is merely an example and does not impose any limitation on the functionality and scope of use of the embodiments of this disclosure.

[0175] like Figure 11 As shown, electronic device 1100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1110, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1120 or a program loaded from storage device 1180 into random access memory (RAM) 1130. RAM 1130 also stores various programs and data required for the operation of electronic device 1100. Processing device 1110, ROM 1120, and RAM 1130 are interconnected via bus 1140. Input / output (I / O) interface 1150 is also connected to bus 1140.

[0176] Typically, the following devices can be connected to I / O interface 1150: input devices 1160 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1170 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1180 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1190. Communication device 1190 allows electronic device 1100 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 11 An electronic device 1100 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 1100 may alternatively implement or have more or fewer devices.

[0177] For example, according to embodiments of this disclosure, the weight deployment method described above can be implemented as a computer software program. For instance, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the weight deployment method described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1190, or installed from a storage device 1180, or installed from a ROM 1120. When the computer program is executed by the processing device 1110, the functions defined in the weight deployment method provided by embodiments of this disclosure can be implemented.

[0178] At least one embodiment of this disclosure also provides a computer-readable storage medium for storing non-transitory computer-readable instructions that, when executed by a computer, can implement the weight deployment method described above. Using this computer-readable storage medium can improve the efficiency of weight deployment.

[0179] Figure 12 This is a schematic diagram of a storage medium provided for some embodiments of this disclosure. For example... Figure 12 As shown, storage medium 1200 is used to store non-transitory computer-readable instructions 1210. For example, when the non-transitory computer-readable instructions 1210 are executed by a computer, one or more steps in the weight deployment method described above can be performed.

[0180] For example, the storage medium 1200 can be used in the aforementioned electronic device 1000. For example, the storage medium 1200 can be... Figure 10 The memory 1020 in the illustrated electronic device 1000. For example, a description of the storage medium 1200 can be found here. Figure 10 The corresponding description of the memory 1020 in the illustrated electronic device 1000 will not be repeated here.

[0181] The following points need to be explained:

[0182] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0183] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0184] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.< / m> < / n> < / m>

Claims

1. A weight deployment method for a neural network based on a memristor array, comprising: Obtain the target weights in the neural network to be deployed to the memristor array; Determine whether the target weight is a first type of weight or a second type of weight, wherein the first type of weight is more important to the neural network than the second type of weight; and A weight mapping operation is performed on the memristor array, wherein if the target weight is a first type of weight, the target weight is mapped to the memristor cell in the memristor array corresponding to the target weight; or if the target weight is a second type of weight, the conductance value of the memristor cell in the memristor array corresponding to the target weight is written as the selected target value. Wherein, if the target weight is the first type of weight, mapping the target weight to the memristor cell in the memristor array corresponding to the target weight includes: If the target weight is the first type of weight, obtain the current weight value formed by the first memristor and the second memristor included in the memristor unit corresponding to the target weight; Determine whether the deviation between the current weight value and the target weight is within the error range; In response to the deviation between the current weight value and the target weight being outside the error range, the conductance values ​​of the first memristor and the second memristor are programmed to reduce the deviation to within the error range.

2. The weight deployment method according to claim 1, wherein, Determining whether the target weight is a first-type weight or a second-type weight includes: Determine whether the probability density of the target weight at the selected reference value is greater than the probability density threshold; In response to the probability density of the target weight at the selected reference value being greater than the probability density threshold, the target weight is the second type of weight; or The target weight is the first type of weight when the probability density of the target weight at the selected reference value is less than or equal to the probability density threshold.

3. The weight deployment method according to claim 1, wherein, The neural network is a Bayesian neural network, and the weights in the Bayesian neural network follow a probability distribution. The target weight is the mean of the probability distribution. Determining whether the target weight is a first-type weight or a second-type weight includes: Based on the mean and standard deviation of the probability distribution, determine the signal-to-noise ratio of the probability distribution at a selected reference value; and Determine whether the signal-to-noise ratio is greater than the signal-to-noise ratio threshold; In response to the signal-to-noise ratio being greater than the signal-to-noise ratio threshold, the target weight is the first type of weight; or In response to the signal-to-noise ratio being less than or equal to the signal-to-noise ratio threshold, the target weight is the second type of weight.

4. The weight deployment method according to claim 3, wherein, The probability distribution is a Gaussian distribution.

5. The weight deployment method according to claim 1 further includes: An initialization operation is performed on the memristor array to initialize the conductance values ​​of the memristors in the memristor array.

6. The weight deployment method according to claim 5, wherein, The neural network includes multiple weights, including the target weight. Performing the weight mapping operation on the memristor array includes: If the target weight is the second type of weight, then the next weight among the multiple weights is directly obtained, and the target weight is updated to the next weight; If the target weight is the first type of weight, then after mapping the target weight to the memristor cell in the memristor array corresponding to the target weight, the next weight among the multiple weights is obtained, and the target weight is updated to the next weight.

7. The weight deployment method according to claim 6, wherein, The memristor array includes multiple rows and columns of memristor cells, each memristor cell including a first memristor and a second memristor provided in pairs, the difference in conductance values ​​of the first memristor and the second memristor representing the current weight value.

8. The weight deployment method according to claim 1, wherein, Mapping the target weight to the memristor cell in the memristor array corresponding to the target weight further includes: If the deviation between the current weight value and the target weight is within the error range, the target weight is skipped, the next weight among the plurality of weights is obtained, and the target weight is updated to the next weight.

9. The weight deployment method according to any one of claims 1-8, wherein, The selected target value is 0.

10. A weight deployment device for a neural network based on a memristor array, comprising: The acquisition unit is configured to acquire the target weights in the neural network to be deployed to the memristor array; The determining unit is configured to determine whether the target weight is a first type of weight or a second type of weight, wherein the first type of weight is more important to the neural network than the second type of weight. The mapping unit is configured to perform a weight mapping operation on the memristor array, wherein if the target weight is a first type of weight, the target weight is mapped to the memristor cell in the memristor array corresponding to the target weight, or if the target weight is a second type of weight, the conductance value of the memristor cell in the memristor array corresponding to the target weight is written as the selected target value. Specifically, when performing the step of mapping the target weight to the memristor cell in the memristor array corresponding to the target weight if the target weight is the first type of weight, the mapping unit is further configured to: If the target weight is the first type of weight, obtain the current weight value formed by the first memristor and the second memristor included in the memristor unit corresponding to the target weight; Determine whether the deviation between the current weight value and the target weight is within the error range; In response to the deviation between the current weight value and the target weight being outside the error range, the conductance values ​​of the first memristor and the second memristor are programmed to reduce the deviation to within the error range.

11. An electronic device, comprising: processor; Memory, which stores one or more computer program instructions; Wherein, the one or more computer program instructions, when executed by the processor, are used to implement the weight deployment method according to any one of claims 1-9.

12. A computer-readable storage medium that non-transitoryly stores computer-readable instructions, wherein, When the computer-readable instructions are executed by a processor, they are used to implement the weight deployment method according to any one of claims 1-9.

Images