Deep neural network training

Abstract

In a method of training a deep neural network, a processor initializes an element of an A matrix. The element can include a resistive processing unit. The processor determines an incremental weight update by updating the element with a chopper value multiplied by an activation value from a weight matrix and an error value. The processor reads an update voltage from the element. The processor determines a chopper product by multiplying the update voltage by the chopper value. The processor stores an element of a hidden matrix. The element of the hidden matrix can include a sum of successive iterations of the chopper product. The processor updates a corresponding element of the weight matrix based on the element of the hidden matrix reaching a threshold state.

Description

Background Technology

[0001] This invention relates generally to deep neural network (DNN) training, and more specifically, to techniques for adding chopper values ​​to filtered signals of resistor processing unit (RPU) devices to reduce noise.

[0002] Deep neural networks (DNNs) can be embodied in analog crosspoint arrays of resistive devices such as resistor processing units (RPUs). An RPU typically includes a first terminal, a second terminal, and an active region. The conductance state of the active region identifies the weight values ​​of the RPU, which can be updated / adjusted by applying signals to the first / second terminal.

[0003] DNN-based models have been used for a variety of cognitive-based tasks, such as object and speech identification and natural language processing. Performing such tasks requires DNN training to provide a high level of accuracy. Training large DNNs is a computationally intensive task. The most popular methods for DNN training (such as backpropagation and stochastic gradient descent (SGD)) require the RPU to be “symmetric” to work accurately. A symmetric analog resistive device changes its conductance symmetrically when subjected to positive and negative voltage pulses. However, in reality, RPU devices can exhibit nonlinear and asymmetric switching characteristics. For example, when a voltage pulse is applied to adjust the weights up or down, an imbalance often exists between the upward and downward adjustments. Summary of the Invention

[0004] This invention provides a technique for training deep neural networks (DNNs) that use resistive processing units (RPUs) to track and update weight values. The technique described herein overcomes the problems of noise and bias that may be introduced by the RPU. Specifically, the noise introduced by the RPU is addressed by using a hidden matrix that acts as a low-pass filter, while the bias is addressed using a chopper.

[0005] In a method or computer program product, the processor determines incremental weight updates by updating elements of matrix A by multiplying activation values ​​and error values ​​from the weight matrix by chopper values. These elements may include resistor processing units. The processor reads the updated voltage from the elements. The processor determines the chopper product by multiplying the updated voltage by the chopper value. The processor stores elements of the hidden matrix. The elements of the hidden matrix may include the summation of successive iterations of the chopper product. The processor updates the corresponding elements of the weight matrix based on the hidden matrix elements reaching a threshold state.

[0006] In one embodiment, the processor tracks the sum of chopper products of the elements of matrix A in the corresponding elements of the hidden matrix. The chopper product can include the activation and error values ​​from the corresponding elements of the weight matrix multiplied by a chopper value before and after they are applied to matrix A. When the sum of one of these sums reaches a threshold, the processor triggers an update of the corresponding elements of the weight matrix.

[0007] Embodiments may include a deep neural network (DNN) having a matrix A that has a resistor processing unit (RPU) device separating the intersections between conductive row wires and conductive column wires. The RPU device may include processed gradients for weighted connections between neurons in the DNN. The DNN may include a weight matrix of the RPU device separating the intersections between conductive row wires and conductive column wires. The RPU device may include weighted connections between neurons in the DNN. The DNN may include a chopper configured to multiply activation values ​​and error values ​​from the weight matrix by a chopper value before being applied to the matrix A, and to multiply the output vector from the matrix A by the chopper value to produce a chopper product. The DNN may include a computer storage configured to store a hidden matrix comprising an H value for each RPU device in the weight matrix W. The H value may include the sum of the chopper products.

[0008] In one embodiment, training a deep neural network (DNN) may include multiplying the input vector e by a chopper value. i The output vector y' is transmitted as a voltage pulse through the conductive column lines of matrix A and read as the current output from the conductive row lines of matrix A. Matrix A may contain resistor processing unit (RPU) devices that separate the intersections between the conductive column lines and the conductive row lines. Training may include determining the chopper product for each RPU by multiplying the output vector y' by a chopper value. Training may include updating the value of the hidden matrix H by iteratively adding the chopper product, where the hidden matrix includes the H value for each RPU. Training may include, after the H value reaches a threshold, transmitting the input vector e through the conductive column lines of the weight matrix W. i As a voltage pulse, the symbol information of the H value that reaches the threshold is transmitted as a voltage pulse through the conductive rows of the weight matrix W.

[0009] A more complete understanding of the invention, as well as its further features and advantages, will be obtained by referring to the following detailed description and accompanying drawings. Attached Figure Description

[0010] Figure 1A This is a schematic diagram showing a deep neural network (DNN) with weight matrix W, matrix A and hidden matrix H.

[0011] Figure 1B This is a diagram illustrating a deep neural network (DNN) embodied in an analog cross-point array of a resistor processing unit (RPU) device according to an embodiment of the present invention;

[0012] Figure 2A This is a diagram illustrating the ideal switching characteristics of a linear and symmetrical RPU device according to an embodiment of the present invention;

[0013] Figure 2B This is a diagram illustrating the non-ideal switching characteristics of a nonlinear and asymmetric RPU device according to an embodiment of the present invention;

[0014] Figure 3 This is a diagram illustrating an exemplary method for training a DNN according to an embodiment of the present invention;

[0015] Figure 4 This is a diagram illustrating two interconnected arrays (i.e., array W and reference array) corresponding to matrix W according to an embodiment of the present invention, with the reference array filled with conductance values ​​corresponding to the zero weight values ​​of matrix W;

[0016] Figure 5 This is a diagram illustrating the forward cycle y = Wx performed according to an embodiment of the present invention;

[0017] Figure 6 This is a diagram illustrating the executing backward loop z = WTo according to an embodiment of the present invention;

[0018] Figure 7 This is a diagram illustrating an array A updated using x propagating in the forward loop and δ propagating in the backward loop, according to an embodiment of the present invention.

[0019] Figure 8 To illustrate the forward loop y' = Ae performed on the weight matrix according to an embodiment of the present invention i Diagram;

[0020] Figure 9 This is a diagram showing the hidden matrix H updated using the values ​​calculated in the forward cycle of matrix A;

[0021] Figure 10 This is a schematic diagram of the hidden matrix H 902 selectively applied to the weight matrix W 1010 according to an embodiment of the present invention;

[0022] Figure 11 This is a diagram illustrating an exemplary one-hot encoded vector according to an embodiment of the present invention;

[0023] Figure 12This is a diagram illustrating an exemplary Hadamard matrix of order 2 according to an embodiment of the present invention;

[0024] Figure 13 This is a diagram illustrating an exemplary Hadamard matrix of order 4 according to an embodiment of the present invention; and

[0025] Figure 14 This is a diagram illustrating an exemplary apparatus that can be used to perform one or more of the techniques according to embodiments of the present invention. Detailed Implementation

[0026] This paper provides a deep neural network (DNN) training technique with an asymmetric resistive processing unit (RPU) device. The DNN is trained by adjusting the weight values ​​between perceptron layers until the data input passing through the DNN accurately matches the data output of the training dataset fed into the DNN. These weight values ​​can be stored digitally, but in the embodiments disclosed herein, the weight values ​​are stored in the RPU device embodied in the weight matrix. The use of the RPU device improves the speed of the DNN and reduces resource consumption, but may introduce noise and bias inherent in many analog systems. To mitigate the noise and bias of the analog RPU device, the embodiments disclosed herein include a hidden matrix that acts as a low-pass filter to reduce noise, and a chopper that introduces positive or negative chopper values ​​to mitigate bias.

[0027] Now turn to the attached image. Figure 1A This is a schematic diagram illustrating a deep neural network (DNN) 100 with a weight matrix W 102, an A matrix 112, and a hidden matrix H 114. (See diagram for reference.) Figure 1A As indicated by the arrows shown, the weight matrix W 102 is trained iteratively using matrix A 112 and hidden matrix 114. As emphasized above, the weight matrix W 102 can be represented in the simulated crosspoint array of the RPU. See, for example, Figure 1B The diagram shown is shown in the image.

[0028] like Figure 1BAs shown, each parameter (weight wij) of the algorithm (abstract) weight matrix 102 is mapped to a single RPU device (RPUij) on the hardware, namely, the physical crosspoint array 104 of the RPU device. The crosspoint array 104 includes a series of conductive row lines 106 and a series of conductive column lines 108, wherein the series of conductive column lines 108 are oriented orthogonally to and intersect with the conductive row lines 106. The crosspoints between the row lines 106 and column lines 108 are separated by RPUs 110, forming the crosspoint array 104 of the RPU device. Each RPU 110 may include a first terminal, a second terminal, and an active region. The conduction state of the active region identifies the weight value of the RPU 110, which can be updated / adjusted by applying a signal to the first / second terminal. Furthermore, by controlling additional terminals, a three-terminal (or even more terminal) device can be effectively used as a two-terminal resistive storage device.

[0029] Each RPU 110 (RPUij) is uniquely identified based on its position in the intersection array 104 (i.e., the i-th row and j-th column). For example, operating from top to bottom and left to right in the intersection array 104, the RPU at the intersection of the first row line 106 and the first column line 108 is designated as RPU11, the RPU at the intersection of the first row line 106 and the second column line 108 is designated as RPU12, and so on. Furthermore, the mapping of the parameters of the weight matrix 102 to the RPUs of the intersection array 104 follows the same convention. For example, the weights wi1 of the weight matrix 102 are mapped to RPUi1 of the intersection array 104, the weights wi2 of the weight matrix 102 are mapped to RPUi2 of the intersection array 104, and so on.

[0030] The RPU 110 of the crosspoint array 104 effectively acts as a weighted connection between neurons in the DNN. The conduction state (e.g., resistance) of the RPU 110 can be changed by controlling the voltage applied between the individual lines of row line 106 and column line 108, respectively. Data is stored by changing the conduction state of the RPU. The conduction state of the RPU 110 is read by applying voltage and measuring the current through the target RPU 110. All operations involving weights are performed completely in parallel by the RPU 110.

[0031] In machine learning and cognitive science, DNN-based models are a family of statistical learning models inspired by the biological neural networks of animals (particularly the brain). These models can be used to estimate or approximate systems and cognitive functions that depend on numerous inputs and weights of connections that are often unknown. DNNs are often embodied as so-called "neuronal-morphology" systems of interconnected processor elements that act as analog "neurons" exchanging "messages" with each other in the form of electronic signals. The connections in a DNN that carry the electronic messages between the analog neurons are provided with numerical weights corresponding to the strength or weakness of a given connection. These numerical weights can be adjusted and tuned empirically, allowing the DNN to adapt to the input and learn. For example, a DNN used for handwritten digits is defined by a set of input neurons that can be activated by pixels of an input image. After being weighted and transformed by functions determined by the network's designer, the activation of these input neurons is then passed on to other downstream neurons. This process is repeated until an output neuron is activated. The activated output neuron determines which character has been read.

[0032] Figure 1A The DNN 100 shown is trained as follows: weight values ​​Wij are updated via matrix A 112, and the output from matrix A 112 is then summed into hidden matrix 114 until the elements of hidden matrix 114 (i.e., Hij) reach a threshold, as explained in detail below. However, before and after updating the weight values ​​in matrix A 112, chopper 116 multiplies the input and output signals by chopper values. The chopper value at a given time is either positive one (+1) or negative one (-1). Chopper 116 randomly flips between chopper values ​​such that for a portion of the training period, the update is applied to matrix A 114 with the opposite sign. This random sign flipping of chopper 116 means that any “bias” contributed to the weight values ​​by matrix A 112 has one sign (i.e., positive or negative) during some periods of the training time and another sign (i.e., negative or positive) during other periods of the training time. Bias can be inherent in any simulation system, including non-ideal RPUs that can be used in DNN 100.

[0033] Figure 2A This is a diagram illustrating the ideal switching characteristics of a linear and symmetrical RPU device according to an embodiment of the present invention. Figure 2A As shown, an ideal RPU responds linearly and symmetrically to external voltage stimuli. For training purposes, such an ideal device perfectly implements the DNN training process of backpropagation and stochastic gradient descent (SGD). Backpropagation is a training process performed in three cycles: a forward cycle repeated multiple times until the convergence criterion is met, a backward cycle, and a weight update cycle. Stochastic gradient descent (SGD) uses backpropagation to compute the error gradient for each parameter (weight wij).

[0034] To perform backpropagation, a DNN-based model involves learning multiple processing layers with data representations at multiple levels of abstraction. For a single processing layer where N input neurons are connected to M output neurons, the forward loop involves computing a vector-matrix multiplication (y = Wx), where a vector x of length N represents the activity of the input neurons, and a matrix W of size M × N stores the weight values ​​between each pair of input and output neurons. The resulting vector y of length M is further processed by performing nonlinear activation on each of the resistive memory elements, and then passed to the next layer.

[0035] Once the information reaches the final output layer, the backward loop involves calculating the error signal and propagating it backward through the DNN. The backward loop on a single layer also involves vector-matrix multiplication over the transpose of the weight matrix (z = WTδ) (interchanging each row and corresponding column), where the vector δ of length M represents the error calculated by the output neuron, and the vector z of length N is further processed using the derivative of the neuron's nonlinearity before being passed down to the previous layer.

[0036] Finally, during the weight update cycle, the weight matrix W is updated by performing the outer product of the two vectors used in the forward and backward cycles. This outer product of the two vectors is often represented as W←W+η(δx) T ), where η is the global learning rate.

[0037] All operations performed on the weight matrix W during this backward propagation process can be implemented using a crosspoint array 104 of RPUs 110 with a corresponding number of M rows and N columns, where the conductance values ​​stored in the crosspoint array 104 form the matrix W. In the forward cycle, the input vector x is transmitted as a voltage pulse through each of the column lines 108, and the resulting vector y is read as the current output from the row line 106. Similarly, when a voltage pulse is supplied from the row line 106 as input for the backward loop, the vector matrix product is computed on the transpose of the weight matrix WT. Finally, in the update cycle, voltage pulses representing vectors x and δ are simultaneously supplied from the column lines 108 and the row lines 106. In this configuration, each RPU 110 performs local multiplication and summation operations by processing the voltage pulses from the corresponding column lines 108 and row lines 106, thus achieving incremental weight updates.

[0038] As mentioned above, symmetric RPU (see Figure 2A This perfectly achieves backpropagation and SGD. That is, through such an ideal w ij ←w ij +ηΔw ij , where wij is the weight value of the i-th row and j-th column of the cross-point array 104.

[0039] In comparison, Figure 2B This is a diagram illustrating the non-ideal switching characteristics of a nonlinear and asymmetric RPU device according to an embodiment of the present invention. For example... Figure 2B As shown, the actual RPU can respond nonlinearly and asymmetrically to external voltage stimuli. Specifically, during the "up" time period 202, when the RPU is given an "up" pulse, the weight value 204 changes with larger step sizes when the weight value is low. That is, when consecutive "up" pulses are applied to the RPU, the weight value 204 levels off. Similarly, during the "down" time period 206, when the RPU is given a "down" pulse, the weight value 204 changes with larger step sizes when the weight value is high. That is, when consecutive "down" pulses are applied to the RPU, the weight value 204 also levels off.

[0040] Figure 2B It is also shown that the RPU has a single weight value (corresponding to the zero-weight value from the zero-shift technique described in detail below), where the upward and downward adjustments are equal in intensity but unbalanced for the remainder of the weight range in other cases. During time period 208, when a series of equal upward and downward pulses are given to the RPU device, this imbalance means that the device has a tendency to rise or fall towards the symmetry point 210. This device behavior can be converted into an additional energy term (internal energy) originating from the physics of the RPU device that controls changes in conductance within the RPU device. Therefore, when used for backpropagation, these RPU implementations with non-ideal switching characteristics differ significantly from the ideal situation, i.e., w ij ←w ij +ηΔw ij F(w ij )-η|Δw ij |G(w ij ), where |Δw ij |G(w ij F(w) represents the additional energy item (internal energy) exhibited due to the asymmetric switching characteristics of the RPU device, and F(w) represents the additional energy item (internal energy) exhibited due to the asymmetric switching characteristics of the RPU device. ij The term ) is due to the nonlinearity in the switching characteristics.

[0041] It has been found that for resistive memory devices such as the RPU 110, there always exists a single point of symmetry where the slopes of the up and down pulses are exactly the same for each device. This point of symmetry (which may be different for each individual RPU) can be assigned a weight value of zero.

[0042] like Figure 2BAs shown, the symmetry point 210 of each RPU 110 in the crosspoint array 104 can be determined by applying repeated (first, second, third, etc.) upward and downward voltage pulses to the RPU 110 in the weighted array until all RPU 110 in the weighted array converge to their own symmetry point. For example, if the conductance range is from Gmin to Gmax, and the average delta G at a single update event is dCavg, then the number of valid states in the conductance range is (Gmax-Gmin) / dCavg. When the device is at an extreme point and given alternating upward / downward (up / dn) pulses, an update will be taken to reach the center point = (Gmax-Gmin) / dGavg. To ensure convergence, additional cycles of alternating upward / downward pulses can be given, for example, nx(Gmax-Gmin) / dCavg, where n = 1 or greater. These upward and downward voltage pulses can be applied to the RPU 110 randomly (i.e., each pulse is randomly either an upward or downward pulse), in an alternating manner (i.e., if the previous pulse is an upward pulse, then the next pulse is a downward pulse, and vice versa).

[0043] Once all RPUs 110 in the weighted array converge to their own symmetry point, the conductance value from each RPU 110 in the weighted array (at the symmetry point) is copied to a separate reference array interconnected with the weighted array. A one-to-one correlation exists between the devices in the weighted array and the reference array, allowing a unique zero-weight value to be established for a corresponding device in the weighted array within each device in the reference array. Thus, during operation, the outputs of the RPUs 110 in the weighted array are shifted zero-weighted from their corresponding devices stored in the reference array. For example, the same signal can be provided to both the weighted array and the reference array. The outputs from the RPUs 110 in the weighted array (e.g., I1, I2, I3, etc.) can then be zero-shifted by subtracting those values ​​(set as zero-weight values) from the outputs of their corresponding devices in the reference array to achieve the zero-shift result. However, in practice, copying the symmetry point to the reference array may not always produce a perfect representation. Imperfect copying of the symmetry point can complicate attempts to use RPU arrays by introducing a “bias.” That is, when a symmetric point is copied to be higher or lower than the actual symmetric point, a bias is introduced into the system.

[0044] After this initial setup to achieve zero offset results, this technique is used to train the DNN while using chopper values ​​to compensate for any bias that may be introduced when setting the zero offset. Figure 3This is a diagram illustrating an exemplary method 300 for training a DNN according to an embodiment of the present invention. During training, weight updates are first accumulated on a matrix. Matrix A is a hardware component consisting of rows and columns of RPUs, which have symmetric behavior around zero. Weight updates from matrix A are then selectively moved to a weight matrix W. The weight matrix W is also a hardware component consisting of rows and columns of RPUs. The training process iteratively determines the set of parameters (weights wij) that maximizes the accuracy of the DNN. During initialization, a reference array with zero weight values ​​ensures that each RPU in the weight matrix A is at its equilibrium point for values ​​ideally corresponding to zero, but is actually limited by incomplete replication of symmetric points. On the other hand, matrix W is initialized to randomly distributed values ​​using conventional practices applied to DNN training. The hidden matrix H (although some embodiments use an analog hidden matrix H, it is generally stored digitally) is initialized to zero.

[0045] During training, weight updates are performed on matrix A. The information processed by matrix A is then accumulated in the hidden matrix H (a separate matrix that effectively performs the low-pass filter). The value of the hidden matrix H that has reached an update threshold is then applied to the weight matrix W. The update threshold effectively minimizes the noise generated within the hardware of matrix A. However, for elements of matrix A initialized with bias, the update threshold will be reached prematurely because each iteration from that element carries a consistent update (positive or negative) based on the bias rather than on the weight updates associated with the trained DNN. The chopper value negates the bias by flipping the sign of the bias for a specific time period during which the bias is accumulated to the hidden matrix H with the opposite sign. Specifically, some time periods add the weight value to the positive bias of the hidden matrix H, while other time periods add the weight value to the negative bias of the hidden matrix H. The random flipping of the chopper value means that time periods with positive bias tend to align with time periods with negative bias (even out). Therefore, the hardware bias and noise associated with a non-ideal RPU are tolerable (or absorbed by the H matrix), and thus give less test error compared to standard SGD techniques, a single hidden matrix H, or other training techniques using asymmetric devices, even with a smaller number of states.

[0046] Method 300 begins with initializing matrix A, the hidden matrix H, and the weight matrix W (block 302). Initializing matrix A includes, for example, determining a symmetry point for each RPU in matrix A and storing the corresponding conductance value in a reference array. As provided above, the symmetry point of each RPU device can be determined by applying repeated upward and downward voltage pulses (e.g., randomly, alternately, etc.) to these RPUs until all RPUs converge to their own symmetry point. Array A and the reference array can be represented as two interconnected arrays, and their combination forms matrix A. Since physical conductance cannot be negative, the difference between the conductance values ​​in array A and the reference array forms the logical value of matrix A. However, the reference array remains constant after the initial programming steps, and we reference matrix A and array A in an interchangeable manner, such that when matrix A is updated, it is the array A being updated. However, for vector matrices, multiplication operations performed on matrix A always use the difference reads of array A and the reference array. The same method and operating principle also apply to matrix W and array W.

[0047] Figure 4 This diagram illustrates two interconnected arrays (i.e., a physical array and a reference array) corresponding to matrix A according to an embodiment of the present invention, and a reference array filled with zero-weight conductance values ​​of matrix A. Cross-point array 402 of RPU 404 (filled with weight values ​​(wij) of matrix A) and cross-point array 406 of RPU 408 (filled with corresponding zero-weight conductance values ​​(w0′) of the reference matrix) are shown. A unique zero-weight value (w0′) can be established for each RPU in the reference matrix for the corresponding RPU in matrix A. Initialization of the hidden matrix H includes setting the current values ​​stored in the matrix to zero, or allocating digital storage space on the connected computing device. Initialization of the weight matrix W includes loading the weight matrix W with random values, so that training processing for the weight matrix W can begin.

[0048] Once the zero-weight conductance values ​​have been stored in the reference array, DNN training is performed. Instead of using only backpropagation in three cycles (i.e., forward, backward, and weight update cycles) to compute the error gradient, the operation is filtered here by a hidden matrix H, which updates the weight values ​​only after an iterative combination of H values ​​grows to a value greater than a threshold. This iterative combination is combined with chopper values ​​configured to counteract biases that may arise from imperfect zero-shifts when mapping symmetric points for each RPU 404.

[0049] Method 300 includes determining activation values ​​by performing a forward loop using a weight matrix W (box 304). Figure 5This is a diagram illustrating a forward loop being executed according to an embodiment of the present invention. The forward loop involves computing a vector-matrix multiplication (y = Wx), where the activation values ​​of the input vector x represent the activity of the input neurons, and the weight matrix W stores the weight values ​​between each pair of input and output neurons. Figure 5 The forward loop vector-matrix multiplication operation is shown to be implemented in the crosspoint array 502 of the RPU device, where the conductance values ​​stored in the crosspoint array 502 form a matrix.

[0050] The input vector x is transmitted as a voltage pulse through each of the conductive column lines 512, and the resulting output vector y is read as the current output from the conductive row lines 510 of the crosspoint array 502. An analog-to-digital converter (ADC) 513 converts the analog output vector 516 from the crosspoint array 502 into a digital signal. A zero-weighted conductance value copied from the reference matrix 406 is used to transfer the output value of the RPU device in the crosspoint array 502 relative to the symmetry point of the RPU device, to account for deviations in its switching behavior and to encode negative logic matrix values. For this purpose, the voltage pulse applied to the crosspoint array 502 is also applied to the reference array. The output vector y of the crosspoint array 502 is then subtracted from the output vector y of the reference array.

[0051] Method 300 also includes determining the error value by performing a backward loop on the weight matrix W (box 306). Figure 6 This is a diagram illustrating a backward loop performed according to an embodiment of the invention. Typically, a backward loop involves calculating an error value δ and propagating that error value δ backward through the weight matrix W via a vector-matrix multiplication on the transpose of the weight matrix W (i.e., z = WTδ), where the vector δ represents the error calculated by the output neuron, and the vector z is further processed using the derivative of the neuron's nonlinearity, and then passed down to the previous layer.

[0052] Figure 6 This illustrates a backward loop vector-matrix multiplication operation implemented in the crosspoint array 502. The error value δ is sent as a voltage pulse through each of the conductive row lines 510, and the resulting output vector z is read as the current output from the conductive column lines 512 of the crosspoint array 502. When a voltage pulse is supplied from row line 510 as input to the backward loop, the vector-matrix product on the transpose of the weight matrix W is then calculated.

[0053] For example Figure 6As shown, ADC 513 is used to convert the (analog) output vector 518 from crosspoint array 502 into a digital signal. Similar to the forward loop described above, zero-weighted conductance values ​​are transferred relative to their symmetry points to the output values ​​of the RPU devices in crosspoint array 502 to account for deviations in their switching behavior and to encode negative logic matrix values. For this purpose, a voltage pulse applied to crosspoint array 502 is also applied to the reference array. The output vector z of crosspoint array 502 is then subtracted from the output vector z of the reference array.

[0054] Method 300 further includes applying chopper values ​​to these activation values ​​and / or these error values ​​(box 308). Chopper values ​​may be applied by a chopper (e.g., chopper 116 from FIG. 1), which is included in each row and each column line of matrix A 502. In some embodiments, the crosspoint array 502 may have choppers only on column line 506 or only on row line 504. After applying chopper values ​​to the activation values ​​and / or error values, method 300 further includes updating matrix A with the activation values, error values, (input vectors x and δ), and chopper values ​​(box 310). Figure 7 This is a diagram illustrating an array A 502 updated according to an embodiment of the invention using x propagating in the forward loop and δ propagating in the backward loop. Each row and column has a chopper value 550 applied to the corresponding line. For a positive chopper value (i.e., no change in the activation or error value), the chopper value 550 is represented by a "+", or for a negative chopper value (i.e., a change in the sign of the activation or error value), it is represented by "X". This update is achieved in the crosspoint array 502 by transmitting voltage pulses representing vector x (from the forward cycle) and vector δ (from the backward cycle) supplied simultaneously from conductive column line 506 and conductive row line 504, respectively. In this configuration, each RPU in the crosspoint array 502 performs local multiplication and summation operations by processing the voltage pulses from the corresponding conductive column line 506 and conductive row line 504, thus achieving incremental weight updates. The forward loop (box 304), the backward loop (box 306), and updating matrix A with the input vectors from the forward and backward loops (box 3010) can be repeated multiple times to improve the updated value of matrix A.

[0055] Method 300 also includes reading the chopper product by performing a forward loop on matrix A using the input vector ei (i.e., y' = Aei) and the chopper value (box 312). At each time step, the new input vector e is used... i And the sub-index i represents the time index. As will be described in detail below, according to an exemplary embodiment, the input vector e iIt is a one-hot encoded vector. For example, as is known in the art, a one-hot encoded vector is a set of bits that has only a single high (1) bit and all other bits are low (0). For illustrative purposes, and to use a simple, non-limiting example, assuming a matrix of size 4×4, the one-hot encoded vector would be one of the following vectors: [1 0 0 0], [0 1 0 0], [0 0 1 0], and [0 0 01]. At each time step, a new one-hot encoded vector is used, and the sub-index i represents that time index. However, it is worth noting that other methods for selecting the input vector ei are also envisioned in this paper. For example, the input vector e i Alternatively, you can choose from columns of a Hadamard matrix, a random matrix, etc.

[0056] Figure 8 This illustrates an embodiment of the invention by performing a forward period y' = Ae on an A matrix having chopper values. i To read the diagram of the chopper product. Input vector e i A voltage pulse is transmitted through each of the conductive column lines 506, and the resulting output vector y' is read as the current output from the conductive row lines 504 of the crosspoint array 502. Each column line 506 and row line 504 is read with the same chopper value (i.e., positive or negative) as the updated A matrix. For example, the first column line 506i1 has Figure 7 and Figure 8 The positive chopper value (+) in the second column line 506i2 has Figure 7 and Figure 8 The negative chopper value (X) in the first row 5041i has Figure 7 and Figure 8 The negative chopper value (X) is calculated. When a voltage pulse is supplied as input from column line 506 to this forward loop, the vector matrix product is then calculated.

[0057] Method 300 includes using a chopper product (i.e., the output vector y' and the input vector e). i Update the hidden matrix H (box 314) by multiplying by the chopper value. Figure 9This is a diagram illustrating the hidden matrix H 902 updated using values ​​computed during the forward cycle of matrix A 904. The hidden matrix H 902 is most commonly a digital matrix (rather than a physical device like matrix A and weight matrix W) storing the H value 906 (i.e., Hij) for each RPU in matrix A (i.e., each RPU at Aij). When the forward loop is performed, an output vector y'eiT is produced and multiplied by a chopper value to determine a chopper product 908, and the hidden matrix H adds the chopper product 908 to each H value 906. Thus, the hidden matrix H 902 changes each time the output vector is read. For those RPUs with low noise levels, the H value 906 will continuously increase, depending on the output vector y'e. i T The value of y'e can increase in either the positive or negative direction. i T Including significant noise, it may be positive for one iteration and negative for another. Positive and negative output vectors y'e i T This combination of values ​​means that the H value of 906 will grow more slowly and inconsistently.

[0058] Method 300 also includes flipping the sign of the chopper value by a flip percentage (box 316). In some embodiments, the chopper value is flipped only after the chopper product has been added to the hidden matrix H. That is, the chopper value is used twice: once when the activation and error values ​​are written to matrix A; and once when the current cycle is read from matrix A. The chopper value should not be flipped before the chopper product is calculated. The flip percentage can be defined as a user preference such that after each chopper product is added to the hidden matrix H, the chopper has a percentage chance to flip its value. For example, the user preference could be fifty percent, such that after the chopper product is calculated, half of the chopper values ​​have a chance to change sign (i.e., positive to negative or negative to positive).

[0059] As the H value 906 increases, method 300 includes tracking whether the H value 906 has increased beyond a threshold (box 318). If the H value 906 at a specific location (i.e., Hij) is not greater than the threshold (box 318 "No"), method 300 repeats from executing a forward loop (box 304) until the hidden matrix H is updated (box 314), potentially flipping the chopper value (box 316). If the H value 906 is greater than the threshold (box 310 "Yes"), method 300 continues to process the input vector e. i The data is transferred to the weight matrix W, but only for a specific RPU (box 320). As mentioned above, the H value 906 can increase in either the positive or negative direction, therefore the threshold can also be positive or negative. Figure 10This is a schematic diagram of the hidden matrix H 902 of the weight matrix W 1010 selectively applied according to an embodiment of the present invention. Figure 10 The diagram shows the first H value 1012 and the second H value 1014 that have exceeded the threshold and are transferred to the weight matrix W 1010. The first H value 1012 reaches the positive threshold and therefore carries a positive one for its row in the input vector 1016: "1". The second H value 1014 reaches the negative threshold and therefore carries a negative one for its row in the input vector 1016: "-1". The remaining rows in the input vector 1016 carry zeros because those values ​​(i.e., H value 906) have not become greater than the threshold. The threshold can be much larger than y'eiT added to the hidden matrix H. For example, the threshold could be ten or one hundred times the expected strength of y'eiT. These high thresholds reduce the frequency of updates performed on the weight matrix W. However, the filtering function performed by the H matrix reduces the error of the neural network's objective function. These updates can only be generated after processing many data examples, and therefore also increase the confidence level in the updates. This technique enables the training of neural networks using noisy RPU devices with only a limited number of states. After applying the H value to the weight matrix W, the H value 906 is reset to zero, and the iteration of method 300 continues.

[0060] Using e i After updating the weight matrix W in step 1018, method 300 continues by determining whether training is complete. If training is not complete, for example, if a certain convergence criterion is not met (box 322 "No"), method 300 restarts by executing the forward loop y = Wx. For example, by way of example only, training can be considered complete when no further improvement on the error signal is observed. When training is complete (box 322 "Yes"), method 300 terminates.

[0061] As described above, according to an exemplary embodiment, the input vector e i It is a one-hot encoded vector, which is a group of bits that consists only of combinations of a single high (1) bit and all other bits being low (0). See, for example, Figure 11 .like Figure 11 As shown, given a 4×4 matrix, the one-hot encoded vector will be one of the following vectors: [1 0 0 0], [0 1 0 0], [00 1 0], and [0 0 0 1]. At each time step, a new one-hot encoded vector is used, denoted by the sub-index i at that time index. According to another exemplary embodiment, the input vector e i The columns are selected from the Hadamard matrix. As is known in the art, the Hadamard matrix is ​​a square matrix with ±1 terms. For example, see... Figure 12 (Hadamard matrix of order 2) and Figure 13(A Hadamard matrix of order 4). This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.

[0062] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital universal disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punched cards, or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0063] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0064] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may be personalized to execute computer-readable program instructions by utilizing state information from the computer-readable program instructions in order to perform aspects of this invention.

[0065] The present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0066] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other apparatus to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0067] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other apparatus to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other apparatus to produce computer-implemented processing, such that the instructions executed on the computer, other programmable apparatus, or other apparatus perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0068] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than indicated in the figures. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0069] Turn now Figure 14 The diagram illustrates a block diagram of an apparatus 1400 for implementing one or more methods proposed herein. By way of example only, the apparatus 1400 may be configured to control input voltage pulses applied to the array and / or process output signals from the array.

[0070] Device 1400 includes a computer system 1410 and removable media 1450. Computer system 1410 includes a processor device 1420, a network interface 1425, memory 1430, a media interface 1435, and an optional display 1440. Network interface 1425 allows computer system 1410 to connect to a network, while media interface 1435 allows computer system 1410 to interact with media such as a hard drive or removable media 1450.

[0071] Processor device 1420 can be configured to implement the methods, steps, and functions disclosed herein. Memory 1430 can be distributed or local, and processor device 1420 can be distributed or single. Memory 1430 can be implemented as electrical memory, magnetic memory, or optical memory, or any combination of these or other types of storage devices. Furthermore, the term "memory" should be interpreted broadly enough to include any information that can be read from or written to an addressable space accessible by processor device 1420. Using this definition, information about a network accessible via network interface 1425 remains within memory 1430, as processor device 1420 can retrieve that information from the network. It should be noted that each distributed processor comprising processor device 1420 typically contains its own addressable memory space. It should also be noted that some or all of the computer system 1410 can be integrated into application-specific or general-purpose integrated circuits.

[0072] The optional display 1440 is any type of display suitable for human user interaction with the device 1400. Typically, the display 1440 is a computer monitor or other similar display.

[0073] While illustrative embodiments of the invention have been described herein, it should be understood that the invention is not limited to those precise embodiments, and various other changes and modifications can be made by those skilled in the art without departing from the scope of the invention.

Claims

1. A computer-implemented method for training a deep neural network (DNN), the method comprising: Incremental weight updates are determined by updating the elements of matrix A by multiplying the activation and error values ​​from the weight matrix by chopper values, wherein the chopper values ​​are randomly selected from a group consisting of positive and negative ones, and wherein the elements include analog resistor processing units (RPUs). Read the updated voltage from the element; The chopper product is determined by multiplying the updated voltage by the chopper value; The elements of the hidden matrix are stored, wherein the elements of the hidden matrix include the sum of successive iterations of the chopper product; as well as The corresponding elements of the weight matrix are updated based on the elements of the hidden matrix reaching a threshold state, wherein the hidden matrix generates a low-pass filter to reduce noise in the analog resistor processing unit (RPU), and the chopper value reduces bias.

2. The method according to claim 1, wherein, The probability that the chopper value flips between a positive and a negative value is user-defined.

3. The method according to claim 1, wherein, Updating the corresponding elements of the weight matrix includes transmitting voltage pulses through the conductive column wiring of the A matrix, and simultaneously transmitting the sign information of the elements of the hidden matrix as voltage pulses through the conductive row wiring of the A matrix.

4. The method according to claim 1, wherein, The chopper value is applied to the conductive column lines of the A matrix.

5. The method according to claim 1, wherein, The chopper value is applied to the conductive rows of the A matrix.

6. A computer-implemented method for training a deep neural network, comprising: The sum of chopper products of the elements of matrix A in the corresponding elements of the tracking hidden matrix, wherein the chopper product includes the activation value and error value of the corresponding element of the weight matrix multiplied by a chopper value, wherein the chopper value includes a random selection of a group of positive and negative ones before and after being applied to matrix A, in order to mitigate the bias of matrix A. When the sum of one of the sums reaches a threshold, the corresponding element of the weight matrix is ​​updated to reduce the noise in the A matrix.

7. The method according to claim 6, wherein, The probability that the chopper value flips between a positive and a negative value is user-defined.

8. The method according to claim 6, wherein, The sum is tracked digitally.

9. A deep neural network (DNN), comprising: A matrix includes an analog resistance processing unit (RPU) device that separates the intersections between conductive row lines and conductive column lines, wherein the analog resistance processing unit (RPU) device includes processed gradients of weighted connections between neurons in the DNN. The weight matrix includes a resistance processing unit (RPU) device that separates the intersections between conductive row lines and conductive column lines, wherein the resistance processing unit (RPU) device includes weighted connections between neurons in the DNN. The processor is configured to multiply the activation and error values ​​from the weight matrix by the chopper value before the chopper value is applied to the A matrix, and to multiply the output vector from the A matrix by the chopper value to produce a chopper product, wherein the chopper value comprises a random selection from a group consisting of positive and negative ones, and the multiplication of the chopper values ​​before and after the chopper value reduces the bias of the A matrix; as well as A computer storage device is configured to store a hidden matrix, the hidden matrix including the H value of each resistor processing unit (RPU) device in the weight matrix, wherein the H value includes the sum of the chopper products.

10. The DNN according to claim 9, wherein, The chopper is assigned to a selection from a group that includes one of the column lines of the A matrix and one of the row lines of the A matrix.

11. The DNN according to claim 9, wherein, The chopper value flips between positive and negative values ​​with a user-defined probability.

12. A computer program product for reducing bias in an array of crosspoints, the computer program product comprising: One or more computer-readable storage media, and program instructions commonly stored on the one or more computer-readable storage media, the program instructions comprising: Program instructions for initializing elements of matrix A, wherein the elements include analog resistor processing units (RPUs); Program instructions for determining incremental weight updates by updating the elements by multiplying the activation values ​​and error values ​​from the weight matrix by a chopper value, wherein the chopper value comprises a random selection from a group consisting of positive and negative ones; Read the program instructions for updating the voltage from the element; The program instructions determine the chopper product by multiplying the updated voltage by the chopper value; Program instructions for storing elements of a hidden matrix, wherein the elements of the hidden matrix comprise the sum of successive iterations of the chopper product; and The program instructions update the corresponding elements of the weight matrix based on the elements of the hidden matrix reaching a threshold state, wherein the hidden matrix generates a low-pass filter to reduce noise in the analog resistor processing unit (RPU), and the chopper value reduces bias.

13. The computer program product according to claim 12, wherein, The probability that the chopper value flips between a positive and a negative value is user-defined.

14. The computer program product according to claim 12, wherein, Updating the corresponding elements of the weight matrix includes transmitting voltage pulses through the conductive column wiring of the A matrix, and simultaneously transmitting the sign information of the elements of the hidden matrix as voltage pulses through the conductive row wiring of the A matrix.

15. The computer program product according to claim 12, wherein, The incremental weight update includes matrix multiplication of the output vector and the input vector of the operation on matrix A.

16. The computer program product according to claim 12, wherein, The instructions for updating the weight matrix include: instructions for transmitting a voltage pulse through the conductive column lines of the A matrix when an element of the hidden matrix reaches a threshold state, and simultaneously transmitting the sign information of the element of the hidden matrix that has reached the threshold state as a voltage pulse through the conductive row lines of the A matrix.

17. A computer-implemented method for training a deep neural network (DNN), comprising: The input vector e, multiplied by the chopper value, is transmitted through the conductive column lines of matrix A. i The output vector y' is obtained as a voltage pulse and read as the current output from the conductive row lines of the matrix A, wherein the matrix A includes an analog resistance processing unit (RPU) device that separates the intersections between these conductive column lines and these conductive row lines, and the chopper value includes a random selection from a group consisting of positive and negative ones. The bias of the A matrix is ​​mitigated by multiplying the output vector y' by the chopper value to determine the chopper product of each analog resistance processing unit (RPU) device. The H value of the hidden matrix is ​​updated by iteratively adding the chopper product, wherein the hidden matrix includes the H value of each analog resistor processing unit (RPU) device; and When the H value reaches the threshold, the input vector e is transmitted through the conductive column lines of the A matrix. i As a voltage pulse, the symbol information of the H value reaching the threshold is transmitted simultaneously through the conductive rows of the A matrix.

18. The method according to claim 17, wherein, The input vector includes activation values ​​and error values ​​from a weight matrix, which is derived from forward and backward cycles of operations performed on the weight matrix.

19. The method of claim 17, wherein, The probability that the chopper value flips between a positive and a negative value is user-defined.

20. The method according to claim 18, wherein, Updating the corresponding elements of the weight matrix includes transmitting voltage pulses through the conductive column wiring of the A matrix, and simultaneously transmitting the sign information of the elements of the hidden matrix as voltage pulses through the conductive row wiring of the A matrix.

21. The method according to claim 17, wherein, The input vector e i This includes selection from a group that includes vectors with one-hot encoding and Hadamard matrices.

Images