Filtering hidden matrices to train DNN

Abstract

In one aspect, a method (300) for training a DNN includes: transmitting an input vector x via a weighting matrix W and reading the resulting output vector y (304); transmitting an error signal δ; transmitting an input vector x with the error signal δ via a transmission row wiring of matrix A (504); and transmitting an input vector e i The resulting output vector y′ is read as the current output (310). Training also includes multiplying by the input vector e i The transpose of the output vector y′ is iteratively added to each H value to update the hidden matrix H(902), which includes the H value (906) for the RPU device (908). Training also includes: when the H value reaches a threshold, the input vector e is... i As a voltage pulse, it is transmitted through the conductive column wiring of matrix W (512), and at the same time, the symbol information of the H value (906) that reaches the threshold is transmitted as a voltage pulse through the conductive row wiring matrix W (510).

Description

Background Technology

[0001] This invention relates generally to deep neural network (DNN) training, and more specifically to techniques for filtering signals from resistor processing unit (RPU) devices to reduce noise.

[0002] Deep neural networks (DNNs) can be embodied in an array of analog crosspoints of a resistor device, such as a resistor processing unit (RPU). An RPU typically includes a first terminal, a second terminal, and an active region. The conductance state of the active region identifies the weight values ​​of the RPU, which can be updated / adjusted by applying signals to the first / second terminal.

[0003] DNN-based models have been used for a variety of cognitive-based tasks, such as object and speech recognition and natural language processing. Performing such tasks requires DNN training to provide a high level of accuracy. Training large DNNs is a computationally intensive task. The most popular methods for DNN training (such as backpropagation and stochastic gradient descent (SGD)) require the RPU to be "symmetric" to work accurately. A symmetric analog resistive device changes its conductance symmetrically when subjected to positive and negative voltage pulses. However, in practice, RPU devices can exhibit nonlinear and asymmetric switching characteristics. For example, when a voltage pulse is applied to adjust the weights up or down, an imbalance often exists between the upward and downward adjustments. Summary of the Invention

[0004] This invention provides a technique for training deep neural networks (DNNs). The method includes transmitting an input vector x as a voltage pulse through the conductive column wiring of a weighting matrix W, and reading the resulting output vector y as a current output from the conductive row wiring of the weighting matrix W. The weighting matrix W may include resistor processing unit (RPU) devices that separate the intersections between the conductive column wiring and the conductive row wiring, such that the RPU devices act as weighted connections between neurons in the DNN. The training may further include: transmitting an error signal δ calculated from the output vector y as a voltage pulse through the conductive row wiring, and reading the resulting output vector z as a current output from the conductive column wiring. The training may further include transmitting the input vector x as a voltage pulse through the conductive column wiring of matrix A, while simultaneously transmitting the error signal δ through the conductive row wiring of matrix A. The training may further include transmitting the input vector x as a voltage pulse through the conductive column wiring of matrix A, while simultaneously transmitting the error signal δ through the conductive row wiring of matrix A. The training may further include... i The voltage pulse is transmitted through the conduction column wiring of the crosspoint array A, and the resulting output vector y′ is read as the current output from the conduction row wiring of the crosspoint array A. Training can further include multiplying by the input vector e i The transpose of the output vector y′ is iteratively added to each H value to update the H value for each RPU device. ijThe hidden matrix H. Training may also include: when the value of H reaches a threshold, transmitting the input vector e through the conductive column wiring of matrix W. i As a voltage pulse, the symbol information of the H value that reaches the threshold is transmitted as a voltage pulse through the conductive row wiring matrix W.

[0005] Embodiments of the present invention also include a deep neural network (DNN). The DNN includes a matrix A having resistor processing unit (RPU) devices that separate the intersections between row and column wirings, whereby the RPU devices include processed gradients for weighted connections between neurons in the DNN. The DNN may also include a weight matrix W, which includes RPU devices that separate the intersections between row and column wirings, whereby the RPU devices include weighted connections between neurons in the DNN. The DNN may further include a hidden matrix H having an H value for each RPU device in the weight matrix W. H values ​​can be iteratively added from matrix A until at least one H value exceeds a threshold.

[0006] A more complete understanding of the invention, as well as its further features and advantages, will be obtained by referring to the following detailed description and accompanying drawings. Attached Figure Description

[0007] Embodiments of the invention will now be described by way of example only with reference to the accompanying drawings, in which:

[0008] Figure 1 This is an illustration of a deep neural network (DNN) embodied in an analog crosspoint array of a resistor processing unit (RPU) device according to an embodiment of the present invention;

[0009] Figure 2A This is a diagram illustrating the ideal switching characteristics of a linear and symmetrical RPU device according to an embodiment of the present invention;

[0010] Figure 2B This is a diagram illustrating the non-ideal switching characteristics of a nonlinear and asymmetric RPU device according to an embodiment of the present invention;

[0011] Figure 3 This is an illustration of an exemplary method for training a DNN according to an embodiment of the present invention;

[0012] Figure 4 This is a diagram illustrating two interconnected arrays (i.e., array W and reference array) corresponding to matrix W according to an embodiment of the present invention, with the reference array filled with conductance values ​​corresponding to the zero weight values ​​of matrix W.

[0013] Figure 5 This is a diagram illustrating a forward loop executed according to an embodiment of the present invention;

[0014] Figure 6 This illustrates the backoff cycle performed according to an embodiment of the present invention;

[0015] Figure 7 This is a diagram illustrating an array A updated according to an embodiment of the present invention, utilizing Δ propagating in the forward loop and δ propagating in the backward loop;

[0016] Figure 8 This illustrates the execution of a forward loop y′=Ae on the weight matrix according to an embodiment of the present invention. i Diagram;

[0017] Figure 9 This is a diagram showing the hidden matrix H updated by the values ​​calculated in the forward loop of matrix A;

[0018] Figure 10 This is a schematic diagram of the hidden matrix H902 selectively applied to the weight matrix W1010 according to an embodiment of the present invention;

[0019] Figure 11 This is a diagram illustrating an exemplary hot-coded vector according to an embodiment of the present invention;

[0020] Figure 12 This is a diagram illustrating an exemplary Hadamard matrix of order 2 according to an embodiment of the present invention;

[0021] Figure 13 This is a diagram illustrating an exemplary Hadamard matrix of order 4 according to an embodiment of the present invention; and

[0022] Figure 14 This is a diagram illustrating an exemplary apparatus that can be used to perform one or more of the technologies according to an embodiment of the present invention. Detailed Implementation

[0023] This paper presents a deep neural network (DNN) training technique with an asymmetric resistive processing unit (RPU) device. As emphasized above, the DNN can be embodied in a simulated crosspoint array of the RPU. See, for example... Figure 1 The schematic diagram shown.

[0024] like Figure 1 As shown, each parameter (weight wij) of the algorithm (abstract) weight matrix 102 is mapped to a single RPU device on the hardware. ijThis refers to the physical crosspoint array 104 of the RPU device. The crosspoint array 104 includes a series of conductive row wirings 106 and a series of conductive column wirings 108, the column wirings 108 being oriented orthogonally to and intersecting with the row wirings 106. The crosspoints between the row wirings 106 and column wirings 108 are separated by the RPU 110, forming the crosspoint array 104 of the RPU device. Each RPU may include a first terminal, a second terminal, and an active region. The conduction state of the active region identifies the weight value of the RPU, which can be updated / adjusted by applying a signal to the first / second terminal. Furthermore, by controlling additional terminals, a three-terminal (or even more terminal) device can be effectively used as a two-terminal resistive memory device.

[0025] Each RPU (RPU) ij The RPUs are uniquely identified based on their positions in the intersection array 104 (i.e., the i-th row and j-th column). For example, operating from top to bottom and left to right in the intersection array 104, the RPU at the intersection of the first row wiring 106 and the first column wiring 108 is designated as RPU 11, the RPU at the intersection of the first row wiring 106 and the second column wiring 108 is designated as RPU 12, and so on. Furthermore, the mapping of the parameters of the weight matrix 102 to the RPUs of the intersection array 104 follows the same convention. For example, the weight wi1 of the weight matrix 102 is mapped to the RPU of the intersection array 104. il The weights wi2 of the weight matrix 102 are mapped to the RPU of the crosspoint array 104. i2 ,etc.

[0026] The RPU 110 of the crosspoint array 104 effectively acts as a weighted connection between neurons in the DNN. The conduction state (e.g., resistance) of the RPU 110 can be changed by controlling the voltage applied between the individual wires of the row wiring 106 and column wiring 108, respectively. Data is stored by changing the conduction state of the RPU. The conduction state of the RPU 110 is read by applying voltage and measuring the current through the target RPU 110. All operations involving weights are performed entirely in parallel by the RPU 110.

[0027] In machine learning and cognitive science, DNN-based models are a family of statistical learning models inspired by the biological neural networks of animals (particularly the brain). These models can be used to estimate or approximate systems and cognitive functions that depend on numerous inputs and weights of connections that are often unknown. DNNs are often implemented as so-called "neuromnographic" systems of interconnected processor elements that act as analog "neurons" exchanging "messages" with each other in the form of electronic signals. The connections in a DNN that carry the electronic messages between the analog neurons are provided with numerical weights corresponding to the strength or weakness of a given connection. These numerical weights can be adjusted and tuned empirically, allowing the DNN to adapt to the input and learn. For example, a DNN used for handwriting recognition is defined by a set of input neurons that can be activated by pixels of an input image. After being weighted and transformed by functions determined by the network's designer, the activation of these input neurons is then passed on to other downstream neurons. This process is repeated until an output neuron is activated. The activated output neuron determines which character has been read.

[0028] Figure 2A This is a diagram illustrating the ideal switching characteristics of a linear and symmetrical RPU device according to an embodiment of the present invention. Figure 2A As shown, an ideal RPU responds linearly and symmetrically to external voltage stimuli. For training purposes, such an ideal device perfectly implements the DNN training process using backpropagation and stochastic gradient descent (SGD). Backpropagation is a training process performed in three cycles: a forward cycle repeated multiple times until the convergence criterion is met, a backward cycle, and a weight update cycle. Stochastic gradient descent (SGD) uses backpropagation to compute the error gradient for each parameter (weight Wij).

[0029] To perform backpropagation, a DNN-based model consists of multiple processing layers that learn data representations with multiple levels of abstraction. For a single processing layer where N input neurons are connected to M output neurons, the forward loop involves computing a vector-matrix multiplication (y = Wx), where a vector X of length N represents the activity of the input neurons, and a matrix W of size M × N stores the weight values ​​between each pair of input and output neurons. The resulting vector y of length M is further processed by performing nonlinear activation on each of the resistive memory elements, and then passed to the next layer.

[0030] Once the information reaches the final output layer, the backward loop involves calculating the error signal and propagating it backward through the DNN. The backward loop on a single layer also involves the weight matrix (z = W). TVector-matrix multiplication on the transpose of δ (interchanging each row and corresponding column), where the vector δ of length M represents the error computed by the output neuron, and the vector z of length N is further processed using the derivative of the neuron's nonlinearity before being passed down to the previous layer.

[0031] Finally, during the weight update cycle, the weight matrix W is updated by performing the outer product of the two vectors used in the forward and backward cycles. This outer product of the two vectors is often expressed as W←W+η(δx) T ), where n is the global learning rate.

[0032] All operations performed on the weight matrix W during this backpropagation process can be implemented using the crosspoint array 104 of the RPU 110, which has a corresponding number of M rows and N columns, wherein the conductance values ​​stored in the crosspoint array 104 form the matrix W. In the forward cycle, the input vector X is transmitted as a voltage pulse through each of the column wirings 108, and the resulting vector y is read as the current output from the row wirings 106. Similarly, when a voltage pulse is supplied from the row wirings 106 as the input to the backward cycle, the weight matrix W... T The vector matrix product is calculated on the transpose of the vectors. Finally, during the update cycle, voltage pulses representing vectors x and δ are simultaneously supplied from column wiring 108 and row wiring 106. In this configuration, each RPU 110 performs local multiplication and summation operations by processing the voltage pulses from the corresponding column wiring 108 and row wiring 106, thus achieving incremental weight updates.

[0033] As mentioned above, symmetric RPU (see Figure 2A This perfectly implements backpropagation and SGD. That is, utilizing this ideal RPU w ij ←w ij +ηΔw ij , where Wij is the weight value of the i-th row and j-th column of the cross-point array 104.

[0034] In comparison, Figure 2B This is a diagram illustrating the non-ideal switching characteristics of a nonlinear and asymmetric RPU device according to an embodiment of the present invention. For example... Figure 2BAs shown, a real RPU can respond nonlinearly and asymmetrically to external voltage stimuli. That is, when a voltage pulse is applied to adjust the weights up or down (referred to herein as "up" and "down" pulses, respectively), the amount of weight update (i.e., the size of the step) depends linearly on the current weight value. Therefore, only a single weight value exists (corresponding to the zero-point weight value from the zero-point drift technique described in detail below), where the up and down adjustments are equal in intensity but unbalanced for the remainder of the weight range in other cases. Assuming a set of up and down pulse sequences are given to the RPU device, this imbalance means that the device has a tendency to rise or fall towards a symmetrical point where the up and down pulses are equal in intensity. This device behavior can be translated into an additional energy term (internal energy) derived from the physics of the RPU device that controls the changes in conductance within the RPU device. Therefore, when used for backpropagation, these RPUs with non-ideal switching characteristics realize something very different from the ideal situation, i.e., w ij ←w ij +ηΔw ij F(w ij )-η|Δw ij |G(w ij ), where |Δw ij |G(w ij ) represents the additional energy term (internal energy) that arises due to the asymmetric switching characteristics of the RPU device, and (w ij The term ) arises due to the nonlinearity of the switching characteristics.

[0035] It has been found that for resistive memory devices such as the RPU 110, there always exists a single point of symmetry where the slopes of the up and down pulses are exactly the same for each device. This point of symmetry (which may be different for each individual RPU) can be assigned a weight value of zero.

[0036] The symmetry point of each RPU 110 in the crosspoint array 104 is determined by applying repeated (first, second, third, etc.) voltage pulses up and down to the RPU 110 in the weighted array until all RPU 110 in the weighted array converge to their own symmetry point. For example, if the conductivity range is from Gmin to Gmax, and the average DELTAG (delta G) at a single update event is dCavg, then the number of valid states in the conductivity range is (Gmax-Gmin) / dCavg. When the device is at an extreme point and given alternating rising / falling (dn) pulses, an update is taken to reach the center point = (Gmax-Gmin) / dGavg. To ensure convergence, additional cycles of alternating rising / dn pulses can be given, for example, nx(Gmax-Gmin) / dGavg, where n = 1 or greater. These upward and downward voltage pulses can be applied to the RPU 110 randomly (i.e., each pulse is randomly either an upward or downward pulse), in an alternating manner (i.e., if the previous pulse is an upward pulse, then the next pulse is a downward pulse, and vice versa).

[0037] Once all RPUs 110 in the weighted array converge to their own symmetry point, the transmitted value from each RPU 110 in the weighted array (at the symmetry point) is replicated to a separate reference array interconnected with the weighted array. A one-to-one correlation exists between the devices in the weighted array and the reference array, such that for a corresponding device in the weighted array, a unique zero-weight value can be established in each device in the reference array. Thus, during operation, the outputs of the RPUs 110 in the weighted array are shifted zero-weighted from their corresponding devices stored in the reference array. For example, the same signal can be provided to both the weighted array and the reference array. The outputs from the RPUs 110 in the weighted array (e.g., I1, I2, I3, etc.) can then be zero-shifted by subtracting those values ​​(set as zero-weight values) from the outputs of their corresponding devices in the reference array to achieve the zero-shift result.

[0038] After this initial setup for achieving the zero-shift result, this technique is used to train the DNN. Figure 3This is a diagram illustrating an exemplary method 300 for training a DNN according to an embodiment of the present invention. During training, weight updates are first accumulated on matrix A. Matrix A is a hardware component consisting of rows and columns of RPUs with symmetric behavior around zeros. Then, the weight updates from matrix A are selectively moved to weight matrix W. Weight matrix W is also a hardware component consisting of rows and columns of RPUs. The training process iteratively determines the set of parameters (weights Wij) that maximize the accuracy of the DNN. During initialization, a reference array with zero weight values ​​ensures that each RPU in weight matrix A is at a balance point of its corresponding zero value. On the other hand, matrix W is initialized to randomly distributed values ​​using conventional practices applied to DNN training. The hidden matrix H (although some embodiments use an analog hidden matrix H, it is generally stored digitally) is initialized to zero. During training, weight updates are performed on matrix A. The information processed by matrix A is then accumulated in the hidden matrix H (a separate matrix that effectively performs a low-pass filter). The value of hidden matrix H that has reached an update threshold is then applied to weight matrix W. The update threshold effectively minimizes the noise generated within the hardware of matrix A. Advantageously, this weight matrix W minimizes the objective function of the neural network, and matrix A is used to process the gradients of the objective function, which are then passed through a digital low-pass filter before being fed back into W. Therefore, it tolerates (or is absorbed by the H matrix) the hardware bias and noise associated with a non-ideal RPU, and thus gives less test error compared to standard SGD techniques or other training techniques using asymmetric devices (even with a smaller number of states).

[0039] Method 300 begins with initializing matrix A, the hidden matrix H, and the weight matrix W (box 302). Initializing matrix A includes, for example, determining the symmetry point of each RPU in matrix A and storing the corresponding conductance value in the reference array. As provided above, the symmetry point of each RPU device can be determined by applying repeated upward and downward voltage pulses (e.g., randomly, alternately, etc.) to these RPUs until all RPUs converge to their own symmetry point. Array A and the reference array can be implemented as two interconnected arrays, and their combination forms matrix A. Since physical conductance cannot be negative, the difference between the conductance values ​​in array A and the reference array forms the logical value of matrix A. However, the reference array remains constant after the initial programming steps, and we reference matrix A and array A in an interchangeable manner because array A is updated when matrix A is updated. However, vector-matrix multiplication operations performed on matrix A always use the difference reads of array A and the reference array. The same method and operating principle also apply to matrix W and array W.

[0040] Figure 4This diagram illustrates two interconnected arrays (i.e., a physical array and a reference array) corresponding to matrix A according to an embodiment of the present invention, and a reference array filled with zero-weight conductance values ​​of matrix A. Cross-point array 502 of RPU 404 (filled with weight values ​​(Wij) of matrix A) and cross-point array 406 of RPU 408 (filled with corresponding zero-weight conductance values ​​(W0′) of the reference matrix) are shown. A unique zero-weight value (W0′) can be established for the corresponding RPU in matrix A in each RPU of the reference matrix. Initialization of the hidden matrix H includes setting the current values ​​stored in the matrix to zero, or allocating digital storage space on the connected computing device. Initialization of the weight matrix W includes loading the weight matrix W with random values, so that training processing for the weight matrix W can begin.

[0041] Once the zero-weight conductance values ​​have been stored in the reference array, DNN training is performed. Instead of using backpropagation in just three cycles (i.e., forward cycle, backward cycle, and weight update cycle) to compute the error gradient, the operation is filtered here by a hidden matrix H, which updates the weight values ​​only after the iterative combination of H values ​​grows to a value greater than a threshold.

[0042] Method 300 includes using a weight matrix W to perform a forward loop (box 304). Figure 5 This is a diagram illustrating a forward loop being executed according to an embodiment of the present invention. The forward loop involves calculating vector-matrix multiplication (y = Wx), where the input vector x represents the activity of the input neuron, and the weight matrix W stores the weight values ​​between each pair of input and output neurons. Figure 5 The forward loop vector-matrix multiplication operation is shown to be implemented in the cross-point array 502 of the RPU device, where the conductance values ​​stored in the cross-point array 502 form a matrix.

[0043] The input vector X is transmitted as a voltage pulse through each of the conductive column wirings 512, and the resulting output vector y is read as the current output from the conductive row wirings 510 of the crosspoint array 502. An analog-to-digital converter (ADC) 513 converts the analog output vector 516 from the crosspoint array 502 into a digital signal. Zero-weight conductance values ​​copied from the reference matrix 406 are used to offset the output values ​​of the RPU device in the crosspoint array 502 relative to the symmetry point of the RPU device, to address deviations in its switching behavior and to encode negative logic matrix values. For this purpose, voltage pulses applied to the crosspoint array 502 are also applied to the reference array. The output vector y of the crosspoint array 502 is then subtracted from the output vector y of the reference array.

[0044] Method 300 also includes performing a backward loop on the weight matrix W (box 306). Figure 6This is a diagram illustrating a backward cycle performed according to an embodiment of the present invention. Generally, the backward loop involves calculating an error signal δ and propagating that error signal δ backward through the weighting matrix W via a vector-matrix multiplication on the transpose of the weighting matrix W (i.e., z = W). T δ), where the vector δ represents the error calculated by the output neuron, and the vector z is further processed using the derivative of the neuron's nonlinearity, and then passed down to the previous layer.

[0045] Figure 6 This illustrates a backward loop vector-matrix multiplication operation implemented in the crosspoint array 502. An error signal δ is transmitted as a voltage pulse through each of the conductive row wirings 510, and the resulting output vector z is read as the current output from the conductive column wirings 512 of the crosspoint array 502. When a voltage pulse is supplied from the row wirings 510 as input to the backward loop, the vector-matrix product on the transpose of the weight matrix W is then calculated.

[0046] For example Figure 6 As shown, ADC 513 is used to convert the (analog) output vector 518 from crosspoint array 502 into a digital signal. Similar to the forward loop described above, zero-weight conductance values ​​are transferred relative to their symmetric points to the output values ​​of the RPU devices in crosspoint array 502 to resolve deviations in their switching behavior and to encode negative logic matrix values. For this purpose, a voltage pulse applied to crosspoint array 502 is also applied to the reference array. The output vector Z of crosspoint array 502 is then subtracted from the output vector Z of the reference array.

[0047] Method 300 also includes updating matrix A with input vector x and δ (box 308). Figure 7 This diagram illustrates an array A 502 updated according to an embodiment of the invention, utilizing x propagating in the forward loop and δ propagating in the backward loop. This update is achieved in the crosspoint array 502 by transmitting voltage pulses representing vector x (from the forward cycle) and vector δ (from the backward cycle) simultaneously supplied from the conduction column line 506 and conduction row line 504, respectively. In this configuration, each RPU in the crosspoint array 502 performs local multiplication and summation operations by processing the voltage pulses from the corresponding conduction column line 506 and conduction row line 504, thus achieving incremental weight updates. The forward loop (box 304), the backward loop (box 306), and the update of matrix A with input vectors from the forward and backward loops (box 308) can be repeated multiple times to improve the updated value of matrix A.

[0048] Method 300 also includes using the input vector e i (That is, y′=Ae) iPerform a forward loop on matrix A (box 310). At each time step, use the new input vector e i And sub-index i represents the time index. As will be described in detail below, according to an exemplary embodiment, the input vector e is a one-hot encoded vector. For example, as is known in the art, a one-hot encoded vector is a set of bits that has only a single high (1) bit and all other bits are low (0). For illustrative purposes, and for the purpose of using a simple, non-limiting example, assuming a matrix of size 4×4, a one-hot encoded vector would be one of the following vectors: [1 0 0 0], [0 1 0 0], [0 0 1 0], and [0 0 0 1]. At each time step, a new one-hot encoded vector is used, and sub-index i represents the time index. However, it is worth noting that this document also contemplates a method for selecting the input vector e. i Other methods. For example, input vector e i Alternatively, you can choose from columns of a Hadamard matrix, a random matrix, etc.

[0049] Figure 8 This illustrates the forward loop y′=Ae performed on matrix A according to an embodiment of the present invention. i The diagram shows the input vector e. i As a voltage pulse is transmitted through each of the conductive column wirings 512, the resulting output vector y′ is read as the current output from the conductive row wirings 510 of the crosspoint array 502. When the voltage pulse is supplied as input from the column wirings 512 to this positive cycle, the vector matrix product is calculated.

[0050] Method 300 includes using the output vector y′ and the input vector e. i Update the hidden matrix H (box 312). Figure 9 This is a diagram illustrating the hidden matrix H 902 updated using values ​​computed in the forward cycle of matrix A 904. The hidden matrix H 902 is often a digital matrix (rather than a physical construct like matrix A and the weight matrix W), storing the H value 906 (i.e., Hij) for each RPU 908 in matrix A (i.e., each RPU located at Aij). ij When the forward loop is executed, it produces the output vector y'e. i T The hidden matrix H is then added to this value along with each H value 906. Thus, the hidden matrix H 902 changes each time the output vector is read. For RPUs with low noise levels, the H value 906 will continuously increase, depending on the output vector y'e. i T The value of y'e can increase in either the positive or negative direction. iT Including significant noise, it may be positive for one iteration and negative for another. Positive output vector y'e i T Value and negative output vector y'e i T This combination of values ​​means that the H value of 906 will grow more slowly and inconsistently.

[0051] As the H value 906 increases, method 300 includes tracking whether the H value 906 has increased beyond a threshold (box 314). If at a specific location (i.e., H... ij If the H value 906 at point (box 314 "No") is not greater than the threshold (box 314 "No"), then method 300 repeats by updating the hidden matrix H (box 312), starting from executing the forward loop (box 304). If the H value 906 is greater than the threshold (box 310 "Yes"), then method 300 continues to update the hidden matrix H (box 312) by updating the input vector e. i The data is transmitted to the weight matrix W, but only for a specific RPU (box 316). As mentioned above, the H value 906 can increase in either the positive or negative direction, so the threshold can also be positive or negative.

[0052] Figure 10 This is a schematic diagram of the hidden matrix H902 of the selectively applied weight matrix W1010 according to an embodiment of the present invention. Figure 10 The diagram shows the first H value 1012 and the second H value 1014 that have exceeded the threshold and are passed to the weight matrix W1010. The first H value 1012 reaches the positive threshold and therefore carries a positive value for its row in the input vector 1016: "1". The second H value 1014 reaches the negative threshold and therefore carries a negative one for its row in the input vector 1016: "-1". The remaining rows in the input vector 1016 carry zeros because those values ​​(i.e., H value 906) did not become greater than the threshold. The threshold can be much larger than y'e added to the hidden matrix H. i T For example, the threshold could be y'e i T The expected intensity is ten or one hundred times higher. These high thresholds reduce the frequency of updates performed on the weight matrix W. However, the filtering function performed by the H matrix reduces the error of the neural network's objective function. These updates can only be generated after processing many data examples, and thus also increase the confidence level in the updates. This technique enables the training of neural networks using a noisy RPU device with only a finite number of states. After applying the H value to the weight matrix W, the H value 906 is reset to zero, and iterations of method 300 continue.

[0053] Using e iAfter updating the weight matrix W in step 1018, method 300 continues by determining whether training is complete. If training is not complete, for example, if a certain convergence criterion is not met (box 318 "No"), method 300 restarts by executing the forward loop y = Wx. For example, by way of example only, training can be considered complete when no further improvement to the error signal is observed. When training is complete (box 320 "Yes"), method 300 terminates.

[0054] As described above, according to an exemplary embodiment, the input vector e is a one-hot encoded vector, which is a group of bits that consists only of combinations of a single high (1) bit and all other bits being low (0). See, for example, Figure 11 .like Figure 11 As shown, given a 4×4 matrix, a one-hot encoded vector will be one of the following vectors: [1 0 0 0], [0 1 0 0], [00 1 0], and [0 0 0 1]. At each time step, a new one-hot encoded vector is used, denoted by the sub-index i at that time index. According to another exemplary embodiment, the input vector e is selected from the columns of a Hadamard matrix. As is known in the art, a Hadamard matrix is ​​a square matrix with ±1 terms. For example, see... Figure 12 (Hadamard matrix of order 2) and Figure 13 (A Hadamard matrix of order 4). This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.

[0055] Computer-readable storage media can be tangible means for retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital universal disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0056] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0057] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may be personalized to execute computer-readable program instructions by utilizing state information from the computer-readable program instructions in order to perform aspects of this invention.

[0058] The present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0059] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other apparatus to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0060] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other apparatus to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other apparatus to produce computer-implemented processing, such that the instructions executed on the computer, other programmable apparatus, or other apparatus perform the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0061] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a non-linear order. For example, depending on the function involved, two consecutively shown blocks may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0062] Turn now Figure 14 The diagram illustrates a block diagram of an apparatus 1400 for implementing one or more methods proposed herein. By way of example only, the apparatus 1400 may be configured to control input voltage pulses applied to the array and / or process output signals from the array.

[0063] Device 1400 includes a computer system 1410 and removable media 1450. Computer system 1410 includes a processor 1420, a network interface 1425, memory 1430, a media interface 1435, and an optional display 1440. Network interface 1425 allows computer system 1410 to connect to a network, while media interface 1435 allows computer system 1410 to interact with media such as a hard drive or removable media 1450.

[0064] Processor device 1420 can be configured to implement the methods, steps, and functions disclosed herein. Memory 1430 can be distributed or local, and processor device 1420 can be distributed or single. Memory 1430 can be implemented as electrical memory, magnetic memory, or optical memory, or any combination of these or other types of storage devices. Furthermore, the term "memory" should be interpreted broadly enough to include any information that can be read from or written to an addressable space accessible by processor device 1420. Using this definition, information about a network accessible via network interface 1425 remains within memory 1430, as processor device 1420 can retrieve that information from the network. It should be noted that each distributed processor comprising processor device 1420 typically contains its own addressable memory space. It should also be noted that some or all of the computer system 1410 can be integrated into application-specific or general-purpose integrated circuits.

[0065] Optional display 1440 is any type of display suitable for human user interaction with device 1400. Typically, display 1440 is a computer monitor or other similar display.

[0066] While illustrative embodiments of the invention have been described herein, it should be understood that the invention is not limited to those precise embodiments, and various other changes and modifications can be made by those skilled in the art without departing from the scope of the invention.

Claims

1. A method of training a deep neural network (DNN), the method comprising: initializing a state of each element of an A matrix; updating each element of the A matrix to determine an incremental weight update; storing elements of a hidden matrix, wherein each element of the hidden matrix corresponds to a sum of the incremental weight updates of one of the elements of the A matrix; updating a corresponding element of a weight matrix W based on a corresponding element of the hidden matrix reaching a threshold state, wherein updating each element of the A matrix comprises transferring an error signal d through conductive row wires and reading a resulting output vector z as a current output from conductive column wires.

2. The method of claim 1, wherein, the incremental weight update comprises a matrix multiplication of an output vector and an input vector operating on the A matrix.

3. The method of claim 1, wherein, updating the weight matrix W comprises sending a voltage pulse through a conductive column wire of the weight matrix W simultaneously with a sign information of the element of the hidden matrix reaching the threshold state as a voltage pulse through a conductive row wire of the weight matrix W when the element of the hidden matrix reaches the threshold state.

4. The method of claim 1, wherein, the threshold state comprises ten times an average of the incremental weight updates.

5. The method of claim 1, comprising: resetting the corresponding element of the hidden matrix to zero when the corresponding element of the weight matrix W is updated.

6. A method for forming a deep neural network DNN, the method comprising: forming an A matrix comprising resistance processing unit RPU devices separating crossings between conductive row wires and conductive column wires, wherein the resistance processing unit RPU devices comprise processed gradients of weighted connections between neurons in the deep neural network DNN; forming a weight matrix W comprising resistance processing unit RPU devices separating crossings between conductive row wires and conductive column wires, whereby the resistance processing unit RPU devices comprise weighted connections between neurons in the deep neural network DNN; and forming a hidden matrix comprising H values of each resistance processing unit RPU device in the weight matrix W, wherein the H values of the each resistance processing unit RPU device are iteratively added to values from an A matrix until at least one H value is above a threshold value.

7. The method of claim 6, comprising a reference array interconnected with the crosspoint array, the reference array storing zero weight conductance values of the resistance processing unit RPU devices in the crosspoint array.

8. The method of claim 6, wherein, the hidden matrix H comprises digital stored values of each resistance processing unit RPU device in the weight matrix W.

9. A computer program product, comprising: one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions comprising instructions for: initializing a state of each element of an A matrix; updating each element of the A matrix to determine an incremental weight update; storing elements of a hidden matrix, wherein each element of the hidden matrix corresponds to a sum of the incremental weight updates of one of the elements of the A matrix; updating a corresponding element of a weight matrix W based on a corresponding element of the hidden matrix reaching a threshold state, wherein the instructions for updating each element of the A matrix include instructions for transmitting an error signal δ through a conductive row wire and reading a resulting output vector z as a current output from a conductive column wire.

10. The computer program product of claim 9, wherein, The delta weight update includes a matrix multiplication of an output vector and an input vector operated on by the A matrix.

11. The computer program product of claim 9, wherein, The instructions for updating the weight matrix W include instructions for transmitting a voltage pulse through a conductive column wire of the weight matrix W simultaneously with sign information of an element of the hidden matrix that reached the threshold state as a voltage pulse through a conductive row wire of the weight matrix W when the element of the hidden matrix reached the threshold state.

12. The computer program product of claim 9, wherein, The threshold state includes ten times an average value of the delta weight update.

13. The computer program product of claim 9, wherein, The program instructions include instructions for resetting a corresponding element of the hidden matrix to zero when a corresponding element of the weight matrix W is updated.

14. A computer-implemented method for training a deep neural network (DNN), comprising: An input vector e is transmitted as voltage pulses via the conductive column wires of the A matrix i and the resulting output vector y' is read as current output from the conductive row wires of the A matrix, wherein the A matrix comprises resistive processing unit RPU devices separating the cross-points between the conductive column wires and the conductive row wires; by iteratively adding the output vector y' multiplied by the transpose of the input vector e i to each H value, wherein the hidden matrix comprises the H values of each resistive processing unit RPU; and When the H value reaches the threshold value, the input vector e i The H value that reaches the threshold value is transmitted as a voltage pulse through the conductive column wiring of the weight matrix W, and the sign information of the H value that reaches the threshold value is transmitted as a voltage pulse through the conductive row wiring of the weight matrix W.

15. The method of claim 14, wherein, deriving the input vector and error signal from a forward pass and a backward pass operating on the weight matrix W.

16. The method of claim 14, comprising: transmitting an input vector x as a voltage pulse through a conductive column wire of the weight matrix W and reading a resulting output vector y as a current output from a conductive row wire of the weight matrix W, wherein the weight matrix W includes a resistive processing unit (RPU) device separating a cross point between the conductive column wire of the weight matrix W and the conductive row wire of the weight matrix W; and before transmitting the input vector e i The A matrix is updated before transmitting the input vector e by transmitting an error signal δ calculated from the output vector y as a voltage pulse via the conductive row wiring and reading the resulting output vector z as a current output from the conductive column wiring.

17. The method of claim 14, wherein, The input vector e i includes a one-hot encoded vector.

18. The method of claim 14, wherein, The input vector e i selected from Hadamard matrices.

19. The method of claim 14, wherein, digitally storing the H values.

Images