Hidden matrix training DNN filtering

Abstract

In one aspect, a method for training a DNN includes sending an input vector x through a weight matrix W and reading the resulting output vector y; sending an error signal δ; sending the input vector x with the error signal δ through a conductive row line of a matrix A; and sending the input vector e i The training also involves transmitting the input vector e i The training also includes updating the hidden matrix H containing the H values ​​of the RPU device by iteratively adding the output vector y′ multiplied by the transpose of y′ to each H value. The training also includes transmitting the sign information of the H value that has reached the threshold as a voltage pulse through the conductive row lines of the weight matrix W when the H value reaches a threshold, and simultaneously transmitting the input vector e i as a voltage pulse.

Description

[Technical field]

[0001] The present invention relates generally to deep neural network (DNN) training, and more particularly to techniques for filtering signals from resistive processing unit (RPU) devices to reduce noise. [Background technology]

[0002] A deep neural network (DNN) can be embodied in an analog cross-point array of resistive devices, such as a resistive processing unit (RPU). The RPU device generally comprises a first terminal, a second terminal, and an active area. The conductance state of the active area identifies the weight value of the RPU, which can be updated / adjusted by applying a signal to the first terminal / second terminal.

[0003] DNN-based models have been used for a variety of different cognitive-based tasks, such as object and speech recognition, as well as natural language processing. Training of DNNs is necessary to provide a high level of accuracy when performing such tasks. Training large-scale DNNs is a computationally intensive task. Most of the common methods of DNN training, such as backpropagation and stochastic gradient decent (SGD), require the RPU to be “symmetric” to operate accurately. Symmetric analog resistive devices change their conductance symmetrically when subjected to positive and negative voltage pulses. However, in practice, RPU devices may exhibit nonlinear and asymmetric switching characteristics. For example, when voltage pulses are applied to adjust weights up and down, there is often an imbalance between the up and down adjustments. Summary of the Invention [Means for solving the problem]

[0004] The present invention provides a technique for training a deep neural network (DNN). The method includes transmitting an input vector x as a voltage pulse through conductive column lines of a weight matrix W, and reading a resulting output vector y as a current output from conductive row lines of the weight matrix W. The weight matrix W may include a resistive processing unit (RPU) device that separates the crossings between the conductive column lines and the conductive row lines, such that the RPU device serves as a weighted connection between neurons in the DNN. The training may further include transmitting an error signal δ calculated from the output vector y as a voltage pulse through the conductive row lines, and reading a resulting output vector z as a current output from the conductive column lines. The training may further include transmitting the input vector x as a voltage pulse through conductive column lines of the matrix A simultaneously with the error signal δ through conductive row lines of the matrix A. The training may further include transmitting an input vector e through the conductive column lines of the matrix A as a voltage pulse. i as voltage pulses and reading the resulting output vector y′ as current outputs from the conductive column lines of the cross-point array A. i The output vector y' multiplied by the transpose of ij The training may further include updating a hidden matrix H with an H value for each RPU by iteratively adding to the input vector e t , the input vector e t , and ... i as a voltage pulse.

[0005] One embodiment of the present invention also includes a deep neural network (DNN). The DNN includes a matrix A including resistive processing unit (RPU) devices that separate intersections between conductive row lines and conductive column lines, where the RPU devices include processed gradients for weighted connections between neurons in the DNN. The DNN may also include a weight matrix W consisting of RPU devices that separate intersections between conductive row lines and conductive column lines, where the RPU devices include weighted connections between neurons in the DNN. The DNN may further include a hidden matrix H including an H value for each RPU device in the weight matrix W. The H value may iteratively add values ​​from the matrix A until at least one H value exceeds a threshold.

[0006] A more complete understanding of the present invention, as well as further features and advantages of the present invention, will be obtained by reference to the following detailed description and drawings.

[0007] One embodiment of the invention will now be described, by way of example only, with reference to the accompanying drawings. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a deep neural network (DNN) implemented in an analog crosspoint array of a resistive processing unit (RPU) device according to one embodiment of the present invention. [Figure 2A] FIG. 2A illustrates the ideal switching characteristics of a linear and symmetric RPU device according to one embodiment of the present invention. [Figure 2B] FIG. 2B illustrates the non-ideal switching characteristics of a non-linear and asymmetric RPU device according to one embodiment of the present invention. [Diagram 3] FIG. 3 illustrates an exemplary methodology for training a DNN, according to one embodiment of the present invention. [Figure 4]FIG. 4 is a diagram showing two interconnected arrays (i.e., array W and a reference array) corresponding to matrix W, where the reference array is populated with conductance values ​​corresponding to the zero weight values ​​of matrix W, in accordance with one embodiment of the present invention. [Diagram 5] FIG. 5 illustrates a forward cycle y=Wx performed according to one embodiment of the present invention. [Figure 6] FIG. 6 is a diagram illustrating a backward cycle z=WTδ performed according to one embodiment of the present invention. [Figure 7] FIG. 7 is a diagram illustrating an array A being updated with χ propagated in the forward cycle and δ propagated in the backward cycle according to one embodiment of the present invention. [Figure 8] FIG. 8 is a diagram illustrating a forward cycle y′=Aei performed on a weight matrix according to one embodiment of the present invention. [Figure 9] FIG. 9 illustrates how the hidden matrix H is updated with the values ​​calculated in the forward cycle of the matrix A. [Figure 10] FIG. 10 is a schematic diagram illustrating that hidden matrix H 902 is selectively applied to weight matrix W 1010 according to one embodiment of the present invention. [Figure 11] FIG. 11 is a diagram illustrating an exemplary one hot encoded vector according to one embodiment of the present invention. [Figure 12] FIG. 12 is a diagram illustrating an exemplary Hadamard matrix of order 2, in accordance with one embodiment of the present invention. [Figure 13] FIG. 13 is a diagram illustrating an exemplary Hadamard matrix of order 4, in accordance with one embodiment of the present invention. [Figure 14] FIG. 14 illustrates an exemplary apparatus that can be used in implementing one or more of the present techniques, in accordance with one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A deep neural network (DNN) training technique using an asymmetric resistive processing unit (RPU) device is provided herein.As highlighted above, the DNN can be embodied in an analog cross-point array of RPUs.See, for example, the schematic diagram shown in FIG.

[0010] As shown in FIG. 1, each parameter (weight w ij ) is a single RPU device on the hardware (RPU ij ), i.e., a physical cross-point array 104 of the RPU device. The cross-point array 104 comprises a series of conductive row lines 106 and a series of conductive column lines 108 oriented to be orthogonal and intersect the conductive row lines 106. The intersections between the row lines 106 and the column lines 108 are separated by RPUs 110 forming the cross-point array 104 of the RPU device. Each RPU may comprise a first terminal, a second terminal, and an active area. The conductive state of the active area identifies the weight value of the RPU, which can be updated / adjusted by application of a signal to the first / second terminals. Furthermore, a three terminal (or more terminal) device can effectively function as a two terminal resistive memory device by controlling the extra terminal.

[0011] Each RPU (RPU ij ) is uniquely identified based on its location in the cross-point array 104 (i.e., the i-th row and the j-th column). For example, working from top to bottom and left to right in the cross-point array 104, the RPU at that intersection of the first row line 106 and the first column line 108 is RPU 11 and the RPU at the intersection of the first row line 106 and the second column line 108 is RPU 12, etc. Furthermore, the mapping of the parameters of weight matrix 102 to the RPUs of cross-point array 104 follows the same convention. For example, the weights w i1 is the RPU of the crosspoint array 104 i1 and the weights w i2 is the RPU of the crosspoint array 104 i2 , and so on.

[0012] The RPUs 110 of the cross-point array 104 essentially function as weighted connections between neurons in the DNN. The conduction state (e.g., resistance) of the RPUs 110 can be changed by controlling the voltages applied between the individual row lines 106 and column lines 108, respectively. Data is stored by changing the conduction state of the RPUs. The conduction state of the RPUs 110 is read by applying a voltage and measuring the current passing through the RPU 110 of interest. All operations involving weights are performed by the RPUs 110 in full parallel.

[0013] In machine learning and cognitive science, DNN-based models are a family of statistical learning models inspired by biological neural networks of animals, especially the brain. These models can be used to estimate or approximate systems and cognitive functions that depend on many inputs and generally unknown connection weights. DNNs are often embodied as so-called "neuromorphic" systems of interconnected processor elements that act as simulated "neurons" that exchange "messages" with each other in the form of electronic signals. The connections in DNNs that transmit electronic messages between the simulated neurons are equipped with numerical weights that correspond to the strength or weakness of a given connection. These numerical weights can be adjusted and tuned based on experience, allowing the DNN to adapt to and learn from the inputs. For example, a DNN for handwriting recognition is defined by a set of input neurons that can be activated by pixels of an input image. After being weighted and transformed by a function determined by the network's designer, the activations of these input neurons are passed on to other downstream neurons. This process is repeated until an output neuron is activated. The activated output neurons determine which letter was read.

[0014] FIG. 2A is a diagram illustrating the ideal switching characteristics of an RPU device, which is linear and symmetrical according to one embodiment of the present invention. As shown in FIG. 2A, an ideal RPU responds linearly and symmetrically to an external voltage stimulus. For training purposes, such an ideal device fully implements the DNN training process of backpropagation and stochastic gradient descent (SGD). Backpropagation is a learning process that repeats three cycles, i.e., forward cycle, backward cycle, and weight update cycle, multiple times until a convergence criterion is met. Stochastic gradient descent (SGD) uses backpropagation to iterate over each parameter (weight w ij ) error gradient.

[0015] To perform backpropagation, DNN-based models consist of multiple processing layers that learn representations of data with multiple levels of abstraction. For a single processing layer with N input neurons connected to M output neurons, the forward cycle computes vector-matrix multiplication (y=Wx), where vector x of length N represents the activity of the input neurons, and matrix W of size M×N stores the weight values ​​between each pair of input and output neurons. The resulting vector y of length M is further processed by performing nonlinear activation in each of the resistive memory elements and then passed to the next layer.

[0016] Once the information reaches the final output layer, the backward cycle involves computing the error signal and backpropagating the error signal through the DNN. The backward cycle in a single layer also involves computing the weight matrix (z=W T The method involves vector-matrix multiplication on the transpose (swapping each row with its corresponding column) of n (δ), where the length M vector δ represents the error computed by the output neurons, and the length N vector z is further processed using the derivative of the neuron's nonlinearity and then passed back to the previous layer.

[0017] Finally, in the weight update cycle, the weight matrix W is updated by performing a cross product of the two vectors used in the forward and backward cycles. This cross product of the two vectors is often expressed as W←W+η(δχ T ), where η is the global learning rate.

[0018] All operations performed on the weight matrix W during this backpropagation process can be implemented in the crosspoint array 104 of the RPU 110 with a corresponding number of M rows and N columns, where the stored conductance values ​​in the crosspoint array 104 form the matrix W. In the forward cycle, an input vector x is sent as a voltage pulse through each of the column lines 108, and the resulting vector y is read as a current output from the row lines 106. Similarly, when a voltage pulse is provided from the row lines 106 as an input to the backward cycle, a vector-matrix product is performed to obtain the weight matrix W. T Finally, in the update cycle, voltage pulses representing vectors x and δ are simultaneously provided from column lines 108 and row lines 106. In this configuration, each RPU 110 performs local multiplication and summation operations by processing the voltage pulses coming from the corresponding column lines 108 and row lines 106, thus achieving incremental weight updates.

[0019] As highlighted above, a symmetric RPU (see FIG. 2A) fully implements backpropagation and SGD. That is, such an ideal RPU has w ij ←w ij +ηΔw ij where w ij is the weight value of the i-th row and j-th column of the cross point array 104.

[0020] In contrast, FIG. 2B illustrates the non-ideal switching characteristics of an RPU device, which is non-linear and asymmetric, according to one embodiment of the present invention. As shown in FIG. 2B, a real RPU can respond non-linearly and asymmetrically to an external voltage stimulus. That is, when applying a voltage pulse to adjust the weights up or down (referred to herein as an "up" pulse and a "down" pulse, respectively), the weight update amount (i.e., the step size) is linearly dependent on the current weight value. Thus, there is only a single weight value (corresponding to a zero weight value from the zero-shift technique described in detail below) where the up and down adjustments are equal in strength, but are otherwise unbalanced for the rest of the weight range. Assuming that the RPU device is given a set of sequences of up and down pulses, this imbalance means that the device has a tendency to go up or down toward a symmetric point where the strength of the up and down pulses is equal. This device behavior can be translated into an additional energy term (internal energy) that comes from the physics of the RPU device that governs the conductance change in the RPU device. Therefore, when used for backpropagation, these RPUs with non-ideal switching characteristics implement something quite different from the ideal case, i.e.

number

[0021] It turns out that in a resistive memory device, such as RPU 110, there is always a single symmetrical point where the slope of the up pulse and the slope of the down pulse of each device are exactly the same. This symmetrical point (which can be different for each individual RPU) can be assigned a weight value of 0.

[0022] The symmetry point for each RPU 110 in the cross point array 104 is determined by repeatedly (first, second, third, etc.) applying up and down voltage pulses to the RPUs 110 in the weight array until all of the RPUs 110 in the weight array converge to their own symmetry points. For example, if the conductance range is Gmin to Gmax, and the average delta G at one update incident is dGavg, then the number of valid states in the conductance range is (Gmax-Gmin) / dGavg. If the device is at an extreme point, and alternating up / down pulses are given, it takes # of updates to reach the center point = (Gmax-Gmin) / dGavg. To ensure convergence, for example, additional cycles of alternating up / down pulses can be given, i.e., n x (Gmax-Gmin) / dGavg, where n = 1 or more. The up and down voltage pulses can be applied to RPU 110 randomly (i.e., each pulse is randomly either an up pulse or a down pulse), in an alternating manner (i.e., if the preceding pulse is an up pulse, then the next pulse is a down pulse, and vice versa), etc.

[0023] Once all of the RPUs 110 in the weight array have converged to their own symmetrical points, the conductance values ​​for each RPU 110 from the weight array (at the symmetrical points) are copied to another reference array interconnected with the weight array. A one-to-one correlation exists between the devices in the weight array and the devices in the reference array, so that a unique zero weight value can be established in each device in the reference array for a corresponding one device in the weight array. Thus, during operation, the output of the RPUs 110 in the weight array is shifted by the zero weight value from the corresponding device stored in the reference array. For example, the same signal can be provided to both the weight array and the reference array. Then, the outputs (e.g., I1, I2, I3, etc.) from the RPUs 110 in the weight array can be zero-shifted by subtracting their values ​​from the output of the corresponding device in the reference array (set to zero weight value) to achieve a zero-shifted result.

[0024] After this initial setup to achieve zero-shifted results, the technique is used to train the DNN. FIG. 3 illustrates an exemplary method 300 for training a DNN, according to one embodiment of the present invention. During training, weight updates are first stored on a matrix A, which is a hardware component made up of rows and columns of the RPU with symmetric behavior around the zero point. Then, the weight updates from matrix A are selectively moved to a weight matrix W, which is also a hardware component made up of rows and columns of the RPU. The learning process selects a set of parameters (weights w ij) is iteratively determined. During initialization, the reference array with zero weight values ​​ensures that each RPU in the weight matrix A is at an equilibrium point for a value corresponding to zero. Meanwhile, the matrix W is initialized to randomly distributed values ​​using common practices applied to training DNNs. The hidden matrix H (generally stored digitally, but some embodiments use an analog hidden matrix H) is initialized to zero. During training, the weight updates are performed in the matrix A. The information processed by the matrix A is then stored in the hidden matrix H (a separate matrix that effectively performs a low pass filter). The values ​​of the hidden matrix H that reach an update threshold are then applied to the weight matrix W. The update threshold effectively minimizes the noise generated in the hardware of the matrix A. Advantageously, this weight matrix W is used to minimize the objective function of the neural network, and the matrix A is used to process the gradient of the objective function and then pass it through a digital low pass filter before feeding it back to W. Therefore, hardware biases and noise associated with a non-ideal RPU are tolerated (or absorbed by the hidden matrix H), thus resulting in lower test errors even for small numbers of states compared to standard SGD techniques or other training techniques that use asymmetric devices.

[0025] The method 300 begins by initializing matrix A, hidden matrix H, and weight matrix W (block 302). Initializing matrix A includes, for example, determining the symmetry points of each RPU in matrix A and storing the corresponding conductance values ​​in a reference array. As provided above, the symmetry points for each RPU device can be determined by repeatedly applying up and down voltage pulses (e.g., random, alternating, etc.) to the RPU until all of the RPUs converge to their own symmetry points. Array A and the reference array can be embodied in two interconnected arrays, and their combination forms matrix A. Since physical conductance cannot be a negative quantity, the difference between the conductance values ​​in array A and the conductance values ​​in the reference array forms the logic value for matrix A. However, the reference array is kept constant after the initial programming step, and since it is array A that is updated when matrix A is updated, matrix A and array A are referred to in an interchangeable manner. However, the differential readings of array A and the reference array are always used for vector-matrix multiplication operations performed on matrix A. The same methodology and principles of operation apply for matrix W and array W.

[0026] 4 is a diagram illustrating two interconnected arrays (i.e., a physical array and a reference array) corresponding to matrix A and a reference array populated with zero-weight conductance values ​​of matrix A, according to one embodiment of the present invention. ij ) and the crosspoint array 406 of the RPU 408 (populated with the corresponding zero-weight conductance values ​​(w0′) of the reference matrix). 0′) may be established in each RPU in the reference matrix for one corresponding RPU in matrix A. Initializing the hidden matrix H includes zeroing out the current values ​​stored in the matrix or allocating digital storage space on the connected computing device. Initializing the weight matrix W includes loading random values ​​into the weight matrix W so that the training process of the weight matrix W can begin.

[0027] Once the zero weight conductance values ​​are stored in the reference array, training of the DNN is performed. In addition to calculating the error gradient using backpropagation in three cycles (i.e., forward cycle, backward cycle, and weight update cycle), the operation is now filtered by a hidden matrix H that updates the weight values ​​only after a repeated combination of that H value grows larger than a threshold.

[0028] The method 300 includes performing a forward cycle (block 304) with a weight matrix W. FIG. 5 illustrates a forward cycle performed according to one embodiment of the present invention. The forward cycle includes computing a vector-matrix multiplication (y=Wx) where an input vector x represents the activity of an input neuron, and a weight matrix W stores the weight values ​​between each pair of input and output neurons. FIG. 5 illustrates that the vector-matrix multiplication operation of the forward cycle is implemented in a cross-point array 502 of an RPU device, where the conductance values ​​stored in the cross-point array 502 form a matrix.

[0029] An input vector x is sent as a voltage pulse through each of the conductive column lines 512, and the resulting output vector y is read as a current output from the conductive row lines 510 of the cross-point array 502. An analog-to-digital converter (ADC) 513 is used to convert the analog output vector 516 from the cross-point array 502 to a digital signal. The zero-weighted conductance values ​​copied from the reference matrix 406 are used to shift the output values ​​of the RPU devices in the cross-point array 502 relative to their symmetrical points to account for biases in their switching behavior, and to encode negative logic matrix values. To do so, the voltage pulses applied to the cross-point array 502 are also applied to the reference array. The output vector y of the cross-point array 502 is then subtracted from the output vector of the reference array.

[0030] The method 300 also includes performing a backward cycle on the weight matrix W (block 306). FIG. 6 illustrates a backward cycle performed in accordance with one embodiment of the invention. In general, the backward cycle includes computing an error signal δ and performing a vector-matrix multiplication on the transpose of the weight matrix W (i.e., z=W T The method involves backpropagating the error signal δ through the output neuron δ, where the vector δ represents the error computed by the output neuron, and the vector z is further processed using differentiation of the neuron nonlinearity and then passed back to the previous layer.

[0031] 6 shows that the backward-cycle vector-matrix multiplication operation is implemented in cross-point array 502. The error signal δ is sent as a voltage pulse through each of the conductive row lines 510, and the resulting output vector z is read as a current output from conductive column lines 512 of cross-point array 502. When a voltage pulse is provided from row lines 510 as an input for the backward cycle, the vector-matrix product is calculated on the transpose of weight matrix W.

[0032] As also shown in FIG. 6, ADC 513 is used to convert the (analog) output vector 518 from the crosspoint array 502 into a digital signal. Similar to the forward cycle described above, the zero weight conductance value shifts the output values ​​of the RPU devices in the crosspoint array 502 relatively to their symmetrical points to account for the bias in their switching behavior, and encodes a negative logic matrix value. To do so, the voltage pulse applied to the crosspoint array 502 is also applied to the reference array. The output vector z of the crosspoint array 502 is then subtracted from the output vector of the reference array.

[0033] The method 300 also includes updating matrix A with input vectors x and δ (block 308). FIG. 7 illustrates an array A 502 updated with x propagated in a forward cycle and δ propagated in a backward cycle according to an embodiment of the present invention. The update is implemented in the cross-point array 502 by sending voltage pulses representing vector x (from the forward cycle) and vector δ (from the backward cycle) simultaneously fed from conductive column lines 506 and conductive row lines 504, respectively. In this configuration, each RPU in the cross-point array 502 performs local multiplication and summation operations by processing the voltage pulses coming from the corresponding conductive column lines 506 and conductive row lines 504, thus achieving incremental weight updates. The forward cycle (block 304), backward cycle (block 306), and update of matrix A with input vectors from the forward cycle and the backward cycle (block 308) may be repeated multiple times to improve the updated values ​​of matrix A.

[0034] The method 300 also includes the step of: i (i.e., y' = Ae i) to perform a forward cycle on matrix A (block 310). At each time step, a new input vector e i is used, and the subindex i indicates its time index. As will be described in more detail below, according to an exemplary embodiment, the input vector e i is a one-hot encoded vector. For example, as known in the art, a one-hot encoded vector is a set of bits that has only a single high (1) bit and all other bits are low (0) combinations. To use a simple non-limiting example for illustrative purposes, assuming a matrix A of size 4×4, the one-hot encoded vector would be one of the following vectors: [1 0 0 0], [0 1 0 0], [0 0 1 0], [0 0 0 1]. At each time step, a new one-hot encoded vector is used, and the sub-index i indicates its time index. However, in this specification, the input vector e i It should be noted that other methods for selecting input vector e are contemplated. For example, i It is noteworthy that can alternatively be selected from columns of a Hadamard matrix, a random matrix, etc.

[0035] FIG. 8 illustrates a forward cycle y′=Ae performed on a matrix A in accordance with an embodiment of the present invention. i 1 is a diagram showing an input vector e i is sent as a voltage pulse through each of the conductive column lines 512, and the resulting output vector y′ is read as a current output from the conductive row lines 510 of the cross point array 502. As voltage pulses are provided from the column lines 512 as inputs to the forward cycle, a vector-matrix product is calculated.

[0036] The method 300 calculates an output vector y′ and an input vector e i9 illustrates that hidden matrix H 902 is updated with values ​​calculated in the forward cycle of matrix A 904. Hidden matrix H 902 is updated in most cases with the values ​​calculated in the forward cycle of matrix A 904 for each RPU 908 in matrix A (i.e., A ij For each RPU located at ij ) is stored in a digital matrix (not in a physical device like matrix A and weight matrix W). When the forward cycle is performed, the output vector y'e i T Then, the hidden matrix H adds its value to each H value 906. Thus, every time the output vector is read, the hidden matrix H 902 changes. For an RPU with a low noise level, the H value 906 grows steadily. The growth of the value is due to the fact that the output vector y′e i T Depending on the value of , it can be in the positive or negative direction. i T If y contains significant noise, it is possible that y′e is positive for one iteration and negative for another. i T This combination of values ​​means that the H value 906 will grow more slowly and more consistently.

[0037] As the H value 906 grows, the method 300 includes tracking (block 314) whether the H value 906 becomes greater than a threshold value. ij If the H value 906 is greater than the threshold (block 314, “yes”), the method 300 repeats from performing the forward cycle (block 304) to updating the hidden matrix H (block 312). If the H value 906 is greater than the threshold (block 314, “yes”), the method 300 repeats from performing the forward cycle (block 304) to updating the hidden matrix H (block 312). i to the weight matrix W, but only for the particular RPU (block 316). As discussed above, the growth of the H value 906 can be in a positive or negative direction, and therefore the threshold is also a positive or negative value.

[0038] FIG. 10 is a schematic diagram of the selective reapplication of hidden matrix H 902 to weight matrix W 1010 according to an embodiment of the present invention. FIG. 10 shows that a first H value 1012 and a second H value 1014 that reach above a threshold are sent to weight matrix W 1010. The first H value 1012 reaches a positive threshold, and therefore a positive threshold value is sent: "1" for that row in the input vector 1016. The second H value 1014 reaches a negative threshold, and therefore a negative threshold value is sent: "-1" for that row in the input vector 1016. The remaining rows in the input vector 1016 have zeros since their values ​​(i.e., H values ​​906) are not greater than the threshold. The thresholds are added to the hidden matrix H by y'e i T For example, the threshold may be much larger than y'e i T , which may be 10 or 100 times the expected strength of . These high thresholds reduce the frequency of updates performed on the weight matrix W. However, the filtering function performed by the hidden matrix H reduces the error in the neural network's objective function. These updates are generated only after processing many data examples, thus also increasing the confidence in the updates. This approach allows training neural networks on noisy RPU devices that have only a limited number of states. After the H value is applied to the weight matrix W, the H value 906 is reset to zero and the iterations of method 300 continue.

[0039] The weight matrix W is e iAfter updating at 1018, method 300 continues by determining whether training is complete. If the training is not complete, e.g., if some convergence criteria is not met (block 318 "NO"), method 300 repeats from the beginning by performing a forward cycle y=Wx again. For example, and by way of example only, the training may be considered complete if no further improvement in the error signal is observed. If the training is complete (block 318 "YES"), method 300 ends.

[0040] As highlighted above, according to an exemplary embodiment, the input vector e i is a one-hot encoded vector, which is a set of bits that has only bit combinations with a single high (1) bit and all other bits low (0) (see, for example, FIG. 11). Given a matrix of size 4×4 as shown in FIG. 11, the one-hot encoded vector may be any one of the following vectors: [1 0 0 0], [0 1 0 0], [0 0 1 0], [0 0 0 1]. At each time step, a new one-hot encoded vector is used, and the sub-index i is indicated by its time index. According to another exemplary embodiment, an input vector e i are selected from the columns of a Hadamard matrix. As known in the art, a Hadamard matrix is ​​a square matrix with entries ±1 (see, for example, FIG. 12 (Hadamard matrix of order 2) and FIG. 13 (Hadamard matrix of order 4)).

[0041] The present invention may include a system, method, or computer program product, or any combination thereof. The computer program product may comprise one or more computer-readable storage media having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0042] The computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of the computer-readable storage medium includes the following: a portable computer diskette; (登録商標) , hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or raised structures in grooves in which instructions are stored, or any suitable combination of these. As used herein, the computer-readable storage medium should not be construed to be a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a wave guide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted over an electrical wire.

[0043] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing device / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may be comprised of copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing device / processing device receives the computer-readable program instructions from the network and transmits the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing device / processing device.

[0044] The computer readable program instructions for carrying out the operations of the present invention may be either assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, such as object-oriented programming languages, such as Smalltalk, C++, and the like, conventional procedural programming languages ​​(e.g., the "C" programming language or similar programming languages). The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, partially on the user's computer as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any kind of network, such as a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry to perform aspects of the invention.

[0045] Aspects of the present invention are described herein with reference to flowchart illustrations or block diagrams of a method, an apparatus (system), and a computer program product or computer program product, or a combination thereof, according to one embodiment of the present invention. It will be understood that each block of the flowchart illustrations or block diagrams or combinations thereof, and combinations of blocks in the flowchart illustrations or block diagrams or combinations thereof, can be implemented by computer readable program instructions.

[0046] These computer readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / operations specified in one or more blocks of the flowchart illustrations or block diagrams, or combinations thereof, to produce a machine. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer programmable data processing apparatus or other device, or combinations thereof, to function in a particular manner, such that a computer readable storage medium having stored thereon includes an article of manufacture including instructions that implement aspects of the functions / operations specified in one or more blocks of the flowchart illustrations or block diagrams, or combinations thereof.

[0047] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device such that the instructions, which execute on a computer, other programmable data processing apparatus, or other device, implement the functions / acts identified in one or more blocks of the flowchart illustrations or block diagrams, or combinations thereof, to cause the computer, other programmable apparatus, or other device to perform a series of operational steps to generate a computer-implemented process.

[0048] The flowchart diagrams and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products or computer programs according to various embodiments of the present invention. In this regard, each block in the flowchart diagrams or block diagrams may represent a module, segment, or portion of an instruction, which includes one or more executable instructions for implementing one or more specified logical functions. In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the drawings. For example, two blocks shown in succession may actually be accomplished as one step performed simultaneously, substantially simultaneously, partially or fully in a time-overlapping manner, depending on the functions involved, or the blocks may be performed in the reverse order. It should be noted that each block of the block diagrams or flowchart diagrams or combinations thereof, as well as combinations of multiple blocks of the block diagrams or flowchart diagrams or combinations thereof, may be implemented by a special purpose hardware-based system that performs the specified functions or operations, or may execute a combination of special purpose hardware and computer instructions.

[0049] 14, there is shown a block diagram of an apparatus 1400 for implementing one or more of the methodologies presented herein. By way of example only, the apparatus 1400 can be configured to control input voltage pulses applied to the array, or to process output signals from the array, or to both control input voltage pulses applied to the array and process output signals from the array.

[0050] The apparatus 1400 includes a computer system 1410 and a removable media 1450. The computer system 1410 includes a processor device 1420, a network interface 1425, memory 1430, a media interface 1435, and an optional display 1440. The network interface 1425 allows the computer system 1410 to connect to a network, and the media interface 1435 allows the computer system 1410 to interact with media, such as a hard drive or removable media 1450.

[0051] The processor device 1420 can be configured to implement the methods, steps, and functions disclosed herein. The memory 1430 can be distributed or local, and the processor device 1420 can be distributed or unitary. The memory 1430 can be implemented as an electrical memory, a magnetic memory, an optical memory, or any combination of these or other types of storage devices. Moreover, the word "memory" should be interpreted broadly enough to encompass any information that can be read from or written to an address in an addressable space accessed by the processor device 1420. In this definition, information on a network accessible through the network interface 1425 is still in the memory 1430 because the processor device 1420 can obtain the information from the network. It should be noted that each distributed processor that constitutes the processor device 1420 typically includes its own addressable memory space. It should also be noted that part or all of the computer system 1410 can be incorporated into an application specific or application general integrated circuit.

[0052] Optional display 1440 is any type of display suitable for interacting with a human user of device 1400. Typically, display 1440 is a computer monitor or other similar display.

[0053] Although exemplary embodiments of the present invention have been described herein, it is to be understood that the invention is not limited to those precise embodiments, and that various other changes and modifications may be made by those skilled in the art without departing from the scope of the present invention.

Claims

1. 1. A method of training a deep neural network (DNN), comprising: Initialize the state for each element of matrix A; updating each element of the matrix A to determine incremental weight updates; storing elements of a hidden matrix, where each element of the hidden matrix corresponds to a sum of incremental weight updates for one of a plurality of the elements of matrix A; updating a corresponding element of the weight matrix based on the corresponding element of the hidden matrix reaching a threshold state; The method comprising:

2. 2. The method of claim 1 , wherein updating each element of the matrix A comprises sending an error signal δ over a conductive row line and reading the resulting output vector z as a current output from a conductive column line.

3. The method of claim 1 , wherein the incremental weight update comprises a matrix multiplication of the output vector and the input vector computed on the matrix A.

4. 2. The method of claim 1 , wherein updating the weight matrix comprises transmitting sign information of an element of the hidden matrix that has reached a threshold state as a voltage pulse through a conductive row line of the weight matrix when the element of the hidden matrix reaches the threshold state, and simultaneously transmitting a voltage pulse through a conductive column line of the weight matrix.

5. The method of claim 1 , wherein the threshold state comprises 10 times the average of the incremental updates of the weights.

6. The method of claim 1 , comprising resetting corresponding elements of the hidden matrix to zero when the corresponding elements of the weight matrix are updated.

7. 1. A computer-implemented method for training a deep neural network, comprising: keeping track of the sum of incremental weight updates obtained from corresponding elements of matrix A in corresponding elements of the hidden matrix; triggering an update for one of the corresponding elements of the hidden matrix when the sum of incremental weight updates reaches a threshold value for the corresponding element of the hidden matrix. The method comprising:

8. 8. The method of claim 7, wherein the incremental weight update comprises a matrix multiplication of an output vector and an input vector operated on the corresponding elements of the matrix A.

9. The method of claim 7 , wherein the sum comprises a positive or negative value, and the update comprises sign information corresponding to the sign of the sum.

10. The method of claim 7 , wherein the total is tracked digitally.

11. A deep neural network (DNN), a matrix A including resistive processing unit (RPU) devices isolating intersections between conductive row lines and conductive column lines, where the RPU devices include processed gradients for weighted connections between neurons in the DNN; a weight matrix including an RPU device that separates intersections between conductive row lines and conductive column lines, where the RPU device includes weighted connections between neurons in the DNN; and a hidden matrix including an H value for each RPU device in the weight matrix W, where the H value iteratively adds values ​​from the matrix A until at least one H value exceeds a threshold; The DNN is provided with:

12. The DNN of claim 11 , further comprising a reference array interconnected with the cross-point array for storing zero-weight conductance values ​​of the RPU devices in the cross-point array.

13. The DNN of claim 11 , wherein the hidden matrix H includes digitally stored values ​​for each RPU device in the weight matrix W.

14. A computer program comprising: Initialize the state for each element of matrix A; updating each element of the matrix A to determine incremental weight updates; storing elements of a hidden matrix, where each element of the hidden matrix corresponds to a sum of incremental weight updates for one of a plurality of the elements of matrix A; updating a corresponding element of the weight matrix based on the corresponding element of the hidden matrix reaching a threshold state; The computer program causes a computer to execute each step of the method comprising the steps of:

15. 15. The computer program product of claim 14, wherein updating each element of the matrix A comprises sending an error signal δ over a conductive row line and reading the resulting output vector z as a current output from a conductive column line.

16. 15. The computer program product of claim 14, wherein the incremental weight update comprises a matrix multiplication of the output vector and the input vector computed on the matrix A.

17. 15. The computer program product of claim 14, wherein updating the weight matrix comprises transmitting sign information of an element of the hidden matrix that has reached a threshold state as a voltage pulse through a conductive row line of the weight matrix when the element of the hidden matrix reaches a threshold state, and simultaneously transmitting a voltage pulse through a conductive column line of the weight matrix.

18. The computer program product of claim 14 , wherein the threshold state comprises 10 times an average of the incremental updates of the weights.

19. 15. The computer program product of claim 14, further comprising resetting a corresponding element of the hidden matrix to zero when the corresponding element of the weight matrix is ​​updated.

20. 1. A computer-implemented method for training a deep neural network (DNN), comprising: The input vector e is fed through the conductive column lines of the matrix A. i as voltage pulses, and reading the resulting output vector y′ as current output from conductive row lines of said matrix A, said matrix A comprising a resistance processing unit (RPU) that separates crossings between said conductive column lines and said conductive row lines. The input vector e i updating the H values ​​of a hidden matrix by iteratively adding the output vector y′ multiplied by the transpose of x′ to each H value, where the hidden matrix includes the H values ​​for each RPU; and When the H value reaches a threshold, the sign information of the H value that has reached the threshold is transmitted as a voltage pulse through the conductive row line of the weight matrix W, and at the same time, the input vector e i Transmitting the voltage pulse The method comprising:

21. 21. The method of claim 20, wherein the input vector and the error signal are derived from forward and backward cycles operated on the weight matrix.

22. sending an input vector x as a voltage pulse through conductive column lines of the weight matrix W and reading a resulting output vector y as a current output from conductive row lines of the weight matrix W, the weight matrix W comprising a resistive processing unit (RPU) device that separates crossings between the conductive column lines of the weight matrix W and the conductive row lines of the weight matrix W; and The input vector e i updating matrix A by transmitting an error signal δ calculated from said output vector y as a voltage pulse through said conductive row lines before transmitting 21. The method of claim 20, comprising:

23. The input vector e i 21. The method of claim 20, wherein x comprises a one-hot encoded vector.

24. The input vector e i 21. The method of claim 20, wherein is selected from the Hadamard matrices.

25. The method of claim 20, wherein the H values ​​are stored digitally.

Images