ELECTRONIC DEVICE AND METHOD FOR OVERCOMMING NONIDEALITY OF ReRAM CROSSBAR ARRAY

Abstract

An electronic device according to one embodiment includes a memory in which computer-executable instructions are stored; and a processor that accesses the memory and executes the instructions, wherein the instructions can train the deep neural network by generating first extended input data from a first training input vector based on the dimension of a first weight matrix of the deep neural network, applying tensor data generated based on the combination of the first weight matrix and the first extended input data to a weight prediction model, thereby obtaining a target weight matrix in which the first weight matrix is ​​changed according to the analog non-ideality of the RCA (ReRAM crossbar arrays) circuit, and back-propagating using the loss of the deep neural network calculated through an operation including forward propagation based on the first training input vector and the target weight matrix.

Description

Technology Field

[0001] Below, technology is provided regarding a method and apparatus for overcoming the non-ideality of a resistive random access memory crossbar array. Background Technology

[0002] Matrix-vector multiplication using deep neural networks is a core operation in deep neural network computation. In relation to this operation, RCA (ReRAM crossbar arrays) circuits are used as hardware for deep neural network accelerators as circuits capable of implementing matrix-vector multiplication quickly and efficiently.

[0003] Specifically, the RCA circuit may include a structure in which analog components are arranged in a two-dimensional grid. For example, if the weight matrix of a deep neural network is mapped to the conductance value of each resistive component of the RCA circuit, and the value of the input vector of the deep neural network is applied to the RCA circuit as a voltage, the result of the matrix-vector product can be represented as an output value in the form of a current value.

[0004] However, RCA circuits have a problem in that they produce inaccurate results due to the nonideality of analog components. For example, the analog components of RCA circuits have problems such as IR drop or IV nonlinearity, where the voltage applied to the analog components is lower than the ideal expected value due to the influence of wire resistance. Therefore, the results derived from the nonideality of RCA circuits can be a problem that degrades the performance of the deep neural network mentioned above.

[0005] Therefore, a method is required to solve the aforementioned problem by predicting in advance the effect of non-ideality on RCA circuits during the learning process of deep neural networks.

[0006] The background technology described above is possessed or acquired by the inventor in the process of deriving the content of the disclosure of the present application, and cannot necessarily be considered as prior art disclosed to the general public prior to the filing of this application. means of solving the problem

[0007] An electronic device according to one embodiment includes a memory in which computer-executable instructions are stored; and a processor that accesses the memory and executes the instructions, wherein the instructions can train the deep neural network by generating first extended input data from a first training input vector based on the dimension of a first weight matrix of the deep neural network, applying tensor data generated based on the combination of the first weight matrix and the first extended input data to a weight prediction model, thereby obtaining a target weight matrix in which the first weight matrix is ​​changed according to the analog non-ideality of the RCA (ReRAM crossbar arrays) circuit, and back-propagating using the loss of the deep neural network calculated through an operation including forward propagation based on the first training input vector and the target weight matrix.

[0008] The processor can generate a number of replication vectors identical to the first training input vector equal to the number of dimensions of the first training input vector, and combine the plurality of replication vectors in a predetermined direction based on the dimensions of the first weight matrix to generate the first extended input data.

[0009] The processor can train the weight prediction model by applying the second training input vector and the training weight matrix to a hardware simulation implementing the RCA circuit-based deep neural network accelerator, thereby obtaining a real-value weight matrix representing weights changed according to the analog non-ideality of the RCA circuit, and outputting the real-value weight matrix from the second training input vector and the training weight matrix.

[0010] The processor may obtain average data of training input vectors in a set of training input vectors including at least two training input vectors based on the case where the element data of the first training input vector includes binary data, and generate a random input vector including element data that follows the same distribution as the distribution of the set of training input vectors based on the obtained average data.

[0011] The processor extracts a weight matrix of a deep neural network trained in a manner that compensates for errors caused by the analog non-ideality of the RCA circuit, maps the extracted weight matrix to a resistive element of the RCA circuit, maps an input vector applied to the deep neural network to an input bit line of the RCA circuit, and obtains an output vector by means of the RCA circuit through a matrix-vector multiplication operation of the input vector and the trained weight matrix. Brief explanation of the drawing

[0012] FIG. 1 is a diagram illustrating the structure of an RCA (ReRAM crossbar arrays) circuit according to one embodiment. FIG. 2 is a diagram showing an example of a matrix-vector product operation according to one embodiment. FIG. 3 is a diagram showing a distorted current value due to the analog non-ideality of an RCA circuit according to one embodiment. FIG. 4 is a flowchart illustrating a method for training a deep neural network using a weight prediction model according to one embodiment. FIG. 5 is a flowchart illustrating a deep neural network learning step based on a weight matrix obtained through a weight prediction model according to one embodiment. FIGS. 6a and 6b illustrate a method for obtaining a target weight matrix in a weight prediction model according to one embodiment. FIG. 7 is a diagram illustrating the relationship between an RCA circuit and a weight prediction model according to one embodiment. FIGS. 8A and FIGS. 8B are drawings illustrating a learning method for a weight prediction model according to one embodiment. FIG. 9 is a diagram illustrating a method of training a deep neural network by applying a weight prediction model according to one embodiment to a deep neural network. FIG. 10 is a diagram illustrating a method of training a deep neural network by applying a target weight matrix to each RCA unit block according to one embodiment. FIGS. 11a to 11c illustrate a method of applying a weight matrix of a learned deep neural network according to one embodiment to an RCA circuit. FIG. 12 is a diagram showing the results of a hardware simulation of the non-ideality characteristics of an RCA circuit analog element according to one embodiment. FIG. 13 is a graph showing a comparison of 16 prediction results of an IV-IR scenario according to one embodiment. FIG. 14 is a graph showing a comparison of the training times of a deep neural network according to one embodiment. FIG. 15 is a graph showing the results of measuring accuracy when the weight matrix of a learned deep neural network according to one embodiment is applied to an RCA circuit. Specific details for implementing the invention

[0013] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0014] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0015] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between.

[0016] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0017] In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0018] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0019] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0020] FIG. 1 is a diagram illustrating the structure of an RCA (ReRAM crossbar arrays) circuit according to one embodiment.

[0021] An RCA circuit (100) according to one embodiment may be a resistive random access memory crossbar array structure as hardware for a deep neural network accelerator. The RCA circuit (100) may include an input line (110), an output line (120), and an analog element (130). For example, the RCA circuit (100) may include a single resistive element array cell. The input line (110) may represent a line (e.g., a bit line) that transmits input data to the analog element (130) through a first path (115) in the RCA circuit (100). The output line (120) may represent a line that outputs output data transmitted from the analog element (130) through a second path (135) in the RCA circuit (100). The analog element (130) is a device having a resistance corresponding to a weight for deep neural network operations and may be implemented as a non-volatile random access memory. For example, the analog element (130) may be a resistive element, such as a memristor, which operates by changing the resistance of the solid-state material of the dielectric.

[0022] Deep neural network operations may include operations of individually multiplying input values ​​received at any layer of the deep neural network by weights (e.g., connection weights) and operations of summing the products. The aforementioned multiplication and accumulation (MAC) operation constitutes a significant portion of deep neural network operations and can also be referred to as a matrix-vector operation. An RCA circuit (100) according to one embodiment may perform matrix-vector operations. For example, a matrix-vector operation may represent an operation of linearly combining each component of a vector with a column vector that constitutes a matrix.

[0023] A matrix-vector product operation according to one embodiment can be expressed by Equation 1:

[0024]

[0025] Here, can represent the i-th element data of the input vector, and It can represent the element data of the i-th row and k-th column of a matrix, and can represent the k-th element data of the output vector.

[0026] In the matrix-vector operation performed by the RCA circuit (100), the matrix may have weight values ​​of the connecting lines connecting the nodes of the deep neural network, and the vector may have input values. The RCA circuit (100) may receive a signal (e.g., input voltage) corresponding to the input value through the input line (110). The analog component (130) of the RCA circuit (100) may have a component value (e.g., resistance value or conductance value) corresponding to the weight value. The RCA circuit (100) may provide a signal (e.g., output current) corresponding to the result of the aforementioned matrix-vector operation through the output line (120).

[0027] However, the value applied to the analog component (130) of the RCA circuit (100) may differ from the ideal expected value due to the influence of analog characteristics (e.g., wire resistance). The analog component (130) of the RCA circuit (100) may have nonideality due to the voltage drop (IR drop) caused by analog characteristics. Due to the nonideality of the analog component (130), the analog component (130) may exhibit a conductance different from the conductance corresponding to the intended weight. Consequently, weight distortion may occur in the analog component (130), and distortion may also occur in the result of matrix-vector operations.

[0028] FIG. 2 is a diagram showing an example of a matrix-vector product operation according to one embodiment.

[0029] An electronic device according to one embodiment can obtain an output vector (220) through a multiplication operation of an input vector (210) and an input matrix (230). For example, the electronic device can obtain an output vector (220) as the first element of the output vector (220) by multiplying the first element of the input vector (210) and all elements in the first column of the input matrix (230) and adding the result.

[0030] An electronic device according to one embodiment may perform the matrix-vector multiplication operation described above in an RCA circuit (e.g., the RCA circuit (100) of FIG. 1). For example, the input vector (210) may represent a vector applied as input data to the input line of the RCA circuit (e.g., the input line (110) of FIG. 1). The input matrix (230) may represent a matrix mapped to a resistive element of an analog element of the RCA circuit (e.g., the analog element (130) of FIG. 1). The output vector (220) may represent a vector that is output data of the output line of the RCA circuit (e.g., the output line (120) of FIG. 1).

[0031] FIG. 3 is a diagram showing a distorted current value due to the analog non-ideality of an RCA circuit according to one embodiment.

[0032] The expected current (300) according to one embodiment can be expressed by Equation 2:

[0033]

[0034] Here, can represent the i-th voltage value in a voltage vector containing voltage values, and can represent the conductance value at (i,k) in a conductance matrix containing conductance values. Therefore, since current represents the result of the product of voltage and conductance, It can represent the k-th current value in a current vector containing current values.

[0035] An electronic device according to one embodiment may obtain a distorted current (310) due to the non-ideality of an analog component in an RCA circuit (e.g., the RCA circuit (100) of FIG. 1). The distorted current (310) may represent a current different from the expected current (300) value obtained by Equation 2. For example, the RCA circuit may output a distorted current (310) that is different from the expected current (300) compared to the applied voltage and conductance. Specifically, the electronic device may obtain a distorted current (310) that is different from the expected current (300) predicted by Equation 2 because the analog component (e.g., the analog component (130) of FIG. 1) to which the conductance matrix is ​​mapped in the RCA circuit has a characteristic that has a conductance different from the conductance corresponding to the intended weight due to a voltage drop according to the analog characteristics. A method for resolving weight distortion caused by non-ideality in the analog component of the RCA circuit will be described below.

[0036] FIG. 4 is a flowchart illustrating a method for training a deep neural network using a weight prediction model according to one embodiment.

[0037] In step (410), the electronic device may generate first tensor data by combining the first weight matrix and the first extended input data. For example, the first weight matrix may represent the weight matrix of one of the multiple layers of a deep neural network. The first extended input data may represent a matrix containing multiple vectors identical to the first training input vector. Here, the first training input vector may represent training data for training the deep neural network. The tensor data may represent data formed by combining the first weight matrix and the first extended input data into columns of a predetermined dimension. A method for generating tensor data is described later in FIG. 6b below.

[0038] An electronic device according to one embodiment can generate first expanded input data based on the dimension of a first training input vector and the dimension of a first weight matrix. For example, when the dimension of the first training input vector is N-dimensional and the dimension of the first weight matrix is ​​N×N-dimensional, the electronic device can generate N replication vectors. A replication vector may represent a vector identical to the first training input vector. Subsequently, the electronic device can generate first expanded input data by combining a plurality of replication vectors into N×N dimensions identical to the dimension of the first weight matrix and combining the N replication vectors in a predetermined direction. That is, the first expanded input data may represent a matrix formed by combining a plurality of replication vectors that replicate the first training input vector. Consequently, the electronic device can generate first expanded input data having the form of a matrix with the dimension of the weight matrix by combining a plurality of replication vectors. By generating the first expanded input data, the electronic device can preserve the data of the same first training input vector in the weight data of the column dimension of the first weight matrix. The method for generating the first extended input data is described later in Fig. 6b below.

[0039] In step (420), the electronic device can obtain a target weight matrix by applying the tensor data to a weight prediction model. For example, the weight prediction model may represent a neural network that models weights distorted due to the non-ideality of an RCA circuit (e.g., the RCA circuit (100) of FIG. 1). Additionally, the weight prediction model can be expressed by Equation 3:

[0040]

[0041] Here, can represent the weight matrix of a deep neural network, and can represent a training input vector, and can represent the target weight matrix. Also, can represent a weighted prediction model. For example, may be a function representing a weight prediction model that takes tensor data obtained through the combination of a weight matrix and a training input vector as input data and outputs a target weight matrix.

[0042] As described above, when an input signal (e.g., input voltage) corresponding to an input vector is applied to an RCA circuit in which component values ​​(e.g., conductance values ​​or resistance values) corresponding to the weight values ​​of the weight matrix of a deep neural network are set, resistance values ​​different from the intended resistance values ​​may be observed in the analog components of the RCA circuit. In other words, weight distortion may occur in the RCA circuit due to analog characteristics. A weight prediction model can receive tensor data as input and output a target weight matrix corresponding to the component values ​​that have changed or been distorted according to the non-ideality of the RCA circuit. The target weight matrix may represent a matrix containing weight values ​​that have been distorted due to the analog characteristics of the analog components of the RCA circuit (e.g., the analog component (130) of FIG. 1).

[0043] In step (430), the electronic device can apply the target weight matrix and the first training input vector to the deep neural network.

[0044] A deep neural network according to one embodiment can be generated through machine learning. The deep neural network may include a plurality of artificial neural network layers. For reference, the deep neural network may be trained based on training data comprising pairs of training inputs and training targets mapped to those training inputs. During training, the deep neural network may generate a temporary output in response to the training input and may be trained to minimize the loss between the temporary output and the training target. During the training process, parameters of the deep neural network (e.g., connection weights between nodes / layers in a neural network) may be updated according to the loss. This learning may be performed, for example, on the electronic device itself where the machine learning model is executed, or through a separate server. The machine learning model that has completed training may be stored in memory.

[0045] An electronic device according to one embodiment can apply a target weight matrix and a first training input vector to a deep neural network as in Equation 4:

[0046]

[0047] Here, can represent the weight matrix of a deep neural network, and can represent the first training input vector. Also, can represent the operation of a weight matrix and a training input vector in a single layer of a deep neural network. Specifically, instead of a weight matrix, the electronic device, The target weight matrix obtained through a weight prediction model that performs operations can be applied to a deep neural network.

[0048] For example, the electronic device may generate a temporary output based on a first training input vector and a target weight matrix, which are training inputs. Here, the temporary output may be the result of a matrix-vector operation of the input vector and the target weight matrix. The electronic device may train a deep neural network such that the loss between the generated temporary output and the training target is minimized. Here, the training target may be an output corresponding to the first training input vector. During the training process, the electronic device may update the parameters (e.g., weight matrices) of multiple layers of the deep neural network (e.g., layers containing the first weight matrix) according to the acquired loss. The electronic device may train the deep neural network such that it has weight values ​​compensated for distortion due to the non-ideality of the RCA circuit by generating a temporary output based on the target weight matrix instead of generating a temporary output based on the first weight matrix.

[0049] FIG. 5 is a flowchart illustrating a deep neural network learning step based on a weight matrix obtained through a weight prediction model according to one embodiment.

[0050] An electronic device according to one embodiment can train a deep neural network by applying a target weight matrix obtained from a weight prediction model to the training of the deep neural network. For example, the deep neural network may include a plurality of layers, and each layer may include a weight matrix.

[0051] In step (530), the electronic device may obtain an output vector (e.g., a temporary output) of one of the multiple layers of the deep neural network. For example, in step (532), the electronic device may apply a first training input vector and a first weight matrix to a weight prediction model to obtain a target weight matrix of a layer containing the first weight matrix. For reference, the first training input vector may include a training input vector for training the deep neural network or an output vector of a previous layer. In step (534), the electronic device may obtain an output vector through an operation including forward propagation based on the first training input vector and the target weight matrix. In other words, the electronic device may train the deep neural network through an operation including forward propagation based on the target weight matrix obtained from the weight prediction model instead of the first weight matrix. The deep neural network trained as described above may have weight values ​​that compensate for distortions due to the non-ideality of the RCA circuit. Therefore, an RCA circuit with component values ​​set to correspond to the weight values ​​of a deep neural network can perform matrix-vector operations with the intended component values ​​during operation.

[0052] Subsequently, in step (540), if the layer of the aforementioned operation is not the last layer, the electronic device can obtain an output vector through an operation including forward propagation of the next layer by applying the first training input vector of the next layer as the obtained output vector. Additionally, if the layer of the aforementioned operation is the last layer, the electronic device can train the deep neural network through backpropagation.

[0053] In step (550), the electronic device can train a deep neural network based on the difference between the output vector and the training target. For example, the electronic device can calculate the loss of the deep neural network through the difference between the output vector and the training target. Then, the electronic device can train the deep neural network by backpropagating using the calculated loss.

[0054] In step (560), the electronic device may terminate the learning by determining whether the number of learning cycles (e.g., epoch) exceeds a preset number of learning cycles.

[0055] FIGS. 6a and 6b illustrate a method for obtaining a target weight matrix in a weight prediction model according to one embodiment.

[0056] An electronic device according to one embodiment can obtain a target weight matrix (650a) by applying a weight matrix (620a) to a weight prediction model (640a). The electronic device can obtain an output vector y by the mathematical formula (660a) using the obtained target weight matrix (650a) and the training vector x.

[0057] An electronic device according to one embodiment can obtain a target weight matrix (650b) by applying tensor data (630) to a weight prediction model (640b). The electronic device can obtain tensor data (630) by combining expanded input data (610) and a weight matrix (620b). In FIG. 6b, for convenience of explanation, the first matrix of the tensor data is depicted as a weight matrix (620b), but it is not limited thereto, and the electronic device can obtain tensor data (630) in which the first matrix of the tensor data (630) is expanded input data (610). Subsequently, the electronic device can obtain a target weight matrix (650b) by applying the tensor data (630) to a weight prediction model (640b).

[0058] An electronic device according to one embodiment may apply a training input vector (605) and a target weight matrix (650b) to an RCA circuit. Here, the electronic device may apply the training input vector (605) to an input line (660b) of the RCA circuit and the target weight matrix (650b) to an analog component (670b) of the RCA circuit.

[0059] However, for convenience of explanation, FIG. 6b illustrates that the target weight matrix (650b) is applied to the RCA circuit, but it is not limited thereto. The electronic device may apply the weight matrix of the deep neural network learned by applying the target weight matrix to the RCA circuit. The method of applying the weight matrix of the deep neural network learned to the RCA circuit is described later in FIG. 11c below.

[0060] FIG. 7 is a diagram illustrating the relationship between an RCA circuit and a weight prediction model according to one embodiment.

[0061] An electronic device according to one embodiment can obtain a final output current from an RCA circuit based on a weight prediction model (700) and a deep neural network (730). For example, the deep neural network (730) may represent a neural network trained based on a target weight matrix (720) output from the weight prediction model (700). The weight prediction model (700) may represent a neural network that takes tensor data (710) as input data and outputs a target weight matrix (720). The electronic device can provide conductance (750) to the RCA circuit by mapping the weights of the trained deep neural network (730) to analog components of the RCA circuit. Specifically, the electronic device can provide the weights of the trained deep neural network (730) to the RCA circuit so that the weight values ​​compensate for distortions due to non-ideality of the RCA circuit through the target weight matrix (720). The electronic device can apply voltage (740) to the input vector of the deep neural network through the input line of the RCA circuit. Consequently, the electronic device can perform matrix-vector multiplication operations during the operation of the deep neural network in the RCA circuit having weights compensated for distortion due to the non-ideality of the RCA circuit. A method of applying the trained deep neural network (730) to the RCA circuit to obtain a result corresponding to the final output current is described later in FIG. 11c below.

[0062] FIGS. 8A and FIGS. 8B are drawings illustrating a learning method for a weight prediction model according to one embodiment.

[0063] An electronic device according to one embodiment can train a weight prediction model (840a) used for training a deep neural network. The weight prediction model (840a) may include a plurality of artificial neural network layers. For reference, the weight prediction model (840a) may be trained based on training data including pairs of training inputs and training targets mapped to the training inputs. Here, the training input may include a training input vector (800a) and a training weight matrix (810a). The elements of the training input vector (800a) and the elements of the training weight matrix (810a) may represent random binary values. The training target may include a real value weight matrix (830a). The real value weight matrix (830a) may represent a weight matrix generated in an RCA circuit due to the non-ideality of the RCA circuit when the training input is applied to the RCA circuit. The weight prediction model (840a) during training can generate a temporary output in response to the training input and can be trained such that the loss (860a) between the temporary output and the training target is minimized. Here, the temporary output may include a temporary weight matrix (850a). During the training process, the parameters of the weight prediction model (840a) (e.g., connection weights between nodes / layers in a neural network) may be updated according to the loss (860a). This training may be performed, for example, on the electronic device itself where the weight prediction model (840a) is executed, or through a separate server. The weight prediction model (840a) after training is complete may be stored in memory.

[0064] An electronic device according to one embodiment can obtain a real-value weight matrix (830a) by applying a training input vector (800a) and a training weight matrix (810a) to a hardware simulation (820a). Specifically, the electronic device can obtain a real-value weight matrix (830a) by applying tensor data (e.g., tensor data (630) of FIG. 6b) generated through the training input vector (800a) and the training weight matrix (810a) to a hardware simulation (820a). For example, the hardware simulation (820a) may represent a simulation of the operation of an RCA circuit that performs deep neural network operations. For convenience of explanation, the present specification mainly describes a SPICE (simulation program with integrated circuit emphasis) simulation as the hardware simulation (820a).

[0065] A real value weight matrix (830a) according to one embodiment can be obtained by Equation 5:

[0066]

[0067] Here, can represent the final output current vector obtained through hardware simulation (820a), and also The training input vector (800a) may represent a diagonal matrix generated from a voltage vector applied as a voltage value in the hardware simulation (820a). In other words, the real value weight matrix (830a) may include weight distortion due to the non-ideality of the RCA circuit based on the final output current of the hardware simulation (820a) and the input voltage of the hardware simulation (820a). Additionally, the electronic device may obtain a weight prediction model (840a) that outputs a matrix containing weight values ​​distorted due to analog characteristics by training the weight prediction model (840a) based on the real value weight matrix (830a) containing weight distortion.

[0068] An electronic device according to one embodiment can train a weight prediction model (840b) by applying a second extended input data (822b), generated based on the average data (810b) of the elements of a first training input vector (800b), to the weight prediction model (840b). For example, the first training input vector (800b) may be a training input for training the weight prediction model (840b). Specifically, the first training input vector (800b) may include randomly generated binary data. The second extended input data (822b) may represent a matrix containing a plurality of vectors identical to the second random input vector (812b). Here, the second random input vector (812b) may represent a vector containing randomly generated binary data based on the average data (810b) of the elements of the first training input vector (800b). Specifically, the electronic device can calculate the average data (810b) of the binary data based on the case where the element data of the first training input vector (800b) is binary data. The average data (810b) may be the sum of all elements of the first training input vector (800b) divided by the number of elements of the first training input vector (800b). However, the average data (810b) is not limited to this, and the average data (810b) may include the ratio of each of the binary data of the first training input vector (800b).

[0069] Average data according to one embodiment can be calculated as in Equation 6:

[0070]

[0071] For example, here can represent a training input vector, and It can represent the average data of 100 training input vectors. Also, can represent a random input vector, and It can represent the average data of a random input vector.

[0072] Subsequently, the electronic device can generate random binary data based on the calculated average data (810b) to create a vector having the same number of elements as the first training input vector (800b). For example, the electronic device can generate a second random input vector (812b) having the same average data as the average data (810b) of the first training input vector (800b). By generating the second random input vector (812b), the electronic device can train the weight prediction model (840b) independently of the input data (e.g., the first training input vector (800b)). For reference, in FIG. 8b, for convenience of explanation, it is described that the electronic device calculates the average data (810b) based on a single first training input vector (800b), but this is not limited thereto. As another example, the electronic device can generate a random input vector by calculating the average data of multiple training input vectors. That is, the electronic device can generate a random input vector having the same average data as the average data of multiple training input vectors.

[0073] An electronic device according to one embodiment may obtain a temporary weight matrix (852b) by applying a second extended input data (822b) and a weight matrix (830b) generated from a first training input vector (800b) to a weight prediction model (840b). The electronic device may calculate a loss (872b) through the difference between the temporary weight matrix (852b) and the real value weight matrix (860b). Subsequently, the electronic device may train the weight prediction model (840b) to minimize the loss (872b). During the training process, the electronic device may update the parameters of the weight prediction model (840b) (e.g., connection weights between nodes / layers in the weight prediction model (840b)) according to the loss (872b). For example, the electronic device may obtain a temporary weight matrix (856b) by applying the second extended input data (826b) and the weight matrix (830b) generated from the first training input vector (800b) to the weight prediction model (840b). The electronic device may calculate the loss (876b) through the difference between the temporary weight matrix (856b) and the real value weight matrix (860b). Subsequently, the electronic device may train the weight prediction model (840b) to minimize the loss (876b). During the training process, the electronic device may update the parameters of the weight prediction model (840b) (e.g., connection weights between nodes / layers in the weight prediction model (840b)) according to the loss (876b).

[0074] For reference, FIG. 8b mainly describes a method in which an electronic device generates a second random input vector (e.g., a second random input vector (812b)) to train a weight prediction model (840b) independently of the input data, but is not limited thereto. For example, the electronic device may obtain a target weight matrix by applying the second random input vector and the weight matrix to a weight prediction model trained by the method described above. Here, the electronic device may reduce the time required to obtain the target weight matrix by generating random data (e.g., an input vector that can represent the representativeness of the multiple input data as the second random input vector) instead of obtaining a target weight matrix for each of the multiple input data (e.g., a first training input vector (800b)). Additionally, by obtaining the target weight matrix based on the random data (e.g., the second random input vector), the electronic device may apply a target weight matrix, from which dependency on the input data (e.g., the first training input vector (800b)) has been removed, to a deep neural network. In FIG. 9, a method for applying the target weight matrix obtained from the weight prediction model to the training of a deep neural network is described below.

[0075] FIG. 9 is a diagram illustrating a method of training a deep neural network by applying a weight prediction model according to one embodiment to a deep neural network.

[0076] An electronic device according to one embodiment can train a deep neural network (910) by applying a weight prediction model (930) to the training of the deep neural network (910). For example, the weight prediction model (930) may represent a pre-trained weight prediction model. The electronic device may obtain a target weight matrix (940) by applying the weight matrix (920) of the deep neural network (910) to be trained and the input vector (900), which is the training input for training the deep neural network (910), to the pre-trained weight prediction model (930). Subsequently, the electronic device may obtain an output vector (950) by performing a matrix-vector multiplication operation on the target weight matrix (940) and the input vector (900) in a layer containing the weight matrix (920). For example, the output vector (950) may be a temporary output of the layer to which the target weight matrix (940) is applied. The electronic device can calculate a loss (960) representing the difference between the training target corresponding to the output vector (950) and the input vector (900), based on the case where the layer containing the weight matrix (920) is the last layer of the deep neural network (910). For example, since the loss (960) is a loss obtained using the target weight matrix (940) rather than the weight matrix (920), it may represent a loss that reflects the non-ideality of the RCA circuit. Therefore, the electronic device updates the parameters of the deep neural network (910) (e.g., connection weights between nodes / layers) according to the aforementioned loss (960), thereby training the deep neural network (910) to reflect the non-ideality of the RCA circuit. For reference, in FIG. 9, for convenience of explanation, multiple layers of the deep neural network (910) are omitted and an example of applying the weight matrix (920) of a single layer to the weight prediction model (930) is mainly described, but it is not limited to this.

[0077] FIG. 10 is a diagram illustrating a method of training a deep neural network by applying a target weight matrix to each RCA unit block according to one embodiment.

[0078] An electronic device according to one embodiment may apply a target weight matrix predicted by a weight prediction model for each RCA unit block during the training of a deep neural network. This is because, when the size of the deep neural network is larger than the size of the RCA, weights corresponding to a part of the deep neural network (e.g., weights corresponding to the RCA unit block) may be set in the RCA. An RCA unit block is a block determined based on the number of weights that can be set in the RCA; for example, if 64×64 weights can be set in the RCA, the RCA unit block may be a block of size 64×64. For example, if the weights of the deep neural network are 100×100, a target weight matrix based on the aforementioned weight prediction model may be predicted for the weights corresponding to the 64×64 RCA unit block among the 100×100 weights. During the forward propagation process of the deep neural network, the electronic device may repeatedly predict a target weight matrix based on the aforementioned weight prediction model for each RCA unit block and perform a forward propagation operation using the predicted target weight matrix. For convenience of explanation, this specification primarily describes examples where the RCA unit block is a layer, but is not limited thereto.

[0079] The electronic device can train the deep neural network by applying the target weight matrix of the weight prediction model to each layer of the deep neural network. For example, the electronic device can train the deep neural network by applying the target weight matrices obtained through the first weight prediction model (1016), the second weight prediction model (1026), and the third weight prediction model (1046) to the layers of the deep neural network corresponding to the first RCA unit block (1010), the second RCA unit block (1020), and the nth RCA unit block (1040), which correspond to one layer of the deep neural network. For reference, in FIG. 10, the weight prediction models (e.g., the first weight prediction model (1016), the second weight prediction model (1026), and the nth weight prediction model (1046)) are depicted as separate models, but this is not limited thereto. A single weight prediction model may be used repeatedly in the operations of multiple RCA unit blocks.

[0080] An electronic device according to one embodiment may obtain a first target weight matrix (1018) by applying a first training input (1014) and a weight matrix (1012) of a first layer to a first weight prediction model (1016). Here, the first training input (1014) may represent a training input (1000) of a deep neural network. The electronic device may calculate a second training input (1024) representing a training input of a second layer by applying the first training input (1014) and the first target weight matrix (1018) to a layer of a deep neural network corresponding to a first RCA unit block (1010).

[0081] The electronic device can obtain a second target weight matrix (1028) by applying the second training input (1024) and the weight matrix (1022) of the second layer to the second weight prediction model (1026). The electronic device can calculate a third training input (1034) representing the training input of the third layer by applying the second training input (1024) and the second target weight matrix (1028) to the layer of the deep neural network corresponding to the second RCA unit block (1020).

[0082] Based on the case where the deep neural network has n layers, the electronic device can obtain the n-th target weight matrix (1048) by applying the n-th training input (1044) and the weight matrix (1042) of the n-th layer to the n-th weight prediction model (1046). The electronic device can calculate a provisional output (1050) by applying the n-th training input (1044) and the n-th target weight matrix (1048) to the layer of the deep neural network corresponding to the n-th RCA unit block (1040). The electronic device can calculate a loss (1054) through the difference between the training target (1052) and the provisional output (1050). The training target (1052) may represent a training target mapped to the training input (1000). Subsequently, the electronic device can train the deep neural network so that the loss (1054) between the provisional output (1050) and the training target (1052) is minimized. During the training process, the parameters of the deep neural network can be updated according to the loss (1054).

[0083] FIGS. 11a to 11c illustrate a method of applying a weight matrix of a learned deep neural network according to one embodiment to an RCA circuit.

[0084] FIG. 11a illustrates an electronic device according to one embodiment applying an input value (1110a) to a learned deep neural network (1105a) to obtain an output value (1140a). Specifically, the learned deep neural network (1105a) may include a learned weight matrix (1120a). The output value (1140a) is a value mapped when the input value (1110a) is applied to the learned deep neural network (1105a), and may represent a value close to a training target. Additionally, when the electronic device performs the inference process (1100a) of the learned deep neural network (1105a) in software, it can obtain an undistorted output value.

[0085] FIG. 11b illustrates an electronic device according to one embodiment applying an input value (1110b) and a learned weight matrix (1120b) of a learned deep neural network (1105b) to an RCA circuit (1130b) to obtain a distorted output value (1140b). Here, the electronic device may apply the learned weight matrix (1120b) to the RCA circuit (1130b) to perform a matrix-vector multiplication operation. However, due to the non-ideality of the RCA circuit (1130b), the learned weight matrix (1120b) may be applied as a distorted weight matrix when applied to the RCA circuit (1130b). As a result, the electronic device may obtain a distorted output value (1140b) that is different from the output value mapped when the input value (1110b) is applied to the learned deep neural network (1105b). Here, the distorted output value (1140b) may represent an output value different from the ideal output value expected for the input value (1110b) (e.g., the output value (1140a) of FIG. 11a).

[0086] FIG. 11c is a diagram illustrating that an electronic device according to one embodiment obtains an output value (1140c) by applying a learned weight matrix (1120c) of a deep neural network (1105c) learned by an input value (1110c) and a weight prediction model (1115c) to an RCA circuit (1130c).

[0087] An electronic device according to one embodiment may apply a learned weight matrix (1120c) of a deep neural network (1105c), which is learned by applying a weight prediction model (1115c) to an RCA circuit (1130c) to perform a matrix-vector multiplication operation. For example, since the learned weight matrix (1120c) represents learned weights that reflect the non-ideality of the RCA circuit (1130c), it may be applied as a normal weight matrix (e.g., the learned weight matrix (1120a) of FIG. 11a) when applied to the RCA circuit (1130c). As a result, the electronic device may obtain an output value (1140c) that is close to the output value (e.g., the output value (1140a) of FIG. 11a) which is mapped when the input value (1110c) is applied to the learned deep neural network (1105c).

[0088] FIG. 12 is a diagram showing the results of a hardware simulation of the non-ideality characteristics of an RCA circuit analog element according to one embodiment.

[0089] A graph (1200) according to one embodiment may show the effects of line resistance and IV nonlinearity of an analog device (e.g., analog device (130) of FIG. 1). For example, an electronic device may obtain a simulation result of an RCA circuit with a size of 64×64 RCA unit blocks to which a random binary weight matrix and a random binary input voltage signal are applied. The random binary weight matrix may be a weight matrix of a deep neural network. The electronic device may obtain a conductance value corresponding to the random binary weight matrix in a hardware simulation by mapping the random binary weight matrix to the RCA circuit. The random binary input voltage signal may be a voltage signal converted to apply the training input vector of the deep neural network as a voltage to the RCA circuit. The simulation result may show an output current corresponding to the result of matrix-vector operations of the voltage value applied according to the random binary input voltage signal and the conductance value corresponding to the random binary weight matrix in the RCA circuit. The horizontal axis of the graph (1200) illustrated in FIG. 12 may represent the column index of the RCA unit block, and the vertical axis of the graph (1200) may represent the output current. Consequently, the graph (1200) may represent a graph showing the result that the output current of the RCA circuit is influenced by at least one of voltage, resistance, and the size of the RCA unit block.

[0090] FIG. 13 is a graph showing a comparison of 16 prediction results of an IV-IR scenario according to one embodiment.

[0091] A graph (1300) according to one embodiment may be a graph showing the results of comparing prediction methods for 16 different cases of IV-IR (current-voltage non-ideality and voltage drop) scenarios. For example, an IV-IR scenario may represent a scenario in which the prediction results are measured under different conditions, such as the size of the RCA unit block of an RCA circuit, the input voltage, and the resistance exhibited by the analog component. The vertical axis of the graph (1300) may represent RRMSE. RRMSE may represent the square root of the mean of the squares of the errors, which is the difference between the predicted value and the actual value, as a loss function of the regression model. RRMSE may be expressed by Equation 7:

[0092]

[0093] Here can represent the output current measured in an RCA circuit, and It can represent the correct current for the IV-IR scenario.

[0094] A first comparative example (1302) according to one embodiment may represent an example in which GENIEx (Generalized Approach to Emulating Non-Ideality in Memristive Xbars using Neural Networks) is applied to the training of a deep neural network, and the trained deep neural network is applied to an RCA circuit to obtain an output current. For example, GENIEx can obtain a one-dimensional output vector by using input data generated by concatenating a flat vector created by flattening a training input vector and a weight matrix. An electronic device can train a deep neural network by using the output vector as a temporary output in the training of the deep neural network, which is the result of multiplying the training input vector and the weight matrix. Based on the first comparative example (1302), the IV-IR scenario result on the RCA circuit may include a result in which the measured output current differs from the correct current as the size of the unit block of the RCA circuit increases. Additionally, based on the first comparative example (1302), the IV-IR scenario results on the RCA circuit may include results where the measured output current differs from the correct current as the resistance value of the analog component of the RCA circuit increases. Since the first comparative example (1302) does not use the weight matrix in GENIEx as is but converts it into a one-dimensional flat vector to obtain an output vector for training the deep neural network, the IV-IR scenario results obtained by applying the deep neural network trained based on GENIEx to the RCA circuit may show results vulnerable to the non-ideality of the analog component.

[0095] A second comparative example (1304) according to one embodiment may represent an example in which an output current is obtained by applying a stochastic scaling convolutional network (S-SCN) to the training of a deep neural network and applying the trained deep neural network to an RCA circuit. For example, the S-SCN can obtain a two-dimensional output matrix by using a weight matrix as input data. An electronic device can train a deep neural network by using the value obtained by multiplying the output matrix and the training input vector as a temporary output during the training of the deep neural network. Based on the second comparative example (1304), the IV-IR scenario result on the RCA circuit may show a result that is vulnerable to non-ideality of the analog component as the resistance value of the analog component of the RCA circuit increases, because only the spatial information of the weight matrix of the deep neural network is maintained when obtaining the two-dimensional output matrix. In other words, since S-SCN maintains spatial information of the weight matrix of the deep neural network but does not use the training input vector in the weight prediction process (e.g., the process of obtaining a 2D output matrix), as the input voltage value of the RCA circuit increases, it may result in a vulnerability to non-ideality of the analog device.

[0096] In contrast, the result obtained by applying a deep neural network trained by applying the first weight prediction model (1306) and the second weight prediction model (1308) according to one embodiment to an RCA circuit can show superior results regarding the non-ideality of the analog component as the resistance value of the analog component of the RCA circuit increases, unlike the comparative embodiments. For example, the first weight prediction model (1306) may be a weight prediction model trained with an extended input data and weight matrix generated based on the average data of the elements of the training input vector (e.g., average data (810b) in FIG. 8b) as a S-WPM (stochastic weight-centric prediction model). The second weight prediction model (1308) may be a weight prediction model trained with an extended input data and weight matrix generated from the training input vector as a WPM (weight-centric prediction model). The result obtained by applying a deep neural network trained by applying the first weighted prediction model (1306) to an RCA circuit may include superior performance results compared to the second weighted prediction model, which uses the training input vector as is, because the model has reduced dependency on input data by utilizing average data. In addition, the result obtained by applying a deep neural network trained by applying the first weighted prediction model (1306) and the second weighted prediction model (1308) to an RCA circuit may show consistently superior results in the difference between the measured output current and the correct current, unlike the comparative examples. Specifically, the result obtained by applying a deep neural network trained by applying the first weighted prediction model (1306) to an RCA circuit may include results that resolve the non-ideality problem of the analog components of the RCA circuit by up to 90 times compared to the first comparative example (1302) and up to 6 times compared to the second comparative example (1304).

[0097] FIG. 14 is a graph showing a comparison of the training times of a deep neural network according to one embodiment.

[0098] A graph (1400) according to one embodiment may represent a graph including the results of comparing the training time of a deep neural network. The vertical axis of the graph (1400) may represent the training time, and the horizontal axis of the graph (1400) may include the comparison embodiment and the weight prediction model of the present invention.

[0099] For example, the first comparison embodiment (1402) and the second weight prediction model (1408) may result in slow training times due to dependency on input data, because they require re-evaluation (e.g., obtaining a new output vector or target weight matrix every epoch) for every input pair (e.g., training input vector and weight matrix) in order to be applied to the training of a deep neural network. In contrast, the first weight prediction model (1406) may result in fast training times because it generates the target weight matrix only once during the training process of the deep neural network by obtaining the target weight matrix generated based on the expanded input data generated based on the average data of the elements of the training input vector (e.g., average data (810b) in FIG. 8b). Specifically, the result of applying the first weight prediction model (1406) to the training of a deep neural network may include additional speed improvements in the same weights (i.e., target weight matrix) that are reused multiple times for different input data across the row and column directions of the convolution layer.

[0100] FIG. 15 is a graph showing the results of measuring accuracy when the weight matrix of a learned deep neural network according to one embodiment is applied to an RCA circuit.

[0101] A graph (1500) according to one embodiment may show the results when a deep neural network trained to recognize the non-ideality of an RCA circuit by using a weighted prediction model and a simply trained deep neural network are applied to an RCA circuit, respectively. For example, the accuracy in software (1502) may represent the accuracy of the inference results obtained in software using the trained deep neural network. The accuracy in the first RCA (1504) may represent the accuracy of the inference results on the RCA circuit by applying the trained deep neural network to the RCA circuit. Finally, the accuracy in the second RCA (1506), which is an embodiment of the present invention, may represent the accuracy of the inference results on the RCA circuit by applying the deep neural network trained to recognize the non-ideality of an RCA circuit by using a weighted prediction model to the RCA circuit. Therefore, the aforementioned results may indicate that when a weighted prediction model is applied to the training of a deep neural network, the inference performance on the RCA circuit of the trained deep neural network is excellent.

[0102] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0103] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave in order to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0104] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0105] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0106] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0107] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

Claim 1 A method for training a deep neural network performed by a processor comprises: a step of generating first expanded input data from a first training input vector based on the dimension of a first weight matrix of the deep neural network; a step of obtaining a target weight matrix in which the first weight matrix is ​​changed according to the analog non-ideality of an RCA (ReRAM crossbar arrays) circuit by applying tensor data generated based on the combination of the first weight matrix and the first expanded input data to a weight prediction model; and a step of training the deep neural network by back-propagating using the loss of the deep neural network calculated through an operation including forward propagation based on the first training input vector and the target weight matrix, wherein the step of obtaining the target weight matrix comprises: a step of obtaining average data of training input vectors in a set of training input vectors based on the case where the set of training input vectors includes at least two input vectors. A deep neural network learning method comprising the step of generating a random input vector containing element data that follows the same distribution as the distribution of the set of training input vectors, based on the average data obtained above. Claim 2 A deep neural network learning method according to claim 1, wherein the step of generating the first extended input data comprises: generating a replication vector identical to the first training input vector for each dimension of the first training input vector; and combining the plurality of replication vectors in a predetermined direction based on the dimension of the first weight matrix to generate the first extended input data. Claim 3 A deep neural network learning method according to claim 1, further comprising: a step of obtaining a real-value weight matrix representing weights changed according to the analog non-ideality of the RCA circuit by applying a second training input vector and a training weight matrix to a hardware simulation implementing a deep neural network accelerator based on the RCA circuit; and a step of training a weight prediction model to output the real-value weight matrix from the second training input vector and the training weight matrix. Claim 4 delete Claim 5 A deep neural network learning method according to claim 1, further comprising: a step of extracting a weight matrix of a deep neural network learned in a way that compensates for errors caused by the analog non-ideality of the RCA circuit; a step of mapping the extracted weight matrix to a resistive element of the RCA circuit; a step of mapping an input vector applied to the deep neural network to an input bit line of the RCA circuit; and a step of obtaining an output vector by the RCA circuit through a matrix-vector multiplication operation of the input vector and the learned weight matrix. Claim 6 A computer program stored on a computer-readable recording medium in combination with hardware to execute the method of any one of claims 1 through 3 and 5. Claim 7 In an electronic learning device for a deep neural network, a memory in which computer-executable instructions are stored; An electronic device comprising a processor that accesses the memory and executes the instructions, wherein the instructions generate first extended input data from a first training input vector based on the dimension of a first weight matrix of the deep neural network, apply tensor data generated based on the combination of the first weight matrix and the first extended input data to a weight prediction model to obtain a target weight matrix in which the first weight matrix is ​​changed according to the analog non-ideality of the RCA (ReRAM crossbar arrays) circuit, and train the deep neural network by back-propagating using the loss of the deep neural network calculated through an operation including forward propagation based on the first training input vector and the target weight matrix, and wherein the processor obtains average data of the training input vectors from a training input vector set based on the case where the training input vector set includes at least two input vectors, and generates a random input vector including element data that follows the same distribution as the distribution of the training input vector set based on the obtained average data. Claim 8 An electronic device according to claim 7, wherein the processor generates a number of dimensions of the first training input vector, a number of replication vectors identical to the first training input vector, and combines the plurality of replication vectors in a predetermined direction based on the dimensions of the first weight matrix to generate the first extended input data. Claim 9 An electronic device according to claim 7, wherein the processor applies a second training input vector and a training weight matrix to a hardware simulation implementing a deep neural network accelerator based on the RCA circuit to obtain a real-value weight matrix representing weights changed according to the analog non-ideality of the RCA circuit, and trains the weight prediction model to output the real-value weight matrix from the second training input vector and the training weight matrix. Claim 10 delete Claim 11 An electronic device according to claim 7, wherein the processor extracts a weight matrix of a deep neural network learned in a manner that compensates for errors caused by the analog non-ideality of the RCA circuit, maps the extracted weight matrix to a resistive element of the RCA circuit, maps an input vector applied to the deep neural network to an input bit line of the RCA circuit, and obtains an output vector by the RCA circuit through a matrix-vector multiplication operation of the input vector and the learned weight matrix.

Images