RRAM error emulator and RRAM error emulation method using the same

Abstract

An RRAM error emulator and a method for emulating RRAM errors using the same are disclosed. An RRAM error emulator according to one disclosed embodiment is an RRAM error emulator for mimicking the effects of errors occurring in a Resistive Random Access Memory (RRAM) device, and includes an artificial neural network module comprising an artificial neural network implemented in a crossbar array of the RRAM, and an error generation module that generates error noise and injects the generated error noise into the artificial neural network.

Description

Technology Field

[0001] An embodiment of the present invention relates to RRAM error emulation technology. Background Technology

[0003] Crossbar arrays based on the RRAM (Resistive Random Access Memory) hardware platform provide the capability to perform vector-matrix multiplication and outperform previous generation memory in terms of performance, storage density, and scalability. Consequently, RRAM-based hardware solutions are being used in neuromorphic computing and Artificial Neural Networks (ANNs). However, operational variations such as random noise and high sensitivity during ANN computations remain challenging issues for architectures implemented in RRAM devices.

[0004] Figure 1 is a schematic diagram of a typical RRAM crossbar array. In Figure 1, RRAM memory cells are located at each intersection of the bit line and the word line. Additionally, the weights of the artificial neural network can be composed of cell conductances. When performing matrix multiplication, the RRAM acts as an electronic synapse by changing its resistance state according to the input voltage. All word lines are biased by a vector represented by the amplitude of the read voltage, to which a conductance matrix representing a weight map is additionally multiplied.

[0005] Ideally, a specific voltage is applied to every word line, and this voltage is uniform across all memory cells. However, in actual RRAM crossbar arrays, due to parasitic wire resistance, the voltage drops at nodes (i.e., memory cells) far from the power supply. Consequently, the voltage distribution becomes inconsistent, which leads to a reduction in the distribution of output current across all bit lines.

[0006] In this case, as the accuracy of artificial neural networks implemented in RRAM-based crossbar arrays is significantly degraded, a method is required to simulate errors caused by circuit-level internal stochasticity and parasites (e.g., IR drop, read noise, write noise, etc.). Prior art literature

[0008] Korean Registered Patent Publication No. 10-2141385 (2020.08.05) The problem to be solved

[0009] Embodiments of the present invention are intended to provide an RRAM error emulator capable of mimicking errors occurring in an RRAM device and an RRAM error emulation method using the same. means of solving the problem

[0011] An RRAM error emulator according to one disclosed embodiment is an RRAM error emulator for mimicking the effects of errors occurring in a Resistive Random Access Memory (RRAM) device, comprising: an artificial neural network module including an artificial neural network implemented in a crossbar array of the RRAM; and an error generation module that generates error noise and injects the generated error noise into the artificial neural network.

[0012] The error generation module comprises: a random generator that generates a random value which is a number of a preset size; and an error model that generates error noise based on the random value and injects the generated error noise into the artificial neural network, wherein the error noise may be intended to mimic an error occurring in the RRAM device.

[0013] The artificial neural network described above includes a plurality of layers, and the error model may be configured to generate error noise corresponding to each layer of the artificial neural network and to inject the generated error noise into each layer of the artificial neural network.

[0014] The artificial neural network above may be a binary neural network for representing the state of each memory cell of the RRAM as a low resistance state or a high resistance state.

[0015] The above artificial neural network has a plurality of layers sequentially connected so that the output of a predetermined layer is input to the next layer, and the error model can multiply the error noise by the output of the corresponding layer of the artificial neural network and input it to the next layer.

[0016] The above error model can calculate the actual current of each bit line of the RRAM based on the above random value and generate the error noise based on the difference between the actual current of the bit line and the ideal current of the bit line.

[0017] The above error model can calculate the actual current of the bit line using the following mathematical formula.

[0018] (Mathematical formula)

[0019]

[0020]

[0021]

[0022] I j : Actual current of the j-th bit line

[0023] : Random value generated by the random generator

[0024] : Scale factor indicating the effect of noise in RRAM devices

[0025] : Voltage of the j-th bit line

[0026] m: Number of word lines

[0027] : Ideal equivalent resistance of the j-th bit line

[0028] : Bit line wire resistance

[0029] , : Approximation coefficient pre-set to represent the actual equivalent resistance of the j-th bit line

[0030] The above error model can calculate the read accuracy of the corresponding bit line based on the difference between the actual current of the bit line and the ideal current of the bit line, and set the calculated read accuracy as error noise.

[0031] The above error model can calculate the reading accuracy of the above bit line by the following mathematical formula.

[0032] (Mathematical formula)

[0033]

[0034] : Actual current of the j-th bit line

[0035] : Ideal current of the j-th bit line

[0036] A method for emulating an RRAM error according to one disclosed embodiment is performed in a computing device having one or more processors and a memory storing one or more programs executed by said one or more processors, and is a method for emulating an RRAM error to mimic the effect of an error occurring in a Resistive Random Access Memory (RRAM) device, comprising: a step of generating error noise; and a step of injecting said generated error noise into an artificial neural network implemented in a crossbar array of said RRAM. Effects of the invention

[0038] According to the disclosed embodiment, by mimicking errors caused by internal stochastic causes and parasitic problems of the RRAM, the impact of voltage drop or current leakage caused by errors on the performance of the artificial neural network can be accurately estimated, thereby improving the performance and accuracy of the RRAM-based artificial neural network. Brief explanation of the drawing

[0040] Figure 1 is a schematic diagram showing a typical RRAM crossbar array. FIG. 2 is a schematic diagram showing an artificial neural network-based RRAM error emulator according to an embodiment of the present invention. FIG. 3 is a schematic diagram illustrating the process of generating error noise to be injected into each layer of an artificial neural network in an error generation module according to an embodiment of the present invention. FIG. 4 is a diagram illustrating the process of calculating read accuracy in an error model according to an embodiment of the present invention. FIG. 5 is a flowchart illustrating an RRAM error emulation method according to an embodiment of the present invention. FIG. 6 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments. Specific details for implementing the invention

[0041] Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, apparatuses, and / or systems described herein. However, this is merely illustrative and the present invention is not limited thereto.

[0042] In describing the embodiments of the present invention, detailed descriptions of known technologies related to the present invention are omitted if it is determined that such detailed descriptions may unnecessarily obscure the essence of the present invention. Furthermore, the terms described below are defined in consideration of their functions within the present invention, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification. Terms used in the detailed description are intended merely to describe the embodiments of the present invention and should not be limiting in any way. Unless explicitly stated otherwise, expressions in the singular form include the meaning of the plural form. In this description, expressions such as "include" or "comprise" are intended to refer to certain characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof, and should not be interpreted to exclude the existence or possibility of one or more other characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof other than those described.

[0043] Additionally, terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms may be used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0045] FIG. 2 is a schematic diagram showing an artificial neural network-based RRAM error emulator according to one embodiment of the present invention, and FIG. 3 is a schematic diagram showing the process of generating error noise to be injected into each layer of the artificial neural network in an error generation module according to one embodiment of the present invention.

[0046] Referring to FIGS. 2 and 3, the RRAM error emulator (100) may include an error generation module (102) and an artificial neural network module (104). The RRAM error emulator (100) may be intended to mimic the effects of errors caused by internal probabilistic causes and parasitic problems in a Resistive Random Access Memory (RRAM) device. Here, the RRAM device may include an RRAM crossbar array.

[0047] The error generation module (102) can generate error noise that is injected into the artificial neural network module (104). The error generation module (102) may include a random generator (111) and an error model (113).

[0048] The random generator (111) can randomly generate numbers of a preset size. In one embodiment, the random generator (111) can randomly generate 16-bit numbers. The random generator (111) can randomly generate 16-bit numbers such that the distribution of the generation process is a Gaussian normal distribution.

[0049] The random generator (111) may include a random number generator (111a) and a distribution inverter (111b). The random number generator (111a) can generate a 16-bit random number of uniform distribution according to a preset random number generation algorithm. The distribution inverter (111b) applies an ICDF (Inversion of Cumulative Distribution Function) to the generated random number to invert the Gaussian normal distribution into a random value ( It can be output as ).

[0050] The error model (113) can generate error noise based on random values ​​(random values) generated by the random generator (111). Here, the error noise may be intended to mimic errors caused by internal stochastic causes and parasitic problems (e.g., parasitic wire resistance or parasitic capacitor, etc.) occurring in the RRAM crossbar array. The error model (113) can generate error noise at each preset clock cycle.

[0051] Meanwhile, the artificial neural network module (104) may include an artificial neural network implemented in a crossbar array of RRAM (Resistive Random Access Memory). In one embodiment, the artificial neural network module (104) may be an RRAM-based artificial neural network accelerator. That is, the artificial neural network module (104) may be intended to accelerate the computation of an application based on RRAM.

[0052] In one embodiment, an artificial neural network R the state of each memory cell of RRAM LRS (Low Resistance State) or R HRS A binary neural network can be used to represent the (High Resistance State). Here, the low resistance state or the high resistance state can be distinguished based on a preset reference resistance. That is, if the resistance of the RRAM memory cell is lower than the preset reference resistance, it is considered a low resistance state, and if the resistance of the memory cell is higher than the preset reference resistance, it is considered a high resistance state.

[0053] The artificial neural network module (104) may include a multi-layer artificial neural network. Each layer of the artificial neural network may continuously process data and generate an output at every clock cycle. Multiple layers of the artificial neural network may be connected sequentially. That is, data output from each layer of the artificial neural network may be connected to be input to the next layer. In this case, the layer of the artificial neural network may be a convolution layer that performs convolution on the input data.

[0054] Here, the error model (113) can generate error noise corresponding to each layer of the artificial neural network included in the artificial neural network module (104) and inject the generated error noise into each layer of the artificial neural network of the artificial neural network module (104). The error model (113) can simultaneously generate error noise to be injected into each layer of the artificial neural network. The error model (113) may include error layers that generate error noise corresponding to each layer of the artificial neural network.

[0055] In one embodiment, each error noise may be multiplied by the output of each convolution layer of the artificial neural network and injected into the next layer. The output of each convolution layer is multiplied by the injected error noise to emulate the effect of errors occurring during matrix multiplication in the convolution operation.

[0056] The error model (113) is based on the actual current (I) of the j-th bit line based on the random value (random value) generated by the random generator (111). j ) can be calculated by the following mathematical formula 1.

[0057] (Mathematical Formula 1)

[0058]

[0059] Here, And, am.

[0060] represents a random value generated by a random generator (111). is a scale factor indicating the effect of noise in the RRAM device. is the voltage of the j-th bit line. m represents the number of word lines. represents the ideal equivalent resistance of the j-th bit line. represents the wire resistance of each bit line. and is an approximation coefficient that is set to represent the actual equivalent resistance of the j-th bit line.

[0061] The error model (113) can calculate the difference between the actual current and the ideal current in each bit line. The error model (113) can set the difference between the actual current and the ideal current in each bit line as the error noise of the corresponding bit line.

[0062] The error model (113) can calculate the read accuracy of a corresponding bit line based on the difference between the actual current and the ideal current of each bit line. The error model (113) can also set the read accuracy for a bit line as the error noise of that bit line. The error model (113) can calculate the read accuracy for the j-th bit line through the following mathematical formula 2.

[0063] (Mathematical Formula 2)

[0064]

[0065] : Actual current of the j-th bit line

[0066] : Ideal current of the j-th bit line

[0067] FIG. 4 is a diagram illustrating the process of calculating read accuracy (error noise) in an error model according to an embodiment of the present invention. Referring to FIG. 4, the error model (113) uses write noise (ε based on initial voltage and current). WN ) and read noise (ε RN New R affected by ) LRS (Low Resistance State), R HRS The (High Resistance State) can be calculated respectively. Here, R LRS , R HRS is the node state distribution of the RRAM crossbar array, each having a uniform distribution.

[0068] The error model (113) has a uniform distribution R LRS , R HRS Based on, the equivalent resistance (R) of all nodes (i.e., memory cells) of the RRAM crossbar array eq i,j ) and voltage (V ij ) can be calculated sequentially. Next, the error model (113) can calculate a conductance map (G) containing noise effects and voltage at all nodes. ij ) can be calculated. For each bit line representing the convolution output, the emulated output current is It can be derived from the conductance map and the corresponding voltage as shown.

[0069] Meanwhile, the ideal output current (I) of each bit line o_ideal j ) is R of a uniform distribution LRS and R HRS Based on this, it can be directly calculated through the resistance of the RRAM junction. Finally, the error model (113) can calculate the read accuracy of the corresponding bit line based on the ideal current and the actual current of each bit line.

[0070] According to the disclosed embodiment, by mimicking errors caused by internal stochastic causes and parasitic problems of the RRAM, the impact of voltage drop or current leakage caused by errors on the performance of the artificial neural network can be accurately estimated, thereby improving the performance and accuracy of the RRAM-based artificial neural network.

[0071] In this specification, the term "module" may refer to a functional and structural combination of hardware for carrying out the technical concept of the present invention and software for driving said hardware. For example, the "module" may refer to a logical unit of a specific code and a hardware resource for executing said code, and does not necessarily refer to physically connected code or a single type of hardware.

[0073] FIG. 5 is a flowchart illustrating an RRAM error emulation method according to an embodiment of the present invention. Although the method is described in the illustrated flowchart in a plurality of steps, at least some of the steps may be performed in a different order, combined with other steps and performed together, omitted, divided into detailed steps, or performed with one or more steps not illustrated added.

[0074] Referring to FIG. 5, the RRAM error emulator (100) can generate a random value which is a number of a preset size (S 101). Next, the RRAM error emulator (100) can generate error noise based on the generated random value (S 103). Next, the RRAM error emulator (100) can inject the generated error noise into each layer of the artificial neural network (S 105).

[0076] FIG. 6 is a block diagram illustrating a computing environment (10) including a computing device suitable for use in exemplary embodiments. In the illustrated embodiments, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those described below.

[0077] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be an RRAM error emulator (100).

[0078] The computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) can cause the computing device (12) to operate according to the exemplary embodiment described above. For example, the processor (14) can execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, and the computer-executable instructions may be configured to cause the computing device (12) to perform operations according to the exemplary embodiment when executed by the processor (14).

[0079] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by a processor (14). In one embodiment, the computer-readable storage medium (16) may be memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other forms of storage media that are accessed by a computing device (12) and capable of storing desired information, or a suitable combination thereof.

[0080] The communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and the computer-readable storage medium (16).

[0081] The computing device (12) may also include one or more input / output interfaces (22) and one or more network communication interfaces (26) that provide interfaces for one or more input / output devices (24). The input / output interfaces (22) and network communication interfaces (26) are connected to a communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) through the input / output interfaces (22). An exemplary input / output device (24) may include an input device such as a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices and / or imaging devices, and / or an output device such as a display device, a printer, a speaker and / or a network card. An exemplary input / output device (24) may be included inside the computing device (12) as a component constituting the computing device (12), or it may be connected to the computing device (12) as a separate device distinct from the computing device (12).

[0083] Although representative embodiments of the present invention have been described in detail above, those skilled in the art will understand that various modifications can be made to the above-described embodiments without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims set forth below as well as equivalents thereof. Explanation of the symbols

[0085] 10: Computing Environment 12: Computing device 14 : Processor 16: Computer-readable storage media 18: Communication bus 20 : Program 22 : Input / Output Interface 24 : Input / Output Devices 26 : Network communication interface 100 : RRAM error emulator 102 : Error generating module 104 : Artificial Neural Network Module 111 : Random Generator 111a : Random number generator 111b : Distribution inverter 113 : Error model

Claims

Claim 1 An RRAM error emulator for mimicking the effects of errors occurring in an RRAM (Resistive Random Access Memory) device, comprising: an artificial neural network module including an artificial neural network implemented in a crossbar array of the RRAM; and an error generation module that generates error noise and injects the generated error noise into the artificial neural network, wherein the error generation module includes a random generator that generates a random value which is a number of a preset size; and an error model that generates error noise to mimic errors occurring in the RRAM device based on the random value and injects the generated error noise into the artificial neural network, wherein the error model calculates the actual current of each bit line of the RRAM based on the random value and generates the error noise based on the difference between the actual current of the bit line and the ideal current of the bit line, wherein the actual current of the bit line is calculated by reflecting an approximation coefficient preset to represent the wire resistance of the corresponding bit line, the ideal equivalent resistance of the corresponding bit line, and the actual equivalent resistance of the corresponding bit line. Claim 2 delete Claim 3 An RRAM error emulator according to claim 1, wherein the artificial neural network comprises a plurality of layers, and the error model is configured to generate error noise corresponding to each layer of the artificial neural network and to inject the generated error noise into each layer of the artificial neural network. Claim 4 In claim 3, the artificial neural network is a binary neural network for representing the state of each memory cell of the RRAM as a low resistance state or a high resistance state, an RRAM error emulator. Claim 5 In claim 3, the artificial neural network has a plurality of layers sequentially connected so that the output of a predetermined layer is input to the next layer, and the error model is an RRAM error emulator in which the error noise is multiplied by the output of the corresponding layer of the artificial neural network and input to the next layer. Claim 6 delete Claim 7 The RRAM error emulator of claim 1, wherein the error model calculates the actual current of the bit line by the following mathematical formula. (Mathematical formula) I j : Actual current of the j-th bit line : Random value generated by the random generator : Scale factor indicating the effect of noise in RRAM devices : Voltage of the j-th bit line m: Number of word lines : Ideal equivalent resistance of the j-th bit line : Bit line wire resistance , : Approximation coefficient pre-set to represent the actual equivalent resistance of the j-th bit line Claim 8 An RRAM error emulator according to claim 1, wherein the error model calculates the read accuracy of the bit line based on the difference between the actual current of the bit line and the ideal current of the bit line, and sets the calculated read accuracy as error noise. Claim 9 In claim 8, the error model is an RRAM error emulator that calculates the read accuracy of the bit line by the following mathematical formula. (Mathematical formula) : Actual current of the j-th bit line : Ideal current of the j-th bit line Claim 10 A method for emulating an RRAM error to mimic the effects of an error occurring in a Resistive Random Access Memory (RRAM) device, performed in a computing device having one or more processors and a memory storing one or more programs executed by said one or more processors, comprising: a step of generating error noise; and a step of injecting said generated error noise into an artificial neural network implemented in a crossbar array of said RRAM, wherein the step of generating the error noise comprises: a step of generating a random value which is a number of a preset size; a step of calculating the actual current of each bit line of said RRAM based on said random value; and a step of generating error noise to mimic an error occurring in said RRAM device based on the difference between the actual current of said bit line and the ideal current of said bit line, wherein the actual current of said bit line is calculated by reflecting an approximation coefficient preset to represent the wire resistance of said bit line, the ideal equivalent resistance of said bit line, and the actual equivalent resistance of said bit line. Claim 11 delete Claim 12 A method for RRAM error emulation according to claim 10, wherein the artificial neural network comprises a plurality of layers, the step of generating error noise comprises generating error noise corresponding to each layer of the artificial neural network, and the step of injecting into the artificial neural network comprises injecting the generated error noise into each layer of the artificial neural network. Claim 13 A method for emulating RRAM errors according to claim 12, wherein the artificial neural network is a binary neural network for representing the state of each memory cell of the RRAM as a low resistance state or a high resistance state. Claim 14 A method for emulating RRAM errors according to claim 12, wherein the artificial neural network comprises a plurality of layers sequentially connected such that the output of a predetermined layer is input to the next layer, and the step of injecting into the artificial neural network comprises multiplying the error noise by the output of the corresponding layer of the artificial neural network and inputting it to the next layer. Claim 15 delete Claim 16 An RRAM error emulation method according to claim 10, wherein the actual current of the bit line is calculated by the following mathematical formula. (Mathematical formula) I j : Actual current of the j-th bit line : Random value generated by the random generator : Scale factor indicating the effect of noise in RRAM devices : Voltage of the j-th bit line m: Number of word lines : Ideal equivalent resistance of the j-th bit line : Bit line wire resistance , : Approximation coefficient pre-set to represent the actual equivalent resistance of the j-th bit line Claim 17 A method for RRAM error emulation according to claim 10, wherein the step of generating error noise comprises: a step of calculating a read accuracy of a bit line based on the difference between the actual current of the bit line and the ideal current of the bit line; and a step of setting the calculated read accuracy as error noise. Claim 18 A method for emulating an RRAM error according to claim 17, wherein the read accuracy of the bit line is calculated by the following mathematical formula. (Mathematical formula) : Actual current of the j-th bit line : Ideal current of the j-th bit line Claim 19 A computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises one or more instructions for performing RRAM error emulation to mimic the effect of an error occurring in a Resistive Random Access Memory (RRAM) device, and when the instructions are executed by a computing device having one or more processors, the computing device performs the steps of: generating error noise; and injecting the generated error noise into an artificial neural network implemented in a crossbar array of the RRAM, wherein the step of generating error noise includes: generating a random value which is a number of a preset size; and calculating the actual current of each bit line of the RRAM based on the random value. A computer program stored in a non-transient computer-readable storage medium, comprising the step of generating error noise to mimic an error occurring in the RRAM device based on the difference between the actual current of the bit line and the ideal current of the bit line, wherein the actual current of the bit line is calculated by reflecting the wire resistance of the bit line, the ideal equivalent resistance of the bit line, and the approximation coefficient pre-set to represent the actual equivalent resistance of the bit line.

Images