Neural network circuits and devices

Abstract

The present invention relates to neural network circuits and devices. [Solution] In one embodiment, a neural network circuit includes a synapse memory cell arranged along an output line and including a resistive memory element having one of a first resistance value and a second resistance value, and in response to receiving an input signal via an input line, generates a column signal based on the resistive memory element and the input signal, a reference memory cell arranged along a reference line and including a reference memory element having the first resistance value or the second resistance value in a ratio predetermined by an application, and generates a reference signal based on the reference memory element and the input signal, and an output circuit that generates an output signal for the output line from the column signal and the reference signal.

Description

[Technical field]

[0001] The following disclosure relates to neural network circuits and devices. [Background technology]

[0002] Currently, in von Neumann computer architectures, the frequent movement of huge amounts of data between the processor and memory causes long delays and high power consumption, limiting the performance of the chip. In machine learning applications, for example, data movement between layers of a neural network is a major aspect of the computational load. Considering the computational cost when implementing machine learning in von Neumann architectures, current software-based deep neural network operations can use artificial intelligence (AI) accelerator hardware. Currently, software-based deep neural network operations use AI accelerator hardware such as high-performance CPUs, GPUs, and ASICs.

[0003] Neuromorphic architectures perform computations immediately at the location of the memory element where data is stored, and can store and update the connection strengths (e.g., synaptic weights) between neuronal circuits in the memory element. Neuromorphic computation methods can be applied to artificial intelligence, big data, sensor networks, pattern / object recognition, etc. Neuromorphic architectures can be realized as hardware using analog memory. Summary of the Invention [Means for solving the problem]

[0004] In one embodiment, a neural network circuit includes a synapse memory cell arranged along an output line, the synapse memory cell including a resistive memory element having one of a first resistance value and a second resistance value, and in response to receiving an input signal via an input line, generates a column signal based on the resistive memory element and the input signal; a reference memory cell arranged along a reference line, the synapse memory cell including a reference memory element having the first resistance value or the second resistance value in a ratio predetermined by an application, the reference memory cell generating a reference signal based on the reference memory element and the input signal; and an output circuit generating an output signal for the output line from the column signal and the reference signal.

[0005] The synapse memory cell includes a resistive memory element having a bit number for expressing a synapse weight value assigned to the synapse memory cell, and the resistive memory elements having the bit number may be arranged along the same input line.

[0006] The reference memory cells may include reference memory elements having a number of bits for expressing the synapse weight values, and the reference memory elements having the number of bits may be arranged along the same input line.

[0007] The reference memory cell may include a reference memory element for representing a code bit.

[0008] The output circuit can generate as the output signal a current corresponding to a positive integer multiple of a net current that is a difference between a first current based on a resistive memory element of the first resistance value and a second current based on a resistive memory element of the second resistance value.

[0009] The output circuit can generate as the output signal a current corresponding to a negative integer multiple of a net current that is a difference between a first current based on a resistive memory element of the first resistance value and a second current based on a resistive memory element of the second resistance value.

[0010] The output circuit may further include a leaky integration and fire circuit that fires other neuron circuits based on a result of comparing the output signal with a threshold value.

[0011] The LIF circuit can increase its output voltage in response to the output signal corresponding to a positive integer multiple of the net current.

[0012] The LIF circuit may reduce its output voltage in response to the output signal corresponding to a negative integer multiple of the net current.

[0013] The LIF circuit may leak the output signal in response to a voltage accumulating on the output signal not reaching a threshold voltage within a critical time.

[0014] The LIF circuit can fire the other neuron circuit in response to a voltage accumulated based on the output signal reaching a threshold voltage within a critical time corresponding to the threshold.

[0015] In one embodiment, the neural network circuit may further include a threshold memory array including a plurality of memory elements, at least one memory element selected from the plurality of memory elements based on the set threshold value has the first resistance value.

[0016] In one embodiment, the neural network circuit may further include an additional reference memory cell sharing a reference word line with the threshold generation circuit and disposed along the reference word line, the additional reference memory element having the second resistance value.

[0017] The output circuit may set a critical time corresponding to the threshold based on a signal generated based on the threshold memory cell and a signal generated based on the additional reference memory cell.

[0018] The output circuit can begin accumulating a current corresponding to a difference between a signal generated based on the threshold memory cell and a signal generated based on the additional reference memory cell, and when a voltage corresponding to the accumulated current exceeds a threshold voltage, output a signal indicating a threshold time corresponding to the threshold.

[0019] The output circuitry can apply a critical time corresponding to the threshold determined based on the threshold memory cell and the additional reference memory cell to the output signal for the output line and other output signals for other output lines.

[0020] The resistive memory elements of the synapse memory cells connected to the same output line can be connected in parallel with each other.

[0021] In one embodiment, the neural network circuit further includes different synapse memory cells arranged along different output lines from the output line, and the output circuit can generate output signals individually using the same reference memory cells for each of the output line and the different output line.

[0022] The output circuit may include a read circuit that generates a column integrated signal by accumulating the column signals for each bit of the synapse memory cells, and generates a reference integrated signal by accumulating the reference signals for each bit of the reference memory cells.

[0023] The read circuit may include a current mirror that copies the column signal for each bit of the synapse memory cell and the reference memory cell with a current multiple corresponding to that bit.

[0024] The output circuitry may generate the output signal corresponding to a difference between the column accumulation signal and the reference accumulation signal.

[0025] The output circuit may include a capacitor through which a current corresponding to the difference between the column accumulation signal and the reference accumulation signal flows by causing a current corresponding to the reference accumulation signal to flow in toward a node and a current corresponding to the column accumulation signal to flow out from the node.

[0026] The output circuit can obtain a multiply and accumulate (MAC) between the input signal received along the input line and the synaptic weight value based on the result of interpreting the output signal, and transmit a node value determined based on the obtained multiply and accumulate (MAC) to another neuron circuit.

[0027] According to one embodiment, a method for operating a neural network circuit includes the steps of: generating a column signal based on a resistive memory element of a synapse memory cell to which an input signal is applied via an input line, among one or more memory cells arranged along an output line; generating a reference signal based on a reference memory element having a reference resistance value of a reference memory cell to which the input signal is applied, among one or more memory cells arranged along a reference line, and the input signal; and generating an output signal for the output line from the column signal and the reference signal, wherein the reference resistance value can be determined based on a predetermined combination of a first resistance value and a second resistance value different from the first resistance value depending on an application. [Brief description of the drawings]

[0028] [Figure 1] 1 illustrates a neural network circuit according to one embodiment. [Diagram 2] 1 illustrates a synapse memory cell and readout circuit according to an embodiment. [Diagram 3] 1 illustrates a Leaky Integration and Fire (LIF) circuit for an output circuit according to an embodiment. [Figure 4]1 illustrates a threshold generation circuit and an additional reference circuit included in a neural network circuit according to an embodiment. [Figure 5A] 1 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to one embodiment. [Figure 5B] 1 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to one embodiment. [Figure 5C] 1 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to one embodiment. [Figure 6A] 10 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to another embodiment. [Figure 6B] 10 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to another embodiment. [Figure 6C] 10 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to another embodiment. [Figure 7] 4 illustrates a critical time generation circuit of an output circuit according to an embodiment. [Figure 8] 1 shows elements included in an LIF circuit in an output circuit according to an embodiment, and a timing diagram for each element. [Figure 9] An example of a firing operation of a neural network circuit according to an embodiment will be described. [Figure 10] 4 is a flowchart illustrating a method of operating a neural network circuit according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] The specific structural or functional description of the embodiments is disclosed for the purpose of illustration only, and may be modified in various forms. Therefore, the embodiments are not limited to the specific disclosed forms, and the scope of the present specification includes modifications, equivalents, or alternatives within the technical spirit.

[0030] Although terms such as first or second may be used to describe multiple components, such terms should be construed only for the purpose of distinguishing one component from the other components. For example, a first component may be named a second component, and similarly, a second component may be named a first component.

[0031] When any component is referred to as being "connected" to another component, it is directly linked or connected to the other component, but it should be understood that there may be other components in between.

[0032] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, the terms "comprise" or "have" and the like indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, and should be understood as not precluding the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0033] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Commonly used predefined terms should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted as ideal or overly formal unless expressly defined in this specification.

[0034] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the drawings, the same components are given the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted.

[0035] FIG. 1 illustrates a neural network circuit according to one embodiment.

[0036] The neural network circuit 100 according to an embodiment can be realized as a circuit that replicates synapses connecting neuron circuits of a previous layer and neuron circuits of a target layer in a neural network. An electronic device including the neural network circuit 100 may be referred to as a neural network device. The neuron circuit may include a circuit that realizes activation of neurons and / or activation functions of a neural network. The neural network includes a number of layers each having a number of nodes, and the previous layer represents a layer previously connected to the target layer in the neural network. The nodes of the neural network correspond to the neuron circuits of the neural network circuit 100. The neural network circuit 100 can transmit node values ​​(e.g., input values) output from the neuron circuits of the previous layer to the neuron circuits of the target layer based on the connection strength (e.g., synaptic weights) between the corresponding neuron circuits. The target neuron circuit of the target layer can have, as a node value, a result of applying an activation function to a weighted sum based on the node values ​​and synaptic weights received from the previous neuron circuits of the previous layer connected to the target neuron circuit. In this specification, the neuron circuit may be implemented in the output circuit 150, and the neuron circuit of the neural network circuit 100 implementing a spiking neural network may include a leaky integration and fire (LIF) circuit, as described below. However, without being limited thereto, the activation function of the neural network may be implemented in a circuit designed according to other methods. Here, the activation value from the activation function or the result of the activation function may be distinguished according to whether the line or operator is in an active or inactive state, for example, whether the line is activated or inactivated. However, if a non-zero activation value output of a previous layer is applied to an input line, for example, the input line is considered to be active or activated, and conversely, if a zero activation value or signal is not applied, the line is considered to be inactive or inactivated.

[0037] Neural network circuit 100 includes a synapse memory array 110, a reference memory array 120, and an output circuit 150. Synapse memory array 110 and reference memory array 120 may be referred to as memory arrays.

[0038] The synapse memory array 110 includes a plurality of synapse memory cells 111 arranged along input lines and output lines. The plurality of synapse memory cells 111 may be arranged in a crossbar array configuration. The input lines are lines receiving inputs, and are shown as K word lines WL0 to WLK-1 in FIG. 1, where K may be an integer equal to or greater than 1. The signals from each word line indicate "0" or "1", respectively, which are outputs of the K nodes in the previous layer. The output lines may output an output signal 159 indicating a sum (e.g., a multiplication and accumulation (MAC) result value or a weighted sum) of a synapse weight value indicated by a resistive memory element of the synapse memory cell 111 arranged along the output line and an operation result (e.g., a product result) between input values ​​indicated by each input signal. The output lines are shown as J lines in FIG. 1, where J may be an integer equal to or greater than 1. Thus, the synapse memory array 110 may include K×J synapse memory cells 111. Given a synapse memory cell 111 in a jth cell column (j is 1 to J) and a kth cell row (k is 1 to K), the value stored in the given synapse memory cell 111 is a synapse weight value between a jth node of a previous layer and a kth node of a current / next layer. In other words, the neural network circuit 100 shown in FIG. 1 may be interpreted as a circuit in which synaptic connections connecting K previous nodes and J next nodes are embodied. If the synapse weight value is expressed as a bits, each output line may include a bit lines, where a may be an integer equal to or greater than 1. An output signal 159 output from any one output line is a difference signal between a signal obtained by accumulating column signals (e.g., bit column signals) generated by bit lines included in the output line and a signal obtained by accumulating a reference signal generated by a reference line.

[0039] The input line may receive an input signal via a pre-synaptic circuit. The pre-synaptic circuit may receive an output of a previous node (e.g., a previous neuron circuit) of the neural network and transmit it to a synaptic memory element of the current node. The pre-synaptic circuit may be referred to as an axon circuit.

[0040] The output line is connected to a post-synaptic circuit and transmits a signal corresponding to the sum of products between the synaptic weights of the synaptic memory elements connected to the output line and the input signal to the post-synaptic circuit. The post-synaptic circuit may fire or transmit an output signal 159 corresponding to the sum of products between the synaptic weights and the input signal to a next node (e.g., a next synaptic circuit). The post-synaptic circuit may be represented as a dendrites circuit, and may be illustratively realized as a LIF circuit, which will be described later.

[0041] An input signal indicates a signal received through an input line. In this specification, when the neural network circuit 100 receives an input signal corresponding to a bit value "1" from any one of the input lines, an input voltage may be applied to the input line. As a different example, when the input signal of any one of the input lines indicates a bit value "0", the neural network circuit 100 may inactivate the input line. In other words, the neural network circuit 100 may apply no voltage or a voltage of 0V to the input line. However, the input voltage applied for each bit value indicated by the input signal is not limited to the above. In the example shown in FIG. 1, among the K input lines, an activation value (for example, a voltage corresponding to a bit value "1") may be applied to k input lines. k is an integer between 0 and K. That is, for a given input, some input lines become activated "1" and other lines become inactivated "0".

[0042] The synapse memory cells 111 may be arranged along an output line and may include resistive memory elements having one of a first resistance value and a second resistance value. The synapse memory cells 111 may include resistive memory elements having a bit number for expressing a synapse weight value assigned to the synapse memory cells 111. The resistive memory elements having the above-mentioned bit number may be arranged along the same input line.

[0043] In response to receiving an input signal via an input line, the synapse memory cell 111 may generate a column signal based on the resistive memory element and the input signal. The first resistance value and the second resistance value may be mapped to a binary coded value. For example, the first resistance value and the second resistance value may indicate a bit value of "0" or "1", respectively. Exemplarily, the first resistance value being a low resistance indicates a bit value of "0", and the second resistance value being a high resistance indicates a bit value of "1". However, without being limited thereto, the bit values ​​mapped to the first resistance value and the second resistance value may vary depending on the design.

[0044] When the synapse weight value is expressed as a bit sequence having multiple bits, the synapse memory cell 111 may include a sub-cells. For example, the synapse weight value may be expressed as a bit sequence of a bits. For a synapse weight value of a bits, the synapse memory cell 111 may include a sub-cells. The first sub-cell may have a resistance value of a bit value corresponding to a least significant bit (LSB) digit of the bit sequence indicating the synapse weight value, and the a-th sub-cell may have a resistance value of a bit value corresponding to a most significant bit (MSB) digit. Each of the a sub-cells may include a resistive memory element having a resistance value corresponding to a bit value of a corresponding bit digit of the synapse weight value. The resistive memory element included in each of the a sub-cells may be set to have a resistance value mapped to a bit value of a corresponding bit digit. It may be set to have one of a first resistance value and a second resistance value. In FIG. 2 below, an example in which the synapse memory cell 111 is embodied as a 3-bit cell will be described.

[0045] The column signal is a signal obtained by accumulating signals output from the synapse memory cells 111 arranged along any one of the column lines in the synapse memory array 110. For example, the column signal may be a signal corresponding to the sum of products between the bit values ​​corresponding to the resistive memory elements of the subcells connected to the same bit line in the synapse memory array 110 and the input value of the input signal. In other words, the column signal of any one of the bit lines may be a signal corresponding to the sum of products of the bit digits indicated by the bit line in the output line. As will be described later, the neural network circuit 100 can obtain a signal obtained by multiplying each of the column signals of the bit lines included in the same output line by a weighting value (hereinafter, "bit weighting value") corresponding to the bit digit of the bit line. For example, the bit weighting value corresponding to the LSB may be 1 / 4, the bit weighting value corresponding to the first bit digit from the LSB may be 1 / 2, and the bit weighting value corresponding to the second bit digit from the LSB may be 1. However, the bit weighting value of the second bit digit from the LSB, which is the MSB, is designed assuming that the bit weighting value is 1, and the bit weighting value may be changed according to the design, rather than being input thereto. For convenience of explanation, certain columns are shown herein as mainly vertical lines, but the shape of the column lines may vary depending on the design, without being limited thereto.

[0046] As described above, the resistive memory element is an element having one of a first resistance value and a second resistance value. For example, the resistive memory element may be a magnetic ram (MRAM). The MRAM may have one of two states of a magnetic tunnel junction (MTJ). The MRAM in a parallel state (P state) may have a first resistance value, and the MRAM in an anti-parallel state (AP state) may have a second resistance value. The second resistance value may be greater than the first resistance value. The second resistance value may be referred to as a high resistance value, and the first resistance value may be referred to as a low resistance value. The AP state may be referred to as a high resistance state, and the P state may be referred to as a low resistance state. The resistance ratio of the second resistance value to the first resistance value of the MRAM may be 2. However, the resistance ratio of the resistive memory element is not limited to the above, and may vary depending on the type of element. In one embodiment, the neural network circuit 100 can generate a net signal using a reference memory array 120 described below in order to distinguish signals output according to the first resistance value and the second resistance value of the resistive memory element.

[0047] For reference, the present specification will mainly describe resistive memory elements in a P state and an AP state for convenience of description, but is not limited thereto. Unless contrary to the description, the P state may be interpreted as a state having a first resistance value, and the AP state may be interpreted as a state having a second resistance value.

[0048] The reference memory array 120 includes a plurality of reference memory cells 121 arranged along the reference lines. As an example shown in FIG. 1, the reference memory cells 121 included in the reference memory array 120 may be arranged along the reference lines. The reference memory array 120 may include the same number of reference memory cells 121 as the number of input lines.

[0049] The reference memory cell 121 may include a reference memory element arranged along a reference line and having a programmable resistance value according to an application. Here, having a "programmable" resistance value means that the resistance value can be dynamically changed. The reference memory cell 121 may include reference memory elements having a number of bits for expressing a synapse weight value. The reference memory elements having the above-mentioned number of bits may be arranged along the same input line.

[0050] Each reference memory cell 121 according to an embodiment may include reference sub-cells, in other words, resistive memory elements, the number of which is equal to the number of bits, similar to the above-described synapse memory cell 111. Here, the reference memory cell 121 may include resistive memory elements set to a predetermined resistance value among a first resistance value and a second resistance value.

[0051] In another embodiment, each reference memory cell 121 may be a one-bit cell that stores a single bit of information (eg, a sign bit), regardless of the number of bits of the synapse memory cell 111 .

[0052] The reference memory cells 121 may generate a reference signal based on the reference memory elements and the input signal. The reference signal is an accumulated signal of signals output from the reference memory cells 121 arranged along the reference lines in the reference memory array 120. The reference lines may also include reference bit lines as many as the number of bits. For example, the reference signal is a signal corresponding to the sum of products between the input value of the input signal and the bit values ​​corresponding to the resistive memory elements of the reference sub-cells connected to the same reference bit line in the reference memory array 120. In other words, the reference signal of any one of the reference bit lines may be a signal corresponding to the sum of products of the bit digits indicated by the reference bit line.

[0053] The output circuit 150 generates an output signal 159 for the output line from the column signal and the reference signal. For reference, the neural network circuit 100 may further include different synapse memory cells arranged along different output lines from the output line. In this case, the output circuit 150 can generate output signals separately for the output line and the different output lines using the same reference memory cells. In other words, the output circuit 150 can use the same reference column signal to generate different output signals for each output line. The operation of the output circuit 150 will be described later.

[0054] In the neural network circuit 100 according to an embodiment, during synapse operation, when K word lines for accessing each synapse memory element are input, all signals (e.g., current signals) generated for each column are integrated. The output lines include bit lines corresponding to bit digits, and a column integration signal I is output from the output lines. Cells may be a signal obtained by integrating a signal to which a bit weight is applied to a column signal of an individual bit line. The column accumulated signal is expressed as the following Equation (1).

[0055]

number

[0056] In the above formula (1), I P denotes the current flowing through a resistive memory element in the P state (e.g., a first resistance value), and I AP indicates the current flowing through a resistive memory element in the AP state (e.g., a second resistance value). The column accumulation signal I Cells I P and I AP The synapse readout circuit 151 of the output circuit 150 outputs the column accumulation signal I Cells The reference accumulation signal I REF can be calculated as shown in equation (2).

[0057]

number

[0058] The above-mentioned equation (2) represents a signal in which a bit weight value for each bit digit is applied to a current signal flowing through a resistive memory element to which an input signal is applied among resistive memory elements included in the reference memory array 120. REF is a signal obtained by applying a bit weighting value to the reference signal for each bit digit and integrating it. Here, X may be determined to be any one of integers smaller than or equal to N and larger than or equal to 0 according to the application. When the reference memory cell 121 is a 1-bit cell, X may have a value of 0 or N.

[0059] The reference read circuit 152 of the output circuit 150 outputs the reference accumulated signal I REF In the above-mentioned formula (1) and formula (2), the coefficients N and M of the linear combination are expressed as the following formulas (3) and (4).

[0060]

number

[0061]

number

[0062] N indicates a value determined based on the subcells connected to the activated word lines. k indicates the number of activated word lines (e.g., input lines) as described above. a indicates the number of subcells included in each memory cell, and is the number of bits of the synapse weight value. For example, N is the sum of powers of 2, the exponent of the bit digits indicated by the subcells included in the synapse memory cells 111 connected to the activated word lines among the synapse memory cells 111 connected to any one output line. M is an integer equal to or less than N, and is a value determined based on the subcells in the P state (e.g., the first resistance value) among the synapse memory cells 111 connected to the activated word lines. m ij is 1 if the i-th synapse memory cell of the activated j-th word line has a first resistance value, and is 0 if the i-th synapse memory cell has a second resistance value. For example, M is the sum of k powers of 2 whose exponents are the bit digits indicated by the subcells in the P state among the synapse memory cells 111 connected to the activated word lines among the synapse memory cells 111 connected to any one of the output lines. In summary, N is proportional to the number of bits of the activated input lines, and M is proportional to the number of bits of the activated input lines in the P state (e.g., "1").

[0063] Output signal 159I net、column , i.e., the integrated column accumulated signal I Cells The reference cumulative signal I integrated with REF The net signal is calculated with reference to equation (5).

[0064]

number

[0065] The output circuit 150 of the neural network circuit 100 according to the embodiment outputs the column accumulated signal I Cells and the reference cumulative signal I REF Output signal 159I corresponding to the difference between net、column can be generated.

[0066] A first current I based on the resistive memory element having the first resistance value P and a second current I based on the resistive memory element having the second resistance value. AP Difference between (I P -I AP ) may be denoted as a valid signal or a net signal. P -I AP Since the output signal 159I is a current-type signal, it may be expressed as a net current. net、column corresponds to a factor (MX) of the net signal, where (MX) can be a positive or negative integer depending on the value of X.

[0067] In the present specification, the neural network circuit 100 is mainly described as a spiking neural network (SNN) that realizes a function of simulating a biological neural network that accumulates current according to synaptic weights, but is not limited thereto. The neural network circuit 100 according to an embodiment may be applied to a system that realizes a multiply-accumulate (MAC) operation, such as a computation in memory (CIM) circuit using a memory with a low resistance ratio and a vector matrix multiplication (VMM) circuit. Even if the resistance ratio between the low resistance and the high resistance of the resistive memory element included in the neural network circuit 100 is not high, the output value can be distinguished through the above-mentioned net signal. In the neural network circuit 100 according to an embodiment, even if a resistive memory element with a resistance ratio of a first resistance value to a second resistance value exceeding 1 is used, the output can be distinguished through a net signal. Therefore, constraints in design for detecting and summing currents output from a memory array are reduced.

[0068] Therefore, the neural network circuit 100 according to one embodiment can have a memory array of increased size by compensating for the high resistance state. Furthermore, the neural network circuit 100 can be applied to artificial intelligence processors that handle large amounts of data.

[0069] Furthermore, as will be described in detail below, the neural network circuit 100 according to one embodiment can realize two typical types of reactions in biological neural networks, EPSP (Excitatory Post Synaptic Potential), in which the potential increases due to an input stimulus, and IPSP (Inhibitory Post Synaptic Potential), in which the potential decreases, in one synapse memory array 110. More specifically, the neural network circuit 100 can realize the following two types of reactions in biological neural networks, EPSP (Excitatory Post Synaptic Potential), in which the potential increases due to an input stimulus, and IPSP (Inhibitory Post Synaptic Potential), in which the potential decreases, in one synapse memory array 110. net、column If is proportional to a positive multiple of the net signal, then the ESPS that increases its voltage in response to a stimulus is called the output signal 159I net、column If is proportional to the negative of the net signal, an IPSP can be achieved that reduces voltage in response to a stimulus.

[0070] For reference, the present specification mainly describes an example in which the output circuit 150 includes a LIF circuit as an activation function circuit, but is not limited thereto. For example, the neural network circuit 100 may include an analog-to-digital converter instead of the LIF circuit to convert the aforementioned output signal from an analog signal to a digital value (e.g., output data). An electronic device including the neural network circuit 100 can apply an activation function to the output data to determine a value to be propagated to the next node. In other words, the electronic device can also perform a calculation corresponding to the activation function at a digital level.

[0071] FIG. 2 illustrates a synapse memory cell and readout circuit according to one embodiment.

[0072] An output circuit of a neural network circuit according to an embodiment includes a read circuit 251 and an LIF circuit connected to a synapse memory array 210. Although not shown in Fig. 2, the output circuit may also include a reference read circuit 251 connected to a reference memory array. Although the synapse read circuit 251 is mainly described in Fig. 2, the reference read circuit 251 may also be configured in the same manner as the synapse read circuit 251.

[0073] As shown in Fig. 2, the resistive memory elements of the synapse memory cells 211 connected to the same output line can be connected in parallel to each other. For example, among the synapse memory cells 211 connected to the same output line, the resistive memory elements connected to the same bit line can be connected in parallel to each other. Thus, the current signals output from the resistive memory elements of the same bit line can be integrated along the output line. In Fig. 2, the resistance of the resistive memory element is R DATA As described above, among the resistive memory elements arranged along the bit lines BL0 to BL2, the word lines WL 0 ~WL K-1 A resistive memory element to which an input signal is applied via a word line WL can generate a current signal according to a voltage across the resistive memory element. 0 ~WL K-1 If an input signal is applied along data is turned on.

[0074] Switch sense A switch (e.g., a transistor) can connect the resistive memory element to the sensing line. Here, when a Sel_data signal is applied, the switch SW sense (eg, transistors) are turned on, connecting the bit lines BL0-BL2 and the sense line SL to a supply voltage and ground, respectively.

[0075] The operational amplifier OP has a high gain and can regulate the voltages of the bit lines BL0 to BL2 to V, regardless of the resistance values ​​seen between the sensing line SL and the bit lines BL0 to BL2.amp_ref In other words, the voltage difference between the sensing line SL and the bit lines BL0 to BL2 is kept constant. The resistive memory elements are held constant by applying the same fixed voltage V amp_ref For resistance R DATA may generate a current (eg, a column signal) on the bit lines BL0-BL2 determined by:

[0076] The read circuit 251 of the output circuit includes a current mirror that copies a column signal for each bit of the synapse memory cell 211 and the reference memory cell at a current multiple corresponding to the bit. For example, the current mirror of the read circuit 251 may copy a column signal generated in a bit line to other bit lines BL'0 to BL'2. The magnitude of the copied current may vary depending on the width of the transistor. As described above, the current mirror may copy the column signal at a current copy ratio corresponding to the bit weight. For example, in FIG. 2, the first bit line BL0 is copied at 1 / 4, the second bit line BL1 is copied at 1 / 2, and the third bit line BL3 is copied at 1.

[0077] The read circuit 251 of the output circuit integrates the column signals of the synapse memory cells 211 for each bit to generate a column integrated signal I Cells A reference integrated signal I is generated by accumulating the reference signals for each bit of the reference memory cell. REF For example, the output circuit can sum the replicated currents from the column signals to generate a column accumulation signal I Cells The output circuit can generate a reference accumulated signal I by the same read circuit 251 and operation as described above. REFFor reference, when the resistive memory element is realized as an MRAM, the current flowing through the MTJ is small, so that no read disturbance occurs, and an amplifier capable of processing a relatively low common input is required. The output circuit outputs a column accumulation signal I Cells and the reference cumulative signal I REF and a LIF circuit 260 for processing the generated output signal. The structure and operation of the LIF circuit 260 will be described later.

[0078] The output circuit receives the column accumulation signal I Cells and the reference cumulative signal I REF By using an output signal that is the difference between the P and AP resistive memory elements, it is possible to accurately distinguish between the P and AP resistive memory elements. In addition, a reference memory element having a programmable resistance value is used to generate a reference accumulation signal I REF , the output signal has values ​​corresponding to not only positive integer multiples of the net signal, but also negative integer multiples, and the output circuit can use this to represent IPSPs as well as EPSPs.

[0079] In other words, since the net signal is always positive, the output signal is a positive integer multiple of the net signal, so the column accumulation signal I Cells is the reference cumulative signal I REF It is understood that the output signal is greater than the negative integer multiple of the net signal, and the column accumulation signal I Cells is the reference cumulative signal I REF is understood to be smaller than

[0080] FIG. 2 illustrates an output signal obtained by reading the resistance value of a resistive memory element included in a synapse memory array by a read circuit 251 of the output circuit, and FIG. 3 below illustrates a firing process using the output signal.

[0081] FIG. 3 illustrates a Leaky Integration and Fire (LIF) circuit of an output circuit according to one embodiment.

[0082] The output circuit 150 according to one embodiment includes a LIF circuit 260. The LIF circuit 260 is a circuit for replicating neuron operation and can trigger other neuron circuits based on the result of comparing the output signal 159 with a threshold. The output signal 159 is expressed by the I net、column The LIF circuit 260 includes a leakage subcircuit 361 and an ignition subcircuit 362.

[0083] The post-synaptic potential includes EPSP, which is an increase in voltage in response to a stimulus, and IPSP, which is a decrease in voltage in response to a stimulus. EPSP may be called an excitatory postsynaptic potential, and IPSP may be called an inhibitory postsynaptic potential. In a neuron, if a stimulus exceeding a threshold is transmitted from a pre-neuron to a post-neuron, or if multiple stimuli below the threshold are transmitted several times within a short period of time, an action potential may be generated. The membrane potential of the neuron may then be reset to a base voltage. However, if a stimulus below the threshold is transmitted from a pre-neuron to a post-neuron, a post synaptic potential (PSP) is generated instead of an action potential, and leakage may occur little by little in the PSP until the next stimulus is received. The LIF circuit 260 is a circuit that realizes the above-mentioned neuronal firing or leakage.

[0084] The leakage portion circuit 361 may receive the column accumulation signal and the reference accumulation signal described above with reference to FIG. 2. In FIG. 3, the column accumulation signal and the reference accumulation signal are modeled as current sources, but this is for convenience of explanation and is actually interpreted as a current supplied by the readout circuit shown in FIG. 2. The leakage portion circuit 361 includes a capacitor. For example, the output signal 159 causes a current corresponding to the reference accumulation signal to flow in toward a node of the capacitor, and a current corresponding to the column accumulation signal to flow out from a node such as the capacitor. Due to the inflow of the reference accumulation signal and the outflow of the column accumulation signal described above, a current corresponding to the difference between the column accumulation signal and the reference accumulation signal may flow in the capacitor.

[0085] The output circuit 150 outputs a first current I based on the resistive memory element having a first resistance value. P and a second current I based on the resistive memory element having a second resistance value. AP and generates as output signal 159 a current corresponding to an integer multiple of a net current that is the difference between. Leakage subcircuit 361 is deactivated while reset signal RESET is applied and is activated while reset signal RESET is not applied. In other words, leakage subcircuit 361 can leak an output current due to output signal 159 for a critical time after reset signal RESET is not applied.

[0086] The output current of the output signal 159 is converted into an output voltage by flowing through a capacitor. integ is connected at its output to a window switch, which is connected to a capacitor connected to ground. While a window signal WINDOW is applied to the window switch, the leakage operational amplifier OP integ and a capacitor connected to ground are connected. Therefore, the output current of the output signal 159 leaks into the operational amplifier OP integThe voltage flows to a capacitor connected to ground located at the output terminal of the output terminal of the inverter. The capacitor connected to ground may convert the output current to an output voltage by leaking the charge due to the output current. Here, the voltage charged to the capacitor connected to ground leaks due to the leakage transistor LEAKAGE.

[0087] The leakage transistor LEAKAGE has a V CM_COLOUMN_IN An activation voltage based on the signal WINDOW may be applied to the window switch. The window switch is turned on for a critical time (e.g., leakage time) by the window signal WINDOW. In other words, a capacitor connected to ground leaks output current until a critical time has elapsed after reset.

[0088] According to one embodiment, the leakage subcircuit 361 can transmit the leakage output voltage to the firing subcircuit 362 for the critical time period mentioned above. fire is the preset threshold voltage V TH and the output voltage V COLUMN For example, the comparator OP fire is the accumulated output voltage V during the critical time COLUMN is the threshold voltage V TH If the voltage exceeds the threshold, a firing signal (e.g., 1) is output, and the output voltage V COLUMN is the threshold voltage V TH If it is less than or equal to, a leakage signal (eg, 0) may be output.

[0089] For reference, as will be described later, the critical time is generated by a separate circuit (e.g., a critical time generating circuit) through leakage of a threshold current corresponding to the threshold voltage V TH Since the output current and the threshold current are both currents, it is difficult to compare one current with the other. Therefore, the comparator OP firecan compare the voltage converted from the current leaked during the critical time from the moment the RESET signal becomes 0 to the moment the WINDOW signal becomes 0, which is generated by the WINDOW signal generated in FIG. 7 described later, with the threshold voltage. If the current is small, the amount converted into voltage is also small, and the converted voltage is smaller than the threshold voltage. Conversely, if the current is large, the amount converted into voltage is large, and the converted voltage rises above the threshold voltage, and the comparator OP fire outputs a fire pulse.

[0090] According to one embodiment, the LIF circuit can leak the output signal 159 in response to a voltage accumulated based on the output signal 159 not reaching a threshold voltage within a critical time. For example, the firing subcircuit 362 of the LIF circuit 260 can leak the converted output voltage to the leakage operational amplifier OP if the leakage and converted output voltage are less than the threshold voltage. integ The analog MUX transfers the converted output voltage to the leakage operational amplifier OP. integ The new common mode input V CM_COLUMN The leaky circuit can be provided as follows: when the output signal 159 based on the next neuron input is generated, the leaky circuit can obtain a leaky integrated output voltage by continuing as leakage due to the output current with respect to the common mode input as described above.

[0091] According to one embodiment, the LIF circuit 260 can fire other neuron circuits in response to the voltage accumulating based on the output signal 159 reaching a threshold voltage within a critical time corresponding to the threshold. For example, when the leakage output voltage exceeds the threshold voltage or the leakage voltage of several small output signals 159 exceeds the threshold voltage, the comparator OP of the firing subcircuit 362 can fire the other neuron circuits. fire may generate a firing signal. The firing signal may be a pulse output that indicates firing. The firing signal is generated by an analog amplifier-based comparator OP that is synchronized to a clock signal. fireIt can also be generated as a signal synchronized with the clock signal via a flip-flop circuit.

[0092] The firing subcircuit 362 controls the selection signal SEL of the analog MUX circuit and receives the V CM_COLUMN_IN Leakage operational amplifier OP integ The common mode voltage can be initialized by transmitting the

[0093] The output circuit 150 according to an embodiment performs an integration operation, a comparison operation, and a leakage operation separately for each circuit. Therefore, the output circuit 150 can minimize errors due to restrictions on the size of a capacitor and leakage current. Furthermore, the output circuit 150 can also mimic firing in units of ms, which is the stimulation period of an actual living organism.

[0094] 3, the RESET signal and the SW signal may be generated by a timing generator to which an external clock Clk is input as a control signal. A critical time generation circuit, which will be described later, may generate a window signal WINDOW indicating a critical time based on the presence or absence of firing. The window signal WINDOW provides a critical time (e.g., integration time) that is robust to variations in the set threshold voltage, resistive memory element, and capacitor.

[0095] FIG. 4 illustrates a threshold generation circuit and an additional reference circuit included in a neural network circuit according to one embodiment.

[0096] The neural network circuit according to an embodiment may further include a circuit for setting a threshold value. For example, the neural network circuit may further include a threshold memory array 480 and an additional reference memory array 490. The synapse memory array 110 and the reference memory array 120 have been described above with reference to FIG. 1, and therefore will not be described here.

[0097] The threshold memory array 480 includes a plurality of memory elements. In the threshold memory array 480, at least one memory element designated based on a set threshold among the plurality of memory elements may have a first resistance value (e.g., P state). In FIG. 4, the memory cells included in the threshold memory array 480 are threshold memory cells, and are illustrated as Th. For example, a memory element for expressing a threshold value may be selected in consideration of a bit digit. In other words, the threshold value may be expressed based on the number of memory elements having the first resistance value and the bit digit represented by the memory element.

[0098] The threshold memory array 480 includes threshold memory cells arranged along any one of the column lines. The column line may include multiple bit lines representing multiple bits. Each of the threshold memory cells may include subcells for each bit line. Exemplarily, a threshold memory array having a 6I P If the threshold current is set as (2+4)*I P In order to express 1 =2) and the second bit digit from the LSB (e.g., 2 2 =4) may have a first resistance value.

[0099] The additional reference memory cells share the reference word lines (Ref WL0 to Ref WL-1) with the threshold generation circuit. The additional reference memory cells may include additional reference memory elements disposed along the reference word lines and having a second resistance value. The memory elements of the additional reference memory cells of the additional reference memory array 490 may have a programmable resistance value depending on the application. The additional reference memory cells may be used to represent a specified threshold value as the threshold memory array 480 described above in more detail below, in a net signal.

[0100] For example, a signal accumulated along a column of the threshold memory array 480 (hereinafter, a "threshold accumulated signal") can be expressed as the following Equation (6).

[0101]

number

[0102] In the above formula (6), I P denotes the current flowing through a resistive memory element in the P state (e.g., a first resistance value), and I AP indicates the current through the resistive memory element in the AP state (e.g., the second resistance value). The threshold accumulation signal I Th I P and I AP The second read circuit 452 of the output circuit 450 outputs the threshold accumulation signal I Th Generate.

[0103]

number

[0104] The above-mentioned equation (7) represents a signal in which a bit weight value for each bit digit is applied to a current signal flowing through a resistive memory element to which a reference word signal is applied among resistive memory elements included in the additional reference memory array 490. REF、This a signal obtained by applying a bit weighting value to the reference signal for each bit digit and integrating it. Here, Y may be determined to be any integer equal to or smaller than R and equal to or larger than 0 depending on the application. If the additional reference memory array 490 is composed of 1-bit cells, Y has a value of 0 or Y.

[0105] The second read circuit 452 of the output circuit 450 outputs the additional reference accumulated signal I REF、Th In the above-mentioned Equations (6) and (7), the coefficients T and R of the linear combination can be expressed as the following Equations (8) and (9).

[0106]

number

[0107]

number

[0108] R indicates a value determined based on the subcells connected to the activated reference word line. L indicates the number of activated reference word lines. An example in which all reference word lines are activated will be described with reference to FIG. 4. a is the number of subcells included in each memory cell, and is the number of bits of the synapse weight value. For example, R is the sum of powers of 2, the exponent being the bit digit indicated by each subcell included in the reference memory cell connected to the activated reference word line among the threshold memory cells connected along one column. T is an integer equal to or smaller than R, and is a value determined based on the subcells in the P state (e.g., the first resistance value) among the threshold memory cells connected to the activated reference word line. T corresponds to a preset threshold value. t ijis 1 if the i-th threshold memory cell of the activated j-th word line has a first resistance value, and is 0 if the i-th threshold memory cell has a second resistance value. For example, T is a sum of k powers of 2 whose exponents are the bit digits of the P-state subcells in the threshold memory cells connected in the same column.

[0109]

number

[0110] The output circuit 450 of the neural network circuit according to the embodiment outputs the threshold accumulation signal I Th and additional reference cumulative signal I REF、Th The threshold value corresponding to the difference between the net signal I net、Th In the above-mentioned formula (5), the output circuit 450 generates an output signal I net、column and the threshold net signal I obtained by the above equation (10). net、Th However, as described above, current-to-current comparison is difficult to realize, and the output circuit 450 also outputs the output signal I net、column and the threshold net signal I net、Th The critical time generating circuit may further include a critical time generating circuit for indirectly comparing the critical time generating circuit with reference to FIG.

[0111] The output circuit 450 according to an embodiment applies a critical time corresponding to a threshold determined based on the threshold memory cell and the additional reference memory cell to an output signal for an output line and another output signal for another output line. In other words, the output circuit 450 can apply a critical time corresponding to a common threshold to output signals obtained from the output lines of the synaptic memory array.

[0112] 5A-5C illustrate an example of a firing comparison using net signals generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to one embodiment.

[0113] 5A, one column of the synapse memory array is shown as synapse memory column 510. One column of the reference memory array is shown as reference column 520. One column of the threshold memory array is shown as threshold generation column 580. One column of the additional reference memory array is shown as additional reference column 590. For ease of explanation, an example is shown in which each column includes three bit lines and an input signal is applied to one input line.

[0114] For example, the third subcell of the synaptic memory column 510 may have a resistive memory element set to a second resistance value, and the first and second subcells may have a resistive memory element set to a first resistance value. The first and third subcells of the reference column 520 may have a resistive memory element set to a second resistance value, and the second subcell may have a resistive memory element set to a first resistance value. However, the resistance values ​​of the subcells of the reference column 520 are not limited to the above examples, and may be dynamically changed by an application.

[0115] Each sub-cell can generate a current signal to which a bit weighting value corresponding to the LSB (e.g., 1x), a bit weighting value corresponding to the first bit digit from the LSB (e.g., 2x), and a bit weighting value corresponding to the second bit digit from the LSB (e.g., 4x) are applied. Cells is (2+4)*I P +1*I AP The reference accumulated signal I REF is 2*I P +(1+4)*I AP The output signal I net、column is 4(I P -I AP ) may be used. The output signal I net、column Since is a positive number, the output signal I net、columnis an output signal for performing EPSP actions.

[0116] Similarly, the threshold generation column 580 generates the threshold accumulation signal I Th is 2*I AP +(1+4)*I P =5I P +2I AP The additional reference accumulated signal I generated by the additional reference column 590 may be REF、Th is 2*I P +(1+4)*I AP =2I P +5I AP In FIG. 5A, the threshold net signal I net、Th is 3(I P -I AP The output circuit may output an output signal I net、column is the threshold net signal I net、Th Because it is larger, it can generate a firing signal.

[0117] 5B, the third subcell of the synaptic memory column 510 has a resistive memory element set to a first resistance value, and the first and second subcells have a resistive memory element set to a second resistance value. In the case of the reference column 520, the first and third subcells may have a resistive memory element set to a second resistance value, and the second subcell may have a resistive memory element set to a first resistance value, as in FIG. 5A. However, the resistance value of each of the subcells of the reference column 520 is not limited to the above example, and may be dynamically changed by the application.

[0118] Each sub-cell generates a current signal to which a bit weighting value corresponding to the LSB (e.g., 1x), a bit weighting value corresponding to the first bit digit from the LSB (e.g., 2x), and a bit weighting value corresponding to the second bit digit from the LSB (e.g., 4x) are applied. Cells is 1*I P +(2+4)*I AP The reference accumulated signal I REF is 2*I P +(1+4)*I AP The output signal Inet、column is -1(I P -I AP ) may be used. The output signal I net、column is negative, so the output signal I net、column is an output signal for performing IPSP operation.

[0119] FIG. 5C shows an example where an input signal is applied to two input lines.

[0120] The output signal I in FIG. net、column is the sum of the output signals of FIG. 5A and FIG. 5B (I P -I AP )-(I P -I AP )=3(I P -I AP ) may be used. Similarly, the threshold net signal I net、Th is 6(I P -I AP The output circuit may output an output signal I net、column is the threshold net signal I net、Th Since it is smaller than , it can perform leaky operation.

[0121] 6A-6C illustrate an example of a firing comparison using net signals generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to another embodiment.

[0122] In Figure 6A, any column of the synapse memory array is shown in synapse memory column 610. A column of the reference memory array is shown in reference column 620. A column of the threshold memory array is shown in threshold generation column 680. Any column of the additional reference memory array is shown in additional base column 690. For ease of explanation, an example is shown in which each column includes three bit lines and an input signal is applied to one input line.

[0123] Exemplarily, the third sub-cell of the synaptic memory column 610 may have a resistive memory element set to a first resistance value, and the first and second sub-cells may have a resistive memory element set to a second resistance value. The reference column 620 may be a 1-bit cell that stores a single bit of information (e.g., a sign bit) regardless of the number of bits of the synaptic memory column 610. More specifically, the reference column 620 may have a resistive memory element set to a second resistance value or a first resistance value in the 1-bit cell depending on whether the operation to be replicated is an EPSP or an IPSP. In FIG. 6A, the operation to be replicated is an EPSP, and the 1-bit cell of the reference column 620 may have a resistive memory element set to a second resistance value.

[0124] Each subcell of the synapse memory column 610 can generate a current signal to which a bit weighting value corresponding to the LSB (e.g., 1x), a bit weighting value corresponding to the first bit digit from the LSB (e.g., 2x), and a bit weighting value corresponding to the second bit digit from the LSB (e.g., 4x) are applied. The column accumulation signal I Cells is (4+2)*I P +1*I AP may be also possible.

[0125] A current signal is generated by applying a predetermined bit weight (for example, 7 times) to the 1-bit cell of the reference column 620. REF is 7*I AP The output signal I net、column is 6(I P -I AP ) may be used. The output signal I net、column Since is a positive number, the output signal I net、column is known to be an EPSP.

[0126] Similarly, the threshold generation column 680 generates the threshold accumulation signal I Th is 2*I AP +(1+4)*I P =5I P +2I APThe additional reference accumulated signal I generated by the additional reference column 690 may be REF、Th is 7*I AP In FIG. 6A, the threshold net signal I net、Th is 5(I P -I AP The output circuit may output an output signal I net、column is the threshold net signal I net、Th Since the voltage Vcc is greater than 1, a firing signal can be generated.

[0127] 6B, the third subcell of the synapse memory column 610 may have a resistive memory element set to a second resistance value, and the first and second subcells may have a resistive memory element set to a first resistance value. In FIG. 6B, the operation to be replicated is IPSP, and the 1-bit cell of the reference column 620 may have a resistive memory element set to a first resistance value.

[0128] Each subcell of the synapse memory column 610 generates a current signal to which a bit weighting value corresponding to the LSB (e.g., 1x), a bit weighting value corresponding to the first bit digit from the LSB (e.g., 2x), and a bit weighting value corresponding to the second bit digit from the LSB (e.g., 4x) are applied. The column accumulation signal I Cells is (2+4)*I P +1*I AP may be also possible.

[0129] A current signal is generated by applying a predetermined bit weight (for example, 7 times) to the 1-bit cell of the reference column 620. REF is 7*I P The output signal I net、column is -1(I P -I AP ) may also be used.

[0130] FIG. 6C illustrates an example where signals are applied to two input lines.

[0131] The output signal I in FIG. net、columnis the sum of the output signals of FIG. 6A and FIG. 6B (I P -I AP )-(I P -I AP )=5(I P -I AP ) may be used. Similarly, the threshold net signal I net、Th is 10(I P -I AP The output circuit may output an output signal I net、column is the threshold net signal I net、Th Since it is smaller than , it can perform leaky operation.

[0132] FIG. 7 illustrates a critical time generating circuit of an output circuit according to one embodiment.

[0133] An output circuit according to an embodiment can set a critical time 791 corresponding to a threshold based on a signal generated based on the threshold memory cell and a signal generated based on an additional reference memory cell. Referring to Fig. 7, a critical time generating circuit 753 for setting the critical time 791 in the output circuit will be described. The first read circuit 451, the second read circuit 452, and the LIF circuit have been described above, and therefore will not be described here.

[0134] As described above, since it is difficult to directly compare the output current with the current set as the threshold, the critical time generation circuit 753 can set the critical time 791 for current comparison. The critical time generation circuit 753 may set the time taken to reach the threshold voltage through current leakage corresponding to the threshold as the critical time 791. As described above, the output circuit compares whether the time taken to reach the same threshold voltage through output current leakage is smaller than the critical time 791. For example, the output circuit can indirectly compare the threshold current with the output current by determining whether the output voltage converted through the output current leakage exceeds the threshold voltage within the set critical time 791.

[0135] The output circuit discloses an accumulation of currents corresponding to the difference between the signal generated based on the threshold memory cell and the signal generated based on the additional reference memory cell, and outputs a signal indicating a critical time 791 corresponding to the threshold when the voltage corresponding to the accumulated current exceeds the threshold voltage.

[0136] The critical time generating circuit 753 includes a configuration similar to that of the leakage portion circuit described above with reference to FIG. 3. For example, the critical time generating circuit 753 applies a threshold accumulation signal I Th and an additional reference accumulated signal I REF、Th Therefore, the threshold net signal I net、Th The capacitors and the operational amplifiers may be designed to have the same type and size.

[0137] The reset signal RESET and the hold signal HOLD shown in the timing diagram 790 are generated by a timing generator. When the reset signal RESET=1, the threshold net signal I net、Th The capacitor is initialized because the voltages across the capacitor are all VCM and TWG. When the reset signal RESET=0 and the hold signal HOLD=1, the capacitor is applied with a threshold net signal I net、Th The voltage converted through accumulation by V TWG The converted voltage V is stored. TWG is the threshold net signal I net、Th It gradually increases due to the accumulation of . Window comparator OP window is the converted voltage V TWG is the threshold voltage V TH If the threshold voltage V exceeds 0, the output is 0. Therefore, the critical time 791 is defined as the time from when the reset signal RESET=0 to when the window signal WINDOW=0. CM、TWGはVCM_COLUMN_IN For example, the output signal I net、column and the threshold net signal I net、Th are the same (for example, I net、column =7I P -7I AP , I net、Th =7I P -7IAP ), V CM、TWG The threshold voltage V TH From V CM_COLUMN_IN The converted voltage V TWG The high voltage of 7I P -7I AP In this case, it can be confirmed as ignition.

[0138] FIG. 8 shows elements included in an LIF circuit in an output circuit according to an embodiment, and a timing diagram for each element.

[0139] The reset flip-flop 810 may generate a reset signal RESET in response to a clock signal divided by 4 and a neuron input (input signal). The hold reset flip-flop 820 may generate a hold signal HOLD in response to a clock signal divided by 8 and a neuron input (e.g., an input signal). The compare reset flip-flop 830 may generate a compare signal COMPARE in response to a clock signal, a supply voltage VDD, and an inverted hold signal HOLDB. The compare signal COMPARE is a signal that starts a comparison operation of the comparator when the accumulated output signal is completely transferred to the input of the comparator. The leakage reset flip-flop 840 generates a leakage signal LEAKAGE in response to a clock signal, a supply voltage VDD, and an inverted comparison signal COMPAREB. The leakage signal LEAKAGE is a signal for performing a leakage operation of the output voltage after the above-mentioned comparison operation. The SW pull ripple loop 870 can generate a SW signal to transmit to the leakage subcircuit, if there is a leakage voltage, to sum the leakage voltage into the output based on the next input. As shown in the timing diagram 890 for the above signals, the critical time 891 is the time period after reset before the window signal WINDOW is inactivated.

[0140] FIG. 9 illustrates an example of a firing operation of a neural network circuit according to one embodiment.

[0141] The neural network circuit includes a memory array 910, an output circuit 950, a divider and timing generator 940, a word line driver 970, and a write driver 980. The memory array 910 includes a synapse memory array, a reference memory array, a threshold memory array, and an additional reference memory array. The output circuit 950 may include a read circuit 951, a critical time generation circuit 953, and an LIF circuit 960. The memory array 910 and the output circuit 950 have been described above, so their description will be omitted. The divider and timing generator 940 generates signals used in the circuit (e.g., clock signals and control signals for individual elements). The word line driver 970 drives the word lines (e.g., input lines) of the memory array 910. The write driver 980 can set the resistance values ​​of resistive memory elements of memory cells arranged along the bit lines of the memory array 910 and drive the bit lines and sensing lines.

[0142] The timing diagram 990 shown in FIG. 9 illustrates an exemplary operation of the neural network circuit with the threshold set as T=6. The threshold net signal I net、Th is 6(I P -I AP ) may be used. <0> The current flowing through P -I AP ) →2(I P -I AP )→7(I P -I AP )+(-3)(I P -I AP ) and the column <1> The current flowing through P -I AP ) → -2(I P -I AP ) → 3(I P -I AP )+(-3)(I P -I AP) as described above. If a stimulus below the threshold is delivered, no firing signal is output, and the accumulated voltage V_integrated is delivered during the period when the leakage pulse (LEAKAGE pulse) is high. The level of the accumulated voltage gradually decreases during the period when the accumulation pulse (LEAKAGE pulse) is high. If a stimulus above the threshold is delivered, a firing signal is output, and the accumulated voltage V_integrated is initialized. Also, if a stimulus below the threshold is delivered and leaks, and further stimuli are delivered so that the sum of the two stimuli exceeds the threshold, a firing signal is output. In the example shown in FIG. 9, since the threshold value T=6, the column <0> When a current flows through the column, it ignites, leaks, and leaks in that order. <1> When a current flows through it, it ignites, leaks, and then leaks again.

[0143] FIG. 10 is a flow chart illustrating a method of operating a neural network circuit according to one embodiment.

[0144] First, in step 1010, the neural network circuit generates a column signal based on the input signal and the resistive memory element of a synapse memory cell to which an input signal is applied via an input line among one or more memory cells arranged along an output line. For example, when an input signal (e.g., a spike signal) is received via a pre-synapse circuit, the neural network circuit accesses a word line of the memory array. A read circuit of the neural network circuit can generate a column signal (e.g., a current) according to the synapse weight value and the input signal.

[0145] Then, in step 1020, the neural network circuit generates a reference signal based on the input signal and a reference memory element having a reference resistance value of a reference memory cell to which the input signal is applied, among one or more memory cells arranged along a reference line. The reference resistance value may be determined based on a predetermined combination of the first resistance value or the second resistance value by an application.

[0146] Next, in step 1030, the neural network circuit generates an output signal for the output line from the column signal and the reference signal. The neural network circuit generates an output signal corresponding to the difference between the column accumulated signal and the reference accumulated signal to cancel the current in the high resistance state.

[0147] Then, in step 1040, the neural network circuit fires or accumulates based on the output signal. For example, the neural network circuit may leak the output signal for a critical time. The neural network circuit fires or accumulates depending on the result of comparing the leaked output voltage to a threshold voltage. The neural network circuit may fire if the output voltage exceeds the threshold voltage. The neural network circuit accumulates if the output voltage is equal to or less than the threshold voltage, and holds the voltage until the next neuron input arrives.

[0148] However, the output signal processing of the neural network is not limited to the above-mentioned step 1040. The output circuit of the neural network circuit can obtain a multiply and accumulate (MAC) between the input signal received along the input line and the synapse weight value based on the result of interpreting the output signal (e.g., an analog-digital converted value), and transmit a node value (e.g., an activation value) determined based on the obtained multiply and accumulate to another neuron circuit.

[0149] The embodiments described above may be implemented using hardware components, software components, or a combination of hardware and software components. For example, the devices and components described herein may be implemented using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or other devices that execute and respond to instructions. The processing device executes an operating system (OS) and one or more software applications that run on the operating system. The processing device also accesses, stores, manipulates, processes, and generates data in response to the execution of the software. For ease of understanding, the processing device may be described as being one in which only one processing device is used, but one skilled in the art will appreciate that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0150] The software may include computer programs, codes, instructions, or any combination of one or more thereof, capable of configuring or instructing a processing device to operate as desired, either independently or in combination. The software and / or data may be embodied in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, either permanently or temporarily, to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed across computer systems coupled to a network, and may be stored and executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

[0151] The method according to the present invention may be embodied in the form of program instructions to be executed by various computer means and recorded on a computer-readable recording medium. The recording medium may include program instructions, data files, data structures, and the like, alone or in combination. The recording medium and the program instructions may be specially designed and constructed for the purposes of the present invention, or may be well known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language code, such as that generated by a compiler, but also high-level language code executed by a computer using an interpreter, etc.

[0152] The hardware devices described above may be configured to operate as one or more software modules to perform the operations illustrated in the present invention, and vice versa.

[0153] Although the embodiments have been described above with reference to limited drawings, those skilled in the art may apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than described, and may be replaced or substituted with other components or equivalents to achieve appropriate results.

[0154] Accordingly, other implementations, embodiments, and equivalents of the claims are intended to be within the scope of the following claims. [Explanation of symbols]

[0155] 100 Neural network circuit 110 Synapse memory array 111 Synapse Memory Cell 120 Reference Memory Array 121 Reference Memory Cell 150 Output circuit 151 Synapse Readout Circuit 152 Reference Readout Circuit

Claims

1. A neural network circuit, comprising: a synapse memory cell disposed along the output line, the synapse memory cell including a resistive memory element having one of a first resistance value and a second resistance value, the synapse memory cell responsive to receiving an input signal via an input line to generate a column signal based on the resistive memory element and the input signal; a reference memory cell arranged along a reference line, the reference memory cell including a reference memory element having the first resistance value or the second resistance value in a ratio predetermined by an application, the reference memory element and the reference memory cell generating a reference signal based on the reference memory element and the input signal; an output circuit for generating an output signal for the output line from the column signal and the reference signal; A neural network circuit comprising:

2. The synapse memory cell includes a resistive memory element having a number of bits for expressing a synapse weight value assigned to the synapse memory cell, The neural network circuit of claim 1 , wherein the resistive memory elements of said number of bits are arranged along the same input line.

3. the reference memory cell includes a reference memory element having a number of bits for expressing the synapse weight value; 3. The neural network circuit of claim 2, wherein the reference memory elements of said number of bits are arranged along the same input line.

4. 3. The neural network circuit of claim 2, wherein the reference memory cells include reference memory elements for representing a sign bit.

5. 2. The neural network circuit of claim 1, wherein the output circuit generates as the output signal a current corresponding to a positive integer multiple of a net current that is a difference between a first current based on a resistive memory element of the first resistance value and a second current based on a resistive memory element of the second resistance value.

6. 2. The neural network circuit of claim 1, wherein the output circuit generates as the output signal a current corresponding to a negative integer multiple of a net current that is a difference between a first current based on a resistive memory element of the first resistance value and a second current based on a resistive memory element of the second resistance value.

7. 2. The neural network circuit according to claim 1, wherein the output circuit further includes a leaky integration and fire (LIF) circuit that fires other neuron circuits based on a result of comparing the output signal with a threshold.

8. 8. The neural network circuit of claim 7, wherein the LIF circuit increases its output voltage in response to the output signal corresponding to a positive integer multiple of the net current.

9. 8. The neural network circuit of claim 7, wherein the LIF circuit reduces its output voltage in response to the output signal corresponding to a negative integer multiple of the net current.

10. 8. The neural network circuit of claim 7, wherein the LIF circuit leaks the output signal in response to a voltage accumulating based on the output signal not reaching a threshold voltage within a critical time.

11. 8. The neural network circuit of claim 7, wherein the LIF circuit fires the other neuron circuit in response to a voltage accumulated based on the output signal reaching a threshold voltage within a critical time corresponding to the threshold.

12. 8. The neural network circuit of claim 7, further comprising a threshold memory array including a plurality of memory elements, at least one memory element of the plurality of memory elements designated based on the set threshold value has the first resistance value.

13. 13. The neural network circuit of claim 12, further comprising an additional reference memory cell sharing a reference word line with the threshold generation circuit and disposed along the reference word line, the additional reference memory element having the second resistance value.

14. 14. The neural network circuit of claim 13, wherein the output circuit sets a critical time corresponding to the threshold based on a signal generated based on a threshold memory cell and a signal generated based on the additional reference memory cell.

15. 15. The neural network circuit of claim 14, wherein the output circuitry begins accumulating a current corresponding to a difference between a signal generated based on the threshold memory cell and a signal generated based on the additional reference memory cell, and when a voltage corresponding to the accumulated current exceeds a threshold voltage, outputs a signal indicating a threshold time corresponding to the threshold.

16. 15. The neural network circuit of claim 14, wherein the output circuitry applies a critical time corresponding to the threshold determined based on the threshold memory cell and the additional reference memory cell to the output signal for the output line and other output signals for other output lines.

17. 2. The neural network circuit of claim 1, wherein the resistive memory elements of the synapse memory cells connected to the same output line are connected in parallel with each other.

18. further comprising a different synapse memory cell disposed along a different output line than the output line; 2. The neural network circuit of claim 1, wherein the output circuit generates output signals using the same reference memory cell for the output line and each of the different output lines individually.

19. 2. The neural network circuit of claim 1, wherein the output circuit includes a read circuit that generates a column accumulation signal by accumulating the column signals for each bit of the synapse memory cells, and generates a reference accumulation signal by accumulating the reference signals for each bit of the reference memory cells.

20. 20. The neural network circuit according to claim 19, wherein the read circuit includes a current mirror that copies the column signal for each bit of the synapse memory cell and the reference memory cell with a current multiple corresponding to the bit.

21. 20. The neural network circuit of claim 19, wherein the output circuit generates the output signal corresponding to a difference between the column accumulated signal and the reference accumulated signal.

22. 22. The neural network circuit of claim 21, wherein the output circuit includes a capacitor through which a current corresponding to the difference between the column accumulation signal and the reference accumulation signal flows by directing a current corresponding to the reference accumulation signal into a node and a current corresponding to the column accumulation signal out of the node.

23. 2. The neural network circuit of claim 1, wherein the output circuit obtains a sum of products between an input signal received along the input line and a synaptic weight value based on a result of interpreting the output signal, and transmits a node value determined based on the obtained sum of products to another neuron circuit.

24. The neural network circuit of claim 1 , wherein the ratio is predetermined based on an application.

25. 1. A method of operating a neural network circuit, comprising: generating a column signal based on a resistive memory element of a synapse memory cell to which an input signal is applied via an input line, among one or more memory cells arranged along the output line; and generating a reference signal based on a reference memory element having a reference resistance value of a reference memory cell to which the input signal is applied, among one or more memory cells arranged along a reference line, and the input signal; generating an output signal for the output line from the column signal and the reference signal; Including, A method of operating a neural network circuit, wherein the reference resistance value is determined by an application based on a predetermined combination of a first resistance value and a second resistance value different from the first resistance value.

26. 26. The method of claim 25, wherein the combination is predetermined based on the application.

27. 1. A processor comprising: a memory array including rows and columns of resistive memory cells including respective resistors; each resistive memory cell of the memory array having a resistance programmable to change between a first resistance value and a second resistance value, and a column of the memory array provides an integrated thermal current based on an input signal provided to the resistive memory elements of the column and the resistance of each memory element of the column; a reference array of resistive memory cells each including a resistor; A processor, wherein each resistive memory cell of the reference array is programmed to vary between a first resistance value and a second resistance value, and the reference array provides an integrated reference current based on an input signal provided to the resistive memory elements of the reference array and the resistance of each memory element.

Images