Device for performing in-memory processing, and computing device including the same

Abstract

To provide an apparatus for performing in-memory processing, and a computing device including the same.SOLUTION: An apparatus for performing in-memory processing includes a memory cell array which provides a current sum, when an input signal is applied; a capacitor which charges a sampling voltage corresponding to the current sum; and a processing circuit which compares a currently charged voltage in the capacitor with a reference voltage, and determines a quantization level corresponding to the sampling voltage by performing time-digital conversion, when the currently charged voltage reaches the reference voltage.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to an apparatus for in-memory processing and a computing apparatus including the same, and more particularly, to an apparatus for in-memory processing based on time-digital conversion and a computing apparatus including the same.

Background Art

[0002] A neural network is a computing system implemented with reference to a computational architecture in computer science. Recently, with the development of neural network technology, research has been actively conducted to analyze input data and extract effective information using neural networks in various electronic systems. The processing of the neural network requires a large amount of operations on complex input data. As the data of the neural network increases and the connectivity of the architecture constituting the neural network becomes complex, an excessive increase in the amount of operations and the memory access frequency of the processing apparatus is caused, and inefficient performance in miniaturization and commercialization is shown. For example, the processing of the neural network includes MAC (multiply-accumulate) operations that repeat multiplication and addition. In the processing of the neural network, various attempts have been made for an efficient hardware architecture and a hardware driving method for processing repetitive MAC operations that occupy a large amount of operations with low power and high speed.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The problem to be solved by the present invention is to provide an apparatus for in-memory processing and a computing apparatus including the same. The technical problem to be solved by the present embodiment is not limited to the above-described technical problems, and other technical problems can be inferred from the following embodiments.

Means for Solving the Problem

[0004] According to one aspect, an apparatus for performing in-memory processing includes a memory cell array including a plurality of memory cells that provide a current sum of column currents flowing through respective column lines when an input signal is applied via respective row lines, a sampling circuit including a capacitor that charges a sampling voltage corresponding to the current sum of the respective column lines, and a processing circuit that starts a comparison operation between a voltage currently charged in the capacitor and a reference voltage by applying a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels divided by the charging time of the capacitor, and determines a quantization level corresponding to the sampling voltage by performing time-to-digital conversion when the currently charged voltage reaches the reference voltage.

[0005] According to another aspect, a computing device includes a host processor, a memory device, and an in-memory processing device. The in-memory processing device includes a memory cell array including a plurality of memory cells that provide a current sum of column currents flowing through respective column lines when an input signal is applied via respective row lines, a sampling circuit including a capacitor that charges a sampling voltage corresponding to the current sum of the respective column lines, and a processing circuit that starts a comparison operation between a voltage currently charged in the capacitor and a reference voltage by applying a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels divided by the charging time of the capacitor, and determines a quantization level corresponding to the sampling voltage by performing time-to-digital conversion when the currently charged voltage reaches the reference voltage.

[0006] According to still another aspect, a method of performing in-memory processing includes applying an input signal to a plurality of memory cells via respective row lines of a memory cell array; charging a capacitor connected to each of the column lines with a sampling voltage corresponding to the sum of column currents flowing through the respective column lines of the memory cell array; performing a comparison operation between the voltage currently charging the capacitor and a reference voltage by applying a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels defined by the charging time of the capacitor to a comparator; and determining a quantization level corresponding to the sampling voltage by performing time-to-digital conversion when the currently charging voltage reaches the reference voltage.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0008] In the embodiments of the present invention, the terms used are, as much as possible, general terms that are currently in common use. However, they may also vary depending on the intentions of those skilled in the art, precedents, or the emergence of new technologies. Also, in certain cases, there are terms arbitrarily selected by the applicant, and in such cases, the meaning thereof will be described in detail in the corresponding explanatory part. Therefore, the terms used in the specification must be defined based not only on the simple names of the terms but also on the meaning they have and the content throughout the specification.

[0009] Terms such as "configured to" or "including" used in this embodiment are not necessarily interpreted as including all of the various components or various steps described in the specification. Instead, some of those components or some of those steps may not be included, or may be interpreted as further including additional components or steps.

[0010] In the following, with reference to the accompanying drawings, this embodiment will be described in detail. However, this embodiment can be embodied in various different forms and is not limited to the examples described herein.

[0011] FIG. 1 is a drawing for explaining a biological neuron and its operation.

[0012] Referring to FIG. 1, the biological neuron 10 means a cell existing in the human nervous system and is one of the basic biological computing individuals. The human brain contains about 100 billion biological neurons and about 100 trillion interconnections located therebetween.

[0013] The biological neuron 10 is a single cell and includes a neuron cell body containing a nucleus and various cell organelles. The various cell organelles include mitochondria, a large number of dendrites radiating from the cell body, and an axon traversed by many branched extensions.

[0014] Generally, the axon performs the function of transmitting signals from a neuron to other neurons, and the dendrites perform the function of receiving signals from other neurons. For example, when different neurons are connected to each other, the signal transmitted through the axon of a neuron is also received by the dendrites of other neurons. At this time, between neurons, the signal is transmitted through a specialized connection called a synapse, and various neurons are connected to each other to form a neural network. Based on the synapse, the neuron that secretes a neurotransmitter is called a pre-synaptic neuron, and the neuron that receives the information transmitted through the neurotransmitter is also called a post-synaptic neuron.

[0015] On the one hand, the human brain can learn and memorize an enormous amount of information by transmitting and processing various signals through a neural network formed by connecting a large number of neurons to each other. The enormous number of connections between neurons in the human brain is directly correlated with the massively parallel nature of biological computing. Various attempts have been made to mimic the biological neural network and efficiently process a large amount of information through an artificial neural network. For example, neuromorphic devices have been studied as computing systems designed to implement artificial neural networks at the neuron level.

[0016] On the other hand, the operation of biological neuron 10 is also mimicked by mathematical model 11. The mathematical model 11 corresponding to biological neuron 10 is an example of neuromorphic operation, and includes a multiplication operation of multiplying synaptic weights for information from a large number of neurons, an addition operation (Σ) for the values (ω0x0, ω1x1, ω2x2) multiplied by the synaptic weights, and operations of applying a characteristic function (b) and an activation function (f) to the addition operation result. The neuromorphic operation result can be provided by the neuromorphic operation. Here, values such as x0, x1, x2,... are referred to as axon values, and values such as ω0, ω1, ω2,... are also referred to as synaptic weights.

[0017] FIG. 2 is a drawing for explaining the configuration of a two-dimensional array circuit for performing neuromorphic operation.

[0018] Referring to FIG. 2, the configuration 20 of the two-dimensional array circuit includes N (N is an arbitrary natural number) axon circuits (A1 to A N ) 210, M (M is an arbitrary natural number) neuron circuits (N1 to N M)230, and NxM synaptic arrays (S 11 or S NM )220.

[0019] Each synaptic array (S 11 or S NM )220 is also arranged at the intersection of a first-direction line extending in the first direction from the axon circuit (A1 to A N )210 and a second-direction line extending in the second direction from the neuron circuit (N1 to N M )230. Here, for convenience of explanation, the first direction is illustrated as the row direction and the second direction is illustrated as the column direction, but it is not limited thereto, and the first direction may be the column direction and the second direction may be the row direction.

[0020] Each of the axon circuits (A1 to A N )210 may mean a circuit that mimics the axon projection of the biological neuron 10 (FIG. 1). Since the axon projection of a neuron performs the function of transmitting signals from a neuron to other neurons, each of the axon circuits (A1 to A N )210 can receive an activation (e.g., axons a1, a2,..., aN) and transmit it to the first-direction line. The activation corresponds to a neurotransmitter transmitted through a neuron and may mean an electrical signal input to each of the axon circuits (A1 to A N )210. On the other hand, each of the axon circuits (A1 to A N )210 may include a memory, register, or buffer for storing input information. On the other hand, the activation is also a binary activation having a binary value. For example, the binary activation may include 1-bit information corresponding to a logical value 0 or a theoretical value 1 (or, a logical value -1 or a theoretical value 1). However, it is not necessarily limited thereto, and the activation can have a ternary value and can also have a multi-bit value.

[0021] The synaptic array (S 11 or S NM )220 each synapse can mean a circuit that mimics the synapse between neurons. The synaptic array (S 11 or S NM )220 can store the synaptic load corresponding to the connection strength between neurons. In FIG. 2, for convenience of explanation, as an example of the synaptic load stored in each synapse, w1, w2, …, w M is shown, but other synaptic loads can also be stored in each synapse. The synaptic array (S 11 or S NM )220 each synapse can include a memory element for storing the synaptic load or can be connected to other memory elements that store the synaptic load. Here, such a memory element also corresponds to, for example, a memristor or a resistive memory cell. The memristor or the resistive memory cell can also be implemented by, but is not limited to, SRAM (static random access memory), PCM (phase change memory), OXRAM (oxide based memory), MRAM (magnetoresistive random access memory), STT-RAM (spin-transfer torque random access memory), CBRAM (conductive-bridge random access memory), RRAM (resistive random access memory), FRAM (ferroelectric RAM random access memory), magnetic tunnel junction (MTJ: magnetic tunnel junction) elements, etc.

[0022] The synaptic array (S 11 or S NM)Each of 220 can receive the activation input transmitted from each of 210 via the corresponding first-direction line, and can output the result of the neuromorphic operation between the stored synaptic weight and the activation input. For example, the neuromorphic operation between the synaptic weight and the activation input is also a multiplication operation (i.e., AND operation), but is not limited thereto. That is, the result of the neuromorphic operation between the synaptic weight and the activation input is also a value obtained by any other appropriate operation for simulating the strength or magnitude of the activation adjusted by the connection strength between neurons. N )Each of 210 can receive the activation input transmitted from each of 210 via the corresponding first-direction line, and can output the result of the neuromorphic operation between the stored synaptic weight and the activation input. For example, the neuromorphic operation between the synaptic weight and the activation input is also a multiplication operation (i.e., AND operation), but is not limited thereto. That is, the result of the neuromorphic operation between the synaptic weight and the activation input is also a value obtained by any other appropriate operation for simulating the strength or magnitude of the activation adjusted by the connection strength between neurons.

[0023] By the neuromorphic operation between the synaptic weight and the activation input, the size or strength of the signal transmitted from the axon circuit (A1 to A N )210 to the neuron circuit (N1 to N M )230 can be adjusted. In this way, the operation in which the size or strength of the signal transmitted to the next neuron is adjusted by the connection strength between neurons is simulated by using the synaptic array (S 11 to S NM )220.

[0024] Each of the neuron circuits (N1 to N M )230 can mean a circuit that simulates a neuron including dendrites. The dendrites of a neuron perform the function of receiving signals from other neurons, but each of the neuron circuits (N1 to N M )230 can receive the result of the neuromorphic operation between the synaptic weight and the activation input via the corresponding second-direction line. Each of the neuron circuits (N1 to N M )230 can determine whether to output a spike based on the result of the neuromorphic operation. For example, neuron circuits N1 to N MEach of them can output a spike when the value obtained by accumulating the results of the neuromorphic operation is equal to or greater than a preset threshold value. The neuron circuits (N1 to N M ) The spike output from 230 may correspond to the activation input to the axon circuit in the next stage.

[0025] Note that the neuron circuits (N1 to N M ) 230 are located in the subsequent stage with reference to the synaptic array (S 11 to S NM ) 220, and are also referred to as post-synaptic neuron circuits. The axon circuits (A1 to A N ) 210 are located in the previous stage with reference to the synaptic array (S 11 to S NM ) 220, and are also called pre-synaptic neuron circuits.

[0026] FIG. 3 is a drawing for explaining a processing method of the neuromorphic operation.

[0027] The two-dimensional array circuit for processing the neuromorphic operation can use a current summation method for each column line. For example, the two-dimensional array circuit, due to the activation transmitted from the axon circuits A1 to A N , causes the synapses S 11 , S 21 , …, S (N-1)1 , S N1Sum the currents flowing along column line 310 through it. If the magnitude or intensity of the summed current is equal to or greater than a preset threshold value, output a spike. At this time, in order to obtain the magnitude or intensity of the summed current, peripheral circuits such as an ADC (analog to digital convertor) and a DAC (digital to analog convertor) need to be provided. However, the ADC or the DAC also becomes a factor of inefficient overhead in terms of power and area in the overall circuit configuration. Therefore, in the following, according to this embodiment, a high-integration on-chip system having a circuit design based on a time-digital converter (TDC) is used to perform neuromorphic operations such as MAC operations that repeat addition and multiplication instead of an ADC or a DAC. A method for embodying will be described.

[0028] FIG. 4 is a drawing illustrating an in-memory processing device according to an embodiment.

[0029] Referring to FIG. 4, the in-memory processing device 100 is also configured by a circuit that outputs the result of performing multiplication and addition for neuromorphic operations.

[0030] The in-memory processing device 100 may include a plurality of memory cells 110, a capacitor C, and a time-digital converter (TDC) 140. Further, the in-memory processing device 100 may further include a comparator 130 for generating time information transmitted to the time-digital converter 140. In FIG. 4, for convenience of explanation, only the column line 120 and the row line corresponding to a part of the memory cell array provided in the in-memory processing device 100 are illustrated. Therefore, the in-memory processing device 100 includes a memory cell array in which a plurality of memory cells are arranged at positions where a plurality of column lines and row lines intersect.

[0031] The memory cell 110 is implemented by a memristor or a resistive memory element as described above, and is also an element having variable resistance. In the memory cell 110, voltages V1, V2, V3, …, V m can be applied via respective row lines in response to an input signal. For example, an input voltage signal of the input signal can be directly applied to the memory cell 110, or a supply voltage can be applied by the input signal.

[0032] One end of each of the memory cells 110 is also configured to receive a voltage via a switch SW, and the other end of each of the memory cells 110 is also connected to a capacitor C and a comparator 130. That is, a capacitor C and a comparator 130 are connected to each column line including the memory cell 110.

[0033] Due to the respective resistance values of the memory cells 110 and the voltage values of the input signals applied to each of the memory cells 110, a current having a current value calculated based on Ohm's law flows through the column line 120. Therefore, the current sum I o of the column current flowing through the column line 120 may correspond to the result value of the MAC operation between the corresponding memory cell and the input signal.

[0034] Each of the input signals (that is, input voltages V1, V2, V3, …, V m ) is applied to each of the memory cells 110 in response to a start signal START. For this purpose, the in-memory processing device 100 may include a plurality of switches 101 switched by the start signal START. One end of the switch 101 is also connected to one end of the memory cell 110. For example, the first switch SW1 is connected to one end of the first memory cell R1, …, the mth switch SW m is also connected to one end of the mth memory cell R m Here, m is a natural number of 1 or more. The other end of the switch 101 is an input signal (that is, input voltages V1, V2, V3, …, V m) is also connected thereto. On the other hand, the input signal is not constantly applied to all memory cells 110, and depending on the value of the input signal (input voltage value), the input signal is not applied to some memory cells. At this time, the non-applied input signal may mean the case where the input voltage is 0, but is not limited thereto, and may also be a specific voltage value.

[0035] The input signal may correspond to an individual bit value of an input bit sequence composed of a series of binary values. Specifically, in the in-memory processing device 100, each of a plurality of row lines may correspond to each bit position of the input bit sequence. If the bit value at a certain bit position is 1, an input signal having a voltage value corresponding to the bit value 1 is applied to the row line corresponding to the bit position. On the contrary, if the bit value at a certain bit position is 0, an input signal having a voltage value (for example, 0V voltage) corresponding to the bit value 0 is applied to the row line corresponding to the bit position.

[0036] Each resistance value of the memory cell 110 can have a bit value (for example, weight or synaptic load) multiplied by each bit of the input bit sequence. Since the memory cell 110 is also implemented by a resistive memory element having variable resistance, in the memory cell 110, the memory cell corresponding to the bit value 1 can have a first resistance value, and the memory cell corresponding to the bit value 0 can have a second resistance value. Or, without being limited thereto, the memory cell 110 is also implemented by a circuit configuration in which a switching element is used and a resistor corresponding to the bit value is selected from a plurality of resistors having different resistance values.

[0037] On the other hand, in this embodiment, the case where the bit value is 1 or 0 is assumed, but not limited thereto, and the bit value may also be 1 or -1, or other binary bit values, or ternary bit values, etc.

[0038] The capacitor C is connected to the column line 120 to which the memory cell 110 is connected, and is also charged to a voltage corresponding to the sum of the column currents I flowing through the column line 120. o It is also charged to the voltage corresponding to the sum of the column currents I flowing through the column line 120.

[0039] In the in-memory processing device 100, the capacitor C is connected separately for each column line 120, and the capacitor C samples the voltage corresponding to the sum of the currents I of the column line. o Therefore, in the in-memory processing device 100, the capacitors C connected separately for each column line 120 sample the sum of the currents I of the column line 120. o The sampling voltage V corresponding to the sum of the currents I of the column line 120. o It is possible to configure a sampling circuit for charging the sampling voltage V corresponding to the sum of the currents I of the column line.

[0040] The comparator 130 is also connected to one end of the capacitor C. The comparator 130 can perform a comparison operation of comparing the currently charged voltage with the reference voltage V while the capacitor C is being charged with the sampling voltage V. o The comparator 130 can be implemented, for example, by an operational amplifier (OP amp). The sampling voltage V of the capacitor C is input to one input node of the operational amplifier, and the reference voltage V is input to the other input node. ref The comparator 130 can be implemented, for example, by an operational amplifier (OP amp). The sampling voltage V of the capacitor C is input to one input node of the operational amplifier, and the reference voltage V is input to the other input node. o The sampling voltage V of the capacitor C is input to one input node of the operational amplifier, and the reference voltage V is input to the other input node. ref The sampling voltage V of the capacitor C is input to one input node of the operational amplifier, and the reference voltage V is input to the other input node.

[0041] The sampling voltage V o is the voltage corresponding to the sum of the currents I of the column line 120 connected to the capacitor C, and is also the value corresponding to the MAC operation result of the resistance value of the memory cell 110 and the applied input signal. o is the voltage corresponding to the sum of the currents I of the column line 120 connected to the capacitor C, and is also the value corresponding to the MAC operation result of the resistance value of the memory cell 110 and the applied input signal.

[0042] The capacitor C can have its charging voltage varied over a predetermined time based on the combined resistance value of the column line 120 and the time constant (τ = R*C) determined by the capacitance. Here, the combined resistance value of the column line 120 is the sampling voltage V o of the column line 120, or a value dependent on the sum of currents I o of the column line 120. For example, if the sum of currents I o of the column line 120 is relatively large (or if the sampling voltage V o is relatively small), the time constant τ of the capacitor C becomes small, so the capacitor C is charged more quickly. Conversely, if the sum of currents I o of the column line 120 is relatively small (or if the sampling voltage V o is relatively large), the time constant τ of the capacitor C becomes large, so the capacitor C is charged more slowly.

[0043] The reference voltage V ref is an arbitrary voltage for grasping the voltage change time (for example, the charging time) elapsed while the sampling voltage V0 is charged to the capacitor C, and can mean a voltage serving as a measurement reference for the voltage change time (for example, the charging time). The reference voltage V ref can be arbitrarily set in consideration of circuit element characteristics such as the capacitor C and the memory cell 110, and the time it takes for the voltage charged to the capacitor C to reach the reference voltage V ref . For example, if it is desired that it takes about 40 ns for the voltage charged to the capacitor C to reach the reference voltage V ref based on circuit element characteristics such as the capacitor C and the memory cell 110, the reference voltage V ref is also set to a value between 0.1 V and 0.3 V. However, that is merely an exemplary numerical value for the sake of explanation, and the reference voltage V ref is also a value set in consideration of the circuit configuration of the in-memory processing device 100.

[0044] The comparator 130 is the sampling voltage Vo Upon application of the ref , while the capacitor C is gradually charged, in response to the voltage currently charged in the capacitor C exceeding the reference voltage V ref , an end signal STOP can be output. In other words, the comparator 130 can output the end signal STOP in response to the voltage currently charged in the capacitor C reaching the reference voltage V

[0045] On the other hand, the comparator 130 is activated (or enabled) only while the trigger pulse TRIGGER signal is applied and can perform a comparison operation, which will be described in detail in the relevant part below.

[0046] When the time - digital converter (TDC) 140 receives the end signal STOP from the comparator 130, at the time when the end signal STOP is received, it outputs the counting value T of the counting pulse COUNT applied synchronized with the trigger pulse TRIGGER as a digital value. out The digital value output from the time - digital converter (TDC) 140 can indicate the quantization level corresponding to the sampling voltage V o .

[0047] On the other hand, the memory cell 110, the capacitor C, the comparator 130, and the time - digital converter 140 illustrated in FIG. 4 may correspond to one output line (i.e., one column line) on the memory cell array in the in - memory processing device 100. However, as described above, the memory cell array may be provided with a plurality of output lines (i.e., a plurality of column lines).

[0048] FIG. 5 is a drawing for explaining the mapping relationship between the combined resistance value of one column line and the combined bit value according to an embodiment.

[0049] Referring to FIG. 5, in graph 500, the x-axis represents the sum bit value calculated by one column line. In graph 500, the y-axis represents the normalized combined resistance value in the column line. Here, the combined resistance value in the column line is inversely proportional to the sum of currents in the column line.

[0050] Referring to graph 500, it can be seen that the combined resistance value in one column line is inversely proportional to the sum bit value corresponding to the MAC operation result in the column line. In other words, it can be seen that the sum of currents in one column line is proportional to the sum bit value corresponding to the MAC operation result in the column line. Therefore, in view of such a correlation, it can be seen that the MAC operation result can also be estimated from the sampling voltage V o applied to capacitor C.

[0051] FIG. 6 is a drawing for explaining an in-memory processing device according to an embodiment.

[0052] First, in FIG. 4, a plurality of input lines (row lines) and one output line (column line 120) in the in-memory processing device 100 were described. In FIG. 6, an in-memory processing device 60 including a memory cell array 690 including a plurality of input lines (row lines) and a plurality of output lines (column lines) will be described.

[0053] The memory cell array 690 in the in-memory processing device 60 may include a plurality of input lines (row lines) that can individually receive input signals and a plurality of output lines (column lines) that can individually output output signals. Each of the input lines (row lines) can intersect the output lines (column lines). In FIG. 6, even though the input lines (row lines) and the output lines (column lines) are illustrated as intersecting perpendicularly to each other, it is not limited thereto.

[0054] To each of the input lines (row lines), a switch for switching the application of the input signals by the input signals b1, b2, b3, b4, …, b j , …, b m and the start signal START is connected. The input signals b1, b2, b3, b4, …, b j , …, b m may correspond to, but are not limited to, input voltages (or input currents) indicating binary values. For example, an input signal indicating a bit value of 1 can indicate any voltage, and an input signal indicating a bit value of 0 can indicate a floating voltage.

[0055] A memory cell 610 is provided at each position where the input line (row line) and the output line (column line) intersect.

[0056] Each of the memory cells 610 is also configured to receive an input signal (input voltage) via the input line (row line) among the input lines (row lines) where the memory cell is arranged. For example, the memory cell 610 arranged along the j-th input line 691 is also configured to receive the j-th input signal b j in response to the start signal START.

[0057] The processing circuit 600 in the in-memory processing device 60 may include a capacitor 620 and a comparator 640 connected to one end of each of the output lines (column lines), a time - digital converter (TDC) 650, and an output unit 660.

[0058] The capacitor 620 is individually arranged for each output line (column line), and the capacitor connected to one output line (column line) constitutes a sampling circuit that charges a sampling voltage corresponding to the sum of the currents of the output line (column line).

[0059] The capacitor 620 can be charged while the amount of charge gradually increases by applying a sampling voltage. For example, the capacitor 621 arranged on the i-th output line (the i-th column line) 692 is also charged when a voltage (i.e., the sampling voltage) corresponding to the sum of the resistance values of the memory cells on the i-th output line (the i-th column line) 692 and the current based on the input voltage of the input signal is applied.

[0060] On the other hand, the capacitors 620 connected to the output lines (column lines) form a sampling circuit and can have the same capacitance as each other. Therefore, the difference in the time constants between the capacitors 620 can depend on the difference in the combined resistance values of the respective output lines (column lines), that is, the difference in the sum of the currents of the respective output lines (column lines).

[0061] The comparator 640 can be individually arranged for each output line (column line). Each of the comparators 640 determines whether the voltage currently charged in each of the capacitors 620 of the corresponding output line (column line) has reached the reference voltage V ref . Each of the comparators 640 outputs an end signal STOP to the time-to-digital converter (TDC) 650 at the time when it is compared that the voltage currently charged in each of the connected capacitors 620 has reached the reference voltage V ref .

[0062] The comparator 640 receives a trigger pulse TRIGGER as a start signal for the comparison operation. That is, the comparator 640 is activated (enabled) only when it receives the trigger pulse TRIGGER from the control signal generator 630, and is deactivated (disabled) when it does not receive the trigger pulse TRIGGER. Receiving the trigger pulse TRIGGER means receiving a pulse signal indicating high, but it is not necessarily limited to that.

[0063] When a start signal START instructing the application of an input signal to the memory cell array 690 is applied, the control signal generator 630 can generate a trigger pulse TRIGGER and a counting pulse COUNT. The trigger pulse TRIGGER is also generated at a timing corresponding to a predetermined quantization level at the quantization level (quantization level) divided by the charging time of the capacitor 620 in the sampling circuit. The quantization level will be described in detail with reference to FIGS. 7 and 8. The counting pulse COUNT is a pulse signal generated in synchronization with the trigger pulse TRIGGER when the application of the trigger pulse TRIGGER is started, and is a signal for counting the time when the trigger pulse TRIGGER is applied. The counting pulse COUNT is provided to the time-to-digital converter (TDC) 650.

[0064] The time-to-digital converter (TDC) 650 performs time-to-digital conversion when the voltage currently charged in a certain capacitor reaches the reference voltage V ref at that time.

[0065] Specifically, the time-to-digital converter (TDC) 650 can receive an end signal STOP from each of the comparators 640 connected for each output line (column line). As described above, the end signal STOP is a signal indicating that the voltage currently charged in a certain capacitor has reached the reference voltage V ref at that time. The time-to-digital converter (TDC) 650 latches the counting value of the counting pulse COUNT at the time when the end signal STOP is received.

[0066] For example, when the voltage currently charged in the capacitor 621 of the i-th output line (i-th column line) 692 reaches the reference voltage V ref at that time, the comparator 641 of the i-th output line (i-th column line) 692 outputs the end signal STOP iOutput it to the time-digital converter (TDC) 650. When the time-digital converter (TDC) 650 receives the end signal STOP i it latches the counting value T of the counting pulse COUNT at the time when the end signal STOP is received. out,i

[0067] The output unit 660 can output the counting value related to one output line (column line) output from the time-digital converter (TDC) 650 as the digital value OUT. Specifically, the output unit 660 can output the quantization level corresponding to the counting value among the preset quantization levels. Here, the output quantization level is a value derived from the sampling voltage, and ultimately corresponds to the MAC operation result of the output line (column line).

[0068] The time-digital converter (TDC) 650 outputs the counting value for each output line, and the output unit 660 outputs the counting value related to each output line as the quantization level. On the other hand, as described above, the quantization level will be described in detail with reference to FIGS. 7 and 8.

[0069] FIG. 7 is a drawing for explaining setting the time interval of the quantization level according to the time when the capacitor is charged with voltage in one embodiment.

[0070] Referring to FIG. 7, the graph 700 shows the change in the charging voltage of a certain capacitor constituting the sampling circuit provided in the in-memory processing device. In the graph 700, the x-axis represents the time elapsed after the charging of the sampling voltage to the capacitor starts, and the y-axis represents the change in the charging voltage of the capacitor over time.

[0071] The reference voltage V ref is the voltage input for comparison with the currently charged voltage of the capacitor by the comparator.

[0072] ​ It can be assumed that an in-memory processing device is a device that can provide an arithmetic result with a resolution of k bits (where k is a natural number). Also, it can be assumed that the k-bit resolution has n quantization levels (where n is a natural number).

[0073] As described above, the sum of currents in one column line is also applied to the capacitor as the corresponding sampling voltage. Since all the capacitors provided in the sampling circuit are described as having the same capacitance, it can be understood that the factor causing the time constant difference (τ = R*C) between the capacitors is only the resistance value of the column line to which each capacitor is connected (i.e., the combined resistance value). Based on Ohm's law, the combined resistance value of the column line and the intensity of the column current flowing along that column line (i.e., the sum of currents) are in an inverse proportional relationship. Therefore, it can be derived that the time constant difference between the capacitors depends on the sum of currents of each column line. For example, if the sum of currents in the column line is relatively large (or if the sampling voltage is relatively small), the time constant τ of the capacitor becomes small, so the capacitor will be charged to the reference voltage V ref even earlier. Conversely, if the sum of currents in the column line is relatively small (or if the sampling voltage is relatively large), the time constant τ of the capacitor becomes large, so the capacitor will be charged to the reference voltage V ref even later.

[0074] There is a correlation between the sum of currents in one column line and the MAC operation result in that column line. Therefore, based on such a correlation, it is also estimated that the time constant difference between the capacitors ultimately corresponds to the difference in MAC operation results.

[0075] Due to the k-bit resolution, the total time interval during which the charging voltage changes until the sampling voltage is charged to the capacitor is also quantized into n sub-time intervals. Each of the n sub-time intervals corresponds to each of the n quantization levels.

[0076] Specifically, a case where m memory cells 710 are connected to the column line CL i will be described. Each of the memory cells 710 can have a variable resistance, and each of the memory cells 710 has a first resistance value R 1_max , R 2_max , …, R m_max or a second resistance value R 1_min , R 2_min , …, R m_min and it is assumed that the first resistance value is larger than the second resistance value.

[0077] In the column line CL i , when the resistance value of each of the m memory cells 710 is the first resistance value R 1_max , R 2_max , …, R m_max , the combined resistance value of the m memory cells 710 is the largest. On the contrary, in the column line CL i , when the resistance value of each of the m memory cells 710 is the second resistance value R 1_min , R 2_min , …, R m_min , the combined resistance value of the m memory cells 710 is the smallest.

[0078] Therefore, the combined resistance value related to the column line CL i will be distributed between the maximum combined resistance value calculated when all the memory cells 710 have the first resistance value R 1_max , R 2_max , …, R m_max and the minimum combined resistance value calculated when having the second resistance value R 1_min , R 2_min , …, R m_min .

[0079] When the column line CL i has the maximum combined resistance value, it corresponds to the case where a column current having the minimum current sum flows through the column line CL i . On the contrary, the column line CL iWhen it has the minimum combined resistance value, it corresponds to the case where a column current having the maximum current sum flows through the column line CL i Focusing on such a principle, sub-time intervals corresponding to each of the n quantization levels can be set.

[0080] The sub-time interval 701 corresponds to the case where a column current having the minimum current sum flows through the column line CL i The sub-time interval 702 may correspond to the case where a column current having the maximum current sum flows through the column line CL i For example, the sub-time interval 701 may correspond to the quantization level where each bit value of the k-bit resolution is all 0, and the sub-time interval 702 may correspond to the quantization level where each bit value of the k-bit resolution is all 1.

[0081] The time interval between the sub-time interval 701 and the sub-time interval 702 includes sub-time intervals corresponding to the remaining quantization levels, and is also divided so as to have the same interval according to the number of the remaining quantization levels. However, it is not limited thereto, and the intervals of the sub-time intervals corresponding to the remaining quantization levels may also be divided so as not to be the same. For example, according to the general simulation results related to the MAC operation results, the values of most MAC operation results are also obtained by being normalized by a Gaussian distribution. Therefore, based on the Gaussian distribution, the sub-time interval intervals corresponding to the quantization levels are also set to be different from each other.

[0082] Referring to FIG. 7, in the graph 700, it is also set that the quantization level increases by one level from the sub-time interval 701 to the sub-time interval 702.

[0083] However, without being limited thereto, it may also be set such that the quantization level decreases by one level as it goes from sub-time interval 701 to sub-time interval 702. For example, in FIG. 7, it was described that the case having the maximum combined resistance corresponds to the minimum quantization level and the case having the minimum combined resistance corresponds to the maximum quantization level, but it may also be set the other way around. That is, depending on which logical value each of the variable resistance values (first resistance value and second resistance value) of the memory cell 710 represents, the quantization levels indicated by the respective time intervals may be different.

[0084] FIG. 8 is a drawing for explaining the timing at which a trigger pulse TRIGGER is applied to a comparator according to an embodiment.

[0085] Referring to FIG. 8, the trigger pulse TRIGGER is also generated from a timing (time point) 810 corresponding to a predetermined quantization level. Here, the predetermined quantization level is also a quantization level that is one level higher than the minimum quantization level in terms of the quantization level. However, without being limited thereto, the trigger pulse TRIGGER may also be generated from a predetermined elapsed time point (for example, after 40 ns) after the capacitor starts charging without considering the time point corresponding to the quantization level.

[0086] Since the trigger pulse TRIGGER is applied to the comparator from the timing (time point) 810 corresponding to the predetermined quantization level, power consumption due to the operation of the comparator can be reduced at the minimum quantization level such as "0000". The trigger pulse TRIGGER is also generated so as to be applied to the comparator only up to the time interval corresponding to the quantization level that is one level lower than the maximum quantization level (for example, "1111") (for example, "1110"), thereby further reducing power consumption.

[0087] On the other hand, the pulse width of the trigger pulse TRIGGER is also set to be equal to or less than the interval of the sub-time intervals corresponding to each of the quantization levels.

[0088] In an in-memory processing device, the comparator is also embodied in a latched comparator type. Therefore, the comparator can output a comparison result at the rising edge of the trigger pulse TRIGGER. For example, at time point 820, when the charging voltage of the capacitor reaches the reference voltage V ref when reaching, the comparator can output an end signal STOP indicating the comparison result at the falling edge of the trigger pulse TRIGGER corresponding to time point 820.

[0089] The time point when the end signal STOP is output is also output to the quantization level ("1010") by time-to-digital conversion.

[0090] Due to the sampling voltage applied to the capacitor of one column line (or the sum of the currents of that column line), the time constant of the capacitor is different, and thereby the time point when the charging voltage of the capacitor reaches the reference voltage V ref is also different. As shown in FIG. 8, the sub-time interval 830 of the 4-bit quantization level is also set to correspond to the time point when the charging voltage of the capacitor reaches the reference voltage V ref reaches.

[0091] FIG. 9 is a drawing for explaining time-to-digital conversion performed in an in-memory processing device according to an embodiment.

[0092] Referring to FIG. 9, the control signal generator 910 includes a quantization level determination unit 911, a pulse generator 912, and a counter 913. The pulse generator 912 generates a trigger pulse TRIGGER for starting the comparison operation of the comparator 920. The counter 913 generates a counting pulse COUNT for identifying the time point when the end signal STOP provided from each of the comparators 920 is received. The application of the trigger pulse TRIGGER and the counting pulse COUNT are synchronized with each other, so that at the timing when the trigger pulse TRIGGER is applied to the comparator 920, the counting pulse COUNT is also applied to the time - digital converter (TDC) 930.

[0093] The timing at which the trigger pulse TRIGGER is generated is also determined by the quantization level determination unit 911. The quantization level determination unit 911 can determine the timing at which the trigger pulse TRIGGER is generated in consideration of the total number of set quantization levels (i.e., the number of bits of the digital value), the element characteristics of the capacitor for charging the sampling voltage (e.g., capacitance, time constant), the resistance value of the memory cell connected to each column line, and the like.

[0094] For example, the quantization level determination unit 911 can determine the timing at which the trigger pulse TRIGGER is generated based on the method described in FIGS. 7 and 8 above. Specifically, the quantization level determination unit 911 can control the pulse generator 912 so that the trigger pulse TRIGGER is generated from the timing (time point) 810 (FIG. 8) corresponding to a predetermined quantization level (i.e., a quantization level one level higher than the minimum quantization level). Further, the quantization level determination unit 911 can control the pulse generator 912 so that the trigger pulse TRIGGER is generated only up to the time interval corresponding to a quantization level one level lower than the maximum quantization level. On the other hand, the counter 913 can generate the counting pulse COUNT only while the trigger pulse TRIGGER is applied.

[0095] Comparator 920 receives a trigger pulse TRIGGER as a start signal for a comparison operation, and compares the voltage currently charged in the capacitor connected to each of the comparators 920 with a reference voltage. Each of the comparators 920 outputs an end signal STOP to a time-to-digital converter (TDC) 930 at the time when it is detected that the voltage currently charged in the capacitor connected to each of the comparators 920 has reached the reference voltage.

[0096] Each of the comparators 920 is also implemented as a latch comparator type. Therefore, the comparator 920 can output a comparison result at the rising edge of the trigger pulse TRIGGER. For example, when a comparator s detects at a certain time that the charging voltage of the capacitor has reached the reference voltage, the comparator s can output the end signal STOP to the flip-flop of the time-to-digital converter (TDC) 930 connected to the comparator s at the falling edge of the trigger pulse TRIGGER corresponding to that time.

[0097] On the other hand, in the comparator 920, a signal (done) indicating that the output of the end signal STOP has been completed is fed back to the comparator that has output the end signal STOP, and the comparator to which the signal (done) has been fed back is deactivated.

[0098] The time-to-digital converter (TDC) 930 includes flip-flops 935 connected to each of the comparators 920. The flip-flops 935 are also activated by the counting pulse COUNT received from the counter 913.

[0099] Each of the flip-flops 935, while the counting pulse COUNT is being applied, if an end signal STOP is received from each of the comparators 920, the current counting value T at the time when the end signal STOP is received outLatch it. For example, if a flip-flop connected to comparator s receives an end signal STOP while a counting pulse COUNT is applied, the flip-flop will output the current counting value T of the counting pulse COUNT. out_s Output it.

[0100] In this way, each of the flip-flops 935 provided in the time-to-digital converter (TDC) 930 outputs the counting value T corresponding to the time point when the end signal STOP is received, enabling time-to-digital conversion for each column line (output line) of the memory cell array 690. As described above, the processing circuit 600 can map the digital value of the quantization level corresponding to the counting value T for each column line (output line) and output the digital value of the quantization level (e.g., k-bit resolution) mapped for each column line (output line). out By outputting the counting value T corresponding to each column line (output line) through time-to-digital conversion, time-to-digital conversion can be performed for each column line (output line) of the memory cell array 690. As described above, the processing circuit 600 can map the digital value of the quantization level corresponding to the counting value T for each column line (output line) and output the digital value of the quantization level (e.g., k-bit resolution) mapped for each column line (output line). out Map the digital value of the quantization level corresponding to the counting value T, and output the digital value of the quantization level (e.g., k-bit resolution) mapped for each column line (output line).

[0101] FIG. 10 is a drawing for explaining the output of the counting value for each column line (output line) through time-to-digital conversion according to an embodiment.

[0102] Referring to FIG. 10, each of the time points ta, tb, and tc indicates the time when the voltage currently charged in each of the capacitors C x , C y , C z reaches the reference voltage V ref . When arranged in ascending order of time, the order is time point t b , time point t a and time point t c .

[0103] Column currents having current sums I col_x , I col_y , I col_z flow through column lines 1001, 1002, and 1003 respectively, and the current sums I col_x , I col_y , Icol_z The sampling voltage corresponding thereto is applied to capacitors C x , C y , C z respectively. Comparators 1021, 1022, and 1023 are applied with a trigger pulse TRIGGER from the same point in time. From the point in time when the trigger pulse TRIGGER is applied, the comparators 1021, 1022, and 1023 compare the voltages currently charged in the respective capacitors C x , C y , C z with the reference voltage Vref. At this time, the comparators 1021, 1022, and 1023 are also latch comparators, whereby each of the comparators 1021, 1022, and 1023 can perform a comparison operation at each falling edge of the trigger pulse TRIGGER.

[0104] First, at time t b , comparator 1022 determines that the voltage currently charged in capacitor C y has reached the reference voltage V ref , and outputs an end signal STOP@t b to the flip-flop 1032 of the time-to-digital converter (TDC) 1030. On the other hand, the comparator 1022 whose output of the end signal STOP@t b is completed is deactivated.

[0105] The time-to-digital converter (TDC) 1030 is synchronized with the application of the trigger pulse TRIGGER and is applied with a counting pulse COUNT. The flip-flop 1032 latches the counting value T b of the counting pulse COUNT at the time t b when the end signal STOP@t out @t b is received. Thereby, the time-to-digital conversion for the column line 1002 among the column lines 1001, 1002, and 1003 is first completed. On the other hand, the counting value T out @t b is also output as the digital value of the mapped quantization level.

[0106] Next, at time point t a comparator 1021 outputs the end signal STOP@t a to flip-flop 1031. The comparator 1021 whose output of the end signal STOP@t a is completed is deactivated. Flip-flop 1031 latches the counting value T a of the counting pulse COUNT at the time point t a when the end signal STOP@t out @t a is received. The counting value T out @t a corresponds to a value different from the counting value T out @t b so the digital value of the quantization level mapped to the counting value T out @t a is also different from the digital value of the quantization level mapped to the counting value T out @t b However, at different time points t a ,t b if the counting values T out @t a ,T out @t b are output, and different time points t a ,t b belong to the same sub-time interval corresponding to the same quantization level, the digital value of the same quantization level can be time-digital converted.

[0107] At time point t c comparator 1023 outputs the end signal STOP@t c to flip-flop 1033. The comparator 1021 whose output of the end signal STOP@t c is completed is deactivated. Flip-flop 1033 latches the counting value T c of the counting pulse COUNT at the time point t c when the end signal STOP@t out @t cLatch it.

[0108] The processing circuit in the in-memory processing device can perform MAC operations for each column line (output line) through time-digital conversion as described above. Different from the von Neumann structure in which the memory and the arithmetic unit are separated, the in-memory processing device according to this embodiment can improve the data transfer speed and power consumption. In addition, since the in-memory processing device does not need to be equipped with an ADC for each individual column line, the power consumption and the area occupied in the circuit can be reduced compared to the architecture equipped with an ADC.

[0109] On the other hand, in the described embodiment, it has been described by assuming that the voltage value of the input signal and the resistance value of the memory cell have binary values, such as values corresponding to on (ON) (or logical 1) and off (OFF) (or logical 0), but this embodiment is not limited thereto. The voltage value of the input signal and the resistance value of the memory cell can also have values distinguished in a multi-state. For example, when a 2-bit value is input to 1 input line (row line), for "00", the input signal (input voltage) is floated, for "01", the first voltage value, for "10", the second voltage value larger than the first voltage value, and for "11", the third voltage value larger than the second voltage value can be assigned as the input signal. Also, when the memory cell indicates a 2-bit value, for the memory cell, for "00", the first resistance value, for "01", the second resistance value larger than the first resistance value, for "10", the third resistance value, and for "11", the fourth resistance value can be assigned. The input signal received by each input line (row line) and each memory cell are not limited to indicating a 2-bit multi-state, and can also indicate a multi-state corresponding to more bits, and values according to other systems other than the binary system may be assigned.

[0110] FIG. 11 is a flowchart of a method for performing in-memory processing according to an embodiment. Referring to FIG. 11, since the method for performing in-memory processing relates to the embodiment described in the foregoing drawings, even if the content to be omitted hereinafter, the content described in the drawings above is also applicable to the method of FIG. 11.

[0111] In step 1101, the control signal generator 630 of the processing circuit 600 resets the control signals (trigger pulse TRIGGER and counting pulse COUNT).

[0112] In step 1102, an input signal is applied to the plurality of memory cells 610 via respective row lines of the memory cell array 690.

[0113] In step 1103, each of the capacitors 620 of the processing circuit 600 charges a sampling voltage corresponding to the sum of the column currents flowing through the respective column lines of the memory cell array 690.

[0114] In step 1104, the control signal generator 630 of the processing circuit 600 generates control signals (trigger pulse TRIGGER and counting pulse COUNT) based on the set quantization level. The trigger pulse TRIGGER is applied to the comparator 640, and the counting pulse COUNT is applied to the time-to-digital converter (TDC) 650.

[0115] In step 1105, the comparator 640 of the processing circuit 600, while the trigger pulse TRIGGER is applied, compares the voltage V currently charged in the capacitor 620 connected thereto c with the reference voltage V ref and.

[0116] In step 1106, each of the comparators 640 determines whether the voltage Vc currently charged in the capacitor 620 has reached the reference voltage V ref or not. The currently charged voltage Vc has reached the reference voltage V ref The comparator determined to have reached the reference voltage V outputs an end signal STOP to the time-digital converter (TDC) 650, and 1107 steps are performed. Otherwise, 1106 steps are further performed.

[0117] In step 1107, the comparator that has output the end signal STOP is deactivated.

[0118] In step 1108, the time-digital converter (TDC) 650 of the processing circuit 600 performs time-digital conversion to output the current counting value of the counting pulse COUNT.

[0119] In step 1109, the output unit 660 of the processing circuit 600 outputs a digital value of the quantization level corresponding to the counting value.

[0120] FIG. 12 is a block diagram showing a computing device according to an embodiment.

[0121] Referring to FIG. 12, the computing device 1200 can analyze input data in real time based on a neural network, extract valid information, make a situation judgment based on the extracted information, or control the configuration of an electronic device on which the computing device 1200 is mounted. For example, the computing device 1200 can also be applied to robotic devices such as drones and advanced driver assistance systems (ADAS), smart TVs (televisions), smartphones, medical devices, mobile devices, video display devices, measurement devices, Internet of Things (IoT) devices, etc., and in addition, it is mounted on at least one of a variety of electronic devices.

[0122] The computing device 1200 may include a host processor 1210, a RAM (random access memory) 1220, an in-memory processing device 1230, a memory device 1240, a sensor module 1250, and a communication module 1260. The computing device 1200 may further include an input / output module, a security module, a power control device, and the like. A part of the hardware configuration of the computing device 1200 is also mounted on at least one semiconductor chip. The in-memory processing device 1230 is a device including the in-memory processing device described in the drawings above, and may correspond to a neural network dedicated hardware accelerator itself or a neural network device including the same.

[0123] The host processor 1210 controls the overall operation of the computing device 1200. The host processor 1210 may include a single core or multiple cores. The host processor 1210 can process or execute programs and / or data stored in the memory device 1240. The host processor 1210 can control the functions of the in-memory processing device 1230 by executing a program stored in the memory device 1240. The host processor 1210 is also implemented by a CPU (central processing unit), a GPU (graphics processing unit), an AP (application processor), or the like.

[0124] The RAM 1220 can temporarily store programs, data, or instructions. For example, the programs and / or data stored in the memory device 1240 can be temporarily stored in the RAM 1220 under the control or startup code of the host processor 1210. The RAM 1220 is also implemented by a memory such as DRAM (dynamic random access memory) or SRAM (static random access memory).

[0125] The in-memory processing device 1230 can perform neuromorphic operations, such as MAC operations, described in the drawings above, and output the MAC operation results. However, the in-memory processing device 1230 can also perform various in-memory computing in addition to that.

[0126] The memory device 1240 is a storage location for storing data and can store the OS (operating system), various programs, and various data. In one embodiment, the memory device 1240 can store the data (such as input signal data, weight data, etc.) required in the process of performing operations by the in-memory processing device 1230, and the operation result data (such as MAC operation results (i.e., quantization level data), etc.).

[0127] The memory device 1240 is also a DRAM, but is not limited thereto. The memory device 1240 may include at least one of a volatile memory or a non-volatile memory. The non-volatile memory includes ROM (read only memory), PROM (programmable read only memory), EPROM (erasable programmable read only memory), EEPROM (electrically erasable programmable read only memory), flash memory, PRAM (phase change random access memory), MRAM (magnetic random access memory), RRAM, FRAM, and the like. The volatile memory includes DRAM, SRAM, SDRAM, and the like. In one embodiment, the memory device 1240 may include at least one of an HDD (hard disk drive), SSD (solid static driver), CF, SD, micro-SD, mini-SD, xD, or a memory stick.

[0128] The sensor module 1250 can collect information around the electronic device on which the computing device 1200 is mounted. The sensor module 1250 can sense or receive a signal (for example, a video signal, an audio signal, a magnetic signal, a biological signal, a touch signal, etc.) from the outside of the electronic device and convert the sensed or received signal into data. For this purpose, the sensor module 1250 may include at least one of various sensing devices such as a sensing device, for example, a microphone, an imaging device, an image sensor, a lidar (light detection and ranging) sensor, an ultrasonic sensor, an infrared sensor, a biosensor, and a touch sensor.

[0129] The sensor module 1250 can also provide the converted data as input data to the in-memory processing device 1230. For example, the sensor module 1250 may include an image sensor, capture the external environment of the electronic device to generate a video stream, and sequentially provide consecutive data frames of the video stream as input data to the in-memory processing device 1230. However, without being limited thereto, the sensor module 1250 can provide various types of data to the in-memory processing device 1230.

[0130] The communication module 1260 can be equipped with various wired or wireless interfaces capable of communicating with an external device. For example, the communication module 1260 may include a wired short-range communication network (LAN: local area network), a wireless short-range communication network (WLAN: wireless local area network) such as Wi-Fi (wireless fidelity), a wireless personal area network (WPAN: wireless personal area network) such as Bluetooth, wireless USB (wireless universal serial bus), Zigbee, NFC (near field communication), RFID (radio frequency identification), PLC (power line communication), or a communication interface connectable to a mobile cellular network such as 3G (3rd generation), 4G (4th generation), LTE (long term evolution), 5G (5th generation).

[0131] FIG. 13 is a drawing for explaining an example of a neural network.

[0132] Referring to FIG. 13, the neural network 1300 may correspond to an example of a deep neural network (DNN). For convenience of explanation, the neural network 1300 is illustrated as including two hidden layers, but it may include various numbers of hidden layers. Also, in FIG. 13, the neural network 1300 is illustrated as including a separate input layer 1310 for receiving input data, but the input data may be directly input to the hidden layer.

[0133] In the neural network 1300, the artificial nodes of the layers except the output layer are also connected to the artificial nodes of the next layer via links for transmitting output signals. Through those links, the output of an activation function related to the weighted inputs of the artificial nodes included in the previous layer may be input to the artificial nodes. The weighted input is the input (node value) of the artificial node multiplied by a weight, the input corresponds to the axon value, and the weight corresponds to the synaptic load. The weight is also referred to as a parameter of the neural network 1300. The activation function may include a sigmoid, a hyperbolic tangent (tanh), and a rectified linear unit (ReLU), and the activation function forms non-linearity in the neural network 1300.

[0134] The in-memory processing device described previously in the drawings can be used for in-memory processing or in-memory computing driven by deep learning algorithms. For example, the calculation of weighted inputs transmitted between nodes 1321 of the neural network 1300 is also composed of MAC operations. The output of any one node 1321 included in such a neural network 1300 can be expressed as shown in Equation 1 below.

Equation

[0135] Equation 1 can represent the output value y of the i-th node 1321 related to m input values in any layer. i x j can represent the output value of the j-th node in the previous layer, and w j,i can represent the output value of the j-th node and the weight applied to the i-th node 1321 in the current layer. f() can represent the activation function. As shown in Equation 1, for the activation function, the multiplication-accumulation result of the input value x j and the weight w j,i can be used. In other words, at the desired time point, the operation of multiplying and adding the appropriate input value x j and the weighted value w j,i is repeated (MAC operation). In addition to such applications, there are various application fields that require MAC operations, and for this purpose, a neuromorphic device that can process MAC operations in the analog domain is used.

[0136] A plurality of memory cells of the in-memory processing device can have resistances corresponding to the connection weights of the connection lines connecting the plurality of nodes in a neural network 1300 composed of one or more layers including a plurality of nodes. The input signal provided along the input line (row line) where the memory cell is arranged is the node value x jIt can show the value corresponding to. Therefore, the in-memory processing device can perform at least a part of the operations required for the implementation of the neural network 1300.

[0137] On the other hand, the application of the in-memory processing device is not necessarily limited to neuromorphic operations only. In addition, it can also be utilized for operations that must quickly process multiple input data using analog circuit characteristics with low power.

[0138] FIG. 14 is a flowchart of a method for performing in-memory processing according to an embodiment. Referring to FIG. 14, since the method for performing in-memory processing is related to the embodiment described in the foregoing drawings, hereinafter, even if the content omitted is considered, the content described in the drawings previously is also applicable to the method of FIG. 14.

[0139] In step 1401, an input signal is applied to a plurality of memory cells 610 through each row line of the memory cell array 690.

[0140] In step 1402, the processing circuit 600 charges a capacitor 620 connected to each column line with a sampling voltage corresponding to the sum of the column currents flowing through each column line of the memory cell array 690.

[0141] In step 1403, the processing circuit 600 performs a comparison operation between the voltage currently charged in the capacitor 620 and a reference voltage by applying a trigger pulse generated at a timing corresponding to a predetermined quantization level among the quantization levels divided by the charging time of the capacitor 620 to a comparator.

[0142] In step 1404, the processing circuit 600 determines the quantization level corresponding to the sampling voltage by performing time-digital conversion when the currently charged voltage reaches the reference voltage.

[0143] Incidentally, the above-described embodiments can be created as programs executable on a computer, and can also be implemented by a general-purpose digital computer that uses a computer-readable recording medium and operates the program. Also, the data structures used in the above-described embodiments are recorded on a computer-readable recording medium via various means. The computer-readable recording medium includes recording media such as magnetic recording media (e.g., ROM, floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM (compact disc read only memory), DVD (digital versatile disc), etc.).

[0144] Those skilled in the art in the technical field related to this embodiment will be able to understand that embodiments can also be implemented in modified forms within a range that does not deviate from the essential characteristics described above. Therefore, the disclosed embodiments should be considered from an explanatory perspective rather than a limiting perspective. The scope of rights is shown not in the above description but in the claims, and all differences within the equivalent range should be construed as being included in this embodiment.

Explanation of Reference Numerals

[0145] 10 Biological neuron 11 Mathematical model of biological neuron 20 Configuration of two-dimensional array circuit 60, 100, 1230 In-memory processing device 101 Multiple switches 110, 610 Memory cell 120, 310 Column line 130, 640, 920 Comparator 140, 650, 930, 1030 Time-to-digital converter 210 Axon circuit 220 Synapse array 230 Neuron circuit 600 Processing Circuit 620 Capacitor 630, 910 Control Signal Generator 660 Output Unit 690 Memory Cell Array 911 Quantization Level Determination Unit 912 Pulse Generator 913 Counter 1200 Computing Device 1210 Host Processor 1220 RAM 1240 Memory Device 1250 Sensor Module 1260 Communication Module 1300 Neural Network

Claims

1. An apparatus for performing in-memory processing, comprising: a memory cell array including a plurality of memory cells that provide a sum of column currents flowing through respective column lines when input signals are applied through respective row lines; a sampling circuit including a capacitor that charges a sampling voltage corresponding to the sum of the currents of the respective column lines; a processing circuit that starts a comparison operation between the voltage currently charged in the capacitor and a reference voltage by applying a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels divided by the charging time of the capacitor, and determines a quantization level corresponding to the sampling voltage by performing time-to-digital conversion when the currently charged voltage reaches the reference voltage; wherein the processing circuit receives the trigger pulse as a start signal for the comparison operation, and outputs an end signal at the time when it is determined that the voltage currently charged in the capacitor has reached the reference voltage, a comparator connected separately for each column line; a time-to-digital converter (TDC) that, when the end signal is received, outputs, as a digital value, a counting value of a counting pulse synchronized with and applied at the time when the end signal is received; An apparatus.

2. The processing circuit of claim 1, further comprising: a control signal generator that generates the trigger pulse at the timing corresponding to the predetermined quantization level and generates a counting pulse synchronized with the generation of the trigger pulse; a comparator connected separately for each column line activated by the application of the trigger pulse; The apparatus according to claim 1.

3. Each of the quantization levels corresponds to each of sub-time intervals divided from a total time interval during which a charging voltage changes until the sampling voltage is charged in the capacitor. The apparatus according to claim 1 or 2.

4. The sub-time interval is determined based on at least one of a resistance value of the memory cell and a capacitance of the capacitor. The apparatus according to claim 3.

5. The apparatus according to claim 1, wherein the predetermined quantization level corresponding to the timing at which the trigger pulse is applied is a quantization level that is one level higher than the minimum quantization level at the quantization level.

6. The time-to-digital converter (TDC) The apparatus according to claim 1, further comprising flip-flops connected to respective ones of the comparators that latch a current counting value at a point in time when the end signal is received when the end signal is received.

7. The apparatus according to claim 1, wherein, in the comparator, the comparator that outputs the end signal is deactivated.

8. The apparatus according to claim 1, wherein the comparator is implemented as a latch comparator type.

9. The trigger pulse The apparatus according to claim 1, wherein the trigger pulse is applied up to a time interval corresponding to a quantization level that is one level lower than the maximum quantization level at the quantization level.

10. At the quantization level, the minimum quantization level corresponds to a case where a plurality of memory cells included in one column line have a maximum combined resistance, At the quantization level, the maximum quantization level corresponds to a case where a plurality of memory cells included in one column line have a minimum combined resistance. The apparatus according to claim 1.

11. A computing device, comprising: A host processor; A memory device; An in-memory processing device, The in-memory processing device A memory cell array including a plurality of memory cells that provide a sum of column currents flowing through respective column lines when an input signal is applied through respective row lines; A sampling circuit including a capacitor that charges a sampling voltage corresponding to the sum of the currents of the respective column lines; By applying a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels divided by a charging time of the capacitor, a comparison operation between a voltage currently charged in the capacitor and a reference voltage is started, and when the currently charged voltage reaches the reference voltage, time-to-digital conversion is performed to determine a quantization level corresponding to the sampling voltage. A processing circuit; The processing circuit Receiving the trigger pulse as a start signal for the comparison operation, and outputting an end signal at the time when it is determined that the voltage currently charged in the capacitor has reached the reference voltage, a comparator connected separately for each column line, When the end signal is received, at the time when the end signal is received, outputting the counting value of the counting pulse synchronized with and applied by the trigger pulse as a digital value, a time-to-digital converter (TDC), comprising: A computing device.

12. The processing circuit, At the timing corresponding to the predetermined quantization level, generating the trigger pulse, and generating a counting pulse synchronized with the generation of the trigger pulse, a control signal generator, A comparator connected separately for each column line activated by the application of the trigger pulse, comprising: The computing device according to claim 11.

13. The quantization level corresponds to each of the sub-time intervals divided from the total time interval during which the charging voltage changes until the sampling voltage is charged in the capacitor, the computing device according to claim 11 or 12.

14. The predetermined quantization level corresponding to the timing at which the trigger pulse is applied is, at the quantization level, a quantization level one level higher than the minimum quantization level, the computing device according to claim 11.

15. The time-to-digital converter (TDC), When the end signal is received, including a flip-flop connected to each of the comparators that latches the current counting value at the time when the end signal is received, The computing device according to claim 11.

16. In the comparator, the comparator that outputs the end signal is deactivated, the computing device according to claim 11.

17. A method in which a device performs in-memory processing, The processor of the device applying an input signal to a plurality of memory cells via each row line of the memory cell array, The processor charging a sampling voltage corresponding to the sum of the column currents flowing through each column line of the memory cell array into a capacitor connected to each column line, The processor performs a comparison operation between the voltage currently charged in the capacitor and a reference voltage by applying a trigger pulse generated at a timing corresponding to a predetermined quantization level among the quantization levels divided by the charging time of the capacitor to a comparator; The processor determines a quantization level corresponding to the sampling voltage by performing time-to-digital conversion when the currently charged voltage reaches the reference voltage; The processor generates the trigger pulse at the timing corresponding to the predetermined quantization level; The processor generates a counting pulse synchronized with the generation of the trigger pulse, and includes: The step of determining the quantization level includes: When a time-to-digital converter (TDC) receives an end signal indicating that the voltage currently charged in the capacitor has reached the reference voltage from the comparator, the TDC outputs the counting value of the counting pulse as a digital value at the time when the end signal is received, thereby determining the quantization level. Method.

Images