Apparatus and method with in-memory processing

Abstract

An apparatus and method having in-memory processing are disclosed. An apparatus for performing in-memory processing includes a memory cell array of memory cells configured to output a current sum of column currents flowing in respective column lines of the memory cell array based on input signals applied to row lines of the memory cells, a sampling circuit including a capacitor connected to each column line configured to charge by a sampling voltage of a corresponding current sum of the column line, and a processing circuit configured to compare a reference voltage to a current charging voltage in a corresponding capacitor of the respective capacitors in response to a trigger pulse generated at a time corresponding to a quantization level among quantization levels time partitioned based on a charging time of the capacitor and determine the quantization level corresponding to the sampling voltage by performing a time-to-digital conversion when the current charging voltage reaches the reference voltage.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2020-0061746, filed May 22, 2020, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference for all purposes. TECHNICAL FIELD

[0002] The disclosure relates to an apparatus and method having in-memory processing. BACKGROUND

[0003] A neural network is a computing system implemented based on a computing architecture. The neural network processing requires a large amount of computation for complex input data. As the neural network data increases and the connectivity of the architecture forming the neural network becomes complex, the amount of computation and the memory access frequency of the processing device can excessively increase, resulting in inefficient performance and miniaturization problems. For example, the neural network processing can include a multiply-accumulate (MAC) operation of repeated multiplication and addition, which can require an efficient hardware architecture and a hardware driving method to process a large amount of computation at low power and high speed. SUMMARY

[0004] This summary is provided to introduce a selection of concepts further described below in the detailed description. This summary is neither intended to identify key or essential features of the claimed subject matter nor is it intended for use in determining the scope of the claimed subject matter.

[0005] In one general aspect, an apparatus for performing in-memory processing includes a memory cell array of memory cells configured to output a current sum of column currents flowing in respective column lines of the memory cell array based on input signals applied to row lines of the memory cells, a sampling circuit including a capacitor connected to each column line, the capacitor configured to be charged by a sampling voltage of a corresponding current sum of the column line, and a processing circuit configured to compare a reference voltage to a current charging voltage in the capacitor in response to a trigger pulse generated at a time corresponding to a quantization level among quantization levels time-divided based on a charging time of the capacitor, and determine the quantization level corresponding to the sampling voltage by performing a time-to-digital conversion when the current charging voltage reaches the reference voltage.

[0006] The processing circuit can include a control signal generator configured to generate the trigger pulse at a time corresponding to the quantization level and generate a count pulse in synchronization with the generation of the trigger pulse, and a comparator connected to each column line and activated by application of the trigger pulse.

[0007] Each of the quantization levels can correspond to a sub-time period of a total time period in which the charging voltage changes until the capacitor is charged with the sampling voltage.

[0008] The sub-time period can be determined based on any one or both of a resistance value of the memory cell and a capacitance of the capacitor.

[0009] The quantization level corresponding to the time at which the trigger pulse is generated can be one level greater than a smallest quantization level among the quantization levels.

[0010] The processing circuitry can include a comparator and a time-to-digital converter (TDC). The comparator connected to each column line can be configured to receive the trigger pulse as a start signal of a comparison operation and output a stop signal when the current charging voltage reaches a reference voltage. The TDC can be configured to output a count value of a count pulse as a digital value when the stop signal is received, the count pulse being applied synchronously with the trigger pulse at a time point at which the stop signal is received.

[0011] The TDC can include a flip-flop connected to the comparator and configured to latch a current count value at a time point at which the stop signal is received.

[0012] The comparator outputting the stop signal among the comparators can be deactivated.

[0013] The comparator can be a latch comparator type.

[0014] The trigger pulse can be applied until a time period corresponding to a quantization level that is one level smaller than a largest quantization level among the quantization levels.

[0015] The smallest quantization level among the quantization levels can correspond to the memory cell included in each column line having a largest synthetic resistance among the column lines, and the largest quantization level among the quantization levels can correspond to the memory cell included in each column line having a smallest synthetic resistance among the column lines.

[0016] In another general aspect, a computing device includes an in-memory processing apparatus including an array of memory cells, a sampling circuit, and a processing circuit. The array of memory cells of the memory cells is configured to output a current sum of column currents flowing in respective column lines of the array of memory cells based on an input signal applied to a row line of the memory cells. The sampling circuit including a capacitor connected to each column line is configured to charge a sampling voltage by the corresponding current sum of the column lines. The processing circuit is configured to compare a reference voltage to a current charging voltage in the capacitor in response to a trigger pulse generated at a time corresponding to a quantization level among quantization levels that are time-divided based on a charging time of the capacitor, and determine the quantization level corresponding to the sampling voltage by performing a time-to-digital conversion when the current charging voltage reaches the reference voltage.

[0017] The processing circuit can include a control signal generator configured to generate a trigger pulse at a time corresponding to a quantization level and to generate a count pulse in synchronization with the generation of the trigger pulse, and a comparator connected to each column line and activated by the application of the trigger pulse.

[0018] Each of the quantization levels can correspond to a sub-period of a total period in which the charging voltage is changed until the capacitor is charged with the sampling voltage.

[0019] The quantization level corresponding to the time of the generation of the trigger pulse can be one level greater than the smallest quantization level among the quantization levels.

[0020] The computing device can also include a host processor and a memory storing instructions that, when executed by the host processor, configure the host processor to control the processing apparatus within the memory to output, based on an input signal applied to a row line of the memory cells, a current sum of column currents flowing in respective column lines of an array of the memory cells, a capacitor being charged with a corresponding current sum of the column lines with a sampling voltage, and in response to a trigger pulse generated at a time corresponding to a quantization level, compare a reference voltage to a current charged voltage in the capacitor.

[0021] The processing circuit can include a comparator connected to each column line configured to receive the trigger pulse as a start signal of a comparison operation and to output a stop signal when the current charged voltage reaches the reference voltage, and a time-to-digital converter (TDC) configured to output, as a digital value, a count value of a count pulse applied at a point in time of the reception of the stop signal in synchronization with the trigger pulse when the stop signal is received.

[0022] The TDC can include a flip-flop connected to the comparator configured to latch a current count value at a point in time of the reception of the stop signal when the stop signal is received.

[0023] The comparator outputting the stop signal in the comparator can be deactivated.

[0024] In another general aspect, a method of performing in-memory processing includes applying an input signal to a row line of memory cells of an array of memory cells, charging a capacitor connected to each column line of the memory cells with a corresponding sampling voltage using a current sum of column currents flowing in respective column lines, in response to a trigger pulse generated at a time corresponding to a quantization level among quantization levels that are time-divided based on charging times of the capacitor, comparing a reference voltage to a current charged voltage in the capacitor, and determining the quantization level corresponding to the sampling voltage by performing a time-to-digital conversion when the current charged voltage reaches the reference voltage.

[0025] The method can further include generating a trigger pulse at a time corresponding to the quantization level, and generating a count pulse in synchronization with the generation of the trigger pulse. The determining of the quantization level can include determining the quantization level by outputting, as a digital value, a count value of the count pulse at a time point at which the time-to-digital converter (TDC) receives a stop signal indicating that the current charging voltage in the capacitor reaches the reference voltage from the comparator.

[0026] In another general aspect, an apparatus for performing in-memory processing includes a memory cell, a sampling circuit, a processing circuit, and a time-to-digital converter (TDC). The memory cell has adjacent memory cells connected to each other to form a memory cell array, the memory cell array being configured to output a current sum flowing in respective column lines of the memory cell array based on an input signal applied to a row line of the memory cell. The sampling circuit includes a capacitor connected to each column line, the capacitor being configured to be charged by a sampling voltage of the current sum flowing in the respective column line. The processing circuit is configured to compare a reference voltage with a current charging voltage in the capacitor in response to a trigger pulse corresponding to a quantization level among quantization levels based on a charging time of the capacitor, and determine the quantization level corresponding to the sampling voltage when the current charging voltage reaches the reference voltage. The time-to-digital converter (TDC) is configured to output a count value of a count pulse applied in synchronization with the trigger pulse.

[0027] The TDC can include a flip-flop connected to the comparator and configured to latch a current count value at a time point at which the stop signal is received.

[0028] The comparator outputting the stop signal is deactivated.

[0029] Other features and aspects will be apparent from the following detailed description, the drawings, and the claims. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 An example of a mathematical model related to the operation of a biological neuron is shown.

[0031] Figure 2 A configuration of a two-dimensional array circuit for performing neuromorphic operation according to one or more embodiments is shown.

[0032] Figure 3 A neuromorphic operation processing method according to one or more embodiments is shown.

[0033] Figure 4 An in-memory processing device according to one or more embodiments is shown.

[0034] Figure 5 is a graph showing a mapping relationship between a synthesized resistance value and a sum bit value of a column line according to one or more embodiments.

[0035] Figure 6 shows an in-memory processing device according to one or more embodiments.

[0036] Figure 7 shows a time period for setting a quantization level according to a time for charging a voltage in a capacitor according to one or more embodiments.

[0037] Figure 8 shows a time when a trigger pulse TRIGGER is applied to a comparator according to one or more embodiments.

[0038] Figure 9 shows a time-to-digital conversion performed in an in-memory processing device according to one or more embodiments.

[0039] Figure 10 shows a count value of each column line (output line) output by time-to-digital conversion according to one or more embodiments.

[0040] Figure 11 is a flowchart of a method of performing in-memory processing according to one or more embodiments.

[0041] Figure 12 is a block diagram of a computing device according to one or more embodiments.

[0042] Figure 13 shows an example of a neural network.

[0043] Figure 14 is a flowchart of a method of performing in-memory processing according to one or more embodiments.

[0044] Throughout the drawings and the detailed description, same reference numerals indicate same elements. The drawings can not be to scale, and the relative dimensions, proportions, and depiction of the elements in the drawings can be exaggerated for clarity, illustration, and convenience. DETAILED DESCRIPTION

[0045] The following detailed description is presented to aid the reader in gaining a comprehensive understanding of the methods, apparatuses, and / or systems described herein. However, various changes, modifications, and equivalents can be used, and thus particular embodiments described herein are not to be taken in a limiting sense. For example, although processes are described with regard to the preceding examples, the processes are not limited to the specific orders or sequences for carrying out the processes as described herein, and not all processes are carried out in the same order or sequence as described herein. Additionally, features described herein can be omitted in further implementations. For the sake of brevity, certain aspects of the methods, apparatuses, and / or systems described herein can not be described in detail.

[0046] The features described herein can be implemented in different forms and should not be construed as limited to the examples described herein. Rather, these examples are provided as illustrative of a number of ways in which the methods, apparatuses, and / or systems described herein can be implemented, in view of the disclosure provided herein.

[0047] Throughout the specification, when an element (such as a layer, region, or substrate) is referred to as being "on" another element, "connected to" or "coupled to" another element, it can be directly on, directly connected to, or directly coupled to the other element, or one or more other elements can be interposed therebetween. In contrast, when an element is referred to as being "directly on," "directly connected to," or "directly coupled to" another element, there are no other elements interposed therebetween.

[0048] As used herein, the term "and / or" includes any one of the associated listed items, as well as any combination of any two or more of the associated listed items.

[0049] Although terms such as "first," "second," and "third" can be used herein to describe various elements, components, regions, layers, or sections, these elements, components, regions, layers, or sections should not be limited by these terms. Rather, these terms are only used to distinguish one element, component, region, layer, or section from another element, component, region, layer, or section. Thus, a first element, component, region, layer, or section mentioned in the examples described herein can also be referred to as a second element, component, region, layer, or section, without departing from the teachings of the examples.

[0050] For purposes of the description hereinafter, spatial or directional terms, such as "above", "below", "upper", "lower", "left", "right", "over", "under", "top", "bottom", "vertical", "horizontal", and the like, can be used where appropriate to describe the various examples. Unless otherwise specified, these terms are intended to encompass different orientations of the device in use or operation, in addition to the orientation depicted in the figures. For example, if the device described herein is turned over, and a component previously located "above" another component is now located "below" another component, this is intended to be encompassed by use of these terms. Similarly, the terms "coupled" and "connected" are intended to encompass a direct connection between two components or an indirect connection between two components through one or more other components. In addition, the terms "first", "second", and the like, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The terms "a" and "an" and "the" and similar reference use are intended to include both singular and plural forms, unless otherwise indicated herein. The terms "comprising", "including", "containing", and "having" are intended to be inclusive and mean that there can be additional elements other than the listed elements.

[0051] The terminology used herein is for the purpose of describing various examples only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and "having" specify the presence of stated features, integers, operations, elements, components and / or groups but do not preclude the presence or addition of one or more other features, integers, operations, elements, components and / or groups thereof.

[0052] As will be apparent, the features of the examples described herein can be combined in various ways. Furthermore, although examples described herein have a variety of configurations, other configurations are also possible.

[0053] Figure 1 An example of a neural network mathematical model 11 that models the operation of a biological neuron is shown.

[0054] The mathematical model 11 is an example of a neuromorphic operation that can be simulated by a hardware computing element or processor, including a multiplication operation with respect to information from a plurality of neurons, multiplied by synaptic weights, a summation operation (∑) with respect to values (ω0x0, ω1x1, ω2x2) multiplied by respective synaptic weights, and an operation that applies a feature function (b) and an activation function (f) to the summation operation result. The result of the simulated neuromorphic operation can be provided by the neuromorphic operation. Values such as x0, x1, x2,... can be referred to as axon values, and values such as ω0, ω1, ω2,... can be referred to as synaptic weights. Here, it is noted that the use of the term "may" (e.g., with respect to what an example or embodiment can include or implement) with respect to an example or embodiment indicates that there is at least one example or embodiment that includes or implements such feature, and that all examples and embodiments are not limited to these examples.

[0055] Figure 2A configuration of a two-dimensional array circuit 20 for performing neuromorphic operations according to one or more embodiments is shown.

[0056] Referring to Figure 2 , the two-dimensional array circuit 20 can include N axon circuits A1 to A N 210, M neuron circuits N1 to N M 230, and an N x M synapse array S 11 to S NM 220, where N is a natural number, where M is a natural number. Here, with respect to the examples and descriptions of Figures 1 to 14 and the remaining examples, summaries, and claims, the use of the term "neuron" is not meant to imply that "neuron" has any other meaning beyond the technical meaning, i.e., is not meant to imply that the term "neuron" is the same or similar in structure and operation in hardware and hardware implementations as for chemical and neural neuron implementations hereinafter. Similarly, with respect to the examples and descriptions of Figures 1 to 14 and the remaining examples, summaries, and claims, the use of the term "neuron circuit," "synapse," "synapse circuit," "axon," or "axon circuit" is not meant to imply that "neuron circuit," "synapse," "axon," or "axon circuit" has any other meaning beyond the technical meaning, i.e., is not meant to imply that the term "neuron circuit," "synapse," "axon," or "axon circuit" is the same or similar in structure and operation in hardware and hardware implementations as for chemical and neural neuron implementations hereinafter. For example, an artificial neural network can be hardware configured to have multiple layers of hardware nodes (i.e., "neurons" referred to below).

[0057] The synapses of the synapse array S 11 to S NM 220 can be respectively located at intersections of first direction lines extending in a first direction from the axon circuits A1 to A N 210 and second direction lines extending in a second direction from the neuron circuits N1 to N M 230. For ease of illustration, the first direction is shown as a row direction, and the second direction is shown as a column direction. However, the disclosure is not limited to these examples, the first direction can be a column direction, and the second direction can be a row direction.

[0058] Each of the axon circuits A1 to A N 210 simulating an axon of a neuron can receive an input activation (e.g., axons a1, a2,..., a N ), and transmit the received activation to a first direction line. The activation corresponding to neurotransmitters transmitted through a neuron can represent an input to the axon circuits A1 to A Nan electrical signal of each of 210. Axon circuits A1 to A N Each of 210 can include a memory, a register, or a buffer for storing input information. The activation can be a binary activation having a binary value. For example, the binary activation can include 1-bit information corresponding to a logic value 0 or 1 or a logic value -1 or 1. However, the disclosure is not limited to this example, and the activation can have a ternary value or a multi-bit value.

[0059] synapse array S 11 to S NM 220 can store synaptic weights corresponding to strengths of interconnections between neurons. Although Figure 2 w1, w2, …, w M As an example of synaptic weights to be stored in respective synapses, other synaptic weights can be stored in respective synapses for the convenience of explanation. The synapse array S 11 to S NM Each of 220 can include a memory device for storing a synaptic weight, or can be connected to other memory devices storing synaptic weights. Such a memory device can correspond to, for example, a memristor or a resistive memory cell. The memristor or the resistive memory cell can be implemented by a static random access memory (SRAM), a phase change memory (PCM), an oxide-based memory (OXRAM), a magnetoresistive random access memory (MRAM), a spin-transfer torque random access memory (STT-RAM), a conductive bridge random access memory (CBRAM), a resistive RAM (RRAM), a ferroelectric RAM (FRAM), a magnetic tunnel junction (MTJ) device, or the like, but the disclosure is not limited to these examples.

[0060] synapse array S 11 to S NM Each of 220 can receive an activation input transmitted by a corresponding first direction line from axon circuits A1 to A N Each of 210 inputs an activation input transmitted through a corresponding first direction line, and can output a result of a neuromorphic operation between a stored synaptic weight and the activation input. For example, the neuromorphic operation between the synaptic weight and the activation input can be a multiplication operation (i.e., an AND operation), but the disclosure is not limited to this example. In other words, the result of the neuromorphic operation between the synaptic weight and the activation input can be a value obtained by other appropriate operations reflecting a strength or a magnitude of an activation adjusted based on a strength of an interconnection between neurons.

[0061] The strength of a signal transmitted from axon circuits A1 to A N 210 to neuron circuits N1 to N M 230 can be adjusted. In this way, the signal transmitted from axon circuits A1 to A11 to S NM 220 to implement an operation of adjusting the amplitude or intensity of a signal transmitted to a next neuron according to the strength of interconnection between neurons.

[0062] Each of the neuron circuits N1 to N M 230 can receive a result of a neuromorphic operation between a synaptic weight and an activation input through a corresponding second direction line. Each of the neuron circuits N1 to N M 230 can determine whether to output a spike based on the result of the neuromorphic operation. For example, when a value obtained by accumulating the result of the neuromorphic operation is greater than or equal to a preset threshold value, each of the neuron circuits N1 to N M 230 can output a spike. The spike output from each of the neuron circuits N1 to N M 230 can correspond to an activation input to a next stage axon circuit.

[0063] Each of the neuron circuits N1 to N M 230 can be connected to the synaptic array S 11 to S NM 220, the neuron circuits N1 to N M 230 can be referred to as post-synaptic neuron circuits, and the axon circuits A1 to A N 210 can be referred to as pre-synaptic neuron circuits. N 210 can be referred to as pre-synaptic neuron circuits.

[0064] Figure 3 A neuromorphic operation processing method according to one or more embodiments is illustrated.

[0065] A two-dimensional array circuit for processing a neuromorphic operation can use a current summation method for each column line. For example, the two-dimensional array circuit can sum currents flowing through synapses S N 210 along the column line 310 by activations transmitted from the axon circuits A1 to A 11 , S 21 , S (N-1)1 , and S N1summing the currents and outputting a spike when the amount or intensity of the summed currents is greater than or equal to a preset threshold value. In this state, in order to obtain the amount or intensity of the summed currents, a peripheral circuit (such as an analog-to-digital converter (ADC), a digital-to-analog converter (DAC), etc.) can be provided. However, the ADC or DAC can be a factor of low efficiency in terms of power and size with respect to the overall circuit configuration. Accordingly, in the following description, according to the present example, a method of implementing a system-on-a-chip having high integration with a time-to-digital converter (TDC)-based circuit design is further described instead of an ADC or DAC performing a neuromorphic operation such as a multiply-accumulate (MAC) operation of repeated addition and multiplication.

[0066] Figure 4 An in-memory processing apparatus 100 according to one or more embodiments is illustrated.

[0067] In Figure 4 The in-memory processing apparatus 100 can include a circuit outputting a result of multiplication and addition performed for a neuromorphic operation.

[0068] For example, the in-memory processing apparatus 100 can include a plurality of memory cells 110, a capacitor C, and a TDC 140. In addition, the in-memory processing apparatus 100 can further include a comparator 130 for generating time information to be transmitted to the TDC 140. In Figure 4 In order to facilitate explanation, only column lines 120 and row lines 121 corresponding to a portion of a memory cell array provided in the in-memory processing apparatus 100 are illustrated. The row lines 121 extend in a first direction (for example, a horizontal direction), and the column lines 120 extend in a second direction (for example, a vertical direction) crossing the first direction. A plurality of row lines 121 can be collectively referred to as the row lines 121, and a plurality of column lines 120 can be collectively referred to as the column lines 120.

[0069] Accordingly, the in-memory processing apparatus 100 can include a memory cell array in which a plurality of memory cells are provided at positions where a plurality of column lines and a plurality of row lines cross each other.

[0070] As described above, the memory cell 110 is implemented by a memristor or a resistive memory apparatus, and can be an apparatus having a variable resistance. In response to an input signal, a voltage V1, V2, V3, …, V m The input voltage or the power supply voltage can be applied to the memory cell 110 through the input signal. For example, the input voltage or the power supply voltage can be applied to the memory cell 110 through the input signal.

[0071] One end of each of the memory cells 110 can be configured to be connected to the column line 120 by a switch SW (for example, switches SW1, SW2, SW3, …, SW m) receive a voltage, and the other end of each of the memory cells 110 can be connected to a capacitor C and a comparator 130. In other words, the capacitor C and the comparator 130 are connected to each of the column lines including the memory cells 110.

[0072] According to the resistance value of each of the memory cells 110 and the voltage value of the input signal applied to each of the memory cells 110, a current having a current value calculated based on Ohm's law flows in the column line 120. Accordingly, the current of the column current flowing through the column line 120 and I o may correspond to a result value of a MAC operation between the corresponding memory cell and the input signal.

[0073] Each of the input signals (i.e., the input voltages V1, V2, V3, …, V m may be applied to each of the memory cells 110 in response to a start signal START. To this end, the in-memory processing apparatus 100 can include a plurality of switches 101 switched by the start signal START. One end of each of the switches 101 can be connected to one end of each of the memory cells 110. For example, a first switch SW1 can be connected to one end of a first memory cell R1, …, and an m-th switch SW m may be connected to one end of an m-th memory cell R m , where m is a natural number greater than or equal to 1. The other end of each of the switches 101 can be connected to each of the input signals (i.e., the input voltages V1, V2, V3, …, V m ). The input signals are not always applied to all of the memory cells 110, and the input signals can not be applied to some of the memory cells based on the value (input voltage value) of the input signal. In this state, the input signal not applied can represent an example in which the input voltage is 0, but the disclosure is not limited to this example, and the input signal can have a specific voltage value.

[0074] The input signals can correspond to individual bit values of an input bit sequence including a series of binary values. In more detail, in the in-memory processing apparatus 100, each of the plurality of row lines can correspond to each bit position of the input bit sequence. For example, when a bit value at a bit position is 1, an input signal having a voltage value corresponding to the bit value 1 can be applied to a row line corresponding to the bit position. Unlike the above, when a bit value at a bit position is 0, an input signal having a voltage value corresponding to the bit value 0 (e.g., 0V) can be applied to a row line corresponding to the bit position.

[0075] The resistance value of each of the memory cells 110 can have a bit value (e.g., a weight or a synaptic weight) multiplied by each bit of the input bit sequence. Since the memory cells 110 can be implemented by resistive memory devices having variable resistance, the memory cells corresponding to the bit value 1 of the memory cells 110 can have a first resistance value, and the memory cells corresponding to the bit value 0 can have a second resistance value. Alternatively, the disclosure is not limited to this example, and the memory cells 110 can be implemented by a circuit that is switched by using a switching device to select a resistor corresponding to a bit value from among a plurality of resistors having different resistance values.

[0076] In the present embodiment, although it is assumed that the bit value is 1 or 0, the disclosure is not limited to this example, and the bit value can be 1 or -1, other binary bit values (such as ternary bit values), or the like.

[0077] The capacitor C can be connected to the column line 120 connected to the memory cells 110, and can be charged by a current and I o corresponding to the current flowing in the column line 120.

[0078] In the in-memory processing device 100, the capacitor C is connected to each column line 120, and samples a voltage corresponding to the current and I o flowing in the column line 120. Accordingly, in the in-memory processing device 100, the capacitor C connected to each column line 120 can constitute a sampling circuit that is charged with a sampling voltage V o corresponding to the current and I o flowing in the corresponding column line.

[0079] The comparator 130 can be connected to one end of the capacitor C. In one example, the comparator 130 can perform a comparison operation of comparing the currently charged voltage with a reference voltage V o (reference voltage) while the capacitor C is charged with the sampling voltage V ref . The comparator 130 can be implemented by, for example, an operational amplifier (OP amp). The sampling voltage V o of the capacitor C can be input to one input node of the OP amp, and the reference voltage V ref may be input to the other input node of the OP amp.

[0080] The sampling voltage V o corresponding to the current and I o of the column line 120 connected to the capacitor C can be a value corresponding to the result of the MAC operation between the resistance value of the memory cell 110 and the applied input signal.

[0081] In the capacitor C, the charging voltage can vary over time based on a time constant (τ = RxC) determined by the resultant resistance value of the column line 120 and the capacitance. The resultant resistance value of the column line 120 is a value according to the sampling voltage V o or the current I o of the column line 120. For example, when the current I o of the column line 120 is relatively large (or, the sampling voltage V o is relatively small), the time constant τ of the capacitor C decreases, and thus, the capacitor C can be charged relatively fast. In contrast, when the current I o of the column line 120 is relatively small or the sampling voltage V o is relatively large, the time constant τ of the capacitor C increases, and thus, the capacitor C can be charged relatively slow.

[0082] The reference voltage V ref as a voltage for identifying a voltage variation time (e.g., a charging time elapsed while the capacitor C is charged with the sampling voltage V0) can represent a voltage that is a reference in measurement of a voltage variation time (e.g., a charging time). The reference voltage V ref may be set in consideration of circuit device characteristics (such as the capacitor C, the memory cell 110, etc.) and a time for the voltage charged to the capacitor C to reach the reference voltage V ref . For example, in one non-limiting example, based on circuit device characteristics (such as the capacitor C, the memory cell 110, etc.), when the voltage charged in the capacitor C is expected to take about 40 ns to reach the reference voltage V ref , the reference voltage V ref may be set to a value between about 0.1 V and about 0.3 V. However, these are exemplary figures for ease of explanation, and the reference voltage V ref may be a value set in consideration of a circuit configuration of the in-memory processing apparatus 100.

[0083] The comparator 130 can output the stop signal STOP in response to an example in which the voltage currently charged in the capacitor C exceeds the reference voltage V o while the capacitor C is gradually charged by the application of the sampling voltage V ref . In other words, the comparator 130 can output the stop signal STOP in response to an example in which the voltage currently charged in the capacitor C reaches the reference voltage V ref .

[0084] The comparator 130 can perform a comparison operation by being activated (or enabled) only when the trigger pulse TRIGGER signal is applied, which will be described in detail below.

[0085] When a stop signal STOP is received from comparator 130, TDC 140 synchronizes with the trigger pulse TRIGGER at the time the stop signal STOP is received, and outputs the count value T of the applied counting pulse COUNT. out As a digital value. The digital value output from the TDC 140 indicates the value relative to the sampled voltage V. o The corresponding quantification level.

[0086] Figure 4 The memory cell 110, capacitor C, comparator 130, and TDC 140 shown can correspond to one output line (i.e., one column line) on the memory cell array in the memory processing device 100. However, as described above, the memory cell array can be provided with multiple output lines (i.e., multiple column lines).

[0087] Figure 5 This is a graph 500 showing the mapping relationship between the combined resistance value and the column line and bit values ​​according to one or more embodiments.

[0088] exist Figure 5 In graph 500, the x-axis represents the calculated sum bit value in the column lines. The y-axis represents the normalized combined resistance value in the column lines. The combined resistance value in the column lines is inversely proportional to the sum of the currents in the column lines.

[0089] In graph 500, it can be seen that the combined resistance value in the column lines is inversely proportional to the sum of the bit values ​​corresponding to the MAC operation results in the column lines. In other words, it can be seen that the current sum in the column lines is directly proportional to the sum of the bit values ​​corresponding to the MAC operation results in the column lines. Therefore, considering the above correlation, it can be seen that the sampling voltage V applied to capacitor C... o To estimate the MAC calculation result.

[0090] Figure 6 An in-memory processing apparatus according to one or more embodiments is shown.

[0091] Although above Figure 4 The text describes multiple input lines (row lines) and one output line (column line 120) in the memory processing device 100, but... Figure 6 This illustrates an in-memory processing device 60 equipped with a memory cell array 690 including multiple input lines (row lines) and multiple output lines (column lines).

[0092] The memory cell array 690 in the memory processing device 60 may include input lines (row lines) for individually receiving input signals and output lines (column lines) for individually outputting output signals. The input lines (row lines) may intersect the output lines (column lines) respectively. Although Figure 6The example shows input lines (row lines) and output lines (column lines) intersecting each other perpendicularly, but the disclosure is not limited to this example.

[0093] Used to transmit input signals b1, b2, b3, b4, ..., b j ...b m A switch, connected to each of the input lines (row lines), is used to toggle the application of the input signal, along with the start signal START. Input signals b1, b2, b3, b4, ..., b j ...b m This can correspond to an input voltage (or input current) indicating a binary value, but the disclosure is not limited to this example. For example, an input signal indicating a bit value of 1 can represent voltage, and an input signal indicating a bit value of 0 can represent floating voltage.

[0094] The memory cells 610 are respectively located at the intersection of the input lines (row lines) and the output lines (column lines).

[0095] Each of the memory cells 610 can be configured to receive an input signal (input voltage) via an input line (row line) on which the memory cell is located. For example, a memory cell 610 arranged along the j-th input line 691 can be configured to receive the j-th input signal b in response to a start signal START. j .

[0096] The processing circuitry 600 in the memory processing device 60 may include a capacitor 620 and a comparator 640, a TDC 650 and an output unit 660, with the capacitor 620 and the comparator 640 connected to one end of each of the output lines (column lines).

[0097] Capacitors 620 are individually disposed on their respective output lines (column lines), and the capacitors connected to any one of the output lines (column lines) form a sampling circuit charged with the current and the corresponding sampling voltage of the output line (column line).

[0098] As the amount of charge gradually increases due to the application of their respective sampling voltages, each capacitor 620 can be charged. For example, as current and the corresponding voltage (i.e., sampling voltage) are applied based on the input voltage of the input signal and the resistance value of the memory cell of the i-th output line (i-th column line) 692, the capacitor 621 disposed on the i-th output line (i-th column line) 692 can be charged.

[0099] The capacitors 620 connected to the output lines (column lines) can form a sampling circuit and can have the same capacitance. Therefore, the difference in the time constants between the capacitors 620 can depend on the difference in the combined resistance values ​​between the output lines (column lines) (i.e., the difference in the sum of the currents between the output lines (column lines)).

[0100] The comparators 640 can be individually provided at each output line (column line). Each comparator 640 determines whether the voltage currently charged in the capacitor 620 of the output line (column line) reaches a reference voltage V ref . Each comparator 640 compares the voltage currently charged in each capacitor 620 with the reference voltage V ref , and when the voltage currently charged in each capacitor 620 has reached the reference voltage V ref , the comparator 640 outputs a stop signal STOP to the TDC 650 at the point in time at which the reaching occurs.

[0101] The comparators 640 receive a trigger pulse TRIGGER as a start signal of the comparison operation. In other words, only when the trigger pulse TRIGGER is received from the control signal generator 630, the comparators 640 are activated (enabled), and when the trigger pulse TRIGGER is not received, the comparators 640 are deactivated (disabled). The reception of the trigger pulse TRIGGER can mean that a pulse signal indicating a high level is received, but the disclosure is not limited to this example.

[0102] When a start signal START indicating that an input signal is applied to the memory cell array 690 is received, the control signal generator 630 can generate the trigger pulse TRIGGER and the count pulse COUNT. The trigger pulse TRIGGER can be generated at a time corresponding to a certain quantization level among quantization levels divided according to the charging time of the capacitor 620 in the sampling circuit. The quantization levels are described in more detail below with reference to Figure 7 and Figure 8 The count pulse COUNT is a signal counting the number of times when the trigger pulse TRIGGER is applied, which is generated as a pulse signal synchronized with the trigger pulse TRIGGER at the time when the application of the trigger pulse TRIGGER is started. The count pulse COUNT is provided to the TDC 650.

[0103] The TDC 650 performs time-to-digital conversion at the point in time at which the voltage currently charged in a certain capacitor reaches the reference voltage V ref .

[0104] In detail, the TDC 650 can receive the stop signal STOP from each of the comparators 640 connected to each output line (column line). As described above, the stop signal STOP is a signal indicating that the voltage currently charged in a certain capacitor has reached the reference voltage V ref . The TDC 650 latches the count value of the count pulse COUNT at the point in time at which the stop signal STOP is received.

[0105] For example, when the voltage currently charged in the capacitor 621 of the i-th output line (i-th column line) 692 has reached the reference voltage V ref , the comparator 641 of the i-th output line (i-th column line) 692 outputs a stop signal STOP i to the TDC 650. When receiving the stop signal STOP i , the TDC 650 latches the count value T out,i of the count pulse COUNT at the point in time at which the stop signal STOP is received.

[0106] The output unit 660 can output, as a digital value OUT, the count value with respect to the specific output line (column line) from the TDC 650. In detail, the output unit 660 can output the quantization level corresponding to the count value among the preset quantization levels. The output quantization level is a value derived from the sampling voltage, and ultimately corresponds to the MAC operation result of the output line (column line).

[0107] The TDC 650 outputs the count value of each output line, and the output unit 660 outputs the count value of each output line in quantization levels. As described above, the quantization levels are described in detail below with reference to Figure 7 and Figure 8 .

[0108] Figure 7 A time period in which the quantization levels are set according to the time in which the voltage is charged in the capacitor according to one or more embodiments is illustrated.

[0109] In Figure 7 , the graph 700 illustrates the change in the charging voltage of the capacitor constituting the sampling circuit provided in the in-memory processing device. In the graph 700, the x-axis represents the time elapsed after the capacitor is initially charged with the sampling voltage, and the y-axis represents the change in the charging voltage in the capacitor over time.

[0110] The reference voltage V ref is a voltage input that can be used for comparison with the current charging voltage of the capacitor by the comparator.

[0111] The in-memory processing device can be assumed to be a device capable of providing an operation result with k-bit resolution, where k is a natural number. Further, the k-bit resolution can be assumed to have n quantization levels, where n is a natural number.

[0112] As described above, the current in a certain column line and can be applied to the capacitor as a corresponding sampling voltage. In the example in which the capacitors disposed in the sampling circuit are all described as having the same capacitance, it can be seen that the factor causing a difference in the time constant (τ = R x C) between the capacitors is only the resistance value (i.e., the synthetic resistance value) of the column line to which each capacitor is connected. Based on Ohm's law, the synthetic resistance value of the column line and the intensity of the column current (i.e., the current sum) flowing in the column line are inversely proportional to each other. Therefore, it can be deduced that the difference in the time constant between the capacitors depends on the current sum of each column line. For example, when the current sum of the column line is relatively large (or the sampling voltage is relatively small), as the time constant τ of the capacitor decreases, the capacitor can be relatively quickly charged to the reference voltage V ref . In contrast, when the current sum of the column line is relatively small (or, the sampling voltage is relatively large), as the time constant τ of the capacitor increases, the capacitor can be relatively slowly charged to the reference voltage V ref .

[0113] The current sum of a certain column line corresponds to the result of the MAC operation in the column line (i.e., there is a correlation between the two). Therefore, based on the above-described correlation, the difference in the time constant between the capacitors can be finally estimated as a difference corresponding to the result of the MAC operation.

[0114] For k-bit resolution, the total time period in which the charging voltage varies until the capacitor is charged with the sampling voltage can be quantized into n sub-time periods. Each of the n sub-time periods corresponds to each of n quantization levels.

[0115] In more detail, an example in which m memory cells 710 are connected to a column line CL i is described below. It is assumed that each of the memory cells 710 can have a variable resistance, each of the m memory cells 710 can have any one of a first resistance value R 1_max , R 2_max , …, R m_max or a second resistance value R 1_min , R 2_min , …, R m_min , and the first resistance value is greater than the second resistance value.

[0116] When, in the column line CL i , the resistance values of the m memory cells 710 all have the first resistance value R 1_max , R 2_max , …, R m_max , the synthetic resistance value of the m memory cells 710 is the largest. In contrast, when, in the column line CL i , the resistance values of the m memory cells 710 all have the second resistance value R 1_min , R 2_min , …, R m_minAt this time, the synthesized resistance value of the m memory cells 710 is the smallest.

[0117] Therefore, regarding the column line CL i , the synthesized resistance value can be distributed between the maximum synthesized resistance value calculated from an example in which the memory cells 710 all have the first resistance value R 1_max , R 2_max , …, R m_max and the minimum synthesized resistance value calculated from an example in which the memory cells 710 all have the second resistance value R 1_min , R 2_min , …, R m_min .

[0118] An example of the column line CL i having the maximum synthesized resistance value can correspond to an example in which the column current having the minimum current sum flows in the column line CL i . Conversely, an example of the column line CL i having the minimum synthesized resistance value can correspond to an example in which the column current having the maximum current sum flows in the column line CL i . According to the above principle, the sub time periods corresponding to the respective n quantization levels can be set.

[0119] The sub time period 701 can correspond to an example in which the column current having the minimum current sum flows in the column line CL i , and the sub time period 702 can correspond to an example in which the column current having the maximum current sum flows in the column line CL i . For example, the sub time period 701 can correspond to a quantization level in which all bit values of the k-bit resolution are 0, and the sub time period 702 can correspond to a quantization level in which all bit values of the k-bit resolution are 1.

[0120] The time period between the sub time period 701 and the sub time period 702 can include sub time periods corresponding to other quantization levels, and the sub time periods can be divided to have the same interval according to the number of other quantization levels. However, the disclosure is not limited to this example, and the intervals of the sub time periods corresponding to the number of other quantization levels can not be the same. For example, according to a general simulation result regarding the MAC operation result, the values of most MAC operation results can be obtained by being normalized by a Gaussian distribution. Therefore, the intervals of the sub time periods corresponding to the quantization levels can be set to be different from each other based on the Gaussian distribution.

[0121] In Figure 7 , in the graph 700, the quantization levels can be set to increase by one level from the sub time period 701 to the sub time period 702.

[0122] However, the disclosure is not limited to this example, and the quantization levels can be set to decrease by one level from the sub-time period 701 to the sub-time period 702. For example, in Figure 7 In the above example, while the example having the largest synthesized resistance is described as corresponding to the smallest quantization level, and the example having the smallest synthesized resistance is described as corresponding to the largest quantization level, the opposite case can be set. In other words, the quantization level indicated by each time period can vary according to the logic value indicated by each variable resistance value (first resistance value and second resistance value) of each memory cell 710.

[0123] Figure 8 The time at which the trigger pulse TRIGGER is applied to the comparator according to one or more embodiments is shown.

[0124] In Figure 8 the trigger pulse TRIGGER can be generated from a time (time point) 810 corresponding to a certain quantization level. The certain quantization level can be a quantization level that is one level greater than the smallest quantization level among the quantization levels. In one example, the certain quantization level can be a quantization level that is one level greater than the smallest quantization level among the quantization levels. However, the disclosure is not limited to this example, and the trigger pulse TRIGGER can be generated from a time point at which a certain time (e.g., 40 ns) elapses after the capacitor starts to be charged, regardless of the time point corresponding to the quantization level.

[0125] When the trigger pulse TRIGGER is applied to the comparator from the time (time point) 810 corresponding to the certain quantization level, power consumption according to the operation of the comparator at the smallest quantization level (such as "0000") can be reduced. The trigger pulse TRIGGER can be generated to be applied to the comparator until a time period corresponding to a quantization level (e.g., "1110") that is one level smaller than the largest quantization level (e.g., "1111"), which can result in a reduction in additional power consumption.

[0126] The pulse width of the trigger pulse TRIGGER can be set to be less than or equal to the interval of the sub-time periods corresponding to the respective quantization levels.

[0127] The comparator in the in-memory processing device can be implemented by a latch comparator type. Accordingly, the comparator can output a comparison result at a falling edge of the trigger pulse TRIGGER. For example, when the charging voltage of the capacitor reaches the reference voltage V ref at the time point 820, the comparator can output a stop signal STOP indicating the comparison result at the falling edge of the trigger pulse TRIGGER corresponding to the time point 820.

[0128] The time point at which the stop signal STOP is output can be output as the quantization level "1010" through time-to-digital conversion.

[0129] The time constant of the capacitor varies according to the sampling voltage (or current and) of the capacitor applied to a particular column line, and thus, the time point at which the charging voltage of the capacitor reaches the reference voltage V ref As shown in Figure 8 , the sub-time period 830 of the 4-bit quantization level can be set to correspond to the time point at which the charging voltage of the capacitor reaches the reference voltage V ref .

[0130] Figure 9 A time-to-digital conversion performed in a memory-in processing device according to one or more embodiments is illustrated.

[0131] In Figure 9 , the control signal generator 910 can include a quantization level determination unit 911, a pulse generator 912, and a counter 913. The pulse generator 912 generates a trigger pulse TRIGGER to initiate a comparison operation of the comparators 920 (i.e., comparator 1, …, comparator s, where s is a positive integer greater than 1). The counter 913 generates a count pulse COUNT to identify a time point at which a stop signal STOP provided by each comparator 920 is received. The application of the trigger pulse TRIGGER and the count pulse COUNT are synchronized with each other, and thus, the count pulse COUNT can be applied to the TDC 930 at the time at which the trigger pulse TRIGGER is applied to the comparators 920.

[0132] The time at which the trigger pulse TRIGGER is generated can be determined by the quantization level determination unit 911. The quantization level determination unit 911 can determine the time at which the trigger pulse TRIGGER is generated in consideration of the total number of quantization levels to be set (i.e., the number of bits of the digital value), the device properties (e.g., capacitance or time constant) of the capacitor to be charged by the sampling voltage, the resistance values of the memory cells connected to each column line, etc.

[0133] For example, the quantization level determination unit 911 can determine the time at which the trigger pulse TRIGGER is generated based on the methods described above in Figure 7 and Figure 8 . In detail, the quantization level determination unit 911 can control the pulse generator 912 such that the trigger pulse TRIGGER is generated from the Figure 8The time (time point) 810 at which the quantization level determination unit 911 determines the quantization level is generated. In addition, the quantization level determination unit 911 can control the pulse generator 912 so that the trigger pulse TRIGGER is generated until the time period corresponding to the specific quantization level (i.e., the quantization level that is one level smaller than the maximum quantization level (for example, the quantization level that is one level smaller than the maximum quantization level)). The counter 913 can generate the count pulse COUNT only when the trigger pulse TRIGGER is applied.

[0134] The comparator 920 receives the trigger pulse TRIGGER as a start signal of a comparison operation, and compares the voltage currently charged in the capacitor connected to each comparator 920 with the reference voltage. The voltage currently charged in the capacitor connected to each comparator 920 is determined as the time point at which the reference voltage has been reached, and each comparator 920 outputs the stop signal STOP (for example, STOP1,..., STOP s ) to the TDC 930.

[0135] Each comparator 920 can be implemented by a latch comparator type. Therefore, the comparator 920 can output the comparison result at the falling edge of the trigger pulse TRIGGER. For example, when the comparator s detects that the charged voltage of the capacitor has reached the reference voltage at a specific time point, the comparator s can output the stop signal STOP at the falling edge of the trigger pulse TRIGGER corresponding to the time point as a trigger of the TDC 930 connected to the comparator s.

[0136] The comparator that outputs the stop signal STOP among the comparators 920 can have a feedback of a signal (done) (for example, done1,..., done s ) indicating that the output of the stop signal STOP is completed, and the comparator having the feedback of the signal (done) can be deactivated (disabled).

[0137] The TDC 930 can include a flip-flop (ff) 935 connected to each comparator 920, respectively. The flip-flop 935 can be activated (enabled) by the count pulse COUNT received from the counter 913.

[0138] When the stop signal STOP is received from each comparator 920 during the application of the count pulse COUNT, each flip-flop 935 latches the current count value T out at the time point at which the stop signal STOP is received. For example, when the flip-flop connected to the comparator s receives the stop signal STOP from the comparator s during the application of the count pulse COUNT, the flip-flop outputs the current count value T out_s of the count pulse COUNT.

[0139] Thus, since each flip-flop 935 provided in the TDC 930 individually outputs the count value T corresponding to the reception time point of the stop signal STOP out , time-to-digital conversion can be performed with respect to each column line (output line) of the memory cell array 690. As described above, the processing circuit 600 can map the count value T out corresponding to the quantized level, and output the digital value of the quantized level (e.g., k-bit resolution) mapped with respect to each column line (output line).

[0140] Figure 10 An example of outputting the count value of each column line (output line) by time-to-digital conversion according to one or more embodiments is shown.

[0141] In Figure 10 , each of the time points t a , t b , and t c indicates a time point at which the voltage currently charged in each of the capacitors C x , C y , and C z reaches the reference voltage V ref . The time point t b , the time point t a , and the time point t c are listed in the order of fast time.

[0142] Column currents with currents I col_x , I col_y , and I col_z flow in the column lines 1001, 1002, and 1003, respectively, and the sampling voltages corresponding to the currents I col_x , I col_y , and I col_z are applied to the capacitors C x , C y , and C z , respectively. A trigger pulse TRIGGER is applied to each of the comparators 1021, 1022, and 1023 at the same time point, and the comparators 1021, 1022, and 1023 compare the voltage currently charged in each of the capacitors C x , C y , and C z with the reference voltage V ref from the time point at which the trigger pulse TRIGGER is applied. In this state, each of the comparators 1021, 1022, and 1023 can be a latch comparator, and thus each of the comparators 1021, 1022, and 1023 can perform a comparison operation at each falling edge of the trigger pulse TRIGGER.

[0143] First, at time point t b Comparator 1022 determines the capacitor C y The current charging voltage has reached the reference voltage V. ref And will send a stop signal STOP@t b The output is sent to flip-flop 1032 of the TDC 1030. The stop signal STOP@t has been completed. b The output comparator 1022 can be deactivated (disabled).

[0144] The counting pulse COUNT and the trigger pulse TRIGGER are applied synchronously to TDC 1030. Trigger 1032 receives the stop signal STOP@t. b Time point t b The count value T of the latch count pulse COUNT out @t b Therefore, among column lines 1001, 1002, and 1003, the time-to-digit conversion for column line 1002 is completed first. Count value T out @t b It can be output as a numeric value representing the quantization level of the mapping.

[0145] Next, at time point t a Comparator 1021 outputs the stop signal STOP@ta to flip-flop 1031. The stop signal STOP@t has been completed. a The output comparator 1021 can be deactivated (disabled). The trigger 1031, upon receiving the stop signal STOP@t, a Time point t a The count value T of the latch count pulse COUNT out @t a Due to the count value T out @t a Corresponding to the count value T out @t b Different values, therefore, mapped to the count value T out @t a The numerical value of the quantization level may differ from the value mapped to the count value T. out @t b The numerical value of the quantization level. However, even at different time points t a and t b Output count value T out @t a and T out @t b At different time points t a and t bWhen belonging to the same sub-time period corresponding to the same quantization level, the time-to-digital conversion can be performed to a digital value of the same quantization level.

[0146] Comparator 1023 at time t c Output a stop signal STOP@t to trigger 1033 c The stop signal (STOP@t) has been completed. c Comparator 1021 can be deactivated (disabled). Trigger 1033 responds to the stop signal STOP@t. c Time point t c The count value T of the latch count pulse COUNT out @t c .

[0147] The processing circuitry in the in-memory processing device can perform MAC operations on each column line (output line) via the aforementioned time-to-digital conversion. In the in-memory processing device of this example, unlike the Von Neumann architecture where the memory and operation sections are separate, data transfer speed and power consumption can be improved. Furthermore, compared to architectures with ADCs, the in-memory processing device does not require an ADC in a separate column line, thus reducing power consumption and footprint.

[0148] In the above embodiments, although it is assumed that the voltage value of the input signal and the resistance value of the memory cell have binary values ​​(such as a value corresponding to ON (or logic 1) and a value corresponding to OFF (or logic 0)), the embodiments are not limited to this example. The voltage value of the input signal and the resistance value of the memory cell may have values ​​distinguished by multiple states. For example, when a 2-bit value is input to an input line (row line), the input signal (input voltage) floats relative to "00", and a first voltage value, a second voltage value greater than the first voltage value, and a third voltage value greater than the second voltage value can be assigned as input signals for "01", "10", and "11", respectively. Furthermore, when a memory cell indicates a 2-bit value, a first resistance value, a second resistance value greater than the first resistance value, a third resistance value, and a fourth resistance value can be assigned to the memory cell for "00", "01", "10", and "11", respectively. The input signals received in each input line (row line) and each memory cell are not limited to indicating 2-bit multiple states, but can indicate multiple states corresponding to more bits, and can be assigned according to values ​​in digital systems other than binary systems.

[0149] Figure 11 This is a flowchart illustrating a method for performing in-memory processing according to one or more embodiments. Figure 11In the method of performing in-memory processing, since the method is related to the embodiment described with reference to the above-described drawings, any description provided above but omitted below can be applied to Figure 11 the method.

[0150] In operation 1101, the control signal generator 630 of the processing circuit 600 resets the control signals (i.e., the trigger pulse TRIGGER and the count pulse COUNT).

[0151] In operation 1102, the input signal is applied to the memory cells 610 via each row line of the memory cell array 690.

[0152] In operation 1103, each of the capacitors 620 of the processing circuit 600 is charged using the current of the column current flowing in each column line of the memory cell array 690 and the corresponding sampling voltage.

[0153] In operation 1104, the control signal generator 630 of the processing circuit 600 generates the control signals (i.e., the trigger pulse TRIGGER and the count pulse COUNT) based on the set quantization level. The trigger pulse TRIGGER is applied to the comparators 640, and the count pulse COUNT is applied to the TDC 650.

[0154] In operation 1105, during the application of the trigger pulse TRIGGER, each comparator 640 of the processing circuit 600 compares the current charging voltage V c in the capacitor 620 connected to the comparator 640 with the reference voltage V ref .

[0155] In operation 1106, each comparator 640 determines whether the current charging voltage V c in the capacitor 620 has reached the reference voltage V ref . The comparator that determines that the current charging voltage V c has reached the reference voltage V ref outputs a stop signal STOP to the TDC 650, and operation 1107 is performed. Otherwise, operation 1105 is performed again.

[0156] In operation 1107, the comparator that outputs the stop signal STOP is deactivated (disabled).

[0157] In operation 1108, the TDC 650 of the processing circuit 600 performs time-to-digital conversion of the current count value of the output count pulse COUNT.

[0158] In operation 1109, the output unit 660 of the processing circuit 600 outputs a digital value of the quantization level corresponding to the count value.

[0159] Figure 12 is a block diagram of a computing device 1200 according to one or more embodiments.

[0160] In Figure 12 , the computing device 1200 can extract effective information by analyzing input data in real time based on a neural network, and determine a situation based on the extracted information or a control element of an electronic device in which the computing device 1200 is installed. For example, the computing device 1200 can be applied to a robot device such as a drone or an advanced driver assistance system (ADAS), a smart TV, a smart phone, a medical device, a mobile device, an image display device, a measuring device, an IoT device, etc., and can be installed on at least one of other various types of electronic devices.

[0161] The computing device 1200 can include a host processor 1210, a RAM 1220, an in-memory processing device 1230, a memory device 1240, a sensor module 1250, and a communication module (Tx / Rx module) 1260. The computing device 1200 can further include an input / output module, a security module, a power control device, etc. Some hardware elements of the computing device 1200 can be mounted on at least one semiconductor chip. The in-memory processing device 1230, which is a device including the in-memory processing device described in the above drawings, can correspond to a neural network dedicated hardware accelerator or a neural network device including the same.

[0162] The host processor 1210 can be configured to control the overall operation of the computing device 1200. The host processor 1210 can include one processor core (single core) or a plurality of processor cores (multi-core). The host processor 1210 can process or execute programs and / or data stored in the memory device 1240. The host processor 1210 can control the function of the in-memory processing device 1230 by executing programs stored in the memory device 1240. The host processor 1210 can be implemented by a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), etc.

[0163] The RAM 1220 can temporarily store programs, data, or instructions. For example, programs and / or data stored in the memory device 1240 can be temporarily stored in the RAM 1220 according to the control of the host processor 1210 or a boot code. The RAM 1220 can be implemented by a memory such as a dynamic RAM (DRAM) or a static RAM (SRAM).

[0164] The in-memory processing device 1230 can perform the neuromorphic operation (e.g., the MAC operation) described in the above drawings and output the MAC operation result. However, the in-memory processing device 1230 can perform other various in-memory computations.

[0165] As a storage device for storing data, the memory device 1240 can store an operating system (OS), various programs, and various data. In one embodiment, the memory device 1240 can store a plurality of pieces of data (e.g., input signal data, weight data, etc.) required for performing operations of the in-memory processing device 1230, and operation result data (e.g., MAC operation results (i.e., quantization level data)), etc., as well as.

[0166] The memory device 1240 can be a DRAM, but the disclosure is not limited to this example. The memory device 1240 can include at least one of a volatile memory and a non-volatile memory. The non-volatile memory can include ROM, PROM, EPROM, EEPROM, flash memory, PRAM, MRAM, RRAM, FRAM, etc. The volatile memory can include DRAM, SRAM, SDRAM, PRAM, MRAM, RRAM, FeRAM, etc. In one embodiment, the memory device 1240 can include at least one of an HDD, an SSD, a CF, an SD, a Micro-SD, a Mini-SD, an xD, and a memory stick.

[0167] The sensor module 1250 can collect information around an electronic device in which the computing device 1200 is mounted. The sensor module 1250 can sense or receive a signal (e.g., an image signal, a voice signal, a magnetic signal, a biological signal, a touch signal, etc.) from the outside of the electronic device, and convert the sensed signal or the received signal into data. To this end, the sensor module 1250 can include at least one of various types of sensing devices (e.g., a microphone, an imaging device, an image sensor, a light detection and ranging (LIDAR) sensor, an ultrasonic sensor, an infrared sensor, a biological sensor, a touch sensor, etc.).

[0168] The sensor module 1250 can provide the converted data as input data to the in-memory processing device 1230. For example, the sensor module 1250 can include an image sensor, and can generate a video stream by photographing an external environment of the electronic device, and sequentially provide consecutive data frames of the video stream as input data to the in-memory processing device 1230. However, the disclosure is not limited to this example, and the sensor module 1250 can provide various types of data to the in-memory processing device 1230.

[0169] The communication module 1260 can be provided with various wired or wireless interfaces capable of communicating with external devices. For example, the communication module 1260 can include a wired local area network (LAN), a wireless local area network (WLAN) such as wireless fidelity (Wi-Fi), a wireless personal area network (WPAN) such as Bluetooth, a wireless universal serial bus (USB), Zigbee, near field communication (NFC), radio frequency identification (RFID), power line communication (PLC), or a communication interface capable of accessing a mobile cellular network such as third generation (3G), fourth generation (4G), long term evolution (LTE), or fifth generation (5G), etc.

[0170] Figure 13 An example of a neural network 1300 is illustrated.

[0171] In Figure 13 The neural network 1300 can correspond to an example of a deep neural network (DNN). For ease of explanation, although the neural network 1300 is illustrated as including two hidden layers, the neural network 1300 can include various numbers of hidden layers. Also, although the neural network 1300 is illustrated as including a single input layer 1310 for receiving input data, the input data can be directly input to a hidden layer. Figure 13 The neural network 1300 is illustrated as including a separate input layer 1310 for receiving input data, but the input data can be directly input to a hidden layer.

[0172] In the neural network 1300, artificial nodes of a layer other than an output layer can be connected to artificial nodes of a next layer via a link for transmitting an output signal. An output of an activation function with respect to a weighted input of an artificial node included in a previous layer can be input to the artificial node via the link. The weighted input is obtained by multiplying an input (a node value) of the artificial node by a weight, and the input corresponds to an axon value, and the weight corresponds to a synaptic weight. The weight can be referred to as a parameter of the neural network 1300. The activation function can include a sigmoid, a hyperbolic tangent (tanh), and a rectified linear unit (ReLU), and nonlinearity can be formed in the neural network 1300 by the activation function.

[0173] The in-memory processing apparatus described in the above-described drawings can be used before in-memory processing or in-memory computing for a deep learning algorithm-driven. For example, a calculation of a weighted input transmitted between nodes 1321 of the neural network 1300 can be configured with a MAC operation. An output of one of the nodes 1321 included in the neural network 1300 can be represented by Equation 1 below.

[0174] [Equation 1]

[0175]

[0176] Equation 1 can illustrate an output value y i . x jmay represent an output value of the j-th node in the previous layer, w j,i may represent a weight applied to the output value of the j-th node and the i-th node 1321 of the current layer. f() may represent an activation function. As shown in Equation 1, a multiplication accumulation result of an input value x j and a weight w j,i may be used for the activation function. In other words, an operation of multiplying and adding an appropriate input value x j and a weight w j,i may be repeated at a desired point in time. In addition to the above purpose, there are various application fields requiring MAC operations, for which a neuromorphic device capable of processing MAC operations in an analog region can be used.

[0177] The plurality of memory cells of the in-memory processing device can have resistances corresponding to connection weights of connection lines connecting a plurality of nodes in the neural network 1300 in which one or more layers including a plurality of nodes are provided. An input signal provided according to an input line (row line) in which the memory cells are arranged can indicate a value corresponding to a node value x j . Accordingly, the in-memory processing device can perform at least a part of operations required for implementation of the neural network 1300.

[0178] The application of the in-memory processing device can not necessarily be limited to neuromorphic operations, and can be used for operations that process a plurality of pieces of input data using analog circuit characteristics at low power and fast.

[0179] Figure 14 is a flowchart of a method of performing in-memory processing according to one or more embodiments. In Figure 14 , since the method of performing in-memory processing is related to the embodiments described in the above-described drawings, any description described above but omitted below can be applied to the method of Figure 14 .

[0180] In operation 1401, an input signal is applied to the memory cells 610 via each row line of the memory cell array 690.

[0181] In operation 1402, the processing circuit 600 charges the capacitors 620 connected to each column line using a current of a column current flowing in each column line of the memory cell array 690 and a corresponding sampling voltage.

[0182] In operation 1403, the processing circuit 600 performs a comparison operation between a current charging voltage in the capacitor 620 and a reference voltage by applying a trigger pulse generated at a time corresponding to a quantization level among a plurality of quantization levels divided according to a charging time of the capacitor 620 to a comparator.

[0183] In operation 1404, the processing circuit 600 determines a quantization level corresponding to the sampling voltage by performing a time-to-digital conversion at a time point at which the current charging voltage reaches the reference voltage.

[0184] The operations of methods described in this application can be implemented by a hardware component such as a processor of a mobile device (e.g., a smart phone), a server, a computer, or a computer system, which is configured to perform the operations described in this application. Figures 1 to 14the memory unit 110, the capacitor C 620, the TDC 140, 650, the processing circuitry 600, the comparator 640, the quantization level determination unit 911, the pulse generator 912, the counter 913, the computing device 1200, the host processor 1210, the RAM 1220, the memory device 1240, the sensor module 1250, and the Tx / Rx module 1260, where the hardware components are configured to perform the operations described in this application as performed by the hardware components. Examples of hardware components that can be used to perform the operations described in this application include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more of the hardware components that perform operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer can be implemented by one or more processing elements, such as logic gates arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field programmable gate arrays, programmable logic arrays, microprocessors, or any other device or combination of devices configured to respond to and implement instructions in a defined manner to achieve a desired result. In one example, a processor or computer includes or is connected to one or more memories that store instructions or software for execution by the processor or computer. The hardware components implemented by the processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described in this application. The hardware components can also access, manipulate, process, create, and store data in response to execution of the instructions or software. For simplicity, the singular term “processor” or “computer” can be used in the description of the examples described in this application, but in other examples, multiple processors or computers can be used, or a processor or computer can include multiple processing elements or multiple types of processing elements or both. For example, a single hardware component or two or more hardware components can be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components can be implemented by one or more processors, or a processor and a controller, and one or more other hardware components can be implemented by one or more other processors, or another processor and another controller. The one or more processors, or a processor and a controller, can implement a single hardware component or two or more hardware components.The hardware components can have any one or more of various processing configurations, examples of which include a single processor, multiple processors, a parallel processing processor, a single instruction single data (SISD) multiprocessor, a single instruction multiple data (SIMD) multiprocessor, a multiple instruction single data (MISD) multiprocessor, and a multiple instruction multiple data (MIMD) multiprocessor.

[0185] Figures 1 to 14 The methods of performing the operations described in this application are performed by computing hardware (e.g., by one or more processors or computers) implemented to execute instructions or software as described above to perform the operations described in this application that are performed by the methods. For example, a single operation or two or more operations can be performed by a single processor or two or more processors, or a processor and a controller. One or more operations can be performed by one or more processors, or a processor and a controller, and one or more other operations can be performed by one or more other processors, or further processors and further controllers. The one or more processors, or a processor and a controller, can perform a single operation or two or more operations.

[0186] The instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement the hardware components and perform the methods as described above can be written in a computer program, a code segment, instructions, or any combination thereof, to individually or collectively instruct or configure one or more processors or computers to operate as a machine or special purpose computer to perform the operations performed by the hardware components and methods as described above. In one example, the instructions or software include machine code (such as produced by a compiler) directly executable by the one or more processors or computers. In another example, the instructions or software include high-level code executable by the one or more processors or computers using an interpreter. The instructions or software can be written in any programming language based on the block diagrams and flowcharts shown in the drawings and corresponding descriptions in the specification that disclose algorithms for performing the operations performed by the hardware components and methods as described above.

[0187] Instructions or software that are used to control the computing hardware (e.g., one or more processors or computers) to implement the hardware components and perform the methods as described above, as well as any associated data, data files, and data structures, can be recorded, stored, or fixed in one or more non-transitory computer-readable storage media, or on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random-access memory (RAM), flash memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, magnetic tapes, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state disks, and any other device configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and provide the instructions or software and any associated data, data files, and data structures to one or more processors or computers so that the one or more processors or computers can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed over a networked computer system so that the instructions and software and any associated data, data files, and data structures are stored and accessed in a distributed manner by one or more processors or computers.

[0188] While the present disclosure includes specific examples, it will be clear that various modifications in form and details can be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be considered in a descriptive sense only and not for purposes of limitation. Descriptions of features or aspects within each example should be considered to apply to other examples as appropriate. Suitable results can be achieved if the described techniques are performed in a different order, and / or if the components of the described systems, architectures, devices, or circuits are combined in a different manner, and / or replaced or supplemented by other components or their equivalents. Therefore, the scope of the disclosure is defined not by the specific embodiments described, but by the appended claims and their equivalents, and all variations within the scope of the claims and their equivalents are to be construed as being included in the disclosure.

Claims

1. An apparatus for performing in-memory processing, the apparatus comprising: The memory cell array of the memory cells is configured to output a sum of multiple currents of column currents flowing in multiple column lines of the memory cell array, based on input signals applied to the row lines of the memory cells; The sampling circuit includes a plurality of capacitors respectively connected to the plurality of column lines, each capacitor being configured to be charged by the current and the sampled voltage of the corresponding column line; as well as Processing circuitry, including: The control signal generator is configured to generate a trigger pulse at a time corresponding to a quantization level that is divided into quantization levels based on the charging time of the capacitor, and to generate a counting pulse synchronously with the generation of the trigger pulse. Multiple comparators, each connected to one of the multiple column lines, are configured to: compare a reference voltage with the current charging voltage in a capacitor in response to a trigger pulse; receive the trigger pulse as a start signal for the comparison operation; and output a stop signal when the current charging voltage reaches the reference voltage; and The time-to-digital converter is configured to determine the quantization level corresponding to the sampled voltage by performing time-to-digital conversion when the current charging voltage reaches the reference voltage. The time-to-digital converter is also configured to output the count value of the counting pulse as a digital value when a stop signal is received. The counting pulse is applied synchronously with the trigger pulse at the time the stop signal is received.

2. The apparatus of claim 1, wherein, Each of the plurality of quantization levels corresponds to a sub-time period of the total time period, wherein the charging voltage changes until the capacitor is charged with the sampling voltage during the total time period.

3. The apparatus of claim 2, wherein, The sub-time period is determined based on either or both of the resistance value of the memory cell and the capacitance of the capacitor.

4. The apparatus of claim 1, wherein, The quantization level corresponding to the time of trigger pulse generation is one level larger than the smallest of the plurality of quantization levels.

5. The apparatus of claim 1, wherein, The time-to-digital converter includes: multiple triggers, each connected to the multiple comparators, configured to latch the current count value at the time a stop signal is received.

6. The apparatus of claim 1, wherein, The comparator that outputs the stop signal among the plurality of comparators is deactivated.

7. The device according to claim 1, wherein, The comparators are latch comparators.

8. The device according to any one of claims 1 to 4, wherein, The trigger pulse is applied until the time period corresponds to a quantization level that is one level smaller than the largest quantization level among the plurality of quantization levels.

9. The device according to any one of claims 1 to 4, wherein, The smallest quantization level among the plurality of quantization levels corresponds to the memory cell in each column line having the largest combined resistance among the plurality of column lines, and The largest quantization level among the plurality of quantization levels corresponds to the memory cell in each of the plurality of column lines that has the smallest combined resistance.

10. A computing device, comprising: In-memory processing device, including: The memory cell array of the memory cells is configured to output a sum of multiple currents of column currents flowing in multiple column lines of the memory cell array, based on input signals applied to the row lines of the memory cells; A sampling circuit includes a plurality of capacitors respectively connected to the plurality of column lines, each capacitor being configured to be charged by the current and the sampled voltage of the corresponding column line; and Processing circuitry, including: The control signal generator is configured to generate a trigger pulse at a time corresponding to a quantization level that is divided into quantization levels based on the charging time of the capacitor, and to generate a counting pulse synchronously with the generation of the trigger pulse. Multiple comparators, each connected to one of the multiple column lines, are configured to: compare a reference voltage with the current charging voltage in a capacitor in response to a trigger pulse; receive the trigger pulse as a start signal for the comparison operation; and output a stop signal when the current charging voltage reaches the reference voltage; and The time-to-digital converter is configured to determine the quantization level corresponding to the sampled voltage by performing time-to-digital conversion when the current charging voltage reaches the reference voltage. The time-to-digital converter is also configured to output the count value of the counting pulse as a digital value when a stop signal is received. The counting pulse is applied synchronously with the trigger pulse at the time the stop signal is received.

11. The computing device according to claim 10, wherein, Each of the plurality of quantization levels corresponds to a sub-time period of the total time period, wherein the charging voltage changes until the capacitor is charged with the sampling voltage during the total time period.

12. The computing device according to claim 10, wherein, The quantization level corresponding to the time of trigger pulse generation is one level larger than the smallest of the plurality of quantization levels.

13. The computing device of claim 10, further comprising: A host processor and a memory storing instructions, which, when executed by the host processor, configure the host processor to control processing devices within the memory to: Based on the input signal applied to the row lines of the memory cell, the sum of the multiple currents of the column currents flowing in the multiple column lines of the memory cell array is output; The corresponding capacitor is charged by the sampling voltage of the current and the sum of the currents of each of the multiple columns; as well as In response to a trigger pulse generated at a time corresponding to the quantization level, the reference voltage is compared with the current charging voltage in the capacitor.

14. The computing device according to claim 10, wherein, The time-to-digital converter includes: multiple triggers, each connected to the multiple comparators, configured to latch the current count value at the time a stop signal is received.

15. The computing device according to claim 10, wherein, The comparator that outputs the stop signal among the plurality of comparators is deactivated.

16. A method for performing in-memory processing, the method comprising: The input signal is applied to the row lines of the memory cells in the memory cell array; The capacitors connected to the column lines are charged using the current flowing in the column lines of the memory cell and the corresponding sampling voltage. A trigger pulse is generated at the time corresponding to the quantization level in the quantization level divided by time based on the capacitor charging time, and a counting pulse is generated synchronously with the generation of the trigger pulse. The comparator, in response to the trigger pulse, compares the reference voltage with the current charging voltage in the capacitor. The comparator receives the trigger pulse as the start signal for the comparison operation and outputs a stop signal when the current charging voltage reaches the reference voltage. as well as The time-to-digital converter determines the quantization level corresponding to the sampling voltage when the current charging voltage reaches the reference voltage. When the stop signal is received, the time-to-digital converter outputs the count value of the counting pulse as a digital value. The counting pulse is applied synchronously with the trigger pulse at the time the stop signal is received.

17. A device for performing in-memory processing, comprising: A memory cell having adjacent memory cells connected to each other to form a memory cell array, the memory cell array being configured to output multiple currents flowing in multiple column lines of the memory cell array based on input signals applied to the row lines of the memory cells; The sampling circuit includes a plurality of capacitors respectively connected to the plurality of column lines, each capacitor being configured to be charged by a current flowing in the corresponding column line and a sampled voltage; Processing circuitry, including: The control signal generator is configured to generate a trigger pulse at a time corresponding to a quantization level that is divided into quantization levels based on the charging time of the capacitor, and to generate a counting pulse synchronously with the generation of the trigger pulse. Multiple comparators, each connected to one of the multiple column lines, are configured to: compare a reference voltage with the current charging voltage in a capacitor in response to a trigger pulse; receive the trigger pulse as a start signal for the comparison operation; and output a stop signal when the current charging voltage reaches the reference voltage; and The time-to-digital converter is configured to: determine the quantization level corresponding to the sampling voltage when the current charging voltage reaches the reference voltage; and output the count value of the counting pulse as a digital value when a stop signal is received, wherein the counting pulse is applied synchronously with the trigger pulse at the time the stop signal is received.

18. The device according to claim 17, wherein, The time-to-digital converter includes: multiple flip-flops, each connected to a plurality of comparators in the processing circuit, configured to latch the current count value at the time a stop signal is received.

19. The device according to claim 17, wherein, The comparator that outputs the stop signal among the plurality of comparators is deactivated.

Images