Assembly of a pulse-width-controlled vector-matrix multiplication unit having capacitive elements and method for controlling said assembly
Patent Information
- Application Number
- EP2023804923
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-10-20
- Filing Date
- 2023-10-20
- Publication Date
- 2025-08-27
AI Technical Summary
Existing analog vector matrix multiplication units using capacitive non-volatile memories face challenges in energy efficiency and compact design due to the complexity and size requirements of digital-to-analog converters (ADCs), particularly with high dynamic energy consumption and static power usage in comparators.
The solution involves a pulse-width controlled vector matrix multiplication unit with programmable non-volatile capacitors, a two-ramp ADC, and an improved analog comparator that only consumes dynamic energy at the time of switching, utilizing adjustable capacitors and voltage ramps to reduce energy consumption and compactify the design.
This approach enhances energy efficiency and reduces space requirements by minimizing static power consumption and dynamic energy usage, enabling more compact and efficient analog vector matrix multiplication.
Smart Images
Figure 1.1
Abstract
Description
[0001] Arrangement of a pulse-width controlled vector matrix
[0002] Multiplication unit with capacitive elements and method for controlling it
[0003] The invention relates to an arrangement of a pulse-width controlled vector matrix with a multiplication unit according to the preamble of claim 1.
[0004] The invention also relates to a method for controlling an arrangement according to one of claims 1 to 9
[0005] In recent years, there has been increasing interest in performing analog vector-matrix multiplications using memory cells that represent a multiplicand. The memory cells are arranged in a matrix and represent the coefficients, and the input values are applied to the horizontal lines, which are the word lines. The accumulation operation is usually performed using Kirchhof's law, in which the output currents of the memory cells add up to .
[0006] Applications of such arrangements are mainly in the calculation of artificial neural networks or the solving of differential equations.
[0007] One possibility is the use of non-volatile memories, such as resistive memories (e.g. lelmini et al.: "In-memory computing with resistive switching devices". Nature Electronics, 2018). Here, the multiplication is carried out using Ohm's law, for example by applying a voltage to the word lines of the matrix and measuring the summed currents from the bit line of the matrix.
[0008] With this approach, vector-matrix multiplication is performed in analog form and conversion of digital data into analog input signals for the word lines and conversion of analog output signals into digital data on the bit lines is necessary so that the multiplication unit can be combined with conventional digital processing.
[0009] In patent EP3523805B1 and publication Demasius et al .: “Energy-efficient memcapacitor devices for neuromorphic computing" . Nature Electronics , 2021, in addition to the resistive non-volatile memories just mentioned, capacitive non-volatile memories were also presented . These are based on the fact that the capacitance of a capacitor can be freely adjusted and the capacitance cannot be stored volatilely . Capacitive components have the advantage over resistive components that they hardly have any static power consumption and have a better signal-to-noise ratio . This means that with the same signal quality, a lower readout voltage can be used compared to resistive components, which is also accompanied by lower dynamic energy consumption .
[0010] There are various options for the digital-to-analog and analog-to-digital (ADC) converters required for this: parallel converters, successive approximation register (SAR) ADCs, or integrating converters. In analog vector matrix multiplication, a parallel converter or a SAR ADC is most commonly used (Caselli et al.: "Memory Devices and A / D Interfaces: Design Tradeoffs in Mixed-Signal Accelerators for Machine Learning Applications". IEEE, 2022). The parallel converter has its advantage when high speed is required, and the SAR ADC is advantageous when energy efficiency is a priority. However, both ADC types have the disadvantage of requiring a lot of space and, in the case of SAR ADCs, complicated digital control logic.However, for arrays with analog memory components, the bit lines are arranged quite closely together, and to provide an advantage over digital circuits, it is advantageous if the ADCs connected to the bit lines also have a compact design. This is especially important given that design rules must be adhered to when designing integrated circuits (e.g., minimum spacing between transistors and metal lines), and these design rules become more difficult to adhere to with a close arrangement, the more complex and larger the ADC circuit becomes.
[0011] In US 11409045 B2, a ramp ADC is used in conjunction with resistive components. The bit line output current is converted into a voltage by integration (US 11409045 B2) or an inverting amplifier (US 2021271732 A1). This voltage is then compared with a ramp signal and converted into a pulse length by an analog comparator. The pulse length is converted into a binary signal by a counter.
[0012] ADCs using the two-ramp method have a
[0013] Integration phase in which the current from the bit line is integrated (first phase) and then a deintegration phase (second phase) in which the charge state of the integrator is cleared. The time until the integrator returns to the original voltage is converted into a pulse length using a comparator and counted again. An example implementation with current sources for resistive non-volatile memories can be found in application US2020234111A1. The advantage of two-ramp ADCs compared to single-ramp ADCs is that the integration capacitance is canceled out and there is significantly less aging and temperature dependence. However, this application does not explain the use of capacitive non-volatile memories, which have the advantage of significantly reduced energy consumption because they can no longer be used as a current source.
[0014] In the publication Luo et al.: "Design and Optimization of Non-Volatile Capacitive Crossbar Array for In-Memory Computing". IEEE Transactions on Circuits and Systems, 2022, the use of a programmable capacitive non-volatile memory that integrates accumulated charges with an operational amplifier was published. The publication also includes an array of reference capacitors, which, however, serve to solve the problem with a limiting dynamic (on / off) ratio. A two-ramp ADC with an integration and a discharge phase was not mentioned in the publication.
[0015] In application US 20220027130 A1, an arrangement was published which has an integrator on the bit lines and contains capacitors for representing the weight matrix. The input values can also be applied to the word lines coded as pulse widths, assuming either constant voltage pulses or current pulses. This also discloses the use of an integration and a discharge phase. Both phases are based either on constant voltages applied to a conductance or on current sources, with a reference conductance or a reference current source being used in the discharge phase.
[0016] The aforementioned US 20220027130 A1 does not mention any conversion of the input pulse width into a variable voltage value (it only mentions a digital-to-time converter), nor any capacitive non-volatile memories that can be programmed to an analog capacitance value. Instead, constant reference capacitors are connected in parallel with switches so that different weight states can be achieved. The disadvantage of this arrangement is the large amount of space required to represent many states, and that each memory cell consists of at least one capacitor and one switch. In the patent, the integration with the capacitive weights is also carried out over a number of pulses, which implies high energy consumption. The proposed integration and discharge using current sources or conductances also implies high energy consumption.
[0017] This publication represents the closest prior art. The analog comparator in ramp ADCs can, for example, be another op-amp circuit or a differential amplifier, as described in application US 2020234111 A1. However, this has the disadvantage that static power consumption is required, and this type of ADC no longer offers any energy efficiency advantage. Another option would be a dynamic latch-based comparator, which, however, has to switch 256 times for, say, an 8-bit value, which means high dynamic power consumption. In the publication Jendernalik et al.: “An Ultra-Low-Energy Analog Comparator for A / D Converters in CMOS Image Sensors”. Circuits Syst Signal Process , 2017 an energy-ef fi cient comparator for image sensors was presented, which only has a dynamic power consumption at the switching time.For this purpose, an input transistor is used which, depending on the gate-source voltage difference, generates a discharge current for a precharged capacitor. A connected output transistor switches as soon as the threshold voltage of the input transistor is exceeded. The disadvantage of this comparator is the high dispersion of the input transistor's properties, depending on temperature and manufacturing conditions.
[0018] The object of this invention was therefore to develop an architecture for an analog vector matrix multiplication unit that uses capacitive non-volatile storage elements and adapts the two-ramp ADC for this purpose, as well as increasing the energy efficiency with an improved analog comparator of the two-ramp method. According to the invention, this object is achieved in that the network weights consist of programmable, non-volatile, adjustable capacitors, an additional time-to-voltage converter is arranged between the digital-to-time converter, and a reference capacitor is provided that implements the discharge phase with a reference voltage ramp.
[0019] The non-volatile memory components consist of adjustable capacitors and the input block contains switches which connect a voltage ramp generator to the word lines, wherein the switch is designed to be controllable by the input pulse length and the amplifier in the output block with feedback capacitance acts as a non-inverting amplifier together with the adjustable capacitors of the matrix and the capacitances of the adjustable capacitors are summed, wherein the output voltage of the amplifier is determined by the capacitance ratio, and the reference for a second phase, which is connected to the inverting input of the amplifier, consists of a reference capacitor with a connected inverted voltage ramp generator.
[0020] This means the adjustable capacitors ( C m) are connected in parallel to the non-inverting input of the amplifier and the adjustable capacitors form the elements of the matrix . The adjustable capacitors are connected via the word lines to voltage ramp generators, which are controlled by the input pulses, so that the voltage ramps have a different length at the adjustable capacitors . The output voltage at the inverting amplifiers is thus controlled via the
[0021] Input ramp voltage (V ramp ) and the
[0022] Integration capacitor ( nt ) is determined as follows :
[0023] In a second phase, after the first input ramp, a reverse ramp is applied via a reference capacitor to reduce the previously generated output voltage by means of:
[0024] The duration of this second phase is determined by a comparator and corresponds to the output pulse length.
[0025] In a further advantageous embodiment, the matrix consists of differential elements, each with a positive and a negative adjustable capacitor, and the positive and negative adjustable capacitors are connected to a common bit line and each have a word line, which each have a switch in the input block, which are designed to connect with reversed voltage ramps and are designed to be controllable by the same input pulse length.
[0026] Thus, the weights of a neural network are split into positive and negative values, each with a separate word line, and reversed voltage ramps are connected so that the subtraction is already carried out in the matrix, which is an advantage compared to the state of the art, which usually uses differential amplifiers on two separate bit lines. Expressed in formulas, this means that the output voltage of the
[0027] amplifier for the positive ( V ra mp,+ ') and negative ( V ramp> _ ) voltage ramp , as well as the positive ( C m , + ) and negative ( C m _ ) adjustable capacitors :
[0028] In one embodiment, the reverse voltage ramps are connected to one or more further switches which are designed so that the reverse voltage ramps can be swapped if a different sign of the input pulse length is present.
[0029] In this way, for example, both the negative and positive voltage ramps can be connected to the positive adjustable capacitor, or conversely, to the negative adjustable capacitor. This, together with the positive and negative adjustable capacitors, enables four-quadrant multiplication in the matrix.
[0030] In a further embodiment, a further offset capacitor is connected to the inverting input of the amplifier, which is designed to connect to a further voltage ramp, so that an offset of the output voltage of the amplifier is achieved.
[0031] The purpose of this arrangement is to ensure that the amplifier always has a limited voltage range. If it starts in the lower voltage range in phase 1, it would not be able to cover a negative voltage range. For this purpose, an additional offset capacitor with a voltage ramp is connected to the amplifier input. This shifts the initial output voltage so that negative voltage ranges can also be covered.
[0032] In a further embodiment, one comparator input is connected to a capacitor, which is further connected to a voltage ramp generator via a switch, the switch being designed to be controlled by the comparator output, and the other comparator input being connected to the output of the amplifier.
[0033] A typical comparator has an offset voltage, and the amplifier typically also has an offset voltage, which depends on the selected amplifier topology. This can fluctuate with temperature but also over the manufacturing process. Especially with the comparator structure selected here (next embodiment), the threshold voltage of the input transistor will vary considerably. Varying the offset voltages would lead to a changed output pulse width of the comparator, since it switches at different times. This embodiment describes an arrangement that samples the offset voltage of the comparator and the amplifier on a capacitor using a connected voltage ramp, so that it is subtracted out.
[0034] In a further advantageous embodiment, the comparator consists of an input transistor, in which one input of the comparator is connected to the source terminal and the other input of the comparator is connected to the gate terminal of the input transistor and the drain terminal of the input transistor is connected to a capacitor, which can optionally also be a parasitic capacitance of a transistor, and a gate terminal of a further output transistor, wherein the capacitor and the output of the output transistor are designed in such a way that they can be precharged before a comparison phase.
[0035] Ultimately, the comparator consists of an input transistor that generates a discharge current that depends on the difference voltage between the two inputs (gate and source terminals). This discharge current discharges a capacitor connected to the gate terminal of an output transistor, and at a certain point, the output transistor switches, which corresponds to the end of the output pulse length. This point in time is approximately reached when the difference voltage (VG S ) at the input the threshold voltage (V T ) exceeds :
[0036] The advantage of this comparator is that dynamic power consumption occurs only once, at the switching instant. This distinguishes the comparator from other dynamic comparators. However, the threshold voltage is subject to high variation, which can be resolved by the previous embodiment.
[0037] In another embodiment, a counter is connected to the comparator output. This converts the output pulse length into a binary number.
[0038] In another embodiment, the comparator output is connected to a conversion table, which is controlled by a clock generator. The advantage of this arrangement is that, compared to a binary counter, any number can be written into the conversion table, thus compensating for any non-linearities. It is also conceivable that the non-linear activation function, e.g., ReLU or sigmoid, could be mapped to such a conversion table.
[0039] The following are procedural embodiments explained in more detail:
[0040] In a favorable embodiment, in a first phase, a voltage ramp is applied to the bit lines of the matrix, which is controlled by the input pulse length and the non-volatile adjustable capacitors of the matrix are summed along the bit line. In a second phase, an inverted voltage ramp is applied to the inverting input of the amplifier via a reference capacitor, and the comparator outputs a pulse length until the output voltage of the inverting amplifier has returned to the original voltage. Finally, the input voltage ramp is converted into an output voltage ramp at the amplifier using the following capacitance ratio in a first step:
[0041] In a second phase, this ramp is reduced again with a reference capacitor:
[0042] Since the slope and end point of the amplifier's output voltage ramp can vary depending on the stored adjustable capacitance and input pulse length, but the ramp of the second phase always remains the same, the inverting amplifier returns to its original voltage at different times. This time is determined by the comparator and output as a pulse length.
[0043] In a further method embodiment, the word lines of the positive and the word lines of the negative adjustable capacitors are charged with reversed voltage ramps, but the voltage ramps of the word lines of a corresponding matrix element consisting of a positive and negative adjustable capacitor are controlled with the same input pulse length.
[0044] In another method embodiment, the reverse voltage ramps are swapped, so that the reverse voltage ramp is applied to the word lines of the positive and negative adjustable capacitors when the input signal has a different sign. This enables four-quadrant multiplication, since both positive and negative input values are now possible. This subtracts the positive and negative elements of the matrix within the matrix itself, which is advantageous compared to implementations using a differential amplifier, since this requires significantly more space.
[0045] In another method embodiment, a voltage ramp is connected to one comparator input, which simultaneously charges a capacitor. This voltage ramp is interrupted as soon as the comparator output switches and the capacitor stores the sampled offset voltage. This method compensates for temperature and process fluctuations.
[0046] In a further method embodiment, in a first phase the capacitor and the output of the output transistor are precharged with a reset signal and in a second phase the capacitor is discharged with the aid of the input transistor, which is controlled by the voltage difference from the two inputs, and the output transistor switches over as soon as the threshold voltage of the input transistor is exceeded.
[0047] This type of use of the comparator enables significantly lower energy consumption, since dynamic energy is only consumed at the switching time.
[0048] The following are some of the most advantageous designs:
[0049] Fig . 1 : Overview of the memory matrix with input and
[0050] Output block Fig. 2: Function of the output block
[0051] Fig. 3: Offset voltage sampling with the comparator Fig. 4: Circuit diagram of a possible comparator
[0052] Fig. 1 shows the matrix with non-volatile memory components (4) and connected input (1) and output blocks (6). These are each connected to the word lines (4) and bit lines (5). Within the matrix (4) adjustable capacitors (10) are used, which are split into positive (15) and negative (16) capacitors. The adjustable capacitors (10) are summed along the bit line (5). The word lines are provided with voltage ramps (12) which have an inverse increase for positive (15) and negative (16) capacitors. The length of the voltage ramp is controlled via switches (11) in the input block and the input pulse length (2).
[0053] The output block is explained in more detail in Fig. 2, where the positive (15) and negative (16) capacitors with the switches (11) for the input block are included for explanation. The adjustable capacitors (10) add up along the bit line and, together with the inverting amplifier (7) and a feedback capacitance (8), form an inverting amplifier whose output voltage depends on the value of the adjustable capacitors (10) and the input pulse length (2). In a second phase, a reverse voltage ramp (14) is applied to a further reference capacitor (13) until the previously built-up output voltage has been reduced again, and the length is output as the pulse length by a comparator (9). It is also possible to introduce an offset voltage in the amplifier (7) using an offset capacitor (17) and a further voltage ramp (18).This is particularly useful if both positive and negative voltages are to be output.
[0054] Fig. 3 explains the sampling of the offset voltage: For this, one input (19) of the comparator is connected to a capacitor (20), which is charged by a voltage ramp (22). This is controlled by the output (23) of the comparator and a switch (21). As soon as the comparator (9) switches, the offset voltage is reached on the voltage ramp (22) and stored on the capacitor (20). Since the two inputs are subtracted, the offset voltage is fed into the comparison phase, where the output pulse is generated.
[0055] Fig. 4 shows an advantageous design for a comparator. Since conventional comparators have either a static power consumption or a high dynamic power consumption, a comparator with low dynamic power consumption is used here, which only consumes power at the switching instant (apart from leakage currents). A capacitor (26) is precharged with a reset signal (28) and then discharged via an input transistor (25), the amount of discharge depending on the difference between the input signals applied to the gate and source terminals of the input transistor (25). The output transistor (27) switches as soon as the threshold voltage of the input transistor (25) is exceeded.List of reference symbols - input block - input pulses - word lines - matrix of non-volatile memory components - bit lines - output blocks - amplifier - feedback capacitance - comparator - adjustable capacitors - switch for input block - voltage ramps (generator) - reference capacitor - reverse voltage ramp for reference - positive adjustable capacitor - negative adjustable capacitor - offset capacitor - voltage ramp for offset capacitor - first comparator input - capacitor for comparator - switch for comparator - voltage ramp generator for comparator - comparator output - second comparator input - input transistor - capacitor in comparator - output transistor 28 reset signal.
Claims
Patent claims 1. An arrangement of a pulse-width controlled vector matrix multiplication unit, comprising a digital-to-time converter connected to a reference time and configured to receive a digital input signal and output a signal that is proportional to the digital input signal and modulated in the time domain, a memory, an output interface connected to a matrix of memories and configured to receive its weighted output signal and output a digital value that is proportional to at least the reference time, and a time-to-digital converter, the arrangement being configured to carry out a two-stage process comprising a first accumulation step with an initial condition for a reference discharge phase in the second step, characterized in that the network weights consist of programmable non-volatile adjustable capacitors,another time-to-voltage converter is arranged after the digital-to-time converter and a discharge phase with a reference voltage ramp, Reference capacitor is provided.
2. Arrangement according to claim 1, characterized in that the time-to-voltage converter consists of a reference voltage ramp generator and a switch which is controlled by the input pulse length and whose one switch end is connected to the reference voltage ramp generator and whose second switch end is connected to the input of the matrix.
3. Arrangement according to claim 1 or 2, characterized in that differential elements of the matrix, each with a positive (15) and a negative (16) adjustable capacitor, are used and the positive and negative adjustable capacitors are connected to a common bit line (5) and each have a word line (3), which in the input block each have a switch (11) which is designed to be connected with inverted voltage ramps (12) and is designed to be controllable by the same input pulse length (2).
4. Arrangement according to one of claims 1 to 3, characterized in that the reversed voltage ramps (14) are connected to one or more further switches which are designed such that the reversed voltage ramps (14) can be exchanged if a different sign of the input pulse length (2) is present.
5. Arrangement according to one of claims 1 to 4, characterized in that a further offset capacitor (17) is connected to the inverting input of the amplifier (7), which is connected to a further Voltage ramp (18) is designed to connect so that an offset of the output voltage of the amplifier is achieved.
6. Arrangement according to one of claims 1 to 5, characterized in that one comparator input (19) is connected to a capacitor (20), which is further connected via a switch (21) to a voltage ramp generator (22), the switch being designed to be controlled by the comparator output (23), and the other comparator input (24) is connected to the output of the amplifier (7).
7. Arrangement according to one of claims 1 to 6, characterized in that the comparator (9) consists of an input transistor (25), in which one input of the comparator is connected to the source terminal and the other input of the comparator is connected to the gate terminal of the input transistor (25) and the drain terminal of the input transistor (25) is connected to a capacitor (26), which can optionally also be a parasitic capacitance of a transistor, and a gate terminal of a further output transistor (27), wherein the capacitor (26) and the output of the output transistor (27) are designed such that they can be precharged before a comparison phase.
8. Arrangement according to one of claims 1 to 7, characterized in that a counter is connected to the output of the comparator.
9. Arrangement according to one of claims 1 to 8, characterized in that the output of the comparator is connected to a conversion table which is designed to be controlled by a clock generator.
10. Method for controlling an arrangement according to one of claims 1 to 9, characterized in that in a first phase a voltage ramp (12) is applied to the bit lines (5) of the matrix, which is controlled by the input pulse length (2) and the non-volatile adjustable capacitors (10) of the matrix are summed along the bit line (5), and in a second phase an inverted voltage ramp (12) is applied to the inverting input of the amplifier (7) via a reference capacitor (17) and the comparator (9) outputs a pulse length until the output voltage of the inverting amplifier (7) has returned to the original voltage.
11. Method according to claim 2, characterized in that the word lines (3) of the positive (15) and the word lines of the negative (16) adjustable capacitors are charged with reversed voltage ramps (12), but the voltage ramps (12) of the word lines (3) of a related matrix element consisting of a positive and negative adjustable capacitor are controlled with the same input pulse length (2).
12. Method according to claim 3, characterized characterized in that the reverse voltage ramps (12) can be swapped so that the reverse voltage ramp is applied to the word lines of the positive and negative adjustable capacitors when the input signal (2) has a different sign.
13. Method according to claim 5, characterized in that a voltage ramp (12) is connected to one comparator input (19), which simultaneously charges a capacitor (20) and this voltage ramp (12) is interrupted as soon as the comparator output (23) switches and the capacitor (20) stores the sampled offset voltage.
14. The method according to claim 6, characterized in that in a first phase the capacitor (26) and the output of the output transistor (27) are precharged with a reset signal (28) and in a second phase the capacitor (26) is discharged with the aid of the input transistor (25), which is controlled by the voltage difference between the two inputs, and the output transistor (27) switches over as soon as the threshold voltage of the input transistor (25) is exceeded.