METHOD FOR ANALOGUE MULTIPLATION AND / OR CALCULATION OF A SCALAR PRODUCT USING A CIRCUIT ARRANGEMENT, ESPECIALLY FOR ARTIFICIAL NEURAL NETWORKS

Description

Technical application area

[0001] The present invention relates to a method for analog multiplication and / or for the analog calculation of a scalar product, which is formed by multiplying a first value with a second value of a respective pair of values ​​and summing the results of the multiplication for several pairs of values, comprising a circuit arrangement. The invention also relates to the application of the method in an artificial neural network (ANN).

[0002] In electronic signal processing, most circuit components in the digital domain are now implemented using CMOS technology and binary static logic. Analog-to-digital conversion (ADC) and digital-to-analog conversion (DAC) are shifted to the system periphery as much as possible. This approach of primarily digital signal processing has greatly benefited from the scaling of semiconductor technology to date, i.e., Moore's Law. The technology-driven efficiency gains have balanced the ever-increasing demand for processing power. However, Moore's Law has slowed considerably in the meantime, so this balance is at risk in the future, especially given the increasing demand for signal processing power in the field of artificial intelligence (AI).The demands on the computing power of deep neural networks (DNNs) are increasing significantly faster than the scaling gains of the underlying CMOS technology. Therefore, there is an urgent need for new, energy-efficient signal processing techniques that can be used in artificial neural networks. State of the art

[0003] Artificial neural networks currently primarily employ digital signal processing techniques based on CMOS technology. However, these techniques will soon reach their limits in terms of energy efficiency, given the ever-increasing demands on computing power.

[0004] Approaches are now being pursued that employ mixed-signal processing based on CMOS technology, utilizing switched-capacitor (SC) charge redistribution techniques for analog multiplication or summation. These approaches use dedicated SRAM arrays for storing input values ​​(activations) and weights, and a neuron array for calculating the dot product of the input and weight vectors. In the layout of the corresponding integrated circuit, the memory array and neuron array represent spatially separated units. To further reduce the energy consumption for data transfer between these units, it is also known to integrate both units into a single unit. This approach is referred to as in-memory processing.

[0005] M. Bavandpour et al., "Mixed-Signal Neuromorphic Inference Accelerators: Recent Results and Future Prospects," presented at the 2018 IEEE International Electron Devices Meeting (IEDM), provide an overview of vector-matrix multiplication (VMM) circuit designs. In one of these designs, each weighting factor is stored in floating-gate cells implemented as a voltage-controlled current source. Both the input and output values ​​are encoded as the pulse length of voltage pulses.

[0006] Another example of an analog multiplier, in which a first value is encoded as the gate pulse width and a second value as the floating-gate charge, is known from US4956564. The object of the present invention is to provide a method for multiplication and / or the formation of a scalar product, which can be implemented in CMOS technology and enables energy-efficient operation, as well as an artificial neural network in which the method is used. Description of the invention

[0007] The problem is solved by the methods according to claims 1 and 2 and the artificial neural networks according to claims 10 and 11. Advantageous embodiments of the methods and the artificial neural networks are the subject of the dependent claims or can be found in the following description and the exemplary embodiments.

[0008] The proposed method for analog multiplication, like the method for the analog calculation of a scalar product, employs a circuit arrangement comprising a series connection of a first FET and a second FET or FET array serving as a current source, a charging device, and at least one capacitor. This capacitor can be pre-charged via the charging device and discharged via the series connection of the first FET and the at least one second FET or FET array. The charging device can consist of a simple switch through which the capacitor can be connected to a voltage source. To multiply a first value by a second value, the capacitor is first pre-charged.The first value, encoded as the pulse length of a voltage pulse, is applied to the gate of the first FET, and the second value, encoded as a voltage amplitude, is applied to the gate of the second FET, or as binary voltage amplitudes, to the gates of the parallel-connected second FETs. This causes the capacitor to discharge, at least partially, for a period defined by the pulse length of the voltage pulse applied to the gate of the first FET, with a discharge current defined by the voltage amplitude(s) applied to the gate of the second FET or to the gates of the parallel-connected second FETs. The result of the multiplication can then be determined either from the residual charge or voltage of the capacitor, or—in a configuration described below, taking into account the sign of the first value—from a voltage or charge difference between this capacitor and another capacitor.

[0009] For calculating the scalar product, an analogous circuit arrangement is used, comprising several parallel-connected series circuits consisting of a first FET and a second FET serving as a current source, or a FET array consisting of several parallel-connected second FETs, a charging device, and at least one capacitor that can be pre-charged via the charging device and discharged via the series circuits consisting of the first FET and the second FET or FET array. The calculation of a scalar product here is understood as the multiplication of a first value by a second value of a pair of values ​​and the summation of the results of the multiplication for several pairs of values, as is the case when multiplying two vectors with vector components (as values) in a Cartesian coordinate system. Each of the pairs of values ​​of the scalar product is assigned one of the series circuits.The number of series circuits must at least correspond to the number of value pairs of the scalar product or the vector components of the vectors to be multiplied together. The capacitor is pre-charged for the calculation of the scalar product.For each pair of values, the first value, encoded as the pulse length of a voltage pulse, is applied to the gate of the first FET in the series circuit associated with that pair of values. The second value, encoded as a voltage amplitude, is applied to the gate of the second FET, or as binary voltage amplitudes, to the gates of the parallel-connected second FETs in the associated series circuit. This causes the capacitor to discharge, at least partially, for a period defined by the pulse length of the voltage pulse applied to the gate of the first FET in the respective series circuit. The discharge current is defined by the voltage amplitude(s) applied to the gate of the second FET or to the gates of the parallel-connected second FETs in the respective series circuit. Due to the parallel connection of the series circuits, the discharge currents add up according to Kirchhoff's law.A result of the calculation of the scalar product can then be determined either from the residual charge or voltage of the capacitor or - in a configuration described below, taking into account the sign of the first value - from a voltage or charge difference between this and another capacitor.

[0010] In the proposed methods and neural networks, field-effect transistors (FETs) with an insulating gate ("MISFET"), preferably MOSFETs (metal-oxide-semiconductor FETs), are used. The capacitance to be discharged can be provided by a capacitor or by the parasitic capacitances of the FETs and connecting lines.

[0011] The proposed methods thus employ a circuit arrangement with a nonlinear transfer function, which in its basic configuration consists of two stacked, series-connected FETs and a capacitor. This circuit arrangement is subsequently referred to as an analog mixed-signal multiplier (AMS) due to its function. The first multiplicand value is represented as the pulse width of a voltage pulse applied to the gate of the first FET. The second multiplicand is encoded as an analog voltage applied to the gate of the second FET. An electrical charge packet, proportional to the product of the multiplicands, is accumulated on or subtracted from the capacitor. By connecting further stacked FETs in series to the capacitor, the calculation of a scalar product is made possible with a minimal number of components.Analog multiplication can be performed with low energy consumption, as will be explained in more detail in the later embodiments. The simple design using FETs and capacitors allows the circuit arrangement to be implemented in CMOS technology. In contrast to the digital processing techniques with binary signals that have been preferred in electronic signal processing to date, the proposed methods utilize analog-mixed signal processing, in which selected electrical nodes carry significantly more information in the analog domain, physically limited only by noise and leakage. These methods enable the execution of complex arithmetic operations at individual capacitive circuit nodes, using as few components as possible and drastically reducing the number of required circuit nodes compared to digital techniques. The methods are particularly advantageous when...The circuit arrangements used therein for the computational operations in neurons or neuronal layers in artificial neural networks can be employed. The fundamental functionality of artificial neurons can be mapped to the proposed AMS circuit arrangement with a small number of circuit nodes and components and implemented using advanced CMOS foundry technology.

[0012] In a preferred application of the method for calculating a scalar product in an artificial neural network, the circuit arrangement consisting of the parallel-connected series circuits, the charging device, and the capacitor is each part of an artificial neuron. Each pair of values ​​corresponds to a weight factor and an input value of the artificial neuron. In the preferred embodiment, the weight factor is chosen as the first value of each pair, and the input value as the second. Thus, the weight factors are encoded as the pulse length of a voltage pulse and applied to the gate of the first FET, and the input values ​​are encoded as the voltage amplitude and applied to the gate of the second FET. The weight factors, which are typically digital values, must be stored appropriately, preferably in suitable SRAMs, and are converted into the corresponding voltage pulses by means of a digital-to-time converter (DTC).

[0013] In an alternative embodiment of the proposed method for calculating a scalar product in an artificial neural network, the input value is chosen as the first value of the value pair, and the weighting factor as the second value. In this case, preferably not just one, but several second FETs are used in a parallel circuit, with the weighting factors again being stored digitally. The individual binary digits of the respective weighting factor then control—encoded as a voltage amplitude—the individual second FETs of the FET array, i.e., the parallel connection of the second FETs. This will be explained in more detail in the exemplary embodiment.

[0014] By using two parallel branches with first FETs connected in series to at least one second FET or FET array, and two capacitors, signed weighting factors can also be processed. Signed input values ​​can also be implemented by appropriately extending the circuit topology.

[0015] The proposed method for calculating the scalar product and the circuit arrangement used therein allow the construction of a neural network in which the neurons are formed by the circuit arrangements. Depending on the specific design, digital time converters (DTCs) may also need to be implemented. Suitable transfer circuits for shifting the charge at the output of a neuron to the inputs of neurons in the next layer may also be required, depending on the design. An example of such a transfer circuit can be found in the following embodiments.

[0016] A significant advantage of the proposed methods and the circuit arrangements employed is their very low power consumption. An approximately 500-fold increase in energy efficiency is expected in a 28 nm CMOS AMS compared to a digital 8-bit x 8-bit field multiplier. The circuit arrangements used in the methods can be implemented on commercially available standard CMOS technologies, which are also used for a wide variety of standard and application-specific integrated circuits. This allows the proposed circuit arrangements to be used in a hybrid approach alongside traditional analog, RF, digital, and memory blocks on a single chip in a "system-on-a-chip" approach. An AMS IP library enables the design of an AMS coprocessor IP that can be used together with other blocks on an application-specific integrated circuit (ASIC) or together with standard COTS digital processor ICs.Together with standard smartphone processors, it can be placed to enable highly energy-efficient co-processing of specific KNN-related tasks. By using established standard CMOS logic technology, power-hungry chip-to-chip interfaces for such hybrid systems with conventional digital and new analog signal processing are avoided. The proposed methods are particularly well-suited for applications that do not require exceptional precision. The improved energy efficiency of the proposed method can also support the development of classification tasks based on neural networks, for example, in radar and lidar object detection for autonomous driving in automobiles or in mobile, person-assisted speech and image recognition. Brief description of the drawings

[0017] The proposed methods, in conjunction with an artificial neural network, are explained in more detail below using exemplary implementations and the accompanying drawings. These drawings show: Fig. 1 shows a section of three layers from an artificial neural network (partial figure a) and the structure of an artificial neuron (partial figure b); Fig. 2 shows an example of the circuit arrangement for multiplication (partial figure a) and the circuit arrangement for calculating a scalar product (partial figure b) according to the present invention; Fig. 3 shows an example of a DTC circuit (partial figure a) and the time evolution of the voltages within the circuit (partial figure b); Fig. 4 shows an example of an alternative circuit arrangement for multiplication according to the present invention; Fig. 5 shows examples of the implementation of signed weighting factors in the proposed circuit arrangement; Fig. 6 shows an example of the matrix-like arrangement of several of the proposed multiplication circuit arrangements for implementing a neural layer; Fig.Fig. 7 is an example of a charge transfer circuit for transferring a charge deficit from the output of one neuron to the input of the next neuron; and Fig. 8 is an example of the configuration of a neural network according to the present invention. Ways to implement the invention

[0018] In the following examples, the proposed method and the associated circuit arrangement are used to calculate scalar products in an artificial neural network. Figure 1Figure a shows a section with three layers of an artificial neural network. Figure b depicts the basic structure of an artificial neuron, specifically for the j-th neuron in layer y of the neural network. The input values ​​x1 to xn—that is, the activations from the preceding layer x—are multiplied by the corresponding weighting factors or weights wj1 to wjn, and the multiplication results, along with a constant value bj = x0 · wj0, are added. The resulting sum Sj corresponds to the dot product of the activation vector. X layer x of the neural network and the weight vector WJ , which represents the synaptic weights of the input signals to neuron yj. Furthermore, the sum S j represents the argument of the transfer function φ(S j ) that generates the final neuron activation yj. Each multiplication xi · w ji corresponds to a single synaptic operation.

[0019] The proposed method performs the calculation of the scalar product in a neuron in an energy-efficient manner. Figure 2a The diagram illustrates the core schematic of the proposed circuit arrangement, i.e., the AMS multiplication cell (AMS: analog mixed-signal multiplier), which is based on two stacked FETs (here MOSFETs) and a capacitor used here as a capacitance. Initialization is achieved by pre-charging the capacitor C to the positive supply voltage U DD. The circuit principle is similar to the pre-charge and evaluation function in CMOS domino logic. Figure 2a Figure 1 shows the two MOSFETs Nw and Nx connected in series, the capacitor C, and the pre-charging device connected to the supply voltage UDD, here in the form of a switch. The basic scheme of this AMS multiplication cell is used in two different embodiments of the proposed method.

[0020] In the preferred embodiment, the multiplication result is evaluated as follows. The lower MOSFET Nx operates as a current source transistor, controlled by its analog gate-source voltage uGS,Nx = ux, which is provided via an input value x, the output of the preceding neuron layer. The voltage ux controls the drain current ix via the nonlinear transfer function Ix(Ux) according to the MOSFET's current equation. This nonlinearity is part of the nonlinear transfer function φ of the preceding neuron layer. Since the n-channel MOSFET has a threshold voltage greater than 0 in enhancement mode, a soft rectifier-like transfer function is implemented.

[0021] The drain current ix is ​​drawn from the upper terminal of capacitor C only if the stacked MOSFET Nw is also conducting. By setting its gate voltage to U DD for a time interval TW corresponding to the weighting factor w, the upper MOSFET NW is switched on. The charge Q XW drawn from the output node and the corresponding output voltage UC are: Q xw = T w ⋅ I x U x , U C = U DD − T w ⋅ I x U x C .

[0022] The result of the multiplication thus corresponds to the amount of charge Q XW flowing through the series connection of these two MOSFETs. The time relationships of the voltages and currents in this circuit arrangement are shown in the left part of the Figure 2a depicted.

[0023] Figure 2bThis figure shows the implementation of the proposed circuit arrangement for realizing an AMS scalar product cell by applying Kirchhoff's current law to a common output node, in which all output currents of the AMS multiplication cells are cumulated. The corresponding parallel connection of several series circuits of two MOSFETs, the capacitor C, and the associated charging device are shown in the figure. Figure 2b schematically represented. The output voltage U yi is: U yj = U DD − Q yj C tot , j = U DD − 1 C tot , j ∑ i = 0 n Q xwji = U DD − 1 C tot , j ∑ i = 0 n T wji I xi U xi .

[0024] The artificial neuron function, i.e., a scalar product followed by a nonlinear transfer function, is mapped to simple electrical network principles (i.e., Kirchhoff's laws) in conjunction with established FET device physics (IDS = f(UGS, UDS)). Neuron output activation is implemented along a single line using a series of multipliers.

[0025] Analog multiplication is achieved using only two small MOSFETs. The total capacitance to be charged or discharged during the multiplication process can be limited to just 0.6 fF for 300 nm wide MOSFETs Nx and Ny in 22 nm CMOS. This results in a multiplication power consumption of 0.5 fJ at a supply voltage of 0.8 V. In contrast, the estimated operating power of an 8-bit x 8-bit field multiplier in 28 nm CMOS technology is 8 x 30 fJ = ​​240 fJ (based on 30 fJ for a single 8-bit adder), representing an approximately 500-fold increase in energy efficiency for the proposed AMS.

[0026] In the preferred embodiment above, the neuron input weighting factors wi are represented by the time width T wi of current pulses, where the current amplitude I xi represents the input activations or input values ​​xi (see Figure 1). To minimize energy consumption, the individual weighting factors are preferably stored locally directly adjacent to the corresponding AMS multiplier cells. In a standard CMOS process—the target technology for implementing circuits according to this invention—the most efficient and easiest-to-use memory implementation is formed by sets of static 6-MOSFET memory cells representing binary words or digits. Therefore, a conversion from the digital binary memory words to time pulse widths, i.e., a digital-time converter (DTC), is required.

[0027] Figure 3aFigure 1 shows a circuit that performs this conversion based on the discharge of a parasitic circuit node capacitance C node by a programmable current I dis. The input binary word is represented by the binary signals W 0 to W k supplied by the binary memory cells. These binary signals control the discharge rate of the pre-charged node U out1. The different discharge currents in the various paths via the switching MOSFETs N slvt, at whose gates the binary signals W 0 to W k are applied, are set by the different threshold voltages of the MOSFETs across these switching MOSFETs, indicated by the abbreviations uhvt (ultra-high threshold voltage), llhvt (high threshold voltage for low leakage current), hvt (high threshold voltage), and rvt (regular threshold voltage).Since the path currents are determined by the threshold voltage and not by the channel width, all MOSFETs in this circuit can have a minimum channel width, resulting in very low dynamic power consumption. Two further pre-charged and cascaded dynamic amplifier stages with output nodes Uout2 and Uout3 provide gain and binary signal level regeneration when set to evaluation mode by the signals Urst and Urst2. The two reset / pre-charge signals Urst and Urst2 and the evaluation signal Uev1 are time-shifted, as shown in the diagram. Figure 3b is shown.

[0028] In an alternative embodiment, the weighting and activation inputs, and thus the roles of the lower and upper MOSFETs, are determined in the multiplier evaluation path(s) of Figure 2 swapped, as in Figure 4 The weighting is now represented by a constant source current Iw (see...). Figure 4a ), which is supplied either by a single lower MOSFET or by a programmable set of lower MOSFETs N Wk with drains and sources connected in parallel as the current source Nw. Figure 4b Figure 1 shows an example of the circuit implementation of a current source Nw controlled by a digital word W using an array of parallel lower MOSFETs N Wk. The two-stage time-dependent activation input ux, applied to the gate of the upper MOSFET N x, controls the time pulse width T x of the current discharge current iw (t). In the case of a set of multiple MOSFETs N Wk, the source current I w is in turn controlled by a local binary weight storage device that provides the binary signals W 0 to W k.

[0029] The advantage of the alternative embodiment of the Figure 4 compared to the preferred embodiment of the Figure 2The advantage lies in the fact that no digital time converters or digital pulse-width converters are required for the weighting factors at each position of the mixed-signal multiplier. The disadvantage of the alternative embodiment compared to the preferred embodiment is that charge or voltage pulse-width converters are necessary between the activation outputs (signal domain: analog voltage or charge) of one neural network layer and the subsequent activation input of the next neural layer (signal domain: pulse width). Such a charge pulse-width converter can be implemented after evaluation by recharging the capacitor C with a constant current I charge, starting at a predefined time t 0. A trigger circuit detects the time t 1 of the complete recharging of the capacitor C.Between times t 0 and t 1, a positive voltage uy = U DD is output for the duration ty = t 1 -t 0, where ty = Q y / I charge is proportional to the charge Q y taken from C by the AMS circuit.

[0030] The AMS multiplier circuits according to Figure 2a and Figure 4a They only work with unsigned signals. In the charge equation, both the currents Ix (preferred configuration) or Iw (alternative configuration) and the pulse width Tw (preferred configuration) or Tx (alternative configuration) are positive, resulting in a positive charge Q = I·T, which is drawn from the pre-charged capacitor C in both configurations. Extensions of the circuit topology based on the AMS multipliers according to Figure 2a and Figure 4a They are based on the principle of enabling the use of signed signals.

[0031] In artificial neural networks, the activation value range is often limited to positive values. However, the weights can be positive or negative. Figure 5a The block diagram shows a signed weighting factor using two signals at the neuron's activation output. For a positive weighting factor wji, the two weight components are set to wjip = wji and wjin = 0. For a negative weighting factor wji, the two weight components are set to wjip = 0 and wjin = -wji.

[0032] Figure 5bFigure 1 shows the circuit topology for implementing a signed weight for the preferred embodiment. A MOSFET pair Nwp and Nwn is used, with their common source node connected to the drain of Nx, and the pair Nwp and Nwn replacing the single MOSFET Nw. The drains of Nwp and Nwn are connected to two output voltage lines ucp and ucn, respectively, which have pre-charged capacitors Cn = Cp. The final output signal is the voltage difference ucD = ucp - ucn or the charge difference QD = Qp - Qn. A selector connects the output signal Uout3 of the DTC to the Figure 3a with the corresponding input u wp or u wn of the differential pair N wp / n of the Figure 5b , depending on the sign of the weight factor. The other input of the differential pair is connected to mass.

[0033] Figure 5cFigure 1 shows the circuit topology for implementing a signed weight in the alternative configuration. A MOSFET pair Nxp and Nxn is used, with their common source node connected to the drain of Nw, and the pair Nxp and Nxn replacing the single MOSFET Nx. The drains of Nxp and Nxn are connected to two output voltage lines ucp and ucn, respectively, which feature pre-charged capacitors Cn = Cp. The final output signal is either the voltage difference ucD = ucp - ucn or the charge difference QD = Qp - Qn. A selector connects the activation input signal ux (pulse width domain) from the Figure 3 with the corresponding input u xp or u xn of the difference pair N xp / n from Figure 5c , depending on the sign of the weighting factor. The other input of the differential pair is connected to ground. Here too, N w can be used as a programmable current source according to Figure 4b be implemented.

[0034] To implement both signed weights and signed input activations or input values, the circuit topologies of the Figures 5b and 5c , which represent a single differential topology, are extended to doubly differential topologies and, by cross-connecting their outputs, to u cp and u cn. For the preferred embodiment ( Figure 5b ) the single differential pair N x +(N wp -N wn ) is doubled to form the doubly differential topology (N xp +(N wp -N wn ) p )-(N xn +(N wp -N wn ) n ) as shown in Fig. 5d The diagram shows a uxp and uxn input for a signed differential input activation signal (voltage domain) connected to the gates of the two Nxp and Nxn current source MOSFETs. The weight uwp (pulse width domain) is connected to both Nwp,p and Nwp,n, and the weight uwn is connected to both Nwn,p and Nwn,n.

[0035] For the alternative design ( Figure 5c ) the single differential pair N w +(N xp -N xn ) is doubled to form the doubly differential topology (N wp +(N xp -N xn ) p )-(N wn +(N xp -N xn ) n ) as shown in Fig. 5e As shown, there is a uxp and uxn differential input for a signed differential input activation signal (pulse width domain); uxp is connected to both Nxp,p and Nxp,n, and uxn is connected to both Nxn,p and Nxn,n. Nwp is the active current source for positive weighting factors, and Nwn is the active current source for negative weighting factors.

[0036] A single neuronal layer can be realized through a matrix-like arrangement of several AMS multiplication cells or an arrangement of several scalar product cells next to each other, as is the case in Figure 6 This is an example. Figure 6aThis shows an arrangement of several adjacent AMS scalar product cells with n horizontal lines for the input activation vector. X, m vertical lines for the output activation vector Y and multiplication cells placed at each intersection. Such an arrangement can evaluate a layer of a neural network (see...). Figure 1a ).

[0037] The connection of the AMS multiplication cell with a horizontal and a vertical line and the connection to a local weight storage device (+ digital time converter (DTC)) is in Figure 6b A second superimposed grid of horizontal and vertical lines is used to write weight data from the north and east sides of the matrix array to the local weight memory. A signal flow diagram representation of the circuit is shown. Figure 6b is in Figure 6c depicted.

[0038] The in Figure 6The matrix arrangement of AMS multiplication cells shown is capable of evaluating a single layer in an artificial neural network. For the computations of a complete artificial neural network, several of the operations performed by the matrix arrangement must be cascaded. This can be achieved by feeding the evaluated matrix output signals yi back to the matrix inputs xi or by feeding the evaluated matrix output signals yi to the matrix inputs xi of another (different) matrix. In the preferred embodiment, the output and input signals yi and xi are analogous charge (Q) or voltage (Q / C) amplitude domain signals.

[0039] A very efficient method for transferring the analog amplitude domain signals from the outputs back to the inputs is charge transfer. A corresponding circuit for charge transfer (transfer of a charge deficit) is described in Figure 7This circuit is illustrated as an example. The advantage of this circuit is that it does not use any Class A linear amplifiers with static power consumption. Charge transfer occurs exclusively via switched-mode common-gate circuits with dynamic power consumption.

[0040] Alternatively, charge transfer can also be achieved via analog voltage signal transmission through linear analog buffer amplifiers, i.e., based on operational amplifiers with resistors and / or switched capacitors. Digital signal transmission is also possible through the interposition of analog-to-digital and digital-to-analog converters, preferably implemented using energy-efficient switched-mode (SC)-based conversion principles such as single-stage amplifiers (SAR) and supplemented by means for processing large neural layers and implementing artificial transfer functions. This can be accomplished, for example, via digital memory and digital signal processing blocks.

[0041] In the alternative embodiment of the proposed method, the output signals yi are signals in the charge (Q) or voltage (Q / C) amplitude domain, while the input signals xi are signals in the pulse width domain. Therefore, a charge-to-pulse-width converter, as described in one of the preceding sections, is required for signal transmission from the matrix outputs yi to the matrix inputs xi.

[0042] Figure 8 Figure 1 shows an exemplary implementation of the proposed method in an overall architecture for an integrated neural network coprocessor based on the AMS principles described above (black solid line blocks), supplemented by a parallel digital signal processing path (gray solid line blocks) and an additional external learning unit (dashed lines). The central part is an nxm AMS multiplication and addition matrix, as used in conjunction with Figure 6aAs already explained, at each intersection point there is a forward multiplication and addition unit (AMS multiplication cell, see below). Figures 6b and 6c The array of neurons is arranged in a way that allows the matrix to continuously evaluate the neuronal layers. Control units for the distributed weight storage are located at the north and east corners of the matrix.

[0043] A stack of functional blocks, required for pre-loading and writing the analog horizontal lines and for reading the analog vertical lines of the multiplication and addition matrix, is located on the west and south sides of the matrix, respectively (blocks: preloading and bias injection).

[0044] Neural network layers with more neurons than the matrix row and column number n and m, respectively, can be supported by analog charge addition means using analog charge transfer storage units at the southern output and / or western input edge, as provided by the "transfer gate bank" and "capacitor bank" blocks in the Figure 8 depicted.

[0045] Energy-efficient charge transfer from the neuron layer activation outputs on the south side to the inputs of the next neuron layer on the west side can be achieved by maintaining the analog charge domain using the analog charge transfer circuits described above. Furthermore, power-efficient switched-mode (SC)-based analog-to-digital converters (ADCs) can be connected to the southern activation output edge, and digital-to-analog converters (DACs) to the western activation input edge, enabling hybrid evaluation of the neural network. This allows for parts requiring low precision in the analog path and parts requiring high precision in an additional digital path. This additional digital path can also be used to implement more specialized activation transfer functions.

Claims

1. Method for the analogue multiplication with a circuit assembly, which has a series circuit comprising a first FET and a second FET, or FET array comprising a plurality of parallel-connected second FETs, serving as a current source, a charging device, and at least one capacitance, which can be precharged by way of the charging device, and can be discharged by way of the series circuit comprising the first FET and the second FET, or FET array, in which - the capacitance is precharged for the execution of a multiplication of a first value by a second value, - the first value, encoded as a pulse width of a voltage pulse, is applied to the gate of the first FET, and - the second value, encoded as a voltage amplitude, is applied to the gate of the second FET, or, encoded as binary voltage amplitudes, is applied to the gates of the parallel-connected second FETs, so that the capacitance is discharged for a period of time, which is specified by the pulse width of the voltage pulse applied to the gate of the first FET, with a discharge current, which is specified by the voltage amplitude(s) applied to the gate of the second FET, or to the gates of the parallel-connected second FETs, and a result of the multiplication can be determined from a residual charge or voltage of the capacitance, or from a voltage difference or charge difference between the latter and a further capacitance.

2. Method for the analogue calculation of a scalar product, which is formed by the multiplication of a first value by a second value of a respective value pair, and the summation of results of the multiplications for a plurality of value pairs, with a circuit assembly, which has a plurality of parallel-connected series circuits comprising a first FET and a second FET, or FET array comprising a plurality of parallel-connected second FETs, serving as a current source, a charging device, and at least one capacitance, which can be precharged by way of the charging device, and can be discharged by way of the series circuits comprising the first FET and the second FET, or FET array, wherein - each of the value pairs is associated with one of the series circuits, - the capacitance is precharged for the calculation of the scalar product - for each of the value pairs, the first value, encoded as a pulse width of a voltage pulse, is applied to the gate of the first FET of the associated series circuit, and - the second value, encoded as a voltage amplitude, is applied to the gate of the second FET, or, encoded as binary voltage amplitudes, to the gates of the parallel-connected second FETs of the associated series circuit, such that in each case the capacitance is at least partially discharged for a period of time, which is specified by the pulse width of the voltage pulse applied to the gate of the first FET of the respective series circuit, with a discharge current, which is specified by the voltage amplitude(s) applied to the gate of the second FET, or to the gates of the parallel-connected second FETs of the respective series circuit, and a result of the calculation of the scalar product can be determined from a residual charge or voltage of the capacitance, or from a voltage or charge difference between the latter and a further capacitance.

3. Method according to Claim 2 in an artificial neural network, in which the circuit assembly represents an artificial neuron ,and each value pair is respectively formed by a weight factor and an input value of the artificial neuron.

4. Method according to Claim 3, characterised in that the weight factor is selected as the first value of each value pair, and the input value is selected as the second value.

5. Method according to Claim 3, characterised in that the input value is selected as the first value of each value pair, and the weight factor is selected as the second value.

6. Method according to Claim 4, characterised in that the weight factor is provided as a binary digit sequence, wherein each digit of the digit sequence controls the pulse width at the gate of the first FET by way of a digital-time converter.

7. Method according to Claim 5, characterised in that the weight factor is provided as a binary digit sequence, wherein each digit of the digit sequence, encoded as a voltage amplitude, controls a second FET of the parallel-connected second FETs.

8. Method according to one of the Claims 3 to 7, characterised in that the parallel-connected series circuits, comprising a first FET and a second FET, or an FET array comprising a plurality of parallel-connected second FETs, serving as a current source, are used in a matrix-like manner at crossing points between horizontal connections for an input vector, and vertical connections for an output vector, in a layer of the artificial neural network, so as to execute calculations of a layer of the artificial neural network.

9. Method according to one of the Claims 1 to 8, characterised in that the circuit assembly for processing signed first values in each of the series circuits comprises two parallel circuit branches, which are serially connected to the second FET, or FET array, and in each case comprise a first FET, wherein a first of the two circuit branches is connected to the capacitance, and a second of the two circuit branches is connected to a second capacitance, which can be precharged by way of the charging device, and can be discharged by way of the series circuit comprising the first FET of the second circuit branch and the second FET, or FET array, wherein the respective first value, encoded as the pulse width of a voltage pulse, is applied, depending on its sign, either to the gate of the first FET of the first circuit branch, or to the gate of the first FET of the second circuit branch, and a result of the multiplication or calculation of the scalar product can be determined from a voltage difference or charge difference between the two capacitors.

10. Neural network with one or more layers of artificial neurons, in which the neurons of at least one of the layers in each case comprise a circuit assembly comprising: - a plurality of parallel-connected series circuits comprising a first FET and a second FET, serving as a current source, - a charging device, and - a capacitance, which can be precharged by way of the charging device, and can be discharged by way of the series circuits comprising the first FET and the second FET, wherein components of weight vectors, encoded as pulse widths of a voltage pulse, are applied to gates of the first FETs, and components of input vectors, encoded as voltage amplitudes, are applied to gates of the second FETs.

11. Neural network with one or more layers of artificial neurons, wherein the neurons of at least one of the layers in each case have a circuit array, which comprises: - a plurality of parallel-connected series circuits comprising a first FET, and a second FET, or an FET array comprising a plurality of parallel-connected second FETs, serving as a current source, - a charging device, and - a capacitance, which can be precharged by way of the charging device, and can be discharged by way of the series circuits of the first FET and the second FET, or FET array, wherein components of input vectors, encoded as pulse widths of a voltage pulse, are applied to gates of the first FETs, and components of weight vectors, encoded as voltage amplitudes, are applied to gates of the second FETs, or, encoded as binary voltage amplitudes, are applied to the gates of the parallel-connected second FETs of the series circuits.

12. Neural network according to Claim 10 or 11, characterised in that transfer circuits are designed between the circuit assemblies of successive layers of the neural network, for the transfer of a charge deficit of the capacitance of the respective circuit assembly of the preceding layer to gates of the second FETs of the circuit assemblies of the following layer.

13. Neural network according to one of the Claims 10 to 12, characterised in that a circuit, for the conversion of digital values into pulse widths of a voltage pulse, is arranged upstream of each circuit assembly.

14. Neural network according to one of the Claims 10 to 13, characterised in that the circuit assembly for the processing of signed components of the weight vectors in each of the series circuits has two parallel circuit branches, which are connected to the second FET, or FET array, and in each case have a first FET, wherein a first of the two circuit branches is connected to the capacitance, and a second of the two circuit branches is connected to a second capacitance, which can be precharged by way of the charging means, and can be discharged by way of the series connection of the first FET of the second circuit branch and the second FET, or FET array, wherein the respective component, encoded as the pulse width of a voltage pulse, is applied, depending on its sign, by the control device either to the gate of the first FET of the first circuit branch, or to the gate of the first FET of the second circuit branch.