Neuron circuit

Abstract

A circuit comprising: a plurality of capacitor-based devices, for example, synaptic devices, wherein the plurality of capacitor-based devices are configured to represent and / or store weight information as a plurality of capacitances, wherein each of the plurality of capacitor based devices is configured to be in one of at least a first state and a second state; wherein the plurality of devices are connected to a time-varying, optionally sinusoidal, driving voltage and a reference voltage such that, for each device, said device is connected to the driving voltage in the first state and when in the second state said device of the plurality of devices is connected to the reference voltage, wherein the plurality of devices are grouped into at least a first group and a second group, each group configured to combine the outputs of their respective devices to output a time-varying, optionally sinusoidal, combined output, wherein the output for each group is dependent on the states of the respective plurality of devices of the group and their respective capacitances; a comparison device configured to generate a further output based on at least the output or a further signal derived from the output of the at least two groups.

Description

[0001] Neuron Circuit

[0002] Field

[0003] The invention relates to a circuit for example, a neuron circuit and related method.

[0004] Background

[0005] Artificial Intelligence (Al) and machine learning are becoming increasingly common place in all areas of technology. As the use of Al increases so do the power demands to meet this increasing need. This may lead to creation of vast power hungry data centres.

[0006] Artificial Neural Networks (ANN) are a well-known form of Al model that require large amounts of mathematical operations to be computed. These operations are repetitive and include processing large amounts of data in a short amount of time. The operations occur within each node (neuron) of an ANN.

[0007] Specialized graphical processing units or other hardware may be used for specialist hardware. There is a need for energy-efficient computation for artificial neural networks (ANNs) and other machine learning or artificial intelligence related models.

[0008] S. Maheshwari, A. Serb, C. Papavassiliou, and T. Prodromakis, “An adiabatic capacitive artificial neuron with rram-based threshold detection for energy-efficient neuromorphic computing,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 69, pp. 3512–3525, 9 2022, describes a known adiabatic capacitive artificial neuron circuit. The circuit described in that paper includes a number of limitations.

[0009] Summary

[0010] According to a first aspect, there is provided a circuit comprising: a plurality of capacitor-based devices wherein the plurality of capacitor-based devices are configured to represent and / or store weight information as a plurality of capacitances, wherein each of the plurality of capacitor based devices is configured to be in one of at least a first state and a second state; wherein the plurality of devices are connected to a time-varying, optionally sinusoidal, driving voltage and a reference voltage such that, for each device, said device is connected to the driving voltage in the first state and said device of the plurality of devices is connected to the reference voltage in the

[0011] 55730852-1 second state. The plurality of devices may be grouped into at least a first group and a second group, each group configured to combine the outputs of their respective devices to output a time-varying combined output, wherein the output for each group is dependent on at least the states of the respective plurality of devices of the group and their respective capacitances. The circuit may further comprise a comparison device configured to generate a further output based on at least the output or a further signal derived from the output of the at least two groups.

[0012] The capacitor-based devices may comprise synaptic devices. The driving voltage may comprise a sinusoidal signal. The driving voltage may comprise a power clock signal. The combined output may be sinusoidal.

[0013] The circuit may form part of or may be a module or device.

[0014] The comparison device comprises a comparator. The circuit may comprise a neuron circuit or device. The circuit may be configured to store weights and represent a part, for example, a neuron, of a network, for example, an artificial neural network.

[0015] The capacitor-based device may comprise synaptic circuits or modules.

[0016] The states of the plurality of devices may be controlled by one or more control signals. The control signals may represent at least part of a query input.

[0017] Each device of the plurality of devices may be configured to receive one or more control signals. Each device may be configured to receive more than one control signal. The plurality of devices may be configured to receive a control signal having a bit-size corresponding to or a multiple of the number of devices.

[0018] Each capacitor based device may comprise a capacitor electronically connected to a switching device configured to connect the capacitor to the driving voltage in the first state and to the reference voltage in a second state.

[0019] The switching device may comprise a single pole double throw switch. The switching device may be controlled to be in one of at least a first state and a second state. The switching device may be in the first state one of at least a first state and a second state.

[0020] 55730852-1 The state of the capacitor based device may be controlled by controlling the state of its respective switching device. The switching device may connect the capacitor to the driving voltage when the switching device is in the first state and the switching device may connect the capacitor to the reference voltage when in the second state.

[0021] The switching device may comprise switching logic. The switching logic may represent a logic gate.

[0022] The input signal may comprises a binary signal and wherein each capacitor-based device is controlled by a portion, for example, a bit of the one or more control signals.

[0023] The one or more control signals may comprises a digital signal wherein each bit of the digital signal controls a corresponding switching device. The one or more control signals may comprise one or more query signals.

[0024] Each device of the plurality of devices may comprise at least one capacitor for storing at least part of the weight information as a capacitance.

[0025] The weight information may be stored as one or more capacitance value. The capacitors may be variable may be set and / or adjusted to represent weight information.

[0026] Each device may comprise a ballast capacitor.

[0027] The first group of capacitor based devices may be configured to store positive weight information and the second group of capacitor based devices may be configured to store negative weight information

[0028] The plurality of devices and comparison device may be configured to represent a a partially binarized neural network function

[0029] The comparison device may be configured to perform a comparison between the combined output of the first group and the combined output of the second group and / or signals derived therefrom and output a signal based on the comparison

[0030] 55730852-1 The plurality of devices and / or capacitors may be configured to be charged and / or discharged using a substantially and / or approximately adiabatic process.

[0031] The time-varying signal may be configured to charge and / or discharge one or more of the capacitors in dependence on their state.

[0032] The time varying signal may comprise a rising portion and a falling portion, wherein the time varying signal is configured to charge the plurality of devices and / or their respective capacitors during the upward portion and discharge the plurality of devices and / or their respective capacitors during the downward portion.

[0033] The comparison device may comprise a differential comparator configured to output a signal, optionally a binary signal, based on a difference between the two combined signals and / or signals derived for the combined signals. The comparison device may comprise at least one of: a comparator, a differential amplifier, an operational amplifier, threshold logic circuitry, an operational trans-conductance amplifier.

[0034] The comparison device may comprise a first stage latch based comparator and a second stage comprising threshold logic.

[0035] The comparison device may comprise a first stage comprising a Dynamic Latch Clock Comparator, optionally a zero static power dissipation strong-arm Dynamic Latch Clocked Comparator (DLCC).

[0036] The comparison device may comprise a second stage comprising a clocked set-reset latch, optionally, a NOR-based latch.

[0037] The plurality of capacitor based devices of each group may be provided in a parallel circuit arrangement. The first and second group may form first and a second switchable capacitor network, respectively. The plurality of capacitor based devices may comprise capacitor based synaptic devices.

[0038] The weight information may comprise real-valued positive weight information and real-valued negative weight information. Real-valued positive weight information may be mapped to the capacitance values of the first group of devices and real-valued

[0039] 55730852-1 negative weight information may be mapped to the second group of devices. The weight information may comprise real-valued positive weights and real-valued negative weights. Real-valued positive weights may be mapped to the capacitance values of the first group of devices and real-valued negative weights may be mapped to the second group of devices.

[0040] The weight information may be represented by a weight vector and wherein the direction of the weight vector is preserved by a mapping process.

[0041] Each group of capacitor-based devices may be provided together with a further capacitor or capacitor-based device configured to store a further capacitance, wherein the further capacitance is used to store at least part of the weight information. The further capacitor may comprise a ballast and / or bias capacitor.

[0042] The switching device of each capacitor based device may comprise an inverter device. The switching device may comprise a plurality of transistors arranged to form an inverter device.

[0043] The comparison device may be configured to perform processing on inputs to the comparison device to represent an activation function for the circuit.

[0044] The weight information may comprise a plurality of weights represented by a first vector, and wherein the comparison device is configured to receive an input query represented as a second vector, and configured to process the combined outputs to determine a product, for example, a scalar product, between the first and second vector and further configured to generate an output in response to the product begin above a threshold value.

[0045] The device may comprise at least one further capacitor pair comprising a first further capacitor associated with the first group of devices and a second further capacitor associated with the second group of devices, wherein the capacitance value of each capacitor of the capacitor pair is selected and / or modified to set an operational window for the combined signal and / or to maintain the output voltage within an operational window.

[0046] 55730852-1 The device may comprise at least two further capacitors arranged as a capacitive pillar architecture. Each of the first and / or second group of devices may have an associated ballast and / or bias capacitor and the capacitance of the ballast and / or bias capacitor are controlled to: set an operational window for the combined signal; increase mapped bias and ballast capacitor values

[0047] The circuit may further comprise at least one digital to analogue converter configured to receive the output signal and convert the output signal into a digital signal.

[0048] The circuit may further comprise an analogue to digital converter configured to receive the output signal and convert the output signal into a digital representation. The analogue to digital converter may be a successive approximation register SAR ADC. A flash analogue to digital converter may be used. The SAR ADC may be provided between the first and second groups and the comparison device.

[0049] The plurality of capacitor based devices may be further arranged as a plurality of configurable sub-groups, wherein each sub-group is configurable to select its total capacitance value.

[0050] The sub-groups may comprise a plurality of capacitors, configured to be switched between an active or inactive state, such that, when in the active state, the capacitor contributes to the total capacitance value of the sub-group. The capacitors of each subgroup may have a basis set of capacitance values. The plurality of capacitor-based devices of each sub-group may be arranged in a parallel arrangement. The total capacitance value may be used to represent at least part of the weight information.

[0051] The plurality of capacitor based devices may comprise one or more adjustable or programmable capacitors, optionally memcapacitors. The adjustable or programmable capacitors may be configured to store a capacitance value.

[0052] The circuit may comprise programming circuitry for programing the adjustable or programmable capacitors.

[0053] In accordance with a further aspect there is provided a device or module comprising the circuit of the first aspect.

[0054] 55730852-1 In accordance with a further aspect, there is provided an apparatus configured to represent a network, for example, a neural network, the apparatus comprising a plurality of the circuits according to the first aspect, wherein each circuit is configured to represent a node of the network, for example, a neuron of a neural network.

[0055] In accordance with a further aspect there is provided a method of operating a circuit comprising a plurality of capacitor-based devices grouped into at least a first group and a second group the method comprising: controlling the state of a plurality of capacitorbased devices configured to represent and / or store weight information as capacitances, wherein each of the plurality of capacitor based devices is configured to be in one of at least a first state and a second state; providing a time-varying, optionally sinusoidal, driving voltage to each group of the capacitor based devices, wherein the capacitor-based devices are connected to the driving voltage in the first state and to a reference voltage in the second state; combining the outputs of the capacitor-based devices of each group to output a time-varying first combined signal and second combined signal, wherein the output for each group is dependent on the states of the respective plurality of devices of the group and their respective capacitances; comparing the combined signals or a further signal derived from the combined signals to generate a further output.

[0056] The method may comprise providing a circuit. Providing the circuit may comprise mapping weight information to capacitance values in accordance with a mapping scheme and forming and / or adjusting the circuit such that the capacitor-based devices have said mapped capacitance values. Providing the circuit may comprise forming or otherwise manufacturing the circuit based on the mapping. The method may comprise determining the weight information.

[0057] The scheme may comprise constraining the sum of all capacitance values in both the first and second groups. The scheme may comprise mapping a bias onto at least one of the ballast and / or bias capacitors. The scheme may comprise increasing a value of the ballast and / or bias capacitors by the same fixed amount on each of the first and second groups. The scheme may comprise implementing a capacitive pillar for the ballast and / or bias capacitors. The mapping process may comprise adding a fixed

[0058] 55730852-1 value to the bias and / or ballast capacitors to maintain the voltage of the combined signals within an operation window.

[0059] Features of one aspect may be provided as features of any other aspects. For example, features of the circuit method may be provided as features of the apparatus or method, and vice versa.

[0060] Brief Description

[0061] Various embodiments will now be described by way of example only, and with reference to the accompanying drawings, of which:

[0062] Figure 1 is an illustration of an artificial neural network;

[0063] Figure 2(a) is a circuit diagram of a circuit implementing part of a network, for example, a neuron of an artificial neural network, in accordance with an embodiment and Figure 2(b) illustrates operation of the circuit of Figure 2(a);

[0064] Figure 3 depicts a waveform for driving signal;

[0065] Figure 4 is a circuit diagram of a circuit, in accordance with an embodiment; Figure 5 is a circuit diagram of a synaptic device forming part of a circuit, in accordance with an embodiment;

[0066] Figure 6 is a circuit diagram of a synaptic device, in accordance with a further embodiment;

[0067] Figures 7(a) and Figure 7(b) depict operation of the device of Figure 6;

[0068] Figure 8 is a circuit diagram of a synaptic device, in accordance with a further embodiment;

[0069] Figure 9 is a circuit diagram of a device, in accordance with a further embodiment;

[0070] Figure 10 depicts a comparison device forming part of a neuron circuit, in accordance with an embodiment;

[0071] Figure 11 depicts a comparison device forming part of a neuron circuit, in accordance with a further embodiment;

[0072] Figure 12 is a circuit diagram of a comparison device forming part of a neuron circuit in accordance with an embodiment, in accordance with an embodiment;

[0073] Figure 13 depicts an operational waveform for the circuit of Figure 12;

[0074] Figure 14 illustrates a comparison device, in accordance with an embodiment; Figure 15(a) is a first table and Figure 15(b) is a second table and Figure 15(c) is a plot of energy dissipation;

[0075] 55730852-1 Figure 16 is a table showing a mapping from weights to capacitance values; Figure 17 is a circuit configuration, in accordance with an embodiment;

[0076] Figure 18(a) and Figure 18(b) depict simulation results;

[0077] Figure 19 depicts a table of results from the simulation;

[0078] Figure 20 is a flow-chart of a mapping process for mapping neural network weights to capacitances of the circuit;

[0079] Figure 21 is a circuit diagram of a circuit implementing part of a network, for example, a neuron of an artificial neural network, in accordance with a further embodiment;

[0080] Figure 22(a), 22(b), 22(c) and 22(d) depict four arrangements of circuits;

[0081] Figure 23(a) and 23(b) illustrate configurable synaptic device groups;

[0082] Figure 24 illustrate a programmable neuron circuit, in accordance with an embodiment.

[0083] Detailed Description

[0084] Figure 1 depicts an artificial neural network (ANN). An ANN is a network of connected units or nodes referred to as artificial neurons (for example, node or neuron 12). The neurons are so-called because their operation is modelled on neurons of the brain. The neurons of an artificial neural network are connected by edges (for example, edge 14). The edges are intended to model synapses.

[0085] In use, a neuron receives signals from other connected neurons in the network, processes the signals and sends a further signal to other connected neurons in the network. Each received signal represents a real number, and the output of each neuron is computed by a non-linear function of the sum of its inputs, called the activation function. Each neuron of the artificial neural network therefore receives a plurality of inputs (real numbers) and outputs a non-linear function of the inputs.

[0086] The strength of the signal at each connection is determined by a weight. The weights may be permanently stored or may be adjusted during a training or machine learning process. The neurons are arranged into layers, in this example, an input layer 16, a hidden layer 18 and an output layer 20. Figure 1 depicts a single hidden layer, however, it will be understood that there may be more than one hidden layer 18. Each edge has an associated weight that is a numerical value representing the strength of

[0087] 55730852-1 the connection between neurons in different layers. The weights of a neural network may have positive or negative values. Positive weights can be considered as encouraging a neuron to fire and negative weights can be considered as discouraging a neuron to fire.

[0088] Each neuron of the ANN can be modelled as receiving a plurality of input signals (xi) and is configured to apply a weight to each input signals (each input wi) and then combine the plurality of input signals in accordance with a non-linear activation function. The application of weights to input signals at a neuron can be represented as dot or inner product between two vectors.

[0089] Figure 2(a) depicts a circuit diagram of a circuit, in accordance with an embodiment. The circuit is configured to store or represent weight information. Specifically, the circuit is configured to have capacitive devices that store and / or represent weight information as capacitance values. The circuit is further configured to receive one or more input signals and output a signal based on processing of the weights and the input signals. In embodiments, the circuit can model part of an artificial neural network, specifically a single artificial neuron as described with reference to Figure 1. The circuit can therefore be referred to as a neuron or neural circuit. The neuron circuit can be connected with other circuits to from a device that models an artificial neural network (ANN). The following embodiments relate to a hardware implementation of neural networks, neurons or parts of neurons using electronic means. While referred to a circuit, it will be understood that the circuit may be provided as a module and / or device. As an example, each circuit may be provided as a standalone module or device configured to be assembled together with other modules or devices to represent a neural network.

[0090] As described with reference to Figure 1, in order to model a neuron of an artificial neural network, the neuron circuit is configured to store and / or represent weight information of a neuron. The weight information comprises a plurality of weights (wi) that are mapped to a plurality of capacitance values of the circuit. In the following embodiments, weight information is mapped to the capacitance values of a plurality of capacitors. In general, the neuron circuit is configured to receive a plurality of input signals xiand apply a weight or capacitance value to each of the input signals.

[0091] 55730852-1 As described in the following, the circuit uses charge recovery properties of an adiabatic technique and differential switched capacitor circuits. The circuit may rely on manufacturing technology which is already available on the market, such as CMOS. Without being bound by theory, with reference to a synaptic model, the synapses (i.e. the weighted connection between neurons) are represented by synaptic capacitors, while the comparison device that performs summation and activation functionality represent soma in a biological neuron.

[0092] In the embodiment of Figure 2, a neuron circuit is depicted. In the following embodiments, the artificial neuron is modelled as a single McCulloch-Pitts artificial neuron with a Heaviside activation function. As described above, the neuron has N binary inputs represented mathematically by Xj where i is an element of the indexing set {0... N-1 }. The function has a single binary input y. The output can be expressed by the following relationship:

[0093] 1

[0094]

[0095] 1. if V:,-..- r > T

[0096] , = - ' 0)

[0097] 10, otherwise

[0098] In the equation above, Wj are N real-valued weights from a weight vector w and T is a constant bias value.

[0099] In the following embodiments, the neural network represent by the neuron circuits is a partially binarized neural network. As such, the N weights are divided into two disjoint weights of N+positive valued (excitatory) weights and N' negative valued (inhibitory weights).

[0100] Turning back to Figure 2, the neuron circuit 100 has a number of capacitor-based devices. As set out above, the capacitor-based devices are configured to store or represent weight information as a capacitance. As such, the capacitor-based devices may be referred to as a synaptic devices. As described in the following, in the present embodiment, the capacitor-based devices include a switching device, specifically a single pole, double throw switch and corresponding capacitor.

[0101] 55730852-1 A first synaptic device 102a is labelled in Figure 1 and it will be seen from Figure 1 that a plurality of these devices are provided. The synaptic devices are arranged as a first group 104 and a second group 106. With reference to the weights of the binarized neural network as described above, there are N+synaptic devices in the first group 104 and N' synaptic devices in the second group 106. The number of devices satisfy N = N++ N'. Together the first and second groups comprise N synaptic devices and N capacitors 144.

[0102] In addition to the N capacitors, each group has two associated capacitors: a ballast capacitor and a bias capacitor, described below. Each group has an associated bias capacitor: first bias capacitor 120 Cb+for the first group 104 and second bias capacitor 122 Cb' for the second group. Each group has a respective ballast capacitor: first ballast capacitor 128 Cd+for the first group 104 and second ballast capacitor 130 Cd' for the second group 106. Each group has an associated transmission switch: the first group 104 has a first transmission switch 132 and the second group 106 has a second transmission switch 134. Each group may be referred to as a capacitive tree or capacitive network.

[0103] At the input for the neuron circuit, the neuron circuit also has an input terminal 108 configured to receive an input signal(x). The input terminal is connected to each synaptic device (for example 102a). In the present embodiment, the input signal is an N-bit binary signal such that each bit of the signal is provided to a corresponding synaptic device. In embodiments, the input signal is a time varying digital signal.

[0104] The neuron circuit has a further input terminal 110 configured to receive a time-varying driving voltage signal. In the present embodiment, the signal is a sinusoidal power clock (PC) signal. The further input terminal 110 is connected to the first and second groups via an input node for the first group 111a and an input node for the second group. Regarding the first group, the further input terminal 110 is connected to each synaptic device of the first group via the input node of the first group 111a. Regarding the second group, the further input terminal 110 is connected to each synaptic device of the second group via the input node of the second group 111b.

[0105] At the output of the first and second groups is a comparison device or module 109. In the present embodiment the comparison device comprises a comparator. The

[0106] 55730852-1 comparison device 109 is configured to receive a first combined signal from the first group and the second combined signal from the second group and generate an output based on a comparison of the two combined signals. In the present embodiment, the comparison device outputs a signal based on a comparison of the voltages of the two combined signals.

[0107] Each of the first group and the second group have a combining circuit for generating a combined signal for the comparison device 109. In the present embodiment, the combining circuit for each group includes a first combining node 140 and a second combining node 142. However, it will be understood that in other embodiments, a single combining node may be provided and / or alternative combining devices or circuits may be used to generate a combined signal. The combining node may alternatively be referred to as a charge integration node or charge sharing node.

[0108] In embodiments, the first and / or second combining node may be referred to as a charge integration node. The potentials formed may be referred to as half membrane potentials.

[0109] In the present embodiment, the outputs from the synaptic devices of the first group are combined together with the output from the bias capacitor 120 to form an intermediate combined signal. The intermediate combined signal is then combined with a contribution from the ballast capacitor 128 for the group, to form the first combined signal for the first group. For brevity, the second combined signal for the second group is not described in detail, and will be understood to be formed using a similar combining circuit. The transmission gate 132 is closed to reset the nodes 140 and 142 to known value Vg. The switch is otherwise open at all other times. Similar comments apply to corresponding transmission gate 134. In the present embodiment, the transmission gates (132 and 134) are Single Pole Single Throw (SPST) switches. When closed, the transmission switches connect the output of the plurality of capacitorbased devices of each respective group to an input of the comparator 109.

[0110] The first and second combined signals are time-varying signals and are input to the comparison device 109. The comparison device outputs a binary value (1 or 0) based on the difference between Vm+ and Vm-. The waveform on nodes Vm+ and Vm- may be sampled at the peak of the driving signal (PC peak).

[0111] 55730852-1 In the present embodiment, at a sample time, when the combined signal of the first group (vm+) exceeds the combined signal of the second group (vm-) the comparison device 109 outputs a binary value of 1. Otherwise the comparison device 109 outputs a binary value of 0.

[0112] The output is sampled in accordance with a sampling process. In some embodiments, the comparison samples at the peak of the power clock, Vpc(t) = Vmax, at the end of the evaluation phase (306).

[0113] Sampling is described in further detail with reference to Figure 10 and 11. In some embodiments, the waveform is sampled when the power clock has reached its peak. Depending on that nature of the signal, the sampling may be a single point (i.e. at the peak of sinusoidal wave) or may be over an interval, for example, for a trapezoid waveforms or similar. In this particular case the sampling may be performed using a number of different methods.

[0114] As an example, a delay component and amplifier may be provided. The power clock is sent to the delay component to generate a delayed power clock signal. The power clock signal and delay signal are provided to the amplifier. For delays below a threshold, for example, for a delay of 10ns, both signals are provided to the amplifier. The sign of the difference between the power clock and delayed version changes close to the peaks and the troughs of the sinusoid. As such when the amplifier output changes sign (toggles) a peak is identified.

[0115] First synaptic device 102a is described in further detail as follows and it will be understood that the same comments apply to the other synaptic devices of the device. The device 102a has a switching device 112a and an associated capacitor 114a. The switching device 112a is configured to switch between a first state and a second state in response to receiving a control signal. The first state may be referred to as an active state and the second state may be referred to as an inactive state. In the circuit of Figure 2, in the first state, the switching device connects the capacitor 114a to further input terminal 110 for delivery of the driving signal. In the second state, the switching device 112a connects the capacitor 114a to a reference voltage 116, in this embodiment, a ground voltage. In the present embodiment, the switching device 112a

[0116] 55730852-1 is a single pole double throw SPDT switch. When the switching device is in the first state, the synaptic device 102a is referred to being in an active state. When the switching device is in the second state, the synaptic device 102a is referred to as being in an active state.

[0117] When in the inactive state, the capacitor of the synaptic device is connected to the ground 116 and is therefore in a parallel configuration with the ballast capacitor for that group. For example, for synaptic device 102a, when in the inactive state, a conductive path is created between capacitor 114a and ground terminal 116 and the capacitor 114a is parallel to the ballast capacitor (Cd+) 128. The capacitance values effectively sum together as Ci + Cd+ at the combining node 142. Corresponding comments apply to synaptic devices of the second group that combine to form the second combined signal at the second combining node 143. When in the inactive state, the capacitor of a synaptic device will connect to ground terminal 116 and the capacitor will be parallel to ballast capacitor 130 at the corresponding combining node for the second group.

[0118] When in the active state, the capacitor of the synaptic device is connected to driving signal (power clock). During the evaluation phase, the capacitor charges with the rising power clock signal. The capacitor thus forms part of a capacitive voltage divider. For every “active” synaptic device its corresponding capacitor contributes to a capacitive network. Further description on operation as a voltage divider is provided with reference to Figure 2(b).

[0119] In the circuit of Figure 2(a), the control signals to the plurality of synaptic devices is part of the input binary signal 108. In particular, the control signal for the ithdevice is a corresponding ithcomponent or bit of the input binary signal. The input binary signal may also be referred to as an input query.

[0120] In the above described embodiment, a comparison device 109 is described. The comparator of the comparison device is configured to detect a difference between the two input voltages. Additional power supply voltages are provided for operation of the comparator.

[0121] A weight mapping process as described below maps N weights, for example, of an artificial neuron, to a first set of capacitance values and a second set of capacitance

[0122] 55730852-1 values. In further detail, a set of N+positive valued weights (w*) is mapped to capacitance values (

[0123]

[0124] Q+) of the first group. Likewise, the a set of N' positive valued weights (wf) are mapped to capacitance values of the second group (C ). The mapping algorithm maps ANN weights to circuit capacitance values. The mapping also maps the threshold value, described above, to one of the bias capacitor value (either the bias capacitor of the first group or the bias capacitor of the second group) dependent on whether tau is a positive value or a negative value. The mapping also requires that one of the ballast capacitors is set to a specific value. Once the weights of the ANN are mapped to the circuit, the circuit will give an identical output y to the ANN given the same input. It will be understood that a number of different mapping processes may be implemented.

[0125] In use, each switching device of each synaptic device connects either to the sinusoidal PC supply, or to ground, dependent on the state of each switch. As described above, the state of each switch is controlled by the corresponding bit in the input, Xj where i e I. In use, the binary input is provided the plurality of switches to control their state, as described above. Following the switching process, the driving voltage is provided to the synaptic capacitors of each of the first and second groups. If the switch is controlled to place the synaptic device in the active state then the up-swinging power clock voltage wave is permitted to propagate to the corresponding synaptic capacitor Ci. If the switch is controlled to place the synaptic device in the inactive state then the up-swinging power clock voltage wave is prevented from propagating to the corresponding synaptic capacitor Ci and instead the capacitor is connected to the reference voltage.

[0126] Two combined signals are formed: a first combined signal and a second combined signal. The two combined signals are then provided to the input terminals of the comparison device 109. The comparator of the comparison device is configured to receive the first combined input and the second combined output and generate an output based on the two input voltages. Further description of the operation of the circuit is provided with reference to Figure 2(b).

[0127] The comparison device performs signal processing on the input signals to implement an artificial neural network activation function, in this embodiment, a Heaviside activation function. As described above, the comparison device 109 is configured to receive a first combined signal from the first group and the second combined signal

[0128] 55730852-1 from the second group and generate an output based on the voltage values of the two inputs. As described above, the comparison device 109 outputs a binary value of 1 when the voltage of the first combined signal is greater than the voltage of the second combined signal. Otherwise the comparison device outputs a binary value of 0.

[0129] Figure 2(b) is an illustrative or equivalent circuit diagram corresponding to the circuit of Figure 2(a). Figure 2(b) is provided to illustrate the operation of the circuit of Figure 2. In particular, Figure 2(b) illustrates the formation of one of the two membrane voltages as an input to the comparison device 109 using capacitance voltage division. It will be understood that Figure 2(b) illustrates the first group of synaptic devices (the positive tree) for forming the first combined signal and that the second group is not depicted, for clarity. However, the second group forms the second combined signal in a similar fashion.

[0130] As described with reference to Figure 2(a), Figure 2(b) shows the input for the driving signal 110, combining node 142 and comparison device 109. For the purposes of illustration, Figure 2(b) depicts a first capacitor 152 and a second capacitor 154. The first capacitor may also be referred to as the top capacitor (CTOP) and the second capacitor is also referred to as a bottom capacitor (CBOTTOM).

[0131] These two capacitors do not correspond to two physical capacitor components in the circuit components but rather represent combinations of capacitors of the first group, and the combine capacitance values of those combinations. In further detail, the first capacitor represents the total capacitance of all capacitors of the first group in Figure 2(a) that are active (i.e. have a corresponding switch controlled to be in the active state) together with the capacitance of the bias capacitor Cb. Likewise, the second capacitor represents the total capacitance of all capacitors of Figure 2(b) in the first group that are in the active state (i.e. have a corresponding switch controlled to be inactive state) together with the bias capacitor Cd. As such, the input signal (x) can be considered to control the relative total capacitance values of the first capacitor 152 and second capacitor 154.

[0132] In operation, the driving signal oscillates between a minimum and maximum voltage, in this embodiment between 0 and Vmax. The input signal (x) is provided to the synaptic devices of the first group to control the capacitances of the first and second capacitors.

[0133] 55730852-1 Due to the nature of the voltage divider, in the present embodiment, the combined signal oscillates between a minimum voltage (in this embodiment 0) and a voltage dependent on the combined capacitances. In detail, the upper bound on the voltage oscillation is

[0134] Vmax. CT0PI (CT0P+ C BOTTOM)

[0135] The resulting voltage therefore has the same rate of oscillation and frequency as the incoming driving signal but with an amplitude that is modified dependent on capacitance values due to voltage division. The size of the modulation is dependent on the input signal and the capacitance values of the inactive, active, bias and ballast capacitors of the circuit.

[0136] Figure 2(b) depicts only the first group of synaptic devices. It will be understood that the second group of synaptic devices may be represented by two further top and bottom capacitors.

[0137] In the above described embodiment, a time-varying driving signal was described. Figure 3 depicts such a signal in further detail. The driving signal is an alternating power clock signal aids charge recovery and energy minimization of the system. The use of an alternating power clock, such as a sinusoidal source, allows for adiabatic charge.

[0138] The power clock signal is typically sinusoidal and varies from rail to rail to enable computation and charge recovery. The power clock signal operates in two modes namely: Reset Mode 302 and Operational Mode 304. The operational mode comprises two phases: a first evaluation phase 306 substantially corresponding to a rising portion of the signal and a second recovery phase 308 substantially corresponding to a falling portion of the signal. The reset mode correspond to the power clock signal being substantially at its minima.

[0139] In the reset mode, the circuit is in an idle state. During the reset mode, the transmission gates (first transmission gate 132 and second transmission gates 134) are closed. As

[0140] 55730852-1 such, the first combine signal and second combined signals (also referred to as membrane voltages) have a value equal to the first and second bias voltages (V+B and V'B) respectively during the reset mode. In the Operational Mode, the operation of the circuit is dependent on the respective states of the synaptic devices, as described above.

[0141] Figure 4 depicts a circuit for a synaptic device forming part of a neuron circuit in accordance with an embodiment. It will be understood that for illustrative purposes, Figure 4 depicts a synaptic device in isolation, but as described with reference to Figure 2(a), the neuron circuit includes two groups of such devices. It will be understood that the circuit of Figure 4 has elements corresponding to those of Figure 2(a) including: synaptic device 402, driving signal 410, ground terminal 416, switching device 412, capacitor 414, transmission gate 432, bias capacitor 422, bias voltage 413, ballast capacitor 428, as described with reference to Figure 2(a). The circuit also has a common node 442 at which point a combined signal is formed. The circuit of Figure 4 can be considered to represent a single synapse tree showing synapse, bias and ballast capacitances along with its SPDT and transmission gate (TG) switches.

[0142] In the Reset Mode, the power clock signal is at zero and the transmission gate is closed PC is at zero and the transmission switch is closed setting Vm to the voltage defined by the baseline reference voltage VB. In the Operational Mode, the TG switch is open and the node voltage vm becomes dependent on the state of the SPDT switch and the synapse capacitance. When the input is zero (xi = 0), the SPDT switch is connected to the ground via transistor M3 and when the input is logic T (xi = 1), the SPDT switch is connected to the PC via transistors M1 and M2.

[0143] As described above each group of devices is operable to form a combined voltage (or membrane voltage). The membrane voltages are described in further detail in the following.

[0144] In accordance with an embodiment, the two membrane voltages provided to the comparison device can be expressed as a function of the switch states of the synaptic devices and the capacitor values C* as follows:

[0145] 55730852-1

[0146]

[0147] 1 \ ' Z T ±

[0148] In the above equation,

[0149]

[0150] " T>r and l^,c(t) is the amplitude of the power clock ± = £7^ j,4:;

[0151] at time t. The denominator term, T &!- " includes bias capacitors and ballast capacitors.

[0152] In the specific case where all inputs are zero or are one, the membrane voltages are as follows:

[0153]

[0154] In the above described embodiment, a comparison device in the form of a comparator was described. In further embodiments, the comparator device may include additional circuitry and logic. In accordance with an embodiment, the comparator device further comprises a comparison logic module also referred to as a threshold logic circuitry module. The comparison logic module is configured to output signals based on a comparison between the two signals. In the present embodiment, the comparison logic module is configured to output a 1 under the following condition:

[0155]

[0156] (4)

[0157] The comparison logic module outputs a 0 otherwise.

[0158] The ballast capacitors function to complete the mapping between the determined weights and the capacitance values of the circuit. As described above, weights are mapped to synaptic capacitor values and specific value for the ballast allows for a correct value of y compared to the original ANN. Secondly, the ballast capacitors

[0159] 55730852-1 control of the maximum possible voltage value on the combining (membrane) node 142. This may be referred to as swing control. This may be required in certain circumstances, as the comparator inputs may only work in a lower range of voltages than the power clock.

[0160] In the above-described embodiments, a switching device and associated capacitor is described. Figure 5 depicts a capacitor-based device in accordance with an embodiment and Figure 6 depicts a capacitor-based device in accordance with an alternative embodiment. Figures 7(a) and 7(b) illustrate charging and discharging of the device. Figure 8 depicts an alternative capacitor-based device in which the DC powersupply of the CMOS inverter is changed to an AC power supply (PC). In contrast to Figure 8, Figure 6 has two extra transistor M5 and M6.

[0161] Figure 5 illustrates a capacitor based device with a capacitor 502 and switching device, together with combining node 542 and ballast capacitor 528, as described with reference to Figure 2. The device is configured to receive a control signal 508 and power clock signal 510. The switching device is a transistor based switching device. The device Figure 5 depicts a single capacitive synaptic device with a ballast capacitor 528. As described above, the switching device of Figure 5 is configured to be in a first state in which the capacitor (Ci) is connected to the power clock signal and a second state in which the capacitor (Ci) is connected to a ground terminal. As described above, the switching device of Figure 5, provided as part of a synaptic device and together with a ballast capacitor, allows the formation of the combined signals (Vm) for each group.

[0162] Five transistors are provided (M1, M2, M3, M4, M5). A control signal 508 (Vi) is provided. The control signal corresponds to the control signal described above. For clarity Figure 5 depicts only a single capacitor, however, it will be understood that such a synaptic device can form part of the neuron circuit of Figure 2 and thus form combined signal voltage 530 (via summation node 542).

[0163] In operation, the control signal 508 determines the state of the switching device and therefore whether the power clock signal is provided to the Ci capacitor 502. Vbar 511 is understood as the complement of Vi and is generated locally. It is a derived signal (from Vi). The transistors M4 and M5 forms a CMOS inverter 513.

[0164] 55730852-1 The five transistors combine to form a SPDT switching device that can be in a first state, in which a ground signal is provided to the capacitor and a second state in which the driving signal Vpc is provided to the capacitor.

[0165] The five transistors combine to form a SPDT switching device. When the input signal Vi 508 is logic T, the transistor M1 and M2 switch on. The intermediate node Vs is charged to a time varying signal Vpc510 and while the node Vm542 also charges depending on the voltage division due to the capacitive network. When the input signal is provided with logic ‘0’, the transistor M3 switches on and the node Vs is grounded. The time varying signal cannot pass through and the two capacitors 502 and 528 acts parallel to each other.

[0166] Figure 6 depicts a switching device of a synaptic device in accordance with a further embodiment, in which IN is the control signal 608, PC 610 is the power clock signal and an output signal is (OUT) 611 connect to the associated capacitor of the synaptic device. In this Figure, the capacitor Ci is not shown. Figure 7(a) and 7(b) show operation of the switching device of Figure 6.

[0167] In Figure 7(a), the input (IN) is logic T causing the transistors M2 and M7 to switch on. Intermediate node X then discharges while node OUT charges up via Vpc. The inverter formed by M5 and M6 feeds the inverted output of node ’OUT’ back to node ’X’ such that a correct value is retained. In Figure 7(b), the input (IN) is logic ’0’ causing the transistors M1 and M4 to switch on. Node “OUT” then discharges via M4 while node X charges.

[0168] In the above described embodiment, the comparison device (comprising a comparator) was configured to implement an activation function. In particular, the comparator received first and second combined signals an output a single binary signal (a 1 or a 0). In a further embodiment, as depicted in Figure 9, a further device is provided to receive the combined signals are perform signal processing on the combined signals prior to the signals being provided to the comparator.

[0169] In the embodiment of Figure 9, a SAR circuit 904 is provided between the output terminals of the first and second groups (902a, 902b) and the comparison device 906.

[0170] 55730852-1 The comparison device and the SAR circuit can be considered to form part of the same comparison device or circuit. Such a device is configured to implement multi-bit activation functions, such as a 2-bit activation function. A SAR logic circuit 908 is configured to receive output from the comparison device 906 and is controlled from the SAR ADC device 904. The logic circuit requires SAR sign bit to gate the output so that that y = 0 when the sign bit is T to implement the required non-linearity.

[0171] As described above, a comparison device is connected to the first and second groups. In accordance with embodiment, the comparison device comprises a first comparison module configured to receive two signals and output a signal based on a comparison or difference between the two signals and a second logic module configured to compare the difference between the two signals. In some embodiments, the comparison module comprises a first comparator device and a second latch stage.

[0172] The comparison device may comprise threshold logic circuitry or be referred to as a threshold logic unit. The combined signals, also referred to as membrane voltages, generated by the DTSC synaptic network provide input to the Threshold Logic (TL) circuitry of the comparison device. The threshold logic gate allows implementation of activation functions in the circuit.

[0173] Figure 10 depicts a comparison device 1000 of an embodiment and Figure 11 depicts a comparison device 1100 in accordance with an alternative embodiment. It will be understood that the comparison device 1000 and 1100 may take place of device 109 of Figure 2(a).

[0174] The comparison device of Figure 10 has two stages. At a first stage, the combined signals are provided to a comparator, in this embodiment, a dynamic latch clocked comparator 1002. The comparator may be a latch based comparator. A pMOS variant may offer advantages over nMOS due to the following reasons: a) pMOS consumes less energy due to its lower mobility than nMOS and b) an external bias VB is not required which is used to shift the membrane potential above the subthreshold region which is needed in an nMOS variant and hence consuming more energy.

[0175] At a second stage, a Clocked Set-Reset (CSR) latch 1004 is provided. In alternative embodiments, for an nMOS input comparator a NAND-based SR latch is provided. In

[0176] 55730852-1 Figure 11 a NOR-based SR latch 1104 is provided for a pMOS input comparator to receive output from the DLCC 1102. The second stage may be a threshold logic circuit.

[0177] In use, the DLCC 1002 receives the first and second combined signals (the membrane voltages). The DLCC 1002 outputs two outputs S (set) and R (reset) based on a comparison of the inputs. The CSR latch 1004 receives S and R and outputs two further outputs Q and Qb. To sample Q or Qbar, a selection on which terminal to be sampled is made. Q and Qbar are both valid outputs. It may be useful to use the inverted output or both. Q and Qbar can be considered as two complementary outputs. That is latched on to, for example, a simple D-flipflop (DFF). It is possible to use the signals directly to drive the routing network in the next period of the power clock. In this embodiment, Q is the output such that if Vm+ > Vm- then Q=1 hence y=1. Qb will be =0, so logically y is not equal to Qb. The DFF plays the role of the latch. The "latch" is a general class of components that can "latch onto" inputs and that these have various implementations with different properties. The latch may be implemented as a transparent latch, an SR latch or a DFF latch.

[0178] Figure 10 depicts a comparison device having a second stage as a Clocked Set-Reset (CSR) latch. Figure 11 depicts a comparison device uses a NOR SR latch for a pMOS based DLCC (first stage) and a NAND-based SR latch in the case of an nMOS input DLCC. In general, the comparison device may comprises a first stage having a comparator circuit and a second stage comprising a latch circuit. In some embodiments, the comparison device comprise threshold logic circuitry.

[0179] In accordance with an embodiment, the comparison device of Figure 10 and 11 are configured to perform a comparison and sampling process. Operation of the comparison device of Figure 10 is described as follows. Firstly, the CLK signal in 1002 goes to a low value (in Figure 10) and the DLCC starts comparing the two input voltages Vm+ and Vm-. Based on the strength of the two input signal voltages, the DLCC output swings differentially causing one of the output nodes to take a value of VDD and the other to be set to ground. Those outputs are propagated to the latch stage 1004. Assuming Vm+ is greater than Vm-, the output QS (Figure 10) is pulled to VDD while QR is pulled down to 0V. The first stage output is then provided as an input to the CSR latch 1004. Here, QS is connected to the S (set) terminal and QR to the R

[0180] 55730852-1 (reset) terminal. Since this is a latch stage which latches the output from the first stage, the output node Q charges to VDD while Qb discharges to ground.

[0181] Figure 12 depicts a transistor-level diagram showing the design of a pMOS-based DLCC and the proposed clocked SR (CSR) Latch.

[0182] Operation of the circuit of Figure 12 is described as follows. During the pre-charge phase, the clock (CLK) signal is at VDD. Accordingly, M9 is switched off and M1, M4, M11 and M12 are switched on. As a consequence, the output nodes (QR and QS) of the DLCC are pre-charged to zero and the DC path from the supply to the ground is cut off. As the output from the first stage pre-charges to ground for every clock signal transiting from zero to one, a second stage is necessary to latch the output correctly. During the pre-charge phase, the second stage CLKb signal is zero and the transistors L9 & L10 and L5 & L6 are switched off, thus the CSR latch holds the previous state of Q and Qb giving a stable output for each clock period.

[0183] Next, in the comparison phase, the CLK signal goes low and the transistors M1, M4, M11 and M12 are switched off while switching on the M9 transistor. The DLCC starts comparing the two input voltages: v"mand v+m. Based on the strength of the two input voltages, the DLCC output swings differentially causing one of the output nodes to VDD and the other at ground. Assuming v+m> v−m, the output QS is pulled to VDD and due to the positive feedback transistors, QR is pulled down to 0V. The output from the first stage is fed as input to the CSR latch. Here, QS is connected to the set S terminal and QR to the reset R terminal. As a result, transistor L6 starts conducting whereas L5 is switched off. Since the transistors L9 & L10 are already switched on, the output node Qb discharges to ground and due to the positive feedback, the node Q charges to VDD.

[0184] Figure 13 depicts a waveform comparison between the comparison device described with reference to Figure 10 and Figure 11. Here V+ is applied with a voltage rising from 0V to 500mV for 100μs and then falling to 0V for another 100μS. While V- is set to a threshold voltage of 250mV. Both TL designs shows similar response, whereas, during the falling input, the conventional TL design (Figure 11) shows a large delay and hence offset voltage in comparison to the design of Figure 10.

[0185] 55730852-1 Figure 14 depicts the comparison device implementing threshold logic. On the left hand side of Figure 14 the rising and falling differential inputs of the threshold logic are depicted for Vin changing from -0.1 V to +0.1V and for Vcm = 0.8V. On the right hand side, a test bench for the threshold logic is depicted. Rising and falling offset are calculated and the energy consumption is measured across process corners and temperature ranges from -55°C to 125°C. The setup for measurement is shown in Figure 14 with a CLK frequency of 1MHz and an input voltage Vin range of -0.1V to +0.1 V with a common mode voltage Vcm = 0.8V.

[0186] Figure 15(a) depicts a table of a rising offset voltage (μV) of the conventional (Conv) and the proposed (Prop) TL design at the different process corners, and range of temperature for 1.8V power supply. Figure 15(b) depicts a table of a falling offset voltage of the conventional (Conv) (mV) and the proposed (Prop) (μV) TL design at the different process corners, and range of temperature for 1.8V power supply.

[0187] Based on the Tables of Figure 15(a) and 15(b), the conventional TL shows asymmetry in the falling and rising offset. It has a large falling offset (mV range) in comparison to the rising offset (μV range). On the other hand, the proposed TL has a high degree of offset symmetry having both rising and falling offset in μV range.

[0188] Figure 15(c) is a plot of the energy dissipation across the process corners and temperature sweep was calculated. The proposed threshold latch shows a mean energy of ≈7fJ at −55°C which further increases by 22% at 125°C. This difference in energy is mainly due to the inverter used in the design that generates an inverted clock signal for the clocked SR latch. Finally, the TL is typically sampled at the peak of the PC clock when Vpc(t) = Vmax, at the end of the Evaluation Phase.

[0189] The following description of a non-limiting example hardware implementation of a 12-bit synaptic device, in accordance with an embodiment. In this embodiment the 12 bit device includes a first (positive) group having five capacitor and a second (negative) group having seven capacitors. DTSC ACN is done in the Cadence EDA tool using 0.18μm commercially available CMOS technology at VDD = 1.8 V. The system is set at 1MHz frequency, and the PC is generated using the Power Clock Generator (PCG) circuit described at S. Maheshwari, A. Serb, C. Papavassiliou, and T. Prodromakis, “An adiabatic capacitive artificial neuron with rram-based threshold detection for energy-

[0190] 55730852-1 efficient neuromorphic computing,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 69, pp. 3512–3525, 92022

[0191] The present devices may offer a number of advantages over the neuron device described in that paper. For example, the mapping process is more comprehensive, flexible and efficient. In the original publication SPST switches were used in the synaptic device i.e. when the synaptic device was inactive the synaptic capacitor was floating and not connected to ground, unlike in this new circuit. Mapping this old design may require a large ballast capacitor with a significantly high capacitance value. The SPDT switches described in this circuit make the denominator in Equations 3 and 4 constant regardless of the input, which may allow the mapping to be more tractable and theoretically allowing for an exact and potentially more efficient mapping between original ANN and mapped circuit functionality. In addition, the original publication could only handle positive-valued ANN weights. This may be limiting as ANNs with positive-only weights may be very hard to train. In addition that paper uses a fixed DC threshold voltage on the second comparator input, in contrast to providing a variable membrane voltage, as described above. This means any change in the power clock, such as reduced magnitude or delay, would potentially cause incorrect results unless this DC threshold was dynamically adjusted on the fly. This new circuit automatically compensates for this problem because both trees (groups) are driven by the same single power clock 110. As a consequence if the magnitude of the power clock does change for whatever reason the output y is theoretically completely unaffected.

[0192] All synaptic SPDT switches are kept at a technology minimum. The MIMCAP from the technology has been used for the synaptic capacitances whose values are chosen based on the mapping process described above. The mapping for this specific ACN used the following parameters: Vmax = 1.8V, V ± B = 0V, Cmin = 35.0fF and Vcut = 1.3V. A ballast capacitive pillar Cpd = 542.7fF was also applied to both

[0193]

[0194] so that the swing of both membrane voltages, v±_m(t), remained within a desired range of 0V to Vcut to ensure valid comparator computation.

[0195] A bias capacitive pillar of 35.0fF was used to ensure that C±b≥ Cmin. In the case of this specific ACN note that W+T < W" T with W+T= 3.659 and W−T= 6.529. The total mapped ACN capacitance is 3907.4fF with the total synaptic capacitance CT = 2114.7fF -approximately 54% of the total ACN capacitance.

[0196] 55730852-1 Capacitive pillars can be understood as allowing an ACN designer to raise the value of the ballast capacitors or bias capacitors by a fixed amount without affecting the output. In other words, they are capacitors positioned or arranged in such way as to allow a designer to change the allocations of capacitors, usually in order to achieve certain objectives, such as adjusting the voltage swing of the ACN, without having to worry about the output being corrupted by the change.

[0197] As described above, weights of an artificial neuron of a neural network model are represented by capacitances of the capacitors. The table of Figure 16 shows the mapped configuration for a single DTSC ACN generated from weights extracted from a multi-layer AN network trained in software on a real-world dataset, with a fixed threshold = 0.1 (corresponding to Tau described above)

[0198] Using the parameters defined in the table of Figure 16, the synaptic capacitive trees (first and second groups) are instantiated in the design and integrated with the TL gate in order to complete the computation based on the ACN input signals, xi. Both stages in the TL gate were designed using minimum-size transistors for minimum energy consumption though not optimal. The physical realization of the circuit and RC extraction were done before simulating the circuit for hardware and software comparison. The RC extracted result shows an extra parasitic capacitance of approximately 30fF value on the two vmnodes.

[0199] Figure 17 shows a circuit configuration of a device in accordance with an embodiment. The configuration shows a first group of synaptic devices 1702 and a second group of synaptic devices 1704. The configuration also shows the CSR latch 1706 and DLCC 1708 forming the comparison device.

[0200] Without limitation, a metal insulator metal capacitor (M IM CAP) is designed with the MIMCAP between MIM layers. Each capacitor having equal length and width is oriented in an array format for uniform connectivity and area. The synaptic SPDT switches are on the left side of the layout which will be connected to the 12-bit input signal and the power-clock signal (PC). The TL gate is on the right side of the layout, next to the negative ballast capacitor. Large antenna diodes (reverse diodes) having W / L = 10μm / 10μm were used to reduce antenna effects which helps in dissipating

[0201] 55730852-1 charges to the substrate that will be accumulated during the fabrication process. The non-optimal dimension for 12-bit ACN is 89.05μm × 49.13μm.

[0202] A comparison between software-generated model membrane potential values for the four corner input cases was performed, namely: all zero’s, all one’s, all positive (+ve) synapse tree input one and all +ve tree inputs zero with hardware post-layout simulated values. The table of Figure 18(a) shows results from a post-layout simulation showing a marginal difference in the membrane potentials in comparison to the software values. Figure 18(b) illustrates transient simulation of the post-layout design. The inputs grouped are swept across four binary codes for four corner cases. The hardware-generated membrane potentials are in accord with the software values. The threshold logic gate output response is monitored as y depending on the membrane potentials. Figure 19 depicts a table of results from the simulation.

[0203] Figure 20 depicts an overview of a mapping process to map weights from a pre-trained software neural network model can be mapped onto capacitance values of the device. In the present embodiment, real-value positive and negative weights are mapped to the capacitance values together with a bias.

[0204] In further detail, at step 2002, an artificial neural network is trained to obtain weights for the network. This is based on a training process that uses a training data set 2004. The network is trained using software. The network is a binarized neural network model 2006 that may be implemented in software using a known software package such as tensorflow. In this embodiment, neurons in ANNs that include a binary (Heaviside) activation function can be modelled. However, such networks may have real-value, ternary or binary weights. The training stage may include regularisation, quantisation and, for example, pruning constraints.

[0205] At step 2008, the determined weight of the artificial network are mapped onto capacitance values, in accordance with a mapping process, as described below. A number of input parameters are selected as an input to the mapping process. These input parameters include Vmax, CT, Cmin, Vcut.

[0206] 55730852-1 At step 2010, following the software based mapping process, a hardware configuration process is performed. At this step, the capacitance values of the capacitors of the device are set.

[0207] The following description of a mapping scheme between determined weights of an ANN and capacitance values for the capacitors of the first and second groups is provided.

[0208] The presence of the C± denominator terms means that the mapping is non-trivial. However, it can be shown that the weights wi in (1) can be mapped, under certain conditions, through weight normalization that preserves the direction of the original weight vector. This is possible because C± denominator remains constant regardless of the input Xj due to the use of SPDT (rather than SPST) switches for the synaptic capacitors.

[0209] In the above described embodiment, weight information is stored as capacitance values in a circuit. The weight information may be represented as a weight vector having a direction and a magnitude. In an neuron circuit with a Heaviside (binary) activation function, the output of each neuron may be determined solely by its input (which can be expressed as a vector) and the direction of the weight vector without reference to its magnitude. As such, a mapping process the preserves direction of the original weight vector may be desired.

[0210] In accordance with an embodiment, condition (1) can be split into positive and negative components and rewritten as:

[0211]

[0212] By lettinga'5

[0213]

[0214] 5> wherein?'"T>

[0215]

[0216] :and multiplying through by Vmax the above condition can be rewritten as:

[0217]

[0218] 55730852-1 Further manipulation allows this equation to be expressed as two mapping equations:

[0219]

[0220] These software mapping equations can be compared to the hardware condition of equation (4). There are a number of methods of mapping the above mapping equations to the hardware described above.

[0221] As a first example, VB could be used to map T but this option generates VB values that are functions of the neurons weight vector. As such, this mapping would require different voltages to be supplied to each neuron in an ACN based network. The following example described has a common VB±. With VB+= VB-the DTSC ACN output condition above can itself now be expressed in two similar forms that correspond to the two software mapping equations:

[0222] X • I •• "'7’

[0223]

[0224] A comparison between these two equations and the previous two equations, the mapping can be completed. The number of free variables allows a number of ways to complete the mapping. One constraint on the mapping is that all capacitors must have positive-valued capacitances. For example, assuming T > 0 in this instance, one option is to use Cband C for mapping and to set Cb= 0. Under the condition

[0225] 55730852-1 """T■ «-r*- where **’T~~uand with the total synaptic capacitance give Of, " T as1;the mapping can be completed as:

[0226] When

[0227]

[0228] ... c

[0229] x>

[0230] And when

[0231]

[0232] Cd-= 0

[0233] Further using,

[0234]

[0235] = 0 ensures that the swing of the membrane voltage, v

[0236]

[0237] ^(t) remains within the range 0 to V max.

[0238] The mapping leaves a fixed term CT, the sum of all the synaptic capacitance values in both positive and negative trees, to be selected as an adiabatic design choice. A fixed value of CTto derive capacitance values comes at a disadvantage as small-valued weights in the original weight vector could mean mapped capacitance values below that support by the hardware technology selected. Pruning small weights in the original network, or removing small mapped capacitance values, with some loss of accuracy is a first option. As a further option, to preserve all the weights a value for CTcan be computed for each neuron based on the minimum usable capacitance in the technology Cmin. Using Cmin based mapping, the CTcan be computed for each neuron (neuron circuit) can be computed as CT= CminwT / mini(|wi|). This could led to a potentially larger capacitive costs.

[0239] 55730852-1 As a further option, capacitive pillars can be implemented to increase the value of the ballast or bias capacitors by the same fixed amount on each of the capacitive trees without affecting the circuit output. They can potentially be used to provide further control of the swing

[0240]

[0241] of (t), to raise any small mapped bias and ballast capacitor values above Cmin and for reducing effects of parasitic capacitances on the hardware computation.

[0242] Capacitive pillars are a design tool that allow an hardware circuit designer to control the swing of membrane voltages (combined signals) without theoretically affecting computation (i.e. with the exception of a reduced dynamic range / comparator offset). When applied to the bias capacitors they control the lower limit and when applied to the ballast capacitors they control the upper limit. This can be useful for ensuring the computation occurs in the voltage range best suited to the type of comparator used.

[0243] Capacitive pillars also allow for the situation where the mapped bias and ballast capacitors fall below the minimum allowable capacitance value for the technology in use. For example if the computed mapped bias capacitance values are 30fF and OfF for MIMCAP where Cmin=35fF you can add 35fF capacitive pillar to the bias capacitors i.e. 65fF and 35fF which are now both valid capacitance values.

[0244] Capacitive pillars can be used to ensure that the swing of v±(t) remains within the operation window of the pMOS-based TL comparator of 0V to Vcut where Vcut is the comparators upper threshold voltage for valid computation. Assuming Vcut < Vmax it can be shown that in order for?“KW)

[0245]

[0246] (with v =0) then

[0247]

[0248] A fixed value of CPd can be added to both Cdsuch that the condition

[0249]

[0250] max(vm±(t)) ≤ Vcutholds on both comparator input terminals, or on at least one of the terminals.

[0251] The mapping process described above provides a significant benefit of the DTSC single clock design in that the mapped capacitive values and comparator output are independent of Vpc(t). This is useful, as the magnitude of the Vpc(t) signal can be dynamically scaled post-mapping below Vpc(t) signal can, in theory, be dynamically scaled post-mapping below Vmaxwithout affecting computation. The resulting

[0252] 55730852-1 computations accuracy may depend on resolution and noise effects of the hardware implementation. Furthermore, it may reduce the required accuracy on the comparator sampling time.

[0253] In accordance with an embodiment, a circuit is provided, the circuit including two capacitive trees (positive and negative) and a single sinusoidal Power Clock (PC) also called Double Tree Single Clock (DTSC) are present in the circuit. The DTSC adiabatic capacitive neuron (ACN) includes a set of N synaptic capacitors, each representing a different synaptic connection in the ACN. This includes a subset of N+ synaptic capacitors, Ci+, in the first (positive) capacitive tree, corresponding to positive-valued AN weights. The DTSC ACN also has N- synaptic capacitors, Ci-, in the second (negative) capacitive tree of the ACN with capacitance values that map to the magnitude of each of the negative-valued AN weights.

[0254] A set of N Single Pole Double Throw (SPDT) switches are associated with each of the N synaptic capacitors in the two capacitive trees. The modulation of the ACN inputs with its synaptic capacitance generates two sinusoidal membrane voltages, vm+ and vm- at the input terminals of the comparator. The comparator, which implements the AN activation function, generates the output, y, based on the two input membrane voltage values. In the condition where vm+ exceeds vm-, at the time of sampling, the comparator outputs a binary value of 1, else 0. The DTSC network also includes bias capacitors Cb+ and Cb-, ballast capacitors Cd+ and Cd-, and DC bias voltages VB+ and VB- connected to the positive and negative terminal of the comparator. The Cb+ / -and Cd+ / - capacitors are required to satisfy the weight mapping process but also control the extent of the swing of membrane potentials at the comparator inputs. The maximum number of capacitors in an N-bit ACN is N+4.

[0255] The circuit may achieve remarkable properties thanks to the balance between the first and second trees, such as resilience to power supply variations, and parasitic effects. For reduced design size and improved ANN support the proposed invention may be combined with charge-redistribution ADC designs for multi-bit output. Similarly, the proposed invention may easily be combined with an RRAM-adjusted comparator acting as an offset-compensated neuron.

[0256] 55730852-1 A further description of a method of mapping from determined weights to capacitance values is provided as follows. Assuming that the threshold value is greater or equal to 0, Vg = VBand Cg = 0, then a value of CTis selected. The other parameters may be derived as follows:

[0257]

[0258] The synaptic capacitances and bias capacitor are computed as follows:

[0259]

[0260] When WT+≥ wT-:

[0261]

[0262] When W? < wT:

[0263]

[0264] The electrical circuits described above may be implemented as modules or devices with associated components and circuitry configured to perform a specific application or function. The circuit may be implemented as plug-in modules into application specific hardware of electronic devices. In various other embodiments, the circuits may be implemented as Application Specific Integrated Circuits (ASICs) and other semiconductor chips. The circuit may be extended to a Field Programmable Gate Array module (FPGA) or similar. It will also be understood that all of the specifications, dimensions, and relationships outlined herein (e.g., the number of processors, logic operations, etc.) may be varied without departing from the spirit of the present disclosure, or the scope of the appended claims.

[0265] 55730852-1 In the above-described embodiments an adiabatic process is described. It will be understood that the process is approximately adiabatic because there will be some energy losses. In this context, adiabatic refers to a system notion of a system that change state without energy loss. Charge recovery logic is the technique used to approximate this system. The adiabatic process may be approximately and / or substantially adiabatic. The adiabatic process provides a process of charging capacitors through a resistance without dissipating stored energy on the capacitor. A known neuron circuit design is described in S. Maheshwari, A. Serb, C. Papavassiliou, and T. Prodromakis, “An adiabatic capacitive artificial neuron with rram-based threshold detection for energy-efficient neuromorphic computing,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 69, pp. 3512–3525, 92022.

[0266] Figure 21 depicts an alternative embodiment of a neuron circuit 2100. It will be understood that like numerals of the circuit 2100 of Figure 21 corresponds to like numerals of the circuit 100 Figure 2. For example, comparison device 2109 corresponds to comparison device 109. In the embodiment of Figure 2, the driving power clock signal is only propagated to the capacitor if the input bit received by the synaptic device is a 1. This causes the switch to connect the capacitor of the device to the driving signal. In the further embodiment of Figure 21, each synaptic device of the circuit has two inputs. In this case the switching device of the synaptic device is configured to receive more than one input signal and includes associated switching logic 2148 for switching between the first configuration in which the capacitor is connected to ground and the second configuration in which the capacitor is connected to the power clock.

[0267] In the embodiment of Figure 21, the switching logic 2148 is an exclusive OR gate configured to receive two input signals. The switching device therefore propagates the power clock signal only when the two input control signals are different (i.e. either a 0 and 1 or a 1 and 0). The power clock signal is not propagated if the two inputs are the same. In further embodiments, the switching logic may include alternative logic gates.

[0268] By allowing the SPDT synaptic switching state to be controlled by more than 1 input bit, alternative application may be obtained. The use of the XOR or alternative switching logic may allow for alternative applications of the device. For example, applications

[0269] 55730852-1 beyond artificial neural network. As an example, the embodiment of Figure 21 may be used to compare a first input signal and a reference signal using the top tree and compare a second input signal and reference signal using the second tree. In this application the circuit is configured to fire “neuron” will fire (i.e. the comparison device outputs a y=1) if this second input signal is more similar to the reference signal that the reference signal. Such an application may allow for, for example, block matching for motion estimation in video compression chips.

[0270] In Figure 21, the top positive tree is configured to compute the difference between an input signal and a reference signal in the top positive tree. The XOR SPDT switch allows the PC to propagate only if the two input bits are different. This is completed using an exclusive-XOR logical operation on the two inputs. The bottom negative tree compares another (different) input signal with same reference signal.

[0271] In this embodiment, the circuit will fire (y=1) if this second “test” signal is more similar (more matching bits) to the reference signal compared to the input signal applied to the top best tree. The target application here is block matching for motion estimation in video compression chips. In this specific scenario if the circuit fires (y=1) then the current test input signal will replace the current best tree input signal, ready for the next test signal to be applied. This allows for searching for the closest (minimal bit difference) between a reference signal (e.g. an image) and a multitude of candidate test signals (images).

[0272] In the above described embodiments, it will be understood that the device is designed based on a trained ANN. An ANN is trained (using software) to obtain weights. Weight information mapped onto capacitance values (using one or more different mapping schemes). The capacitance values are then used in a design i.e. the circuit is manufactured using those capacitance values. As such the capacitors store weight information in their capacitance values. In alternative, one or more capacitors of the device may be replaced by memcapacitors or other adjustable capacitive devices configured to retain a capacitance values. Such capacitors may be adjustable programmable to store a desired capacitance value. As such, a configurable device may be provided in which capacitance values can be programmed onto the device after manufacture. An example, embodiment, is provided with reference to Figure 24.

[0273] 55730852-1 Figure 22(a) and 22(b) depict schematically the concept of chaining or otherwise combining more than one neuron circuit, for example, the circuit described with reference to Figure 2. By combining more than one circuit, a circuit that represents an artificial neural network can be constructed.

[0274] In Figure 22(a), a first layer 2202 is connected to a second layer 2204. In this embodiment, the first layer has two 4-bit capacitive circuits (or neuron circuits). Each layer represents a layer of an artificial neural network and each neuron circuit of layer can be considered to represent a node of an artificial neural network. The output of the first layer is input to a second layer having two 2-bit capacitive circuits. Each circuit of each layer may be provided as described above, for example, with reference to Figure 2(a). In this embodiment, layer 1 is fed with the required input and the correct switches are activated and the membrane voltages are provided to a comparison device. Layer 1 outputs Y0L1 and Y1L1.

[0275] The router layer acts to transmit signals from point A to point B. Effectively each circuit corresponds to a neuron circuit. Structurally, the router will have only a positive weight tree (the other side will be a fixed reference) and reconfiguration switches to determine which input should have control of the router. As described above, a neuron circuit can be understood as having at least one capacitive tree and a threshold logic (i.e. a comparator and latch) unit. The purpose of the routing layer is to decouple layer 1 and layer 2. In case of a large network the output from layer 1 will be connected to multiple neurons of the layer 2 which eventually can load layer 1 output causing the system to fail. Secondly, by introducing the routing layer, each neuron layer is operable to work with the same phase of the power-clock (PC1).

[0276] Figure 22(b) illustrates a further embodiment, in which a routing network 2210 is provided. The block diagram for a chaining network with adiabatic routing is shown below. For a two-layer network, the routing network is provided between a first layer ACN 2206 (layerl) and second layer ACN (2208) layer 2. The output from layerl is fed into the routing network, which is then distributed to Iayer2. Here both layers are connected to the same power clock and CLK signals, while the routing layer connects to PC2 and CLKb signal which is 180° phase difference compared to the power-clock used in layer 1 or layer 2.

[0277] 55730852-1 The routing network allows the ACN layers to work using the same power clock signals. That may make the network synchronous and robust against delay variations that can cause functionality failure.

[0278] As depicted in Figure 22(a) and 22(b), each first layer has two 4-bit capacitive circuits and each second layer 2 has 2x 2-bit capacitive circuits. The overall configuration can be considered as a 4x2x2 adiabatic capacitive neural network (where 4 is the number of inputs in layer one 2 is the output from layer 1 and finally 2 is the output from layer 2).

[0279] With reference to Figure 22(c), the router blocks of the router layer may also be implemented with a slightly modified circuit with 2x control switches connected in series. In embodiments, these may be implemented using double-gate transistors. Figure 22(c) shows an example configurable switching device, in accordance with an embodiment. As shown in Figure 22(c) two switching devices are connected to receive a configuration signal 2212 in addition to the input signal (X) described above. The configuration signal 2212 can be understood as a static signal that does not in general change during operation.

[0280] In accordance with embodiment, the router blocks may be implemented as 1:1, in which case the circuit will have a single capacitive synapse, or N:1 in which case there will be N synapses. In the simplest case a neuron of a router may be configured such that if any of the inputs is a 1, then it will "fire" an output. Control of the configuration switches allows for control over which input neuron should control the router. In some embodiments, multiple neurons may take control. The threshold value may also be adjusted. As such, a configurable ACN can be obtained.

[0281] Figure 22(d) shows an implementation of a configurable switching device, having a dual gate, in accordance with an embodiment. The SPDT switch in Figure 22(d) defaults to the GND connection if either the input X of the configuration input config is at 0. If config is 1, then the switch works as normal under the control of the input. This is a standard logical AND operation. At the transistor level it generally means that any transistor receiving input from the input (Vi in figure 8), will now have 2x gates. So M2, M4 and M5 will now become double-gate transistors with 1x gate connected to the input Vi and the other to config.

[0282] 55730852-1 The neuron circuit of Figure 23(a) is an illustrative example of a circuit with two synaptic devices. These two synaptic device have hardcoded values of capacitance. The synaptic devices are manufactured as descried above, including selection of values of capacitance prior to manufacture. As described above, the neuron circuit has a control signal input 2308a and a driving signal input 2310a. In this example, the neuron circuit has two synaptic devices, each having a switch and respective capacitor: first capacitor 2114a and second capacitor 2114b. In this example, the first capacitor has a capacitance 14fF and the second capacitor has a capacitance 10fF. Each configurable group can be controlled to represent a synapse.

[0283] Figure 23(b) depicts a configurable neuron circuit in which the synaptic devices are controlled to form configurable synaptic devices groups. The configurable synaptic device groups are controllable to change a total capacitance value of the group. The group can then be activated and / or deactivated and act as a single synaptic device having the total capacitance value.

[0284] In this embodiment, the circuit has two configurable synaptic device groups. In particular, each configurable synaptic device group includes a basis set of four capacitors allowing a desired capacitance value to be set after manufacture. Each synaptic device has a capacitor and respective switch.

[0285] The synaptic devices may be grouped together, to form configurable synaptic device groups. The configurable synaptic device has a total capacitance value resulting from a sum of active capacitors in the combined synaptic device. The control signal can then be delivered to the combined synaptic device as described above. In the context of Figure 2(a), for example, in which the synaptic devices are arranged in two groups, the configurable synaptic device groups may be referred to as sub-groups of synaptic devices.

[0286] In this embodiment, the neuron circuit has eight synaptic devices, each being controllable by a respective switch. The synaptic devices are arranged into two configurable synaptic groups 2160 and 2161. The first synaptic device group 2160 has a first capacitor 2162a having a capacitance value of 1fF, a second capacitor 2162b

[0287] 55730852-1 having a capacitance value of 2fF, a third capacitor 2162c having a capacitance value of 4fF and a fourth capacitor 2162d having a capacitance value of 8fF. Likewise, the second synaptic device group 2161 has a first capacitor 2164a having a capacitance value of 1fF, a second capacitor 2164b having a capacitance value of 2fF, a third capacitor 2164c having a capacitance value of 4fF and a fourth capacitor 2164d having a capacitance value of 8fF It will be understood that the capacitance values may be different in embodiments. The neuron circuit is configured to receive a control signal input 2308b and a driving signal input 2310b.

[0288] Each capacitor is switchable by a respective switch. As such, a desired capacitance value for each synaptic device can be set through appropriate control of the control signal. In this embodiment, each group of synaptic devices can be configured to form a combined synaptic device with 16 possible values.

[0289] In embodiments, the original N-bit control input signal is combined with a k-bit configuration signal. This can be implemented using a logical AND circuit or other logic. As such there are now k times N inputs to the circuit.

[0290] This allows for each synaptic device group to be configured into 2 to the power k different values. In the embodiment of Figure 23(a) there are 4 x 2 inputs to the circuit. In Figure 23(b) with k=4 the 14fF synapse capacitance in the hardcoded version (top circuit) can be generated in the configurable version (bottom circuit) through enabling the 8fF, 4fF and 2fF capacitors via the external input signal. Changing the external configuration signal allows for different effective synaptic capacitance values to be realized, in each configurable group.

[0291] In this embodiment, the first synaptic device group can be set to have a total capacitance value 14 fF by setting the input signal to “1110” thereby to activate the 8fF capacitor, the 4fF capacitor and the 2fF capacitor and to deactivate the 1fF capacitor. The second synaptic device group can be set to have a total capacitance set to 10fF by setting the input signal to “1010” to activate the 8fF capacitor and the 2fF capacitor. The 4fF capacitor and 1fF are deactivated.

[0292] 55730852-1 As such a control signal “10” for the neuron circuit of Figure 23(a) (i.e. the “1” activating the 14fF capacitor and “0” deactivating the 10fF capacitor) has the equivalent control signal “1110” for the first synaptic device and “0000” for the second synaptic device.

[0293] The control signal “11” for the neuron circuit of Figure 23(a) (i.e. the “1” activating the 14fF capacitor and “1” activating the 10fF capacitor) has the equivalent control signal “1110” for the first synaptic device and “1010” for the second synaptic device.

[0294] The control signal “01” for the neuron circuit of Figure 23(a) (i.e. the “0” deactivating the 14fF capacitor and “1” activating the 10fF capacitor) has the equivalent control signal “0000” for the first synaptic device and “1010” for the second synaptic device.

[0295] Figure 24 depicts a N-bit neuron circuit using memcapacitors. In place of N synaptic devices each having a capacitor as described above, the N synaptic devices each have a memcapacitor. As such the neuron circuit has N memcapacitors 2400, each with a dedicated switching device. In Figure 24, an N-bit control signal 2408 is provided together with a driving signal 2410. Each memcapacitor is provided together with a corresponding switching device. As described above, in operation, a combined voltage 2412 is formed at an output of the group of synaptic devices. It will be understood that only one group of synaptic devices is depicted in Figure 24.

[0296] Further programming of the memcapacitors is possible using dedicated programming circuitry. As an example, control of the PC and Vm node voltages and control signals can allow for isolation and programming of individual synaptic devices (i.e. individual memcapacitors). As an example, in a first phase 1, the PC node is set to 0, the correct switches are activated, and Vm is forced to the memcap write voltage. In the second phase, the PC is set to VDD, the correct switches are activated, and Vm is forced to such a voltage as it writes onto the memcapacitors.

[0297] In further examples, additional programming phases can be provided to allow multiple synapses in the same group to be written to different values. It may be that a maximum of as many phases as the maximum number of synapses sharing a common Vm. In this example, it is assumed that the memcapacitors are bipolar however other schemes can be used for unipolar devices. If it is desired to write to multiple memcaps with the same write voltage and polarity that may be performed in parallel. Other similar

[0298] 55730852-1 programming methods can be used, for example, setting PC to a write voltage and Vm to 0 (in the case of the 2nd phase described above).

[0299] The above embodiments incorporate bias and ballast capacitors. Those capacitors may offer balancing and swing adjustment on the combined membrane voltages. In accordance with embodiments, as described above, the ballast capacitors is used to moderate a swing of the membrane potential during each cycle.

[0300] In accordance with embodiments, the transmission gate switches offer membrane voltage reset mechanism and may counteract leakages and may restore the system to a known initial state.

[0301] By providing circuit in accordance with embodiments, an exact and robust mapping of software compatible trained artificial neural networks may be achieved. The bias capacitors may support software biasing. Specific values of the ballast capacitance Cd may also facilitate a mapping. This may include asymmetric Cd values matching asymmetric values in real networks, as well as asymmetric numbers of positive and negative weights. In addition, by using SPDT switches, a linearity between input and output may be achieved, which may also have a significant impact on the size of Cd.

[0302] In accordance with embodiments, the design may allow for control of the swing (the minimum and maximum values) of alternating membrane voltages with bias and ballast capacitors controlling the extent of the swing. As described above, capacitive pillars that do not affect computation may also be incorporated. These allow for scaling control of the differential input to match an operational voltage window of the threshold logic. In addition, this may offer a minimum offset voltage when all inputs are zero, or sparse, for increased noise floor rejection, and realizable capacitance sizes in the technology used for the mapped (computed) bias and ballast sizes.

[0303] As above described, each capacitor based device is associated with a single tree (or group). In embodiments, variable capacitors may be attached to both groups and have their capacitance values adjusted. In embodiments, variable capacitors may be attached to both groups with one side enforced to 0.

[0304] 55730852-1 In the above described embodiments, certain activation functions may be implemented. In further embodiments, alternative activation functions, such as Rectified Linear Unit function (ReLU) may be implemented. In such embodiments, the comparison device may be modified. In a non-limiting example, the comparison device is formed by multiple comparators, each configured to trigger at a different threshold to represent a quantised ReLU function. In a further non-limiting example, a SAR ADC may be used to produce a digital value based on analogue differential voltage at terminal of the current comparator. In a further non-limiting example, a power-gated analogue amplifier may be used to take a small differential voltage and produce an amplifier voltage that is then converted.

[0305] In the above-described embodiments, bias and ballast capacitors are described. In accordance with embodiments, capacitors, represent neutral bias terms that are used to balance the trees, and therefore may be referred to balance capacitors. In accordance with embodiment, the ballast capacitors may adjust swings and may be referred to as swing capacitors.

[0306] In accordance with embodiments, variable capacitors may be incorporated to allow modifiable synaptic devices. These may take the form of memcapacitors, capacitive banks or MOSCAPs or any other adjustable or programmable capacitor device, le group may be constrained to take a zero value, in the case where additional capacitance on either of the groups may be needed. In such embodiments, the capacitance of the variable capacitors may be set to as close to 0 as possible, or have a main, disconnecting switch that severs them individually or, alternatively, even as a group.

[0307] Although description of particular embodiments has been provided above, it should be understood that these embodiments are illustrative only and that the claims are not limited to those embodiments. Those skilled in the art will be able to make modifications and alternatives to the described embodiments which are contemplated as falling within the scope of the appended claims. Each feature disclosed or illustrated in the present specification may be incorporated in any embodiment, whether alone or in any appropriate combination with any other feature disclosed or illustrated herein. In particular, one of ordinary skill in the art will understand that one or more of the features of the embodiments of the present disclosure described above with

[0308] 55730852-1 reference to the drawings may produce effects or provide advantages when used in isolation from one or more of the other features of the embodiments of the present disclosure and that different combinations of the features are possible other than the specific combinations of the features of the embodiments of the present disclosure described above.

[0309] 55730852-1

Claims

CLAIMS:

1. A circuit comprising:a plurality of capacitor-based devices, for example, synaptic devices, wherein the plurality of capacitor-based devices are configured to represent and / or store weight information as a plurality of capacitances,wherein each of the plurality of capacitor based devices is configured to be in one of at least a first state and a second state;wherein the plurality of devices are connected to a time-varying, optionally sinusoidal, driving voltage and a reference voltage such that, for each device, said device is connected to the driving voltage in the first state and when in the second state said device of the plurality of devices is connected to the reference voltage, wherein the plurality of devices are grouped into at least a first group and a second group, each group configured to combine the outputs of their respective devices to output a time-varying, optionally sinusoidal, combined output, wherein the output for each group is dependent on the states of the respective plurality of devices of the group and their respective capacitances;a comparison device configured to generate a further output based on at least the output or a further signal derived from the output of the at least two groups.

2. The circuit as claimed in claim 1, wherein the states of the plurality of devices are controlled by one or more control signals, wherein the control signals represent at least part of a query input.

3. The circuit as claimed in any preceding claim, wherein each capacitor based device comprises a capacitor electronically connected to a switching device configured to connect the capacitor to the driving voltage in the first state and to the reference voltage in a second state.

4. The circuit of claim 3, wherein the switching device comprises a single pole double throw switch.

5. The circuit as claimed in any preceding claim, wherein the one or more control signals comprises a binary signal and wherein each capacitor-based device is controlled by a portion, for example, a bit of the one or more control signal.55730852-16. The circuit as claimed in any preceding claim, wherein each device of the plurality of devices comprises at least one capacitor for storing weight information as a capacitance.

7. The circuit as claimed in any preceding claim, wherein each device comprises a ballast capacitor.

8. The circuit as claimed in any preceding claim wherein the first group of capacitor based devices are configured to store positive weight information and wherein the second group of capacitor based devices are configured to store negative weight information9. The circuit as claimed in any preceding claim wherein the plurality of devices and comparison device are configured to represent a partially binarized neural network function10. The circuit as claimed in any preceding claim, wherein the comparison device is configured to perform a comparison between the combined output of the first group and the combined output of the second group and / or signals derived therefrom and output a signal based on the comparison11. The circuit as claimed in any preceding claim, wherein the plurality of devices and / or capacitors are configured to be charged and / or discharged using a substantially and / or approximately adiabatic process.

12. The circuit as claimed in any preceding claim, wherein the comparison device comprises a differential comparator configured to output a signal, optionally a binary signal, based on a difference between the two combined signals and / or signals derived for the combined signals.

13. The circuit as claimed in any previous claim, wherein the comparison device comprises a first stage comprising a comparator circuit and a second stage comprising a latch circuit and / or wherein the comparison device comprises threshold logic circuitry.55730852-114. The circuit as claimed in claim 13, wherein the first stage comprises a Dynamic Latch Clock Comparator, optionally a zero static power dissipation strong-arm Dynamic Latch Clocked Comparator (DLCC) and / or wherein the second stage comprises a clocked set-reset latch, optionally, a NOR-based latch.

15. The circuit as claimed in any preceding claim, wherein the plurality of capacitor based devices of each group are provided in a parallel circuit arrangement.

16. The circuit as claimed in any preceding claim, wherein real-valued positive weights are mapped to the capacitance values of the first group of devices and real-valued negative weights are mapped to the second group of devices.

17. The circuit as claimed in any preceding system wherein the weight information is represented by a weight vector and wherein the direction of the weight vector is preserved by a mapping process.

18. The circuit as claimed in any preceding claim wherein each group of capacitorbased devices is provided together with a further capacitor or capacitor-based device configured to store a further capacitance, wherein the further capacitance is used to store at least part of the weight information.

19. The circuit as claimed in any preceding claim, wherein the comparison device is configured to perform processing on inputs to the comparison device to represent an activation function for the circuit.

20. The circuit as claimed in any preceding claim, wherein the device comprises at least one further capacitor pair comprising a first further capacitor associated with the first group of devices and a second further capacitor associated with the second group of devices, wherein the capacitance value of each capacitor of the capacitor pair is selected and / or modified to set an operational window for the combined signal and / or to maintain the output voltage within an operational window.

21. The circuit as claimed in any preceding claim, further comprising at least one analogue to digital converter configured to receive the output signal and convert the55730852-1output signal into a digital signal, wherein the analogue to digital converter comprises a successive approximation register ADC or flash analogue to digital convertor.

22. The circuit as claimed in any preceding claim, wherein the plurality of capacitor based devices are further arranged as a plurality of configurable sub-groups, wherein each sub-group is configurable to select its total capacitance value.

23. The circuit as claimed in any preceding claim, wherein the plurality of capacitor based devices comprise one or more adjustable or programmable capacitors, optionally memcapacitors.

24. An apparatus configured to represent a network, for example, a neural network, the apparatus comprising a plurality of the circuits as claimed in any of the preceding claims, wherein each circuit is configured to represent a node of the network, for example, a neuron of a neural network.

25. A method of operating a circuit comprising a plurality of capacitor-based devices grouped into at least a first group and a second group the method comprising:controlling the state of a plurality of capacitor-based devices configured to represent and / or store weight information as capacitances, wherein each of the plurality of capacitor based devices is configured to be in one of at least a first state and a second state;providing a time-varying, optionally sinusoidal, driving voltage to each group of the capacitor based devices, wherein the capacitor-based devices are connected to the driving voltage in the first state and to a reference voltage in the second state;combine the outputs of the capacitor-based devices of each group to output a time-varying first combined signal and second combined signal, wherein the output for each group is dependent on the states of the respective plurality of devices of the group and their respective capacitances;comparing the combined signals or a further signal derived from the combined signals to generate a further output.55730852-1

Images