Calibration of spiking neural networks

Abstract

A spiking neural network includes: a plurality of input processing circuits, each input processing circuit having an input for receiving a spiking neural network input signal and configured to: apply a transfer function to the input signal to generate a processed input signal; a plurality of offset current generators, each offset current generator configured to: generate an offset current signal at a predetermined level; a plurality of synapses, each synapse connected to receive the processed input signal from one of the plurality of input processing circuits and configured to: apply a predetermined weight to the processed input signal to generate a synapse output signal; a plurality of neurons, each neuron connected to receive a synaptic output signal from the synaptic subset and an offset current signal from one of the plurality of offset current generators, and each neuron configured to: generate a neuron output signal in response to the received synaptic output signal and the offset current signal; and an analog-to-digital converter having an input connectable to receive an offset current signal from one of the plurality of offset current generators and configured to convert the received offset current signal to a corresponding digital output signal.

Description

Technical Field

[0001] The present disclosure relates generally to automatic signal recognition techniques and spiking neural networks, and more particularly to systems and methods for calibrating spiking neural networks. Background Art

[0002] Spiking neural networks (SNNs) are signal processing systems whose design is inspired by biological neural networks. Information is encoded in patterns of spike signals distributed across a network of neurons and synapses, as described in WO2022 / 090542A1 filed by the present applicant. By mimicking the processes that occur in biological brains, spiking neural networks can perform signal processing tasks typically performed by the human brain. Examples include image recognition, sound recognition, and event detection based on input from multiple sensors.

[0003] The functionality of an SNN is determined by its network configuration and weight matrix. The network configuration specifies how many neurons and synapses are used and how they are connected to each other and to the input and output signals. It also defines what kind of pre-processing and post-processing circuits are needed and the complete connections between them. The weight matrix is ​​a set of parameters that determine the parameters of each element of the network. Typically, this includes the gain factor for each synapse, but can also include other parameters such as neuron thresholds.

[0004] The network configuration is determined by the user, who chooses the network topology based on the intended application. The weight matrix, on the other hand, is generated from a training step, in which the SNN is exposed to a large amount of training data and its weights are automatically updated so that the output of the network is close to the desired value.

[0005] Since the training step usually takes a long time to complete, it is preferably performed only once, even if multiple copies of the SNN are made. The weight matrices resulting from the training step are then copied to each individual piece of hardware. However, if the hardware on which the SNN is run is different from the hardware on which the network was trained, the resulting network will behave differently and may not perform its intended function.

[0006] Analog and mixed-signal circuits (usually fabricated as integrated circuits) are a power-efficient way to implement SNNs. One drawback of analog signal processing circuits is that their performance varies with manufacturing tolerances and environmental factors like temperature and supply voltage. Digitally assisted analog mixed-signal circuits can be used as an alternative to provide reliable performance. The amount of assistance needs to be limited and optimized.

[0007] In order to provide reliable computational performance, these implementations of SNNs therefore need to be calibrated to ensure that their performance is determined by their design, independent of manufacturing variations or changes in operating conditions (e.g., voltage and temperature). The calibration process aims to adjust the behavior of the analog circuits in the SNN so that its performance meets predefined specifications.

[0008] One way to ensure that different copies of an analog or mixed-signal SNN perform equally well within acceptable limits is to train each individual copy on the same set of training data. Each copy will then have its own weight matrix optimized for its unique hardware characteristics. While this approach can compensate for manufacturing errors, it does not reduce the variability of the circuit due to temperature or voltage changes. Another disadvantage is that performing the training step for each individual copy can be time-consuming and expensive.

[0009] Another approach described in WO2020 / 260067A1 performs reduced training steps for each individual copy of the SNN, with the goal of adjusting the weights to the specific characteristics of the SNN. However, this still takes a long time and does not guarantee convergence to a viable solution. In addition, it does not give any insight into the type and amount of changes to the internal components, making it difficult to improve the hardware design.

[0010] There is therefore a need for a faster and more powerful calibration process that provides the possibility to tune individual elements of the SNN rather than the entire SNN as a unit. Summary of the invention

[0011] To perform this improved calibration process, circuit parameters can be measured and subsequently adjusted to meet predefined specifications. In complex analog and mixed-signal networks, it may not be practical to directly measure all relevant parameters with sufficient accuracy, so a calibration strategy is needed that optimally utilizes observable parameters to derive values ​​for unobservable parameters. The relevant circuit parameters should be adjustable over a sufficient range to overcome expected variability and adjustable with sufficient resolution to achieve predefined specifications.

[0012] Due to the complexity of SNNs and the various components used in the design of analog or mixed-signal SNN system-on-chip (SoC), the optimal trade-off between performance and amount of calibration under the constraints of the SoC is a non-trivial problem. The present invention provides a system and method for performing such calibration in SNNs with various components, connectivity, and limited observability and measurability.

[0013] In a first aspect, the invention comprises a spiking neural network (SNN). The SNN comprises: a plurality of input processing circuits, each having an input for receiving a spiking neural network input signal and configured to apply a transfer function to the input signal to generate a processed input signal; a plurality of offset current generators, each configured to generate an offset current signal of a predetermined level; a plurality of synapses, each synapse connected to receive a processed input signal from one of the input processing circuits and configured to apply a predetermined weight to the processed input signal to generate a synaptic output signal; a plurality of neurons, each neuron connected to receive a synaptic output signal from a subset of synapses and an offset current signal from one of the offset current generators, and each neuron configured to generate a neuron output signal in response to the received synaptic output signal and the offset current signal; and an analog-to-digital converter having an input connectable to receive an offset current signal from one of the offset current generators and configured to convert the received offset current signal into a corresponding digital output signal.

[0014] The input processing circuit may include input shaping circuits, encoder circuits, signal conditioning circuits, filter circuits, and other types of circuits for processing the spiking neural network input signal. The input signal may include a raw signal or a pre-processed signal from a sensor or other input device, or may include an output from a previous spiking neural network, signal processor, or other type of circuit.

[0015] The bias current generators preferably each comprise a direct current (DC) current source for providing a steady current at a predetermined current level that can be adjusted.The bias current generators may be used to provide a bias current to the neurons during operation of the spiking neural network.

[0016] The synapses are preferably arranged in an array, for example, a first subset of synapses (e.g., a row of synapses) are connected to receive processed input signals from one of the input processing circuits, and a second subset of synapses (e.g., a column of synapses) are connected to provide their synaptic output signals to one of the neurons. The synapse array can be implemented as a crossbar array.

[0017] The spiking neural network can be implemented as an analog signal processing circuit or a mixed signal circuit combining analog and digital signal processing. The offset current generator, synapses, and neurons are implemented as hardware elements (e.g., hardware or hardware with software or firmware), preferably in a single integrated circuit (IC), and can be implemented as a SoC. The input processing circuit can also be implemented in the same IC or SoC as the offset current generator, synapses, and neurons, or can be implemented as one or more external components.

[0018] The analog-to-digital converter may also be implemented in the same IC or SoC as the other components, or may be implemented as one or more external components. A single analog-to-digital converter may be included (reducing chip size and cost), or multiple analog-to-digital converters may be used (to provide a faster calibration process).

[0019] The offset current generators are preferably each individually adjustable to adjust the offset current signal. The spiking neural network may also include a first processing circuit connected or connectable to receive the digital output signal generated by the analog-to-digital converter and configured to: generate an adjustment signal for adjusting a corresponding one of the offset current generators to produce an adjusted offset current signal.

[0020] This adjustment preferably corrects for variations in each offset current generator, and may also correct for leakage current through synapses connected to receive offset current signals from the associated offset current generators. Furthermore, this calibration of the offset current generators eliminates or reduces any variability introduced by the offset current generators, and enables more accurate calibration of other components of the spiking neural network. The first processing circuit may be implemented as a hardware circuit or a processor with software or firmware, and may be implemented in the same IC or SoC as other components of the SNN, or as a separate component that may be permanently connected or connected during calibration.

[0021] The spiking neural network may also include a frequency measurement circuit connected to receive a neuron output signal from one of the neurons and to generate an output indicative of the frequency of the received neuron output signal. The frequency measurement circuit may be implemented using a reference clock signal and components included on the same IC or SoC that includes the synapse and the neuron.

[0022] The neurons of the spiking neural network can be adjusted individually to adjust the input gain, pulse threshold and / or reference potential offset value of the neuron. The spiking neural network can also include a second processing circuit, which is connected or connectable to receive the output from the frequency measurement circuit and is configured to: generate an adjustment signal for adjusting a corresponding neuron.

[0023] The neurons are connected to receive an offset current from an offset current generator, and preferably the offset current generators have been pre-calibrated so that they generate an adjusted offset current having a known predetermined value. Since the input to each neuron is known and the frequency of the pulses generated by the neuron is measured, the transfer function of the neuron can be calculated to generate an adjustment signal to correct for the variation of the neuron. The second processing circuit can be implemented using the same processor as the first processing circuit.

[0024] Each synapse of the SNN can be adjusted to adjust the input gain and / or offset value of the synapse. Each input processing circuit can be adjusted to adjust the input gain, pulse threshold and / or timing value of the input processing circuit.

[0025] The analog-to-digital converter is preferably connectable to receive a synaptic output signal from one of the synapses and is configured to convert the received synaptic output signal into a corresponding digital synaptic output signal. For example, the analog-to-digital converter may have a hardwired connection or a switched connection for receiving the synaptic output signal.

[0026] The SNN may also be configured to provide a calibration mode in which one or more spiking neural network input signals are set to predetermined values. The input signals may include a pulse train having a predetermined frequency and / or amplitude, or a constant input current or voltage having a predetermined value, or some other signal. Applying known input signals enables the transfer functions of various components of the spiking neural network to be determined to assist in calibration.

[0027] The SNN may also include a third processing circuit that is connected or connectable to receive a signal from the frequency measurement circuit during a calibration mode, the signal indicating the frequency of a neuron output signal of one of the neurons, the neuron being connected to receive a synaptic output signal from one of the synapses, the synapse being connected to receive a processed input signal from one of the input processing circuits, wherein the third processing circuit is configured to: generate an adjustment signal for adjusting the input processing circuit and / or the synapse.

[0028] Since the input to the input processing circuit is known and the frequency of the pulses generated by the neuron is measured, the combined transfer function of the input processing circuit and the synapse (which sends the synaptic output signal to the neuron) can be calculated to generate an adjustment signal. The adjustment signal can be used to correct for changes in the input processing circuit and / or the synapse. The third processing circuit can be implemented using the same processor as the first and / or second processing circuit.

[0029] The analog-to-digital converter of the SNN may be connected to receive a synaptic output signal from one of the synapses and may be configured to convert the received synaptic output signal into a corresponding digital synaptic output signal. The SNN may also include a fourth processing circuit connected or connectable to receive the digital synaptic output signal during the calibration mode and configured to generate an adjustment signal for adjusting the input processing circuit and / or the synapse.

[0030] Since the input of the input processing circuit is known and the output of the synapse is measured, the combined transfer function of the input processing circuit and the synapse (which sends the synaptic output signal to the neuron) can be calculated to generate an adjustment signal. The adjustment signal can be used to correct for changes in the input processing circuit and / or the synapse. The fourth processing circuit can be implemented using the same processor as the first, second and / or third processing circuits.

[0031] In a second aspect, the present invention provides a method for calibrating a spiking neural network, the spiking neural network comprising: a plurality of offset current generators configured to generate offset current signals; a plurality of input processing circuits configured to generate processed input signals; a plurality of synapses configured to apply weights to the processed input signals to generate synaptic output signals; and a plurality of neurons connected to receive one of the offset current signals and synaptic output signals from the plurality of synapses, and configured to generate neuron output signals. The method comprises: generating a plurality of offset current signals by the plurality of offset current generators, each offset current signal being at a predetermined level; converting the offset current signals into corresponding digital output signals; and adjusting the offset current generators based on the digital output signals to generate adjusted offset current signals from each offset current generator.

[0032] The calibration method can be used for the SNN of the first aspect of the invention described herein, and the details of the SNN described herein can also be applied to the calibration method. The calibration method includes: measuring the output of each offset current generator, converting it into a digital signal, and adjusting each offset current generator using the resulting digital signal. This process can be used to correct for variations in each offset current generator, and can also correct for leakage current through synapses connected to receive offset current signals from the associated offset current generator. This calibration of the offset current generator eliminates or reduces any variations introduced by the offset current generator, and enables more accurate calibration of other components of the spiking neural network.

[0033] The calibration method may also include: receiving a plurality of adjusted offset current signals from a plurality of neurons; measuring the frequency of a neuron output signal; and adjusting the input gain, pulse threshold, and / or reference potential offset value of each neuron based on the adjusted offset current signal received by the corresponding neuron and the measured frequency of the neuron output signal generated by the corresponding neuron. The method may also include: calculating a transfer function of the neuron, wherein the transfer function of each neuron is based on the adjusted offset current signal received by the neuron and the measured frequency of the neuron output signal of the neuron.

[0034] Calibration of the neurons utilizes an adjusted offset current from an offset current generator, preferably previously calibrated to provide an offset current having a known predetermined value. Since the input to each neuron is known and the frequency of the pulses generated by the neuron is measured, the transfer function of the neuron can be calculated and an adjustment signal can be generated to correct for the variation of each neuron.

[0035] The calibration method may also include: providing a pulse neural network input signal set to a predetermined value to a first input processing circuit in the input processing circuit; setting a weight of a first synapse in the synapses to a predetermined value; receiving the processed input signal from the first input processing circuit by the first synapse, and generating a corresponding synaptic output signal in response; receiving the synaptic output signal from the first synapse and the adjusted offset current signal from the first offset current generator in the offset current generator by a first neuron in the neurons; measuring the frequency of the neuron output signal generated by the first neuron; and adjusting the first input processing circuit and / or adjusting the first synapse based on the pulse neural network input signal and the measured frequency of the neuron output signal to adjust the transfer function of the first input processing circuit and the first synapse.

[0036] The input signal may include a pulse train having a predetermined frequency and / or amplitude, or a constant input current or voltage having a predetermined value, or some other predetermined signal. Since the input to the input processing circuit is known and the frequency of the pulses generated by the neuron is measured, the combined transfer function of the input processing circuit and the synapse can be calculated to generate an adjustment signal. The adjustment signal can be used to correct for changes in the input processing circuit and / or the synapse.

[0037] By performing a calibration method on a subset (e.g., a row) of synapses, the effects of changes in the input processing circuit and the synapses can be separated to a certain extent. The calibration method includes: providing a spiking neural network input signal set to a predetermined value to a first input processing circuit in the input processing circuit; setting the weight of the first synapse subset to a predetermined value; receiving the processed input signal from the first input processing circuit by the first synapse subset and generating a corresponding synaptic output signal from each synapse of the first synapse subset in response; receiving the synaptic output signal from the first synapse subset by a neuron; measuring the frequency of each neuron output signal generated by the neuron; and adjusting the first input processing circuit based on the measured frequency of the neuron output signal to adjust the transfer function of the first input processing circuit.

[0038] This embodiment of the calibration method enables a certain degree of separation of the effects of variations in the input processing circuitry and the synapses. The synaptic output signals from a first subset of synapses (e.g., from a row of synapses in a crossbar array that all receive the same processed input signal) can be averaged to average out variations in the synapses, leaving essentially only variations in the input processing circuitry. An adjustment signal can then be calculated to adjust the input processing circuitry to account for the variation.

[0039] Once the transfer function of the input processing circuit is determined and the input processing circuit is adjusted if possible to account for the variation in the circuit, a further calibration process may be performed to calibrate the individual synapses. The calibration includes: providing a spiking neural network input signal set to a predetermined value to a first input processing circuit in the input processing circuits; receiving the processed input signal from the first input processing circuit by a first synapse in a first subset of synapses and generating a corresponding synaptic output signal in response; receiving the synaptic output signal from the first synapse and an adjusted offset current signal from a first offset current generator in the offset current generators by a first neuron in the neurons; measuring the frequency of the neuron output signal generated by the first neuron; and adjusting the first synapse to adjust the transfer function of the first synapse based on the spiking neural network input signal and the measured frequency of the neuron output signal.

[0040] The calibration method may use a measurement of the neuron output frequency, which may provide an indication of the synaptic output current received by the neuron when the transfer function of the neuron is known. An alternative approach is to measure the synaptic output signal directly. Thus, the calibration method may include: providing a spiking neural network input signal set to a predetermined value to a first input processing circuit in the input processing circuit; setting the weight of a first synapse in the synapse to a predetermined value; receiving the processed input signal from the first input processing circuit by the first synapse and generating a corresponding synaptic output signal in response; converting the synaptic output signal to a corresponding digital output signal; and adjusting the first input processing circuit and / or adjusting the first synapse based on the spiking neural network input signal and the digital output signal to adjust the transfer function of the first input processing circuit and the first synapse. This variation in the calibration method uses an analog-to-digital converter to measure the output signal of the synapse, rather than sending the synaptic output signal to the neuron and measuring the output of the neuron.

[0041] Similarly, in a calibration method, the effects of changes in input processing circuits and synapses are separated, including: providing a spiking neural network input signal set to a predetermined value to a first input processing circuit in the input processing circuits; setting the weights of a first synapse subset to a predetermined value; receiving the processed input signal from the first input processing circuit by the first synapse subset, and generating a corresponding synaptic output signal from each synapse of the first synapse subset in response; converting the synaptic output signal from the first synapse subset into a corresponding digital output signal; and adjusting the first input processing circuit based on the spiking neural network input signal and the digital output signal to adjust the transfer function of the first input processing circuit.

[0042] Similarly, the synapses may be calibrated in a calibration method, the calibration method further comprising: providing a pulse neural network input signal set to a predetermined value to a first input processing circuit in the input processing circuits; receiving, by a first synapse in the first subset of synapses, the processed input signal from the first input processing circuit and generating a corresponding synaptic output signal in response; converting the synaptic output signal from the first synapse into a corresponding digital output signal; and adjusting the first synapse based on the pulse neural network input signal and the digital output signal to adjust a transfer function of the first synapse.

[0043] A single calibration run can be performed for each individual SNN (e.g., each IC or SoC) to compensate for manufacturing tolerances. Further calibration runs can also be performed during operation of the SNN to offset variations in temperature and supply voltage. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Embodiments will now be described, by way of example only, with reference to the accompanying drawings, in which corresponding reference numerals indicate corresponding parts, and in which:

[0045] Figure 1 This is a conceptual diagram of the Minimum Spiking Neural Network (SNN);

[0046] Figure 2 is a simplified schematic diagram of an SNN implemented as a crossbar array;

[0047] Figure 3A Can be used as Figure 2 and Figure 6 A simplified schematic diagram of an input shaping circuit of an input processing circuit in an SNN;

[0048] Figure 3B It is shown Figure 3A A diagram of the input signal of the input shaping circuit;

[0049] Figure 3C It is shown by Figure 3A A diagram of a pulse output signal generated by an input shaping circuit;

[0050] Figure 4 It can be used for Figure 2 and Figure 6 A simplified schematic diagram of synapses in an SNN;

[0051] Figure 5 It can be used for Figure 2 and Figure 6 A simplified schematic diagram of neurons in an SNN;

[0052] Figure 6 yes Figure 2 A simplified schematic diagram of the SNN, including an offset current generator and an analog-to-digital converter;

[0053] Figure 7 It can be used for Figure 6 A simplified schematic diagram of the offset current generator in the SNN;

[0054] Figure 8 is a simplified schematic diagram of the overall structure of an SNN with associated analog-to-digital converters, processors, and memory;

[0055] Fig. 9 is a flow chart showing the steps of a calibration process for an analog-to-digital converter;

[0056] Fig.10 is a flow chart showing various stages of the calibration process;

[0057] Fig.11 is used for Figure 6 or Figure 8 Flow chart of the calibration process of the offset current generator of the SNN;

[0058] Fig.12 yes Figure 6 or Figure 8 A flowchart of the calibration process of the neurons of the SNN; and

[0059] Fig.13 yes Figure 6 or Figure 8 Flowchart of the SNN input processing circuit and synaptic calibration procedure. DETAILED DESCRIPTION

[0060] Some embodiments will be described in more detail below. However, it should be understood that these embodiments should not be interpreted as limiting the scope of protection of the present disclosure.

[0061] Artificial SNN is an electronic system that mimics the signal processing functions of the brain. It is built from components that implement the same network structure as in the biological brain and processes information in a similar way. SNN is particularly suitable for tasks such as audio or video recognition, pattern recognition in large data sets, or anomaly monitoring of sensor data.

[0062] SNNs can perform signal processing tasks with significantly lower power and lower latency than traditional digital electronic systems. Low power consumption is achieved because only those components in the system that actually process the data are active at any one time. The inherent parallel architecture of SNNs ensures low latency from input to output. These two characteristics are important for always-on signal processing applications in power-constrained environments, such as keyword detection in battery-powered devices (e.g., smart watches).

[0063] Like their biological counterparts, SNNs use spike trains as their internal signal representation. Information is encoded in one or more features of these spike trains, such as frequency, phase, pulse width, pulse density, and amplitude. As these spike trains travel along the network, they undergo basic mathematical operations such as addition, multiplication, and integration. These operations, together with the topology of the network, implement specific signal processing functions.

[0064] The exact parameters of all internal components are not set directly by the user, but are derived through a learning process. During this process, the network is exposed to a range of reasonable input signals and the output is observed while the internal parameters are tuned until the output becomes close to the expected value. In this way, the internal parameters will converge to a set of values ​​that enable the network to perform the desired signal processing function.

[0065] Figure 1 is a conceptual diagram of a simple spiking neural network (SNN). The SNN includes an input layer 2 of input processing circuits 3, which are connected to multiple layers of neurons and synapses in neurosynaptic layers 6, 7, 8. One such neurosynaptic layer consists of a set of synapses 4, whose input ports are connected to the previous layer, and whose outputs are connected to a set of neurons 5. The output 9 of the entire system is generated by the last layer of neurons in the network. The output 9 of the last layer is then passed to a decoding layer, which can forward the information to the end user in a meaningful format.

[0066] The input layer 2 receives SNN inputs 1A, 1B, 1C, which are processed by input processing circuits 3A, 3B, 3C. The processed input signals are then provided to neurosynaptic layers 6, 7, 8, each of which includes neurons 5 connected via synaptic connections 4, resulting in output signals 9 generated by the last neurosynaptic layer. Each of these layers can have a digital, analog or mixed signal implementation, as further described below.

[0067] Figure 2is a simplified schematic diagram of an SNN 20 implemented as a crossbar array. The crossbar design is an efficient way to implement a reconfigurable SNN system, especially when fabricated on an integrated circuit. This consists of a rectangular array of synapses with input processing circuitry on one side of the array and neurons on the other side. Figure 2 In the embodiment of FIG. 1 , input processing circuits 24 are arranged in a column, and each circuit drives a row of synapses 25. The synapses 25 are connected in columns, and the outputs of all synapses 25 in a column are added together and used as inputs to neurons 26. By programming appropriate weights in the synaptic array and correctly configuring the interconnection system, a wide variety of network topologies can be realized.

[0068] The signal to be processed (e.g., from a sensor) is typically first preprocessed by an encoder to convert the sensor output signal into a suitable digital or analog signal, and the input signal processing may also include filters in both the time and amplitude domains and different types of input shaping circuits to generate a signal suitable for input to the computational layer of the network. All or part of this input processing may be performed in the input processing circuit 24, or may be performed in whole or in part by a processor associated with the input processing circuit 24. Figure 2 The SNN shown in is implemented with a separate input layer.

[0069] exist Figure 2 In the example, input processing circuit 24 receives input 21 of the SNN via interconnect 23, which can be used to interconnect several SNN arrays, such as Figure 2 Each input processing circuit 24 generates a processed input signal 27, which is received by a row of synapses 25 in the neural synapse array.

[0070] The function of the input processing circuit 24 (e.g., including input shaping circuits and filtering circuits) is to modify the incoming signal 21 so that it can be processed by the neural network. The incoming signal 21 may have information encoded in amplitude, frequency, phase, or any other analog or digital modulation scheme, and needs to be processed to extract useful information for further processing by the neural network. The input processing circuit 24 will typically contain filters, comparators, demodulators and other signal processing circuits at the input end, and pulse generation circuits at the output end. The frequency, amplitude, shape and other parameters of the output pulses are modified to encode the information present in the input signal 21.

[0071] The synapse 25 amplifies or attenuates the incoming signal 27 by a constant factor determined by its weight setting. The weight of each synapse 25 is stored in a memory cell associated with the synapse. The value of all the weights in the network is called the weight matrix and contains the information extracted from the input data during learning.

[0072] exist Figure 2In the embodiment of FIG. 2 , a row of synapses 25 receives processed input signals 27 from an input processing circuit 24 . Each synapse 25 applies a weight to the received processed input signal 27 and generates a synaptic output signal 28 .

[0073] The synaptic output signals 28 from a column of synapses 25 are received by neurons 26. Each neuron 26 integrates the received synaptic output signals 28 and generates a neuron output signal 29 when the integrated value reaches a predetermined threshold, so that the neuron output signal 29 is in the form of a pulse train. The neuron output signal 29 depends on several parameters of the neuron, such as input gain, integration constant and threshold.

[0074] The neuron output signals 29 are provided to the interconnect 23 and can be provided to a decoder. The decoder interprets the calculations done by multiple neurons, as the final answer may be encoded in time or phase, requiring decoding. These can be time measurement circuits or phase detection circuits.

[0075] It will be appreciated that actual implementations of SNNs typically include a very large number of neurons (e.g., hundreds of thousands or millions of neurons) and a correspondingly large number of synapses and input processing circuits. A large SNN can be divided into multiple smaller SNNs, with signals being passed between the smaller SNNs, for example using interconnects 23, such as Figure 2 The interconnect 23 can be configured via the configuration input 22 .

[0076] Depending on the use case that the SNN addresses, there are different possible connections for the design. For example, a high-level connection could be: input, encoder, input processing circuit 24, synapse 25, neuron 26, decoder, output. Within the high-level connections, additional connections are possible, such as feedback connections from each level to another, feed-forward connections from each level to another, different connection structures (e.g., crossbars), and different input and output reference points for different connections from each level. The complex connections at the SoC level of the SNN are use case driven. These complex connections create a scenario where it is a challenge to maintain consistent and predictable operation in the face of device variations.

[0077] SNN Implementation

[0078] SNNs can be implemented as software models running on a processor, or the elements of the SNN (e.g., synapses and neurons) can be manufactured as hardware elements (although SNNs can also utilize some software to implement certain functions). Hardware implementations can be implemented using digital circuits or analog circuits and built from discrete components or manufactured as integrated circuits. Fully digital implementations of SNNs use synthesized digital functions to implement the functions of the neural network. Designs can become more complex, but the trade-off is increased chip area and higher operating power, with increases in chip area and power generally proportional to the complexity of the core logic.

[0079] On the other hand, analog and mixed-signal implementations can be very efficient in terms of power and chip area because the two mathematical operations performed by synapses and neurons, multiplication and integration, can be efficiently implemented using analog circuits. Increasing circuit complexity does not require a large trade-off in chip area and operating power. These analog implementations have a larger design space than digital because, in addition to the signal timing that digital implementations can use, analog circuits can also exploit information encoded in both the phase and amplitude of the input signal. The increase in design space allows chip area and operating power to not scale in proportion to the complexity of the analog design. However, the increase in design space also leads to an increase in problems: limiting the design space for large-scale manufacturing because static timing analysis (STA) for digital implementations is no longer sufficient.

[0080] In addition, digitally enhanced analog mixed-signal designs can be used for SNNs. By enhancing mixed-signal designs with digital logic, an optimal bifurcation can be achieved in the performance, energy usage, and chip area trade-off.

[0081] Variability in SNNs

[0082] All circuit implementations (e.g., complementary metal oxide semiconductor (CMOS) circuits) are subject to variations in their physical structure and changes over time. Based on the design techniques used, there are different suggestions and solutions to make the design work reliably. In the context of this disclosure, these techniques are combined under the framework of calibration.

[0083] SNN IC designs are subject to unexpected variations from the full IC level to the individual transistor level due to technology, lithography, electrical, thermal, and physical effects. Technology effects are related to process corners at the IC level, density variations at the SNN array level, and well proximity and associated stresses at the transistor level. Similarly, lithography affects ICs due to line width variations and transistors due to proximity effects. Circuit defects can cause leakage currents and device mismatches. Electrical effects such as supply voltage variations at the IC level, substrate noise effects on the array, and jitter / crosstalk effects at the transistor level. Thermal effects can cause local and global gradients that lead to dimensional variations. At the transistor level, thermal effects also increase thermal noise. Physical effects also cause gradients, which can have different global and local variations.

[0084] Local and global dimensional variations can in turn lead to temporal variations. These effects can vary from picoseconds to long time scales (e.g., days and years). Smaller time scale effects are due to effects such as power supply noise, substrate noise, and thermal kT noise. Longer time scale effects in the microsecond to millisecond range are due to temperature gradients and flicker noise. Further long term effects in the seconds to minutes range are due to power supply noise, drift, and flicker noise. Effects due to process, lithography, and manufacturing issues are typically static over time.

[0085] A disadvantage of using analog mixed-signal circuits to implement SNN components is that the transfer function of analog circuits may vary due to the process, voltage, and temperature (PVT) variations mentioned above. This means that a single SNN design may not be replicable because subsequent copies of the same circuit will have different performance values, such as gain, bandwidth, offset, and amplitude. In addition, even a single copy may have unstable performance when the supply voltage is unstable or the ambient temperature varies. Therefore, a calibration method is needed to ensure that each copy of a particular analog SNN implementation behaves in the same way, regardless of any tolerances in the manufacturing process or changes in the supply voltage or temperature.

[0086] Circuit parameters that are often sensitive to PVT variations include gain, bandwidth, capacitance, and resistance. Leakage current in a device that is not considered conduction current and mismatches between devices that should be equal are further examples of the effects of PVT on analog circuits.

[0087] In this disclosure, these unintended changes, as well as other unintended changes in the operation of the SNN due to the factors discussed above, are collectively referred to simply as "variations" or "PVT variations," unless otherwise noted.

[0088] Calibration Method

[0089] Calibration of digital SNNs can be performed using static timing analysis (STA). This takes into account spatial and temporal issues by creating hierarchical models that are used together with timing models to constrain errors in different paths of the circuit. Errors constrained within both spatial and temporal constraints ensure that performance metrics (e.g., operating frequency) are met. Further analysis can be performed on digital and mixed-signal designs, such as IR drop analysis, post-layout simulation, and parasitic extraction. These ensure that changes in supply voltage are within the design metrics. Mixed-signal and analog redesigns, such as ADCs, DACs, PLLs, etc., typically calibrate each component using known calibration methods.

[0090] However, for SNN designs containing multiple analog, mixed-signal, and digital blocks that are interconnected in complex connection patterns that depend on the use case, the calibration problem is more difficult. Although there are known methods for calibrating individual blocks used in such SNN designs, the present disclosure provides a means for calibrating a complete SNN SoC with different building blocks, different connectivity, limited observability of internal nodes, and / or limited constrained measurement capabilities of observable values.

[0091] Observability of internal nodes comes at the expense of increased IC area, which increases cost. Additional observability can also limit practical functionality. The present disclosure seeks to provide a method for calibrating an entire SNN SoC with minimal observability and measurements, but still reliably calibrating the entire system.

[0092] Defining the boundaries of variability

[0093] A bottom-up approach can be used to constrain process variability at the level of individual components. How much variation is expected can be determined based on information provided by the device manufacturer. Semiconductor device models typically include information indicating how much variation is expected between manufacturing runs, how much variation is expected between devices manufactured in the same batch, and how each parameter varies with temperature. Circuit-level simulations can be performed to determine the impact of device parameters on network parameters. The result of such an analysis will be a table describing the expected variation of each network parameter. As an example, it can be specified that: synaptic weights (where variability is primarily determined by transistor mismatch) can vary + / -15% for each synapse; neuron thresholds can vary + / -10mV between manufacturing batches due to resistor inaccuracies; or the amplitude of pulses generated by input processing circuits varies + / -30% over a temperature range of -50°C to 150°C due to transistor transconductance variations.

[0094] A top-down approach can be used to constrain errors due to variability from SNN use cases. How much network parameters are allowed to vary is different for each application. Some network architectures are highly sensitive to changes in synaptic weights, while others are more sensitive to changes in neuron thresholds. Statistical simulation can be used to determine the level of sensitivity of a particular SNN implementation to each network parameter. The result of such an analysis will be a table describing the allowed variation for each network parameter. As an example, it may specify: the accuracy of synaptic weights should be within 2%; neuron thresholds should match each other within 5%; and the amplitude of pulses generated by input processing circuits should always be 100nA + / - 10nA. These results determine the resolution required for the adjustment circuits implemented inside the network components: there should be enough resolution to adjust each circuit parameter to within the allowed tolerance range. Similarly, the range of the adjustment circuit should be large enough to offset the amount of variation expected from the circuit-level simulation.

[0095] Observable Points in SNN

[0096] Define observable points within the SNN, i.e. points within the circuit where measurements can be made. Ideally, as few observable points as possible should be used. They increase IC area, make the design more functionally challenging, and also increase power consumption. In principle, observable points can be inputs, internal nodes, and outputs at the SoC level for each building block of the SNN design. Some examples of observable points are as follows:

[0097] A. The output of the neuron. The frequency of the spike train generated by each neuron can be measured. Since the neuron output is digital in nature, this observation point is relatively easy to observe.

[0098] B. Neuronal membrane potential. Since neuronal membrane potential is related to the integration time constant in the neuron, it is difficult to observe the potential without causing functional changes.

[0099] C. Input signal (voltage / current) to the neuron. Neuron input signals can be observed based on the connectivity of the synaptic computation array. Since computation arrays typically have complex connectivity, this creates a practical upper limit on the number of observable points that can be generated in SNN design.

[0100] D. Individual outputs of each synapse. The outputs of synapses form complex connection array structures, such as crossovers and trees. In theory, each individual synapse can be observed, but since there are usually a large number of synapses in a complex routing network, it is difficult to achieve observable points for each synapse.

[0101] E. Internal nodes of synapses. It is difficult to observe the internal nodes of synapses because they are also part of the complex connectivity.

[0102] F. Outputs of input processing circuits. It is possible to observe them for networks where they do not form part of the complex connectivity of a computational array. In many use cases, they are distributed within a synaptic array in a cross or tree-like connectivity, which limits their observation from a routing perspective.

[0103] G. Inputs to the input processing circuits. This can be observed for networks where the input does not form part of the complex connectivity.

[0104] H. Encoder Internal Nodes. Depending on the complexity of the encoder, the internal nodes of the encoder may require calibration.

[0105] Ideally, as few measurement circuits as possible should be used, as they come at the expense of increased IC area and power consumption of the SNN SoC. Frequency measurements can be made using known oscillators, counter circuits, etc. Voltage and current measurements can use analog-to-digital converters (ADCs) to convert analog values ​​to digital values ​​for simpler measurements. The more frequently these measurements are made (either at the frequency of a digital signal or at real-time voltage or current measurements), the greater the IC area and power hit of the system.

[0106] A trade-off can be made between the number of observable points and measurement circuits and the accuracy of calibration and the resulting SNN performance. This trade-off can be evaluated by defining a cost function for the calibration optimization problem that can take into account the chip area used for calibration and measurement circuits, the performance loss from additional observable points, the power consumed by the calibration observable circuits during non-calibration time, the power consumed by the calibration and measurement circuits, and routing complexity. The relative importance of different cost functions should also be considered.

[0107] The design of each subcomponent has performance vs. area and performance vs. power tradeoffs, and SNN use cases should be considered in the tradeoffs of network performance vs. subcomponent performance.

[0108] It is also necessary to consider when to perform calibration. If any calibration steps need to be performed while the SNN is running, they should be performed in a way that does not interfere with the operation of the SNN. One way to do this is to temporarily buffer any incoming signals to ensure that the SNN is not processing any data for a while, and then use this time window to perform one or more calibration steps.

[0109] Another way to calibrate an SNN without interfering with its normal operation is to exploit redundancy in the crossbar architecture. To do this, the SNN synaptic array can be designed to be slightly larger than the largest SNN design expected to be programmed internally, so that there is always at least one unused synaptic column. This spare column can then be calibrated independently of the rest of the chip. Once this is done, the configurable interconnect is modified so that the spare column is placed within the active network, while the column that was once in that position is disconnected and calibrated. A convenient moment to perform this action is the moment when the neurons of that column generate an output signal, because the state of the neurons is reset at this time, and if a different column is inserted in the network in place of the active column, the state of the network will not change.

[0110] Online calibration can be performed periodically, or it can be performed only when environmental conditions change. The supply voltage in a typical battery-powered system changes slowly as the battery discharges or charges, on a time scale of minutes or hours. Passive components (such as capacitors and inductors) and the self-regulating action of the voltage regulator circuit can be used to filter out fast glitches on the supply line. The temperature of most systems is also fairly stable, with time scales typically on the order of tens of seconds or minutes. Monitoring circuits attached to the SNN can be used to measure the supply voltage and temperature while the SNN is active, and trigger a calibration round only when environmental conditions change by a predetermined amount.

[0111] Calibration is required at different timelines for batch-manufactured dies. For example, calibration may be performed during startup, periodically to remove long-term drift, based on temperature measurements to remove temperature effects, based on supply voltage measurements to compensate for supply voltage variation effects, etc. When designing the calibration process, the time spent on calibration and the amount of calibration data that needs to be stored versus the performance of the use case should be considered. A cost function can be used to evaluate the area overhead for redundancy in the process, as well as the power and time overhead for calibration during startup, calibration to remove long-term drift, and calibration to remove temperature effects.

[0112] Figure 3A is a simplified schematic diagram of an input shaping circuit 30, which can be used as Figure 2 SNN 20 and Figure 6 The input processing circuit 23 in the SNN 60. The input shaping circuit 30 extracts useful information from the SNN input signal 33 and generates a processed input signal 39, i.e., a pulse train modulated with the extracted information. Depending on the application, the useful information can be encoded in the amplitude, frequency, phase, duty cycle, digital code or any other characteristic of the input signal.

[0113] As an example, take the frequency f of the input signal in Modulate its output frequency f out The input shaping circuit 30 will have a transfer function f out=f in n+f carrier , where f carrier is the carrier frequency, n is a constant. If the circuit uses the amplitude A of its input signal in Modulate its output amplitude A out , then its transfer function will be A out =A in n, where n is a constant.

[0114] Figure 3A The example input shaping circuit 30 shown includes an information extraction circuit 35 and a pulse generation circuit 38. The information extraction circuit 35 receives the SNN input signal 33 and sends amplitude 36A, frequency 36B and pulse number 36C outputs to the pulse generation circuit 38, which generates a processed input signal 39 including a pulse train. Figure 3B A graph of an example SNN input signal 33 is shown, Figure 3C A graph of an example processed input signal 39 is shown (showing amplitude versus time on the horizontal axis).

[0115] Circuit non-idealities may affect the input shaping circuit 30 in a number of different ways, such as gain errors affecting amplitude, delay errors affecting phase, or local reference errors affecting the processing of the digital input code. Compensation circuits may be implemented to counteract the effects of circuit non-idealities by adding these features as adjustable inputs 34 to adjust gain, frequency or phase, circuit delay, or to adjust the local reference.

[0116] Similarly, the pulse generation circuit 38 will have non-idealities that cause errors in the output pulse train amplitude, decay time, or inter-pulse timing characteristics (e.g., frequency, phase, or density). The pulse generation circuit 38 can be equipped with a compensation circuit that can use adjustment inputs (e.g., rise time 37A and decay time 37B, amplitude 37C, and timing 37D adjustment signals) to modify the parameters that are most sensitive to changes and are related to the desired signal processing function.

[0117] Figure 4 is a simplified schematic diagram of synapse 25, which can be used Figure 2 SNN and Figure 6 The synapse 25 receives the processed input signal 41 (from the input processing circuit 24) and applies the weight 42 (eg, multiplies the input signal 41 by the weight 42) to generate the synaptic output signal 46.

[0118] The transfer function of the synapse is nominally given by I out =I in ·(w / w max ) is given, where I outis the output current, I in is the input current, w is the weight programmed into the synapse, and w max is the maximum possible weight value. Circuit non-idealities will cause additive and multiplicative errors, making the transfer function become I out =α·I in ·(w / w max )+β, where α is the multiplication error and β is the offset error. Adjustment circuitry can be added to synapse 26 to counteract these non-idealities by implementing an adjustable gain factor input 43 that can counteract α and an adjustable offset input 44 that can counteract β. Adjustments to weights 42 can also be used to adjust synapse 25.

[0119] Figure 5 is a simplified schematic diagram of a neuron 26, which can be used to Figure 2 SNN and Figure 6 SNN 60. In this example, the neuron 26 includes an integrator circuit 53 and a comparator circuit 57. The integrator circuit 53 receives one or more synaptic output signals as input 51, integrates (accumulates) the input, and sends the resulting membrane potential signal 54 to the comparator circuit 57. The comparator circuit 57 compares the received membrane potential 54 with a reference potential 55 used as a pulse threshold. When the membrane potential exceeds the pulse threshold, the comparator circuit 57 generates a pulse as a neuron output signal 59. The pulse also serves as a reset signal 58 to reset the integrator circuit 53.

[0120] A typical implementation of an electronic neuron 26 might use a capacitor as the integrating element 53. When manufactured on an integrated circuit, the value of a capacitor can typically vary by about 20% between manufacturing batches. Furthermore, multiple copies of nominally identical capacitors on a single chip can vary by about 5% from one another. Furthermore, capacitance can have a temperature coefficient that, depending on the dielectric material used to make the capacitor, can add several percent variation over a typical commercial or industrial temperature range. All of these variations result in an ill-defined integration constant for the neuron 26, which will result in uncertainty in the signal processing functionality of the SNN.

[0121] The neuron integrates its input signal 51 and generates a neuron output signal 59 when the integrated value reaches the pulse threshold. The output pulse frequency of the neuron can be expressed as f out =g / (C·V)∫i in dt, where g is the input gain, C is the capacitance, V is the threshold voltage, and i inis the input current. Circuit non-idealities can affect the gain g, the slope of the integrator defined by C, or the pulse threshold V. Adjustment circuits can be added to the neuron 26 to counteract the effects of each of these non-idealities. Since both the gain and slope of the integrator affect the transfer function of the neuron in the form of multiplicative errors, both can be compensated for simultaneously by adding an adjustable gain input 52 to the integrator 53. Errors in the pulse threshold can be compensated for by adding a reference potential offset value 55 and / or an adjustable pulse threshold 56 to the comparator 57.

[0122] Figure 6 is a simplified schematic diagram of SNN 60, including Figure 2 The SNN 20 in FIG. 1 also includes an offset current generator 64 and an analog-to-digital converter 66 for calibrating the SNN. Figure 2 Other features of SNN 20 can also be applied to Figure 6 SNN 60. Figure 2 Similar to the embodiment in FIG. 1 , each row of synapses 25 receives a processed input signal 27 from an input processing circuit 24. Each synapse 25 applies a weight to the received processed input signal 27 and generates a synaptic output signal 28.

[0123] The array is extended by adding a row of offset current generators 64, one for each synapse 25 in a row of synapses. Each offset current generator (OCG) 64 generates an offset current 65. Each column of the array has one OCG 64 and a plurality of synapses 25. The offset current 65 from one of the OCGs 64 and the synaptic output signal 28 from a column of synapses 25 are received by one of the neurons 26. Each neuron 26 integrates the received synaptic output signal 28 and generates a neuron output signal 29 when the integrated value reaches a predetermined threshold value, so that the neuron output signal 29 is in the form of a pulse train.

[0124] The OCG 64 can be viewed as an extra row of synapses that add a constant signal to each column of synapses instead of a pulse train. Each OCG 64 generates a constant offset current output 65 controlled by its input code. The ideal transfer function of the OCG 64 is expressed as I out =I ref ·(code / code max ). However, circuit non-idealities may add error terms and factors that change the transfer function to I out =α·I ref ·(code / code max ) + β. To account for these non-idealities, the circuit can be provided with an adjustment mechanism that adds adjustable gain and offset sufficient to cancel the effects of the gain error α and the offset error β.

[0125] Figure 7 An example of an offset current generator 64 is shown that includes a DC current source 71 and has a gain adjustment implemented by adding an additional gain stage 72 (e.g., an analog amplifier or a scaling current mirror) that has an external adjustable gain input 74. The offset adjustment 75 can be implemented by an adjustable constant signal source, the output of which is added to the offset current source output 65. The OCG 64 can be used to provide bias current to the neural synapse array (i.e., synapses 25 and neurons 26) during operation, and can also be used to assist in calibration, as described below. The OCG 64 is preferably included in the same IC as the neural synapse array of the SNN 60, but alternatively, they can be implemented separately and connected to the neural synapse array when needed.

[0126] Return to reference Figure 6 , the SNN 60 also includes an analog-to-digital converter (ADC) 66, which can be connected to the output of any neural synapse column, each column including an OCG 64 and a plurality of synapses 25. The ADC 66 can be connected (e.g., via a switch connection or a multiplexer or circuit that selects inputs within the ADC) to receive an analog signal generated by one of the neural synapse columns, i.e., a combined signal from the OCG 64 and a plurality of synapses 25 in one column of the synapse array. The ADC 66 converts the received analog signal into a digital signal 67 for measurement in the digital domain. The ADC 66 is preferably included in the same IC as the neural synapse array of the SNN 60, but alternatively, it can be implemented separately and connected to the neural synapse array when needed. Instead of having a single ADC 66 that can be switched to receive inputs from different groups (columns) of synapses 25 and OCG 64, the SNN 60 can alternatively be equipped with multiple ADCs that can operate in parallel, each ADC receiving inputs from different groups of synapses and OCGs. Multiple ADCs result in a more complex and expensive implementation, but the calibration measurements can be performed in parallel and are therefore faster.

[0127] The SNN 60 may also include components for measuring the frequency of the neuron output signal 29 from each neuron 26 (ie, measuring the frequency of the pulses generated by the neuron 26). Figure 6 The embodiment in includes a frequency measurement circuit 68 that is connectable (e.g., via a switch connection or a multiplexer or other circuit that selects an input within the frequency measurement circuit 68) to receive a neuron output signal 29 from a neuron 26. The frequency measurement circuit 68 generates an output 69 that indicates the frequency of the pulse train generated by the measured neuron 26. The neuron output frequency can be measured by comparing the neuron output signal 29 to a reference clock using a counter or other circuit.

[0128] Figure 8is a simplified schematic diagram of the overall structure of an SNN, with an analog or mixed-signal SNN 81, an analog-to-digital converter 66 with an external reference 83, a frequency measurement circuit 68, a processing element 84, and a memory 85. These components may all be implemented on the same IC or SoC, or may be implemented in one or more separate ICs.

[0129] Figure 8 The SNN 81 may have Figure 6 The SNN 60 of the embodiment has the same structure and details (except Figure 8 ADC 66 and frequency measurement circuit 68 are shown separately in FIG. 1 ). ADC 66 may be connected to receive an analog signal from SNN 81 as an input (in the same manner as described above with respect to Figure 6 The ADC 66 is used to calibrate the ADC 66 in the same manner as described above and converts them to digital signals using an external reference signal 83. The digital signal output from the ADC 66 is received by a processor 84 which runs software or firmware to perform the calibration process and store the results in a memory 85. As previously described, more than one ADC 66 may be used to speed up the calibration process.

[0130] Frequency measurement circuit 68 may be connected to receive as input the neuron output signal from SNN 81 (in the same manner as described above with respect to Figure 6 The same manner as described above) and generates an output based on the input frequency. The output from the frequency measurement circuit 68 is also received by the processor 84. More than one frequency measurement circuit 68 can be used to speed up the calibration process. Alternatively, the frequency measurement can be performed directly by the processor 84, which receives the neuron output signal 29 directly from the SNN 81.

[0131] The processor 84 may be implemented as a microprocessor, field programmable gate array (FPGA), application specific circuit (ASIC) or other type of processing element suitable for performing the calibration process described herein that executes software. A single processor may be used to perform the calibration process, or multiple processors may be used and the calibration process divided between them.

[0132] The calibration process includes several stages in which relevant parameters are measured by ADC 66 and network parameters are adjusted to offset PVT variations. Network elements (e.g., input processing circuit 24, synapse 25, neuron 26, and OCG 64) may have adjustment mechanisms as described herein that can be used to modify their relevant parameters to correct for component variations. These adjustments are typically controlled through a digital interface. The adjustment values ​​to be applied are then stored in digital memory 85. However, other methods are possible, such as laser trimming (in which small cuts are made in on-chip devices) or analog trimming (in which voltages are stored on well-insulated capacitors).

[0133] The calibration process described can be modified to account for differences in sensitivity to manufacturing variations and environmental factors, depending on the circuit in question. As an example, if the design of the neuron is based on a current mirror type circuit, only a single calibration step is required before the neuron is ready for use. If the neuron is based on a circuit that relies on transistor transconductance, it will need to be calibrated during operation. For each network component type, and therefore for each step of the calibration process, a decision should be made to include it in an initial pre-use calibration step, to include it in an online calibration process, or both.

[0134] Each step of the calibration process has a certain cost in terms of chip area, power consumption, time and memory requirements. These costs need to be considered when designing the various SNN components. For each component, a trade-off analysis needs to be performed to balance the costs incurred with the benefits gained.

[0135] As an example, adding adjustment functionality to synapses would incur a significant chip area penalty for the entire neural synapse array, since synapses typically account for the majority of chip area. Implementing gain and offset adjustment functionality on every single synapse may also become impractical from a routing perspective, leading to the decision to reduce the number of adjustment bits to an absolute minimum or even abolish this functionality entirely, and instead simply measure the non-ideal behavior of each synapse and take it into account when programming the weights. For example, if a particular synapse ends up having a -10% gain error, this error can be offset by increasing the weight value programmed into it by 10%.

[0136] Another example of a trade-off in adjusting circuit design is the threshold of a neuron. In many cases, this will be a fixed voltage that is ultimately derived from an on-chip voltage reference (e.g., a bandgap reference circuit). Making this voltage adjustable for each neuron may require implementing a digital-to-analog converter (DAC) for each individual neuron. High-resolution DACs typically require a larger chip area and larger bias currents to offset the effects of mismatch and noise. The number of DAC bits implemented and the resulting increased chip area and power consumption should be weighted against the expected gain in signal processing accuracy due to the additional threshold voltage accuracy.

[0137] In contrast, when there is only one ADC, it is worthwhile to make the on-chip ADC adjustable to high accuracy so that the die area penalty is not incurred multiple times on the chip. In addition, because the ADC is only used during calibration and can be turned off afterwards, the ADC power requirements can be relatively relaxed.

[0138] When the effects of PVT variations on all circuits in a large array are added together, the overall transfer function of the network cannot be accurately controlled. Therefore, an improved crossbar circuit with calibration elements is proposed, along with a process designed to ensure that the relevant parameters of each network element are controlled to a predetermined level of accuracy.

[0139] The overall process of calibrating an analog or mixed-signal SNN depends on the exact configuration of the network. Figure 6 and Figure 8 The network structure shown describes the following calibration process. The calibration process begins by calibrating the ADC 66 using an external well-characterized signal source 83. Once the input to output transfer function of the ADC is well characterized in this way, it can be stored and used to calibrate components of the SNN 60 / 81.

[0140] Fig. 9 is a flow chart showing the steps of an ADC calibration process 90. In step 91, the external reference 83 is connected to the ADC 66, and the ADC 66 converts the received external reference 83 into a digital output. In step 92, the ADC 66 is adjusted so that the digital output is correct, and in step 93, the external reference 83 is disconnected. In step 94, the SNN 81 is calibrated using the ADC 66, as described below.

[0141] Fig.10 is a flow chart showing the high-level stages of the SNN calibration process. First, the OCG 64 is calibrated (step 101). The neurons 26 are then calibrated (step 102), preferably using a previous calibration of the OCG 64 (i.e., using a previously determined transfer function of the OCG 64, or using an OCG 64 that has been adjusted to reduce PVT variations). Then, the input processing circuit 24 (step 103) and synapse 25 (step 104) are calibrated, preferably using a previously calibrated OCG 64 and neuron 26 (i.e., using a previously determined transfer function of the OCG 64 and neuron 26, or using an OCG 64 and / or neuron 26 that has been adjusted to reduce PVT variations).

[0142] As described below, calibration at each stage involves measuring the transfer functions of the relevant elements of the SNN, and may also involve: adjusting the elements to correct or reduce PVT variations in the elements, if the elements have facilities for individual adjustment. As previously described, the trade-off between reduced PVT variations and greater accuracy versus increased chip area and power consumption may dictate that some elements of the SNN have limited or no facilities for individual adjustment.

[0143] Fig.111 is a flow chart of a process for calibrating the OCGs 64 of the SNN 81. In step 111, the OCGs 64 are set to output offset current signals 65 that are equal to a predetermined reference output current for the calibration process. The ADC 66 is connected to receive the offset current signals 65 from each OCG 64 in turn and convert the received analog signals 65 into digital signals. Due to PVT variations, these offset current signals 65 will differ from the nominal values ​​that the OCGs are designed to produce. The digital signals produced by the ADC 66 are received by the processor 84, which can calculate the transfer function of each OCG 64 and determine adjustment signals, such as the gain adjustment signal 74 and the offset adjustment signal 75, to correct for any PVT variations in the associated OCG 64.

[0144] After these measurements, the OCG 64 can be adjusted to bring its value as close as possible to the desired nominal value. The ultimate limit to the accuracy of the offset current signal 65 generated by the OCG 64 is the accuracy and resolution of the ADC 66 and the range and resolution of the adjustment mechanism of the OCG 64. The transfer function and / or adjustment signal can be stored in a data memory 85.

[0145] Another benefit of calibrating the OCG in this manner is that this step will simultaneously eliminate any leakage current flowing in the synaptic columns. Although the synapses 25 are not active and may be off during this step, so they should not conduct any current, the synapses will actually draw leakage current. Since this leakage current is indistinguishable from any signal current, it should be eliminated to avoid it interfering with the normal operation of the SNN 60. Since the leaky synapses 25 are connected to the OCG 64 and ADC 66 during the OCG calibration step, any leakage current present in the column is measured by the ADC 66 and compensated for when adjusting the OCG.

[0146] Fig.12 is a flow chart of a process for calibrating the neurons 26 of the SNN 81. Once the offset current signals 65 generated by the OCG 64 are well characterized, they can be used as reference inputs for calibrating the neurons 26. In step 121, the (previously calibrated) OCG 64 is set to output an offset current 65 having a predetermined value suitable for the neuron calibration process. In step 122, the output frequency of the neuron 26 (i.e., the frequency of the pulses generated by the neuron) is measured by the frequency measurement circuit 68. Neuron output frequencies can generally be measured accurately because digital systems generally contain accurate clocks that can be used for measurement. By measuring the output frequency of each neuron as a response to a known input signal (i.e., the calibrated offset current signal 65), the processor 84 can accurately determine the transfer function of each neuron 26.

[0147] In step 123, the processor 84 receives the measured neuron output frequency for each neuron 26 and can detect any deviation from the expected value so that the deviation of the transfer function of the neuron 26 caused by the PVT change can be determined. If the neuron 26 has an adjustment mechanism, such as input gain adjustment 52, reference potential offset value 55 and / or pulse threshold adjustment 56, the processor 84 can calculate these adjustments and adjust the individual neurons to account for these PVT changes. The transfer function of the neuron 26 and / or the adjustment signal of the neuron can be stored in the data storage 85.

[0148] Fig.13 is a flow chart of a process for calibrating input processing circuitry 24 and synapses 25 of SNN 81. Once OCG 64 and neurons 26 have been characterized, for example by using the calibration process described above, this information may be used to calibrate synapses 25 and input processing circuitry 24.

[0149] In step 131, a predetermined input signal 61 (e.g., a pulse train having a known amplitude and frequency) is applied to one of the input processing circuits 24, and in step 132, the weight of the synapse 25 is set to a predetermined value. In step 133, the frequency measurement circuit 68 measures the frequency of the neuron output signal 29 generated by the neuron 26, and these measurements are received by the processor 84.

[0150] Since the input signal 61 and the input-to-output transfer function of each neuron 26 are known, the processor 84 can determine the combined transfer function of the input processing circuit 24 and synapse 25 through which the input signal flows to each neuron 26. The transfer function can then be corrected by adjusting the relevant input processing circuit 24, the relevant synapse 25, or both.

[0151] Since both the input processing circuitry 24 and the synapses 25 are affected by PVT variations, both contribute to the inaccuracy of the measurement in step 133. The two contributions can be separated by measuring the average synaptic output signal over a row of synapses 25 connected to a single processing circuit 24. The PVT variations contributed by the synapses 25 will be substantially averaged out of the measured row signal, leaving only the PVT variations contributed by the input processing circuitry 24.

[0152] The synaptic output signals from a row of synapses 25 will be received by a row of neurons 26. The output frequencies of these neurons are measured in step 133 and may be averaged to substantially isolate the PVT variations contributed by the associated input processing circuits 24. In step 134, the input processing circuits 24 may be adjusted, for example by adjusting input gain 34, rise time 37A, decay time 37B, amplitude 37C, and / or timing 37D, so that the transfer function is within a desired tolerance.

[0153] In step 135, after adjusting the input processing circuit 24, the processor 84 may use the neuron output frequency measured for one of the neurons 26 to determine the transfer function of the associated synapse 25 (i.e., the synapse 25 through which the input signal flows from the input processing circuit 24 to the measured neuron 26). In step 136, the transfer function of the associated synapse 25 may be used to adjust the synapse 25, for example using the gain adjustment signal 43 and / or the offset adjustment signal 44 (if the synapse has these facilities) or by adjusting the weight 42. The transfer function of the input processing circuit 24 and the synapse 25 and / or the adjustment signal for the input processing circuit 24 and / or the synapse 25 may be stored in the data memory 85.

[0154] Fig.13 An alternative to the calibration process is that, rather than measuring the frequency of the neuron output signal 29 in steps 133 and 135, the output voltage or current of the associated synapse 25 is routed to the ADC 66 and measured by the ADC in the same manner as in the OCG calibration process. This enables the transfer function of the input processing circuit 24 and the synapse 25 to be measured directly by the ADC 66, rather than using the neuron 26 to generate a pulse train which is then measured by the frequency measurement circuit 68. When this alternative is used, the predetermined input signal 61 applied to the input processing circuit 64 can be a constant signal rather than a pulse train.

[0155] The calibration process described above can be modified to accommodate many other network architectures. As an example, another network topology can be designed in which an encoder replaces the input shaping circuit. This topology would enable the sensor to be connected directly to the SNN without any intermediate signal processing. Calibration of this topology would follow a similar pattern to that just described, but using the input processing circuit calibration process to calibrate the encoder. The content of this step depends on the signal transfer function of the encoder and its specific sensitivity to process and temperature variations.

[0156] In the case where the encoder generates a pulse train whose frequency is proportional to the amplitude of the input signal voltage, the encoder calibration step may include applying a well-defined static input signal to the encoder and measuring the frequency of the resulting current flowing in the synaptic column. The encoder's voltage-to-frequency transfer may then be adjusted until it matches a predetermined value.

[0157] However, it is important to take into account all possible non-ideal behaviors. For example, the encoder may be affected by DC leakage currents whenever no signal is being processed. In this case, the leakage current of the encoder needs to be taken into account during the offset calibration step to ensure that the leakage current does not interfere with any subsequent calibration steps.

[0158] The calibration process cannot distinguish between variations caused by tolerances in the manufacturing process (which are fixed for each individual copy of the circuit) and variations caused by supply voltage and temperature variations (which vary during the life of the circuit). Therefore, the calibration process will need to be performed at least once to compensate for manufacturing variations and will need to be performed again, in whole or in part, when environmental factors change.

[0159] Inaccuracies in manufacturing often result in variations in circuit performance that remain constant over the life of the product. A classic example is the offset in a matched pair of CMOS transistors, often used in subcircuits such as current mirrors. Although the designer intends for the two transistors to be identical, tolerances in manufacturing can cause the two transistors to differ slightly. Common causes of this mismatch are roughness in the lines drawn on the chip, random scattering of implanted atoms, and variations in the thickness of the oxide layer grown on the wafer.

[0160] In a current mirror designed to accurately copy the input current to its output, the expected transfer function is I out =I in . Manufacturing tolerances will cause it to become I out =n*I in , where n is a number other than 1 that is determined by the specific manufacturing inaccuracies present in a single copy of the circuit. Since the physical effects that influence this factor are fixed and do not change, a single calibration step directly after circuit fabrication is sufficient to keep n close to unity over the lifetime of the circuit.

[0161] Other circuit parameters are sensitive to environmental factors such as supply voltage and temperature. An example is the transconductance of a CMOS transistor, which is determined by the gate-source voltage, drain-source voltage, the physical characteristics of the device, and temperature. Since any parameter other than the physical characteristics may change as the transistor is used, it is impossible to pre-compensate for any changes. Any required calibration needs to be performed while the device is operating.

[0162] Certain steps are performed at different points in the chip lifecycle, depending on the expected variation in each parameter and the impact each parameter has on the overall performance of the chip. An initial test process (often called "wafer sort") is performed at the end of the wafer manufacturing process, in which the basic functionality of each chip is tested. At this time, any chip that is determined to have a variation in a critical parameter that is greater than the range of its associated adjustment mechanism is discarded. The remaining chips are then packaged and remeasured in a final test step. This step will include an initial calibration step that aims to offset, as much as possible, all manufacturing variations in circuit parameters that are expected to remain constant over the life of the chip. The resulting value is stored in a memory that can be incorporated into the same chip package, or it can be stored externally but still associated with the individual chip it was measured on by administrative means (e.g., serial number tracking).

[0163] Therefore, the present invention discloses systems and methods for calibrating SNNs.

[0164] Note that the features of any of the embodiments disclosed herein may be combined in any suitable manner.

Claims

1. A spiking neural network, comprising: a plurality of input processing circuits, each input processing circuit having an input for receiving a spiking neural network input signal and configured to: apply a transfer function to the input signal to generate a processed input signal; a plurality of offset current generators, each offset current generator being configured to: generate an offset current signal of a predetermined level; a plurality of synapses, each synapse connected to receive a processed input signal from one of the plurality of input processing circuits and configured to: apply a predetermined weight to the processed input signal to generate a synaptic output signal; a plurality of neurons, each neuron connected to receive synaptic output signals from a subset of the synapses and an offset current signal from one of the plurality of offset current generators, and each neuron configured to: generate a neuron output signal in response to the received synaptic output signal and the offset current signal; and An analog-to-digital converter having an input connectable to receive an offset current signal from one of the plurality of offset current generators and configured to convert the received offset current signal into a corresponding digital output signal.

2. The spiking neural network according to claim 1, wherein: Each of the offset current generators is individually adjustable to adjust the offset current signal.

3. The spiking neural network according to claim 2, further comprising: A first processing circuit is connected to receive the digital output signal generated by the analog-to-digital converter and is configured to generate an adjustment signal for adjusting a corresponding one of the plurality of offset current generators to generate an adjusted offset current signal.

4. A spiking neural network according to any one of the preceding claims, further comprising: A frequency measurement circuit is connected to receive a neuron output signal from one of the plurality of neurons and to generate an output indicative of a frequency of the received neuron output signal.

5. The spiking neural network according to claim 4, wherein: The neurons may be individually adjustable to adjust the input gain, pulse threshold and / or reference potential offset value of the neurons.

6. The spiking neural network according to claim 5, further comprising: A second processing circuit is connected to receive the output from the frequency measurement circuit and is configured to generate an adjustment signal for adjusting a corresponding one of the plurality of neurons.

7. A spiking neural network according to any one of the preceding claims, wherein: Each synapse may be adjustable to adjust an input gain and / or offset value of the synapse.

8. A spiking neural network according to any one of the preceding claims, wherein Each input processing circuit may be adjustable to adjust an input gain, a pulse threshold and / or a timing value of the input processing circuit.

9. A spiking neural network according to any one of the preceding claims, wherein: The analog-to-digital converter may be connected to receive a synaptic output signal from one of the plurality of synapses and configured to convert the received synaptic output signal into a corresponding digital synaptic output signal.

10. The spiking neural network according to any one of the preceding claims, further configured to: A calibration mode is provided in which one or more of the spiking neural network input signals are set to predetermined values.

11. The spiking neural network according to claim 10, further comprising: a third processing circuit connected to receive a signal from the frequency measurement circuit, the signal indicating a frequency of a neuron output signal of one of the plurality of neurons during the calibration mode, the neuron connected to receive a synaptic output signal from a synapse of the synapses connected to receive a processed input signal from one of the plurality of input processing circuits, The third processing circuit is configured to generate an adjustment signal for adjusting the input processing circuit and / or the synapse.

12. The spiking neural network according to claim 10, wherein: The analog-to-digital converter is connectable to receive a synaptic output signal from one of the plurality of synapses and is configured to: convert the received synaptic output signal into a corresponding digital synaptic output signal, and the spiking neural network further comprises: A fourth processing circuit is connected to receive the digital synapse output signal during the calibration mode and is configured to generate an adjustment signal for adjusting the input processing circuit and / or synapse.

13. A method for calibrating a spiking neural network, the spiking neural network comprising: A plurality of offset current generators are configured to: generate offset current signals; A plurality of input processing circuits configured to: generate processed input signals; a plurality of synapses configured to: apply weights to the processed input signals to generate synaptic output signals; and a plurality of neurons connected to receive one of the plurality of bias current signals and synaptic output signals from the plurality of synapses and configured to: generate a neuron output signal, The method comprises: generating a plurality of offset current signals by the plurality of offset current generators, each offset current signal being at a predetermined level; converting the offset current signal into a corresponding digital output signal; and Based on the digital output signal, the offset current generators are adjusted to produce an adjusted offset current signal from each offset current generator.

14. The method according to claim 13, further comprising: receiving a plurality of adjusted offset current signals in the plurality of neurons; measuring the frequency of the neuron output signal; as well as The input gain, pulse threshold and / or reference potential offset value of each neuron are adjusted based on the adjusted offset current signal received by the corresponding neuron and the measured frequency of the neuron output signal generated by the corresponding neuron.

15. The method according to claim 14, further comprising: Transfer functions for the neurons are calculated, wherein the transfer function of each neuron is based on the adjusted offset current signal received by the neuron and a measured frequency of a neuron output signal of the neuron.

16. The method according to any one of claims 13 to 15, further comprising: providing a spiking neural network input signal set to a predetermined value to a first one of the input processing circuits; setting a weight of a first synapse among the synapses to a predetermined value; receiving, by the first synapse, a processed input signal from the first input processing circuit and, in response, generating a corresponding synaptic output signal; receiving, by a first neuron among the neurons, a synaptic output signal from the first synapse and an adjusted offset current signal from a first offset current generator among the offset current generators; measuring a frequency of a neuron output signal generated by the first neuron; as well as Based on the pulse neural network input signal and the measured frequency of the neuron output signal, the first input processing circuit and / or the first synapse are adjusted to adjust the transfer function of the first input processing circuit and the first synapse.

17. The method according to any one of claims 13 to 15, further comprising: providing a spiking neural network input signal set to a predetermined value to a first one of the input processing circuits; setting weights of a first subset of synapses in the synapses to predetermined values; receiving, by the first subset of synapses, a processed input signal from the first input processing circuit and, in response, generating a corresponding synaptic output signal from each synapse of the first subset of synapses; receiving, by the neuron, a synaptic output signal from the first subset of synapses; measuring the frequency of each neuron output signal generated by the neuron; as well as The first input processing circuit is adjusted based on the frequency of the spiking neural network input signal and the measured neuron output signal to adjust a transfer function of the first input processing circuit.

18. The method according to claim 17, further comprising: providing a spiking neural network input signal set to a predetermined value to a first one of the input processing circuits; receiving, by a first synapse in the first subset of synapses, a processed input signal from the first input processing circuit and, in response, generating a corresponding synaptic output signal; receiving, by a first neuron among the neurons, a synaptic output signal from the first synapse and an adjusted offset current signal from a first offset current generator among the offset current generators; measuring a frequency of a neuron output signal generated by the first neuron; as well as The first synapse is adjusted based on the spiking neural network input signal and the measured frequency of the neuron output signal to adjust a transfer function of the first synapse.

19. The method according to any one of claims 13 to 15, further comprising: providing a spiking neural network input signal set to a predetermined value to a first one of the input processing circuits; setting a synaptic weight of a first synapse among the synapses to a predetermined value; receiving, by the first synapse, a processed input signal from the first input processing circuit and, in response, generating a corresponding synaptic output signal; converting the synaptic output signal into a corresponding digital output signal; as well as Based on the spiking neural network input signal and the digital output signal, the first input processing circuit and / or the first synapse are adjusted to adjust a transfer function of the first input processing circuit and the first synapse.

20. The method according to any one of claims 13 to 15, further comprising: providing a spiking neural network input signal set to a predetermined value to a first one of the input processing circuits; setting weights of a first subset of synapses in the synapses to predetermined values; receiving, by the first subset of synapses, a processed input signal from the first input processing circuit and, in response, generating a corresponding synaptic output signal from each synapse in the first subset of synapses; converting synaptic output signals from the first subset of synapses into corresponding digital output signals; as well as The first input processing circuit is adjusted based on the spiking neural network input signal and the digital output signal to adjust a transfer function of the first input processing circuit.

21. The method according to claim 20, further comprising: providing a spiking neural network input signal set to a predetermined value to a first one of the input processing circuits; receiving, by a first synapse in the first subset of synapses, a processed input signal from the first input processing circuit and, in response, generating a corresponding synaptic output signal; converting a synaptic output signal from the first synapse into a corresponding digital output signal; as well as The first synapse is adjusted based on the spiking neural network input signal and the digital output signal to adjust a transfer function of the first synapse.

Images