Calibration of Spiking Neural Networks

Abstract

A spiking neural network comprising: a plurality of input processing circuits, each input processing circuit having an input for receiving a spiking neural network input signal and configured to apply a transfer function to the input signal to generate a processed input signal; a plurality of offset current generators, each offset current generator configured to generate an offset current signal at a predetermined level; a plurality of synapses, each synapse connected to receive a processed input signal from one of the input processing circuits and configured to apply a predetermined weight to the processed input signal to generate a synapse output signal; a plurality of neurons, each neuron connected to receive synapse output signals from a subset of the synapses and an offset current signal from one of the offset current generators, and each neuron configured to generate a neuron output signal in response to the received synapse output signals and the offset current signal; an analog-to-digital converter having an input, the input connectable to receive an offset current signal from one of the offset current generators and configured to convert the received offset current signal into a corresponding digital output signal.

Description

Technical Field

[0001]

[0001] This disclosure generally relates to automatic signal recognition techniques and spiking neural networks, and more particularly to systems and methods for calibrating spiking neural networks.

Background Art

[0002]

[0002] A spiking neural network (SNN) is a signal processing system inspired by biological neural networks. Information is encoded in the pattern of spike signals distributed across a network of neurons and synapses as described in WO2022 / 090542A1 filed by the applicant. By mimicking the processing performed in the biological brain, spiking neural networks can perform signal processing tasks commonly performed by the human brain. Examples include image recognition, speech recognition, and event detection based on inputs from multiple sensors.

[0003]

[0003] The functionality of an SNN is determined by its network configuration and its weight matrix. The network configuration specifies how many neurons and synapses are used and how they are connected to each other as well as to input and output signals. It also defines what types of pre- and post-processing circuits are required and the complete connectivity between them. The weight matrix is a set of parameters that determine the parameters of each network element. Generally, this includes gain factors for each synapse, but other parameters such as neuron thresholds can also be included.

[0004]

[0004] The network configuration is determined by a user who selects the network topology based on the intended application. On the other hand, the weight matrix is generated from training steps, and the SNN is exposed to a large amount of training data, and its weights are automatically updated to bring the output of the network close to the desired values.

[0005]

[0005] Since the training step generally requires a long time to complete, even if multiple copies of the SNN are fabricated, it is preferably performed only once. The weight matrix resulting from the training step is then copied to each individual part of the hardware. However, if the hardware that executes the SNN is different from the hardware on which the network was trained, the resulting network may be implemented differently and may not be able to perform its intended function.

[0006]

[0006] Analog and mixed-signal circuits commonly fabricated as integrated circuits are a power-efficient way to implement SNNs. One drawback of analog signal processing circuits is that their performance can vary due to manufacturing tolerances and environmental factors such as temperature and supply voltage variations. Digitally assisted analog mixed-signal circuits can be used as an alternative to provide reliable performance. The amount of assistance is limited and needs to be optimized.

[0007]

[0007] Therefore, in order to provide reliable computational performance, these implementations of SNNs need to be calibrated to ensure that their performance is determined by their design regardless of manufacturing variations or changes in operating conditions such as voltage and temperature. The calibration process aims to adjust the behavior of the analog circuits in the SNN to match its predefined specifications and performance.

[0008]

[0008] One way to guarantee the same performance within acceptable limits between different copies of an analog or mixed-signal SNN is to train each single copy against the same set of training data. Each copy will then have its own weight matrix optimized for its unique hardware properties. While this method will compensate for manufacturing errors, it does not reduce circuit variations as a result of temperature or voltage fluctuations. Another drawback is that performing training steps for each individual copy can be time-consuming and costly.

[0009]

[0009] Another method described in WO2020 / 260067A1 is to perform a reduced number of training steps for each individual copy of the SNN for the purpose of adjusting the weights towards specific properties of the SNN. However, this can still be time-consuming and is not guaranteed to converge to a working solution. Furthermore, it gives no insight into the type and amount of variations of the internal components, making it difficult to improve the hardware design.

[0010]

[0010] Therefore, a faster and more robust calibration procedure is needed that gives the possibility to adjust the individual elements of the SNN rather than adjusting the SNN as a whole unit.

Summary of the Invention

[0011]

[0011] To implement such an improved calibration procedure, circuit parameters can be measured and then adjusted to comply with predefined specifications. In complex analog and mixed-signal networks, it may not be practical to directly measure all relevant parameters with sufficient accuracy, and a calibration strategy is needed that optimally utilizes observable parameters to infer the values of unobservable parameters. The relevant circuit parameters must be adjustable with a range sufficient to overcome expected variations and a resolution sufficient to achieve the predefined specifications.

[0012]

[0012] Due to the complexity of spiking neural networks (SNNs) and various types of components used in the design of analog or mixed-signal SNN system-on-chips (SoCs), the optimal trade-off between performance and the amount of calibration under SoC constraints is an important issue. The present invention provides a system and method for such calibration in SNNs with various components, connectivity, and limited observability and measurability.

[0013]

[0013] In a first aspect, the present invention comprises a spiking neural network (SNN). The SNN includes a plurality of input processing circuits, each input processing circuit having an input for receiving a spiking neural network input signal and configured to apply a transfer function to the input signal to generate a processed input signal; a plurality of offset current generators, each offset current generator configured to generate an offset current signal at a predetermined level; a plurality of synapses, each synapse connected to receive a processed input signal from one of the input processing circuits and configured to apply a predetermined weight to the processed input signal to generate a synapse output signal; a plurality of neurons, each neuron connected to receive a synapse output signal from a subset of the synapses and an offset current signal from one of the offset current generators, and each neuron configured to generate a neuron output signal in response to the received synapse output signal and offset current signal; an analog-to-digital converter having an input, the input connectable to receive an offset current signal from one of the offset current generators and configured to convert the received offset current signal into a corresponding digital output signal.

[0014]

[0014] The input processing circuit may include an input shaping circuit, an encoder circuit, a signal conditioning circuit, a filter circuit, and other types of circuits for processing the spiking neural network input signal. The input signal may comprise a raw or pre-processed signal from a sensor or other input device, or may comprise an output from a previous spiking neural network, signal processor, or other type of circuit.

[0015]

[0015] Each offset current generator preferably comprises a direct current (DC) current source for providing a stable current at a predetermined current level that can be adjusted. The offset current generator can be used to provide a bias current to a neuron during operation of the spiking neural network.

[0016]

[0016] The synapses are preferably configured in an array. For example, a first subset of synapses (e.g., a row of synapses) is connected to receive an input signal processed from one of the input processing circuits, and a second subset of synapses (e.g., a column of synapses) is connected to provide their synaptic output signals to one of the neurons. The array of synapses can be implemented as a crossbar array.

[0017]

[0017] The spiking neural network can be implemented as an analog signal processing circuit or a mixed-signal circuit combining analog and digital signal processing. The offset current generator, synapses, and neurons are implemented as hardware elements (e.g., hardware with software or firmware), preferably implemented in a single integrated circuit (IC) and can be implemented as a system-on-chip (SoC). The input processing circuit can also be implemented in the same IC or SoC together with the offset current generator, synapses, and neurons, or can be implemented as one or more external components.

[0018]

[0018] The analog-to-digital converter can also be implemented in the same IC or SoC together with other components, or can be implemented as one or more external components. A single analog-to-digital converter can be included (which can reduce chip size and cost), or multiple analog-to-digital converters can be used (to provide a faster calibration process).

[0019]

[0019] Each offset current generator is preferably individually adjustable to adjust the offset current signal. The spiking neural network may further comprise a first processing circuit configured to generate an adjustment signal for adjusting a corresponding one of the offset current generators to generate an adjusted offset current signal, connected or connectable to receive the digital output signal generated by the analog-to-digital converter.

[0020]

[0020] This adjustment preferably corrects for variations in each offset current generator and also for leakage current through the synapses connected to receive the offset current signal from the associated offset current generator. Further, this calibration of the offset current generator removes or reduces any variations introduced by the offset current generator and allows for more accurate calibration of other components of the spiking neural network. The first processing circuit can be implemented as a hardware circuit or processor with software or firmware, and can be implemented in the same IC or SoC as other components of the SNN or as a separate component that can be connected permanently or during calibration.

[0021]

[0021] The spiking neural network may further comprise a frequency measurement circuit connected to receive a neuron output signal from one of the neurons and generate an output indicative of the frequency of the received neuron output signal. The frequency measurement circuit can be implemented using a reference clock signal and components included on the same IC or SoC that includes the synapses and neurons.

[0022]

[0022] The neurons of the spiking neural network can be individually adjustable to adjust the input gain, spike threshold, and / or reference potential offset value of the neuron. The spiking neural network may further comprise a second processing circuit configured to generate an adjustment signal for adjusting a corresponding one of the neurons, connected or connectable to receive an output from a frequency measurement circuit.

[0023]

[0023] The neurons are connected to receive an offset current from an offset current generator, and preferably they are pre-calibrated to generate an adjusted offset current having a known predetermined value of the offset current generator. Since the input to each neuron is known and the frequency of the spikes generated by the neuron is measured, the transfer function of the neuron can be calculated to generate an adjustment signal to correct for the variations of the neuron. The second processing circuit may be implemented using the same processor as the first processing circuit.

[0024]

[0024] Each of the synapses of the SNN can be adjustable to adjust the input gain and / or offset value of the synapse. Each of the input processing circuits can be adjustable to adjust the input gain, spike threshold, and / or timing value of the input processing circuit.

[0025]

[0025] The analog-to-digital converter is preferably connectable to receive a synapse output signal from one of the synapses and is configured to convert the received synapse output signal into a corresponding digital synapse output signal. For example, the analog-to-digital converter may have a wired connection or a switched connection for receiving the synapse output signal.

[0026]

[0026] The SNN can be further configured to provide a calibration mode in which one or more of the spiking neural network input signals are set to a predetermined value. The input signal can comprise a train of spikes having a predetermined frequency and / or magnitude, or a constant input current or voltage having a predetermined value, or some other signal. By applying known input signals, it becomes possible to determine the transfer functions of various components of the spiking neural network to assist with calibration.

[0027]

[0027] The SNN is connected or connectable to a third processing circuit for receiving a signal indicating the frequency of the output signal of one of the neurons during the calibration mode, where the neuron is connected to receive a synaptic output signal from one of the synapses that is connected to receive the input signal processed by one of the input processing circuits, and wherein the third processing circuit is configured to generate an adjustment signal for adjusting the input processing circuit and / or the synapse.

[0028]

[0028] Since the input to the input processing circuit is known and the frequency of the spikes generated by the neuron is measured, the transfer function of the combination of the input processing circuit and the synapse (which sends the synaptic output signal to the neuron) can be calculated to generate an adjustment signal. This adjustment signal can be used to correct for variations in the input processing circuit and / or the synapse. The third processing circuit can be implemented using the same processor as the first and / or second processing circuits.

[0029]

[0029] The analog-to-digital converter of the SNN can be connectable to receive a synaptic output signal from one of the synapses and can be configured to convert the received synaptic output signal to a corresponding digital synaptic output signal. The SNN can further comprise a fourth processing circuit connected or connectable to receive the digital synaptic output signal during the calibration mode and configured to generate an adjustment signal for adjusting the input processing circuit and / or the synapse.

[0030]

[0030] Since the input to the input processing circuit is known and the output of the synapse is measured, the transfer function of the combination of the input processing circuit and the synapse (which sends the synapse output signal to the neuron) can be calculated to generate an adjustment signal. This adjustment signal can be used to correct variations in the input processing circuit and / or the synapse. The fourth processing circuit can be implemented using the same processor as the first, second, and / or third processing circuits.

[0031]

[0031] In a second aspect, the present invention provides a method for calibrating a spiking neural network, the spiking neural network comprising a plurality of offset current generators configured to generate offset current signals, a plurality of input processing circuits configured to generate processed input signals, a plurality of synapses configured to apply weights to the processed input signals to generate synapse output signals, and a plurality of neurons connected to receive one of the offset current signal and the synapse output signal from the plurality of synapses and configured to generate neuron output signals. The method comprises generating, by the plurality of offset current generators, a plurality of offset current signals at respective predetermined levels, converting the offset current signals into corresponding digital output signals, and adjusting the offset current generators based on the digital output signals to generate adjusted offset current signals from each of the offset current generators.

[0032]

[0032] The calibration method can be used in the SNN according to the first aspect of the present invention as described herein, and the details of the SNN described herein can also be applicable to the calibration method. The calibration method comprises measuring the output of each offset current generator, converting this into a digital signal, and using the digital signal obtained to adjust each offset current generator. This procedure can be used to correct for variations in each offset current generator and can also correct for leakage current through the synapses connected to receive the offset current signal from the associated offset current generator. This calibration of the offset current generator removes or reduces any variations introduced by the offset current generator and enables more accurate calibration of the other components of the spiking neural network.

[0033]

[0033] The calibration method can further comprise receiving a plurality of adjusted offset current signals among a plurality of neurons, measuring the frequency of the neuron output signal, and adjusting the input gain, spike threshold, and / or reference potential offset value of each of the neurons based on the adjusted offset current signal received by each neuron and the measured frequency of the neuron output signal generated by each neuron. The method can further comprise calculating a transfer function for the neurons, wherein the transfer function for each neuron is based on the adjusted offset current signal received by the neuron and the measured frequency of the neuron output signal of the neuron.

[0034]

[0034] The calibration of the neurons preferably utilizes an adjusted offset current from a pre-calibrated offset current generator to provide an offset current having a known predetermined value. Since this input to each neuron is known and the frequency of the spikes generated by the neuron is measured, the transfer function of the neuron can be calculated and an adjustment signal can be generated to correct for variations in each neuron.

[0035]

[0035] The calibration method may further include providing a spiking neural network input signal set to a predetermined value in one of the first of the input processing circuits, setting the weight of one of the synapses to a predetermined value, receiving the input signal processed from the first input processing circuit by the first synapse, and generating a corresponding synapse output signal in response thereto, receiving the synapse output signal from the first synapse by one of the first of the neurons, receiving an adjusted offset current signal from one of the first of the offset current generators, measuring the frequency of the neuron output signal generated by the first neuron, and adjusting the first input processing circuit and / or adjusting the first synapse to adjust the transfer function between the first input processing circuit and the first synapse based on the spiking neural network input signal and the measured neuron output signal frequency.

[0036]

[0036] The input signal may comprise a train of spikes having a predetermined frequency and / or magnitude, or a constant input current or voltage having a predetermined value, or some other predetermined signal. Since the input to the input processing circuit is known and the frequency of the spikes generated by the neurons is measured, the transfer function of the combination of the input processing circuit and the synapse can be calculated to generate an adjustment signal. This adjustment signal can be used to correct for variations in the input processing circuit and / or the synapse.

[0037]

[0037] The effects of input processing circuit variations and synaptic variations can be separated to some extent by performing a calibration method on a subset of synapses (e.g., a row). This calibration method involves providing a spiking neural network input signal set to a predetermined value in a first one of the input processing circuits, setting the weights of a first subset of synapses to a predetermined value, receiving an input signal processed from the first input processing circuit by the first subset of synapses, and in response generating a corresponding synaptic output signal from each synapse of the first subset of synapses, receiving the synaptic output signal from the first subset of synapses by a neuron, measuring the frequency of each neuron output signal generated by the neuron, and adjusting the first input processing circuit to adjust the transfer function of the first input processing circuit based on the measured frequency of the neuron output signal.

[0038]

[0038] This embodiment of the calibration method enables the effects of input processing circuit variations and synaptic variations to be separated to some extent. Synaptic output signals from the first subset of synapses (e.g., from a row of synapses in a crossbar array that all receive the same processed input signal) are averaged to average out synaptic variations and can leave only variations for the input processing circuit substantially. An adjustment signal can then be calculated to adjust the input processing circuit to account for the variations.

[0039]

[0039] When the transfer function of the input processing circuit is determined and the input processing circuit is adjusted to account for circuit variations, if possible, further calibration procedures can be performed to calibrate individual synapses. This calibration involves applying a spiking neural network input signal set to a predetermined value to a first one of the input processing circuits, receiving the input signal processed by the first input processing circuit by a first one of a first subset of synapses, and generating a corresponding synapse output signal in response, receiving the synapse output signal from the first synapse by a first one of the neurons, receiving an adjusted offset current signal from a first one of the offset current generators, measuring the frequency of the neuron output signal generated by the first neuron, and adjusting the first synapse to adjust the transfer function of the first synapse based on the spiking neural network input signal and the measured neuron output signal frequency.

[0040]

[0040] The calibration method can use the measurement of the neuron output frequency, which can provide an indication of the synapse output current received by the neuron when the transfer function of the neuron is known. An alternative is to directly measure the synapse output signal. Thus, the calibration method can involve applying a spiking neural network input signal set to a predetermined value to a first one of the input processing circuits, setting the weight of a first one of the synapses to a predetermined value, receiving the input signal processed by the first input processing circuit by the first synapse and generating a corresponding synapse output signal in response, converting the synapse output signal to a corresponding digital output signal, and adjusting the first input processing circuit and / or the first synapse to adjust the transfer function between the first input processing circuit and the first synapse based on the spiking neural network input signal and the digital output signal. This variation of the calibration method uses an analog-to-digital converter to measure the output signal of the synapse rather than sending the synapse output signal to the neuron and measuring the output of the neuron.

[0041]

[0041] Similarly, providing a spiking neural network input signal set to a predetermined value to one of the first of the input processing circuits, setting the weights of the first subset of synapses to the predetermined value, receiving the input signal processed by the first input processing circuit by the first subset of synapses, and in response generating a corresponding synaptic output signal from each synapse of the first subset of synapses, converting the synaptic output signal to a corresponding digital output signal, and adjusting the first input processing circuit to adjust the transfer function of the first input processing circuit based on the spiking neural network input signal and the digital output signal. The effects of variations in the input processing circuit and variations in the synapses that would be separated in a calibration method comprising.

[0042]

[0042] Similarly, a synapse can be further calibrated by a calibration method that further comprises providing a spiking neural network input signal set to a predetermined value to one of the first of the input processing circuits, receiving the input signal processed by the first input processing circuit by one of the first of the first subset of synapses, and in response generating a corresponding synaptic output signal, converting the synaptic output signal to a corresponding digital output signal from the first synapse, and adjusting the first synapse to adjust the transfer function of the first synapse based on the spiking neural network input signal and the digital output signal.

[0043]

[0043] A single calibration round can be performed for each individual SNN (e.g., each IC or SoC) to compensate for manufacturing tolerances. Further calibration rounds can also be performed during operation of the SNN to offset changes in temperature and supply voltage.

[0044]

[0044] Next, embodiments are described merely by way of example with reference to the accompanying drawings that show corresponding reference symbols for corresponding parts.

Brief Description of the Drawings

[0045]

Figure 1

[0045] Conceptual diagram of the smallest spiking neural network (SNN).

Figure 2

[0046] Simplified schematic diagram of an SNN implemented as a crossbar array.

Figure 3A

[0047] Simplified schematic diagram of an input shaping circuit that can be used as an input processing circuit in the SNNs of FIGS. 2 and 6.

Figure 3B

[0048] Diagram showing the input signal to the input shaping circuit of FIG. 3A.

Figure 3C

[0049] Diagram showing the spike output signal generated by the input shaping circuit of FIG. 3A.

Figure 4

[0050] Simplified schematic diagram of a synapse that can be used in the SNNs of FIGS. 2 and 6.

Figure 5

[0051] Simplified schematic diagram of a neuron that can be used in the SNNs of FIGS. 2 and 6.

Figure 6

[0052] Simplified schematic diagram of the SNN of FIG. 2 that also includes an offset current generator and an analog-to-digital converter.

Figure 7

[0053] Simplified schematic diagram of an offset current generator that can be used in the SNN of FIG. 6.

Figure 8

[0054] Simplified schematic diagram of the overall structure of an SNN with a related analog-to-digital converter, a processor, and a memory.

Figure 9

[0055] Flowchart showing the steps of a calibration procedure for an analog-to-digital converter.

Figure 10

[0056] Flowchart showing the stages of a calibration procedure.

Figure 11

[0057] Flowchart of a calibration procedure for the offset current generator of the SNN of FIG. 6 or FIG. 8.

Figure 12

[0058] Flowchart of a calibration procedure for a neuron of the SNN of FIG. 6 or FIG. 8.

Figure 13

[0059] Flowchart of a calibration procedure for an input processing circuit and synapses of the SNN of FIG. 6 or FIG. 8.

Mode for Carrying Out the Invention

[0046]

[0060] Hereinafter, several embodiments will be described in more detail. However, it should be understood that these embodiments should not be construed as limiting the scope of protection of the present disclosure.

[0047]

[0061] An artificial SNN is an electronic system that mimics the signal processing function of the brain. It is constructed from components that implement the same network structure as found in the biological brain and processes information in a similar manner. SNNs are particularly suitable for tasks such as audio or video recognition, pattern spotting in large datasets, or monitoring of sensor data for anomalies.

[0048]

[0062] Compared with conventional digital electronic systems, SNNs can perform signal processing tasks at significantly lower power and with lower latency. Low power consumption is achieved because only the parts of the system that are actually processing data at any given time are active. The essentially parallel architecture of SNNs guarantees low latency from input to output. Both properties are important for always-on signal processing applications in power-constrained environments such as keyword spotting in battery-powered devices like smartwatches.

[0049]

[0063] Like their biological counterparts, SNNs use spike trains as their internal signal representation. Information is encoded within one or more features of these spike trains, such as frequency, phase, spike width, spike density, and amplitude. As these spike trains move along the network, they undergo basic mathematical operations such as addition, multiplication, and integration. These operations, along with the network's topology, implement specific signal processing functions.

[0050]

[0064] The exact parameters of all internal components are not set directly by the user but are derived through a learning process. During this process, the network is exposed to various valid input signals, the outputs are observed, and the internal parameters are adjusted until the output gets close to the expected values. In this way, the internal parameters will converge to a set of values that enable the network to perform the desired signal processing function.

[0051]

[0065] Figure 1 is a conceptual diagram of a simple spiking neural network (SNN). This SNN includes an input layer 2 of an input processing circuit 3 connected to multiple layers of neurons and synapses in neuro-synaptic layers 6, 7, and 8. One such neuro-synaptic layer consists of a set of synapses 4 where the input ports are connected to the previous layer and the outputs are connected to a set of neurons 5. The overall output 9 of the system is generated by the last layer of neurons in the network. The output 9 of the last layer is then passed to a decoding layer that can forward the information towards the end-user in a meaningful format.

[0052]

[0066] The input layer 2 receives SNN inputs 1A, 1B, 1C, which are processed by input processing circuits 3A, 3B, 3C. The processed input signals are then provided to the neuro-synaptic layers 6, 7, 8, each of which comprises neurons 5 connected via synapse connections 4, resulting in an output signal 9 generated by the last neuro-synaptic layer. Each of these layers may have a digital, analog, or mixed-signal implementation, as further described below.

[0053]

[0067] Figure 2 is a simplified schematic diagram of an SNN 20 implemented as a crossbar array. The crossbar design is an efficient way to implement a reconfigurable SNN system, especially when fabricated on an integrated circuit. This comprises a rectangular array of synapses having input processing circuits on one side of the array and neurons on the other side. In the embodiment of Figure 2, the input processing circuits 24 are configured in one column, each driving a row of synapses 25. The synapses 25 are connected in series, and the outputs of all the synapses 25 in the column are summed and serve as an input to the neuron 26. By programming appropriate weights in the synapse array and accurately configuring the interconnect system, a wide variety of network topologies can be implemented.

[0054]

[0068] The signals to be processed (e.g., from a sensor) are generally first preprocessed by an encoder to convert the sensor output signal into a suitable digital or analog signal, and the input signal processing may also include filters in both the time domain and the amplitude domain, and various types of input shaping circuits for generating a signal suitable for input to the computational layers of the network. All or part of this input processing may be performed in the input processing circuits 24, or may be performed in whole or in part by an input layer separate from the SNN shown in Figure 2.

[0055]

[0069] In FIG. 2, the input processing circuit 24 receives the input 21 to the SNN via the interconnection 23, which can be used to interconnect several SNN arrays as shown in FIG. 2. Each input processing circuit 24 generates a processed input signal 27 that is received by a row of synapses 25 in the neuromorphic array.

[0056]

[0070] (For example, including an input shaping circuit and a filtering circuit) The function of the input processing circuit 24 is to modify the incoming signal 21 in such a way that they can be processed by a neural network. The incoming signal 21 can have information encoded in its amplitude, frequency, phase, or any other analog or digital modulation scheme, and needs to be processed to extract useful information that will be further processed by the neural network. The input processing circuit 24 generally includes a filter, a comparator, a demodulator, and other signal processing circuits for their inputs, and will include a spike generation circuit on their outputs. The frequency, amplitude, shape, and other parameters of the output spikes are modified to encode the information present in the input signal 21.

[0057]

[0071] The synapses 25 amplify or attenuate the incoming signal 27 by a constant factor determined by their weight settings. The weight for each synapse 25 is stored in a memory cell associated with the synapse. The values of all the weights in the network are known as the weight matrix and contain information extracted from the input data during the learning process.

[0058]

[0072] In the embodiment of FIG. 2, the row of synapses 25 receives the processed input signal 27 from one input processing circuit 24. Each synapse 25 applies a weight to the received and processed input signal 27 and generates a synapse output signal 28.

[0059]

[0073] The synaptic output signal 28 from the columns of synapses 25 is received by the neurons 26. Each neuron 26 integrates the received synaptic output signal 28 and generates a neuron output signal 29 when the integrated value reaches a predetermined threshold value. Thus, the neuron output signal 29 takes the form of a spike train. The neuron output signal 29 depends on several parameters of the neuron, such as the input gain, the integration constant, and the threshold value.

[0060]

[0074] The neuron output signal 29 is provided to the interconnection 23 and can be provided to the decoder. Since the final response can be encoded in time or phase and needs to be decoded, the decoder interprets the calculations performed by a plurality of neurons. These can be a time measurement circuit or a phase detection circuit.

[0061]

[0075] It will be appreciated that the actual implementation of the SNN generally comprises a very large number of neurons (e.g., hundreds of thousands or millions of neurons) and correspondingly a large number of synapses and input processing circuits. A large SNN can be divided into several smaller SNNs, and signals are communicated between the smaller SNNs using, for example, the interconnection 23 as shown in the embodiment of FIG. 2. The interconnection 23 can be configurable via the configuration input 22.

[0062]

[0076] Depending on the use case being handled by the SNN, there is different connectivity for each possible design. For example, a high level of connectivity can be input, encoder, input processing circuit 24, synapses 25, neurons 26, decoder, output. Additional connectivity is possible within the high level of connectivity such as feedback connections from each stage to other stages, feedforward connections from each stage to other stages, different connectivity structures such as crossbars, and different input and output reference points from different connectivity of each stage. This is a use case driven by the complex connectivity at the SoC level for the SNN. These complex connectivities create scenarios where it is a challenge to maintain consistent predictable behavior in view of device variations.

[0063] Implementation Forms of SNN

[0077] The SNN can be implemented as a software model operating on a processor, or the elements of the SNN (e.g., synapses and neurons) can be fabricated as hardware elements (where the SNN can also utilize some software for some functions). The hardware implementation form is realized using digital or analog circuits and can be constructed from individual components or manufactured as an integrated circuit. The fully digital implementation form of the SNN implements the functions of the neural network using synthesized digital functions. The design can become more complex with the trade-off of increased chip area and higher operating power, and the increase in chip area and power usually scales linearly with the complexity of the core logic.

[0064]

[0078] On the other hand, two mathematical operations performed by synapses and neurons, namely multiplication and integration respectively, can be efficiently implemented using analog circuits. Therefore, analog and mixed-signal implementation forms can potentially be very efficient in terms of power and chip area. The increased circuit complexity does not require a heavy trade-off with chip area and operating power. Since analog circuits can utilize information encoded in both the phase and amplitude of the input signal in addition to the signal timing that digital implementation forms can use, these analog implementation forms have a larger design space than digital ones. The increased design space results in the fact that chip area and operating power do not scale with the complexity of analog design. However, since the static timing analysis (STA) used for digital implementation forms is no longer sufficient, the increased design space also leads to increased problems regarding limiting the design space for mass production.

[0065]

[0079] Furthermore, a digitally enhanced analog mixed-signal design can be used for the SNN. By augmenting the mixed-signal design with digital logic, an optimal trade-off can be achieved in terms of performance vs. energy consumption vs. chip area.

[0066] Variability of SNN

[0080] All circuit implementation forms, such as complementary metal oxide semiconductor (CMOS) circuits, are subject to variations in their physical structure and over time. Based on the design techniques used, there are different implications and solutions to ensure the design operates correctly. Within the context of the present disclosure, these techniques are grouped together under the umbrella of calibration.

[0067]

[0081] SNN IC design is subject to unintended variations from the level of the complete IC to the level of individual transistors due to technological effects, lithography, electrical, thermal, and physical effects. Technological effects are related to process corners at the IC level, density variations at the level of the SNN array, and proximity and stress related at the transistor level. Similarly, lithography can affect the IC due to linewidth variations and can affect the transistors due to proximity effects. Circuit defects can result in leakage current and device mismatches. Supply voltage variations at the IC level, substrate noise can have an adverse effect on the array, and electrical effects such as jitter / crosstalk effects at the transistor level. Thermal effects can give rise to local and global gradients leading to dimensional variations. At the transistor level, thermal effects also add thermal noise. Physical effects can also lead to gradients with different global and local variations.

[0068]

[0082] Local and global dimensional variations can then lead to variations over time. These effects can vary over time scales ranging from picoseconds to long time scales such as days and years. Smaller time scale effects are due to effects such as electrical supply noise, substrate noise, and thermal kT noise. Longer time scale effects in the range from microseconds to milliseconds are due to temperature gradients and flicker noise. Further, longer period effects in the range from seconds to minutes are due to supply noise, drift, and flicker noise. Effects due to processing, lithography, and manufacturing issues are generally static over time.

[0069]

[0083] The drawback of using an analog mixed-signal circuit to implement SNN components is that the transfer function of the analog circuit can change through the processes, voltage, and temperature (PVT) variations described above. This means that subsequent copies of the same circuit may have different values for properties such as gain, bandwidth, offset, and amplitude, so a single SNN design may not be reproducible. Furthermore, when its supply voltage is unstable or the ambient temperature changes, even a single copy can have unstable performance. Therefore, a calibration method is required to ensure that each copy of a particular analog SNN implementation performs similarly regardless of any tolerances in manufacturing or variations in supply voltage or temperature.

[0070]

[0084] Circuit parameters that are generally sensitive to PVT variations include gain, bandwidth, capacitance, and resistance. Leakage current in devices not intended to conduct current and mismatches between devices that should be equal are further examples of PVT effects on analog circuits.

[0071]

[0085] In the present disclosure, these unintended variations due to the factors described above and other unintended variations in the operation of the SNN are simply collectively referred to as "variations" or "PVT variations" unless otherwise specified.

[0072] Calibration method

[0086] Calibration of digital SNNs can be performed using static timing analysis (STA). This takes into account spatial and temporal issues by creating a hierarchical model used with a timing model to limit errors in different paths of the circuit. Errors that are limited within limits in both space and time ensure that performance metrics such as operating frequency are met. Further analysis such as IR drop analysis, post-layout simulation, and parasitic extraction can be performed for both digital and mixed-signal designs. These compensate for fluctuations in the supply voltage being within the design metrics. Mixed-signal and analog-heavy designs such as ADCs, DACs, PLLs, etc. are generally calibrated using known calibration methods for each of the components.

[0073]

[0087] However, in the case of SNN designs with multiple analog, mixed-signal, and digital blocks connected to each other with complex connectivity that is use-case dependent, the calibration problem is more difficult. While there are known methods for calibrating the individual blocks used in such SNN designs, the present disclosure provides means for calibrating a complete SNN SoC with different building blocks, different connectivity, where the observability of internal nodes is limited, and / or with limited measurement capabilities with constraints for the observables.

[0074]

[0088] The observability at internal nodes sacrifices an increase in the area of the IC, which increases the cost. Additional observability can also introduce limitations to the actual functionality. The present disclosure attempts to provide a method for calibrating the entire SNN SoC with the minimum observability and measurements that can still reliably calibrate the entire system.

[0075] Defining the boundaries of variation

[0089] The bottom-up approach can be used to limit process variations at the level of individual components. How much variation can be expected can be determined from information supplied by the device manufacturer. Semiconductor device models generally contain information specifying how much variation is expected during a manufacturing run, how much variation is expected between devices manufactured in the same run, and the variation of each parameter with temperature. Circuit-level simulations can be performed to determine the effect of device parameters on network parameters. The result of such an analysis will be a table explaining the expected variation for each network parameter. As an example, the synaptic weights, where the variation is mostly determined by transistor mismatch, can vary by + / - 15% per synapse, the neuron threshold can vary by + / - 10 mV during a manufacturing run due to resistor inaccuracies, or the amplitude of the spikes generated by the input processing circuit can vary by + / - 30% over a temperature range of -50°C to 150°C due to transistor transconductance variations.

[0076]

[0090] The top-down approach can be used to limit errors due to variations from use cases for SNNs. How much variation in network parameters is acceptable varies for each application. Some network architectures are very sensitive to variations in synaptic weights, while others are more sensitive to variations in neuron thresholds. Statistical simulations can be used to determine the level of sensitivity for each network parameter for a particular SNN implementation. The results of such an analysis will be a table explaining the acceptable variations for each network parameter. As an example, it may be specified that synaptic weights must be accurate within 2%, neuron thresholds must match each other within 5%, and the amplitude of spikes generated by the input processing circuit must always be 100 nA + / - 10 nA. These results determine the resolution required in the adjustment circuits implemented within the network components, and sufficient resolution must be available to adjust each circuit parameter within its tolerance. Similarly, the range of the adjustment circuits must be large enough to offset the amount of variation expected from circuit-level simulations.

[0077] Observable points in SNN

[0091] Observable points within the SNN, i.e., points within the circuit where measurements can be made, are defined. Ideally, as few observable points as possible should be used. They also increase the IC area, make the design functionally more difficult, and increase the power consumption. Observable points can, in principle, be each input, internal node, and output of the building blocks of the SNN design at the SoC level. Some examples of observable points are as follows. A. Output of neurons. The frequency of the spike train generated by each neuron can be measured. Since the neuron output is essentially digital, this observation point is relatively simple to observe. B. Membrane potential of neurons. Since the neuron membrane potential is linked to the integration time constant in neurons, it is more difficult to observe the potential without changing its function. C. Input signal (voltage / current) to neurons. The neuron input signal can be observed according to the connectivity of the synaptic computing array. Since the computing array generally has complex connectivity, this creates a practical upper limit on the number of observable points that can be made in SNN design. D. Individual output of each synapse. The output of the synapse forms a complexly connected array structure such as a crossbar and a tree. Although each individual synapse can theoretically be observed, it is difficult to achieve observable points for individual synapses because there are usually a large number of synapses in a complex routing network. E. Internal nodes of synapses. Since the internal nodes of synapses are also parts of complex connectivity, they are more difficult to observe. F. Output of the input processing circuit. This can be observed for networks where they do not form parts of the complex connectivity of the computing array. In many use cases, they are distributed within the crossbar or tree-like connectivity in the synapse array, which creates limitations for observing them from the perspective of routing. G. Input of the input processing circuit. This can be observed for networks where the input does not form parts of complex connectivity. H. Internal nodes of the encoder. Depending on the complexity of the encoder, the internal nodes of the encoder may require calibration.

[0078]

[0092] Ideally, the measurement circuitry should use as few measurement circuits as possible, as they come at the expense of increased IC area and power consumption of the SNN SoC. Frequency measurements can be performed using known oscillators, counter circuits, etc. Voltage and current measurements can use an analog-to-digital converter (ADC) to convert the analog values to digital values for simpler measurements. The more frequently these measurements are made (regardless of whether it is the digital signal frequency or real-time voltage or current measurements), the greater the IC area and power hit for the system.

[0079]

[0093] A trade-off can be made between the observability points and the number of measurement circuits and the calibration accuracy and the performance of the resulting SNN. This trade-off can be evaluated by defining a cost function for the calibration optimization problem, which can take into account the chip area used for calibration and measurement circuits, the performance hit from additional observability points, the power consumed by the calibration observability circuits during non-calibration times, the power consumed by the calibration and measurement circuits, and the routing complexity. The relative importance of different cost functions must also be considered.

[0080]

[0094] The design of each sub-component has a performance-versus-area and performance-versus-power consumption trade-off, and the SNN use case must be considered in terms of the network performance-versus-sub-component performance trade-off.

[0081]

[0095] The opportunity at which calibration is performed also needs to be considered. If any calibration steps need to be performed while the SNN is operating, they must be done in such a way that the operation of the SNN is not disrupted. One way to do this is to temporarily buffer any incoming signals to ensure that the SNN is not processing any data at the moment, and then use this time window to perform one or more calibration steps.

[0082]

[0096] Another way to calibrate the SNN without disturbing its normal operation is to utilize the redundancy of the crossbar architecture. For this purpose, the SNN neuro-synaptic array can be designed to be slightly larger than the largest SNN design that is expected to be programmed internally, and thus there is always at least one neuro-synaptic column that remains unused. This spare column can then be calibrated independently of the rest of the chip. When this is done, the interconnects that can be configured such that the spare column is placed within the active network are modified, while the column that was previously in that position is disconnected and calibrated. A convenient moment to perform this act is when the state of the neurons is currently reset and the network state will not change when different columns are inserted into the network in place of the active columns, so it is at the moment when the neurons of the column generate an output signal.

[0083]

[0097] Online calibration can be performed at regular intervals or only when the environmental conditions change. In a typical battery-powered system, the supply voltage varies slowly on a time scale of the order of minutes or hours as the battery discharges or is recharged. Quick glitches on the supply line can be filtered using passive components such as capacitors and inductors as well as by the automatic control actions of the voltage regulator circuit. Most systems are also generally moderately stable in terms of temperature on a time scale of the order of tens of seconds or minutes. A monitoring circuit with an SNN can measure the supply voltage and temperature while the SNN is active and be used to trigger a calibration round only when the environmental conditions change by a predetermined amount.

[0084]

[0098] Calibration is required at different timelines of mass-produced dies. For example, calibration can be performed during startup, periodically to cancel long-term drift, based on temperature measurements to cancel temperature effects, based on supply voltage measurements to compensate for supply voltage effects, etc. When designing the calibration process, the time spent on calibration for the performance of the use case and the amount of calibration data that needs to be stored must be considered. A cost function can be used to evaluate the area overhead of redundancy in processing, as well as the power and time overhead for calibration during startup, calibration to cancel long-term drift, and calibration to cancel temperature effects.

[0085]

[0099] FIG. 3A is a simplified schematic diagram of an input shaping circuit 30 that can be used as the input processing circuit 23 in SNN20 of FIG. 2 and SNN60 of FIG. 6. The input shaping circuit 30 extracts useful information from the SNN input signal 33 and generates an input signal 39 that is processed in a state modulated by the information from which the spike train was extracted. Depending on the application example, the useful information can be encoded in the amplitude, frequency, phase, duty cycle, digital code, or any other property of the signal of the input signal.

[0086]

[0100] As an example, for its input signal with frequency f in and its output frequency f out the input shaping circuit 30 that modulates will have a transfer function given by f out = f in ·n + f carrier where f carrier is the carrier frequency and n is a constant. When the circuit modulates its output amplitude A in with its input signal amplitude A out its transfer function will be given by A out = A in ·n, where n is a constant.

[0087]

[0101] The exemplary input shaping circuit 30 shown in FIG. 3A includes an information extraction circuit 35 and a pulse generation circuit 38. The information extraction circuit 35 receives the SNN input signal 33 and sends the output of amplitude 36A, frequency 36B, and several pulses 36C to the pulse generation circuit 38, which generates a processed input signal 39 with a spike train. FIG. 3B shows a graph of an exemplary SNN input signal 33, and FIG. 3C shows a graph of an exemplary processed input signal 39 (showing the variation of amplitude plotted against time on the horizontal axis).

[0088]

[0102] The non-ideality of the circuit can affect the input shaping circuit 30 in many different ways, such as gain errors that affect the amplitude, delay errors that affect the phase, or local reference errors that affect the processing of digital input codes. A compensation circuit can be implemented to cancel the effects of circuit non-ideality by adding features such as an adjustable input 34 to adjust the gain, frequency or phase, circuit delay, or local reference.

[0089]

[0103] Similarly, the pulse generation circuit 38 will have non-idealities that cause errors in the amplitude, decay time, or inter-spike timing properties such as frequency, phase, or density of the output spike train. The spike generation circuit 38 can be equipped with a compensation circuit that uses adjustment inputs such as rise time 37A and decay time 37B, amplitude 37C, and timing 37D adjustment signals to correct the parameters that are most sensitive to variations and related to the intended signal processing function.

[0090]

[0104] FIG. 4 is a simplified schematic diagram of the synapse 25 that can be used in the SNN of FIG. 2 and the SNN60 of FIG. 6. The synapse 25 receives a processed input signal 41 (from the input processing circuit 24) and applies a weight 42 (e.g., multiplies the input signal 41 by the weight 42) to generate a synapse output signal 46.

[0091]

[0105] The transfer function of the synapse is nominally I out =I in· Given by (w / wmax), I out is the output current, and I in is the input current, w is the weight programmed in the synapse, and w max is the maximum possible weight value. The non-ideality of the circuit causes additive and multiplicative errors such that the transfer function becomes I out = α·I in ·(w / w max ) + β, where α is the multiplicative error and β is the offset error. The adjustment circuit can be added to the synapse 26 to cancel these non-idealities by implementing an adjustable gain factor input 43 that can cancel α and an adjustable offset input 44 that can cancel β. The adjustment of the weight 42 can also be used to adjust the synapse 25.

[0092]

[0106] FIG. 5 is a simplified schematic diagram of the neuron 26 that can be used in the SNN of FIG. 2 and the SNN60 of FIG. 6. In this example, the neuron 26 includes an integrated circuit 53 and a comparator circuit 57. The integrated circuit 53 receives one or more synapse output signals as input 51, integrates (accumulates) this input, and sends the resulting membrane potential signal 54 to the comparator circuit 57. The comparator circuit 57 compares the received membrane potential 54 with a reference potential 55 that operates as a spike threshold. The comparator circuit 57 generates a spike as the neuron output signal 59 when the membrane potential exceeds the spike threshold. The spike also operates as a reset signal 58 to reset the integrated circuit 53.

[0093]

[0107] A typical implementation of the electronic neuron 26 may use a capacitor as the integrating element 53. When manufactured on an integrated circuit, the value of the capacitor can generally vary by about 20% during a manufacturing run. Further, multiple copies of nominally the same capacitor on a single chip can vary from each other by about 5%. Additionally, the capacitance can have a temperature coefficient that adds a variation on the order of several percent over a typical commercial or industrial temperature range, depending on the dielectric material used to manufacture the capacitor. All of these variations result in the integration constant of the neuron 26 being incompletely defined, which will introduce uncertainty into the signal processing function of the SNN.

[0094]

[0108] When the integrated value reaches the spike threshold, the neuron integrates its input signal 51 and generates a neuron output signal 59. The output spiking frequency of the neuron is f out =g / (C·V)∫i in dt can be described, where g is the input gain, C is the capacitance, V is the threshold voltage, and i in is the input current. Circuit non-idealities can affect the gain g, the slope of the integrator defined by C, or the spike threshold V. An adjustment circuit can be added to the neuron 26 to cancel the effects of each of these non-idealities. Since both the gain and slope of the integrator affect the transfer function of the neuron as multiplicative errors, both can be compensated simultaneously by adding an adjustable gain input 52 to the integrator 53. Errors in the spike threshold can be compensated by adding a reference potential offset value 55 and / or an adjustable spike threshold 56 to the comparator 57.

[0095]

[0109] FIG. 6 is a simplified schematic diagram of an SNN 60 that includes an offset current generator 64 and an analog-to-digital converter 66 for use in calibrating the SNN and also includes the SNN 20 of FIG. 2. Other features of the SNN 20 of FIG. 2 may also apply to the SNN 60 of FIG. 6. Similar to the embodiment of FIG. 2, each row of synapses 25 receives an input signal 27 processed from one input processing circuit 24. Each synapse 25 applies a weight to the received and processed input signal 27 and generates a synapse output signal 28.

[0096]

[0110] The array is extended by adding a row of offset current generators 64, one for each synapse 25 in a row of synapses. Each offset current generator (OCG) 64 generates an offset current 65. One OCG 64 and a plurality of synapses 25 per column of the array. An offset current 65 from one of the OCGs 64 and a synapse output signal 28 from a column of synapses are received by one of the neurons 26. Each neuron 26 integrates the received synapse output signal 28 and generates a neuron output signal 29 when the integrated value reaches a predetermined threshold, and thus the neuron output signal 29 takes the form of a spike train.

[0097]

[0111] The OCG 64 can be viewed as an additional row of synapses that adds a constant signal instead of a spike train to each neuro-synaptic column. Each OCG 64 generates a constant offset current output 65 controlled by their input code. The ideal transfer function of the OCG 64 is I out =I ref ·(code / code max ). However, circuit non-idealities can add error terms and factors and change the transfer function to I out =α·I ref ·(code / code max )+β. To address these non-idealities, the circuit can be given an adjustment mechanism that adds adjustable gains and offsets sufficient to cancel the effects of the gain error α and the offset error β.

[0098]

[0112] FIG. 7 shows an example of an offset current generator 64 with gain adjustment implemented by adding an additional gain stage 72, such as an analog amplifier or a scaling current mirror, having a DC current source 71 and an externally adjustable gain input 74. The offset adjustment 75 can be implemented by an adjustable constant signal source whose output is added to the offset current source output 65. The OCG 64 can be used to provide a bias current to a neuromorphic array (i.e., synapses 25 and neurons 26) during operation and can also be used to assist in calibration as described below. The OCG 64 is preferably included in the same IC as the neuromorphic array of the SNN 60, but alternatively, they can be implemented and separately connected to the neuromorphic array when needed.

[0099]

[0113] Referring again to FIG. 6, the SNN 60 also includes an analog-to-digital converter (ADC) 66 that can be connected to the output of any of the neuromorphic columns, and each column includes an OCG 64 and a plurality of synapses 25. The ADC 66 is connectable (e.g., via a switching connection or a multiplexer or circuitry within the ADC that selects the input) to receive an analog signal generated by one of the neuromorphic columns, i.e., a composite signal from the OCG 64 and the plurality of synapses 25 in the column of the synapse array. The ADC 66 converts the received analog signal to a digital signal 67 for measurement in the digital domain. The ADC 66 is preferably included in the same IC as the neuromorphic array of the SNN 60, but alternatively, it can be implemented and separately connected to the neuromorphic array when needed. Instead of having a single ADC 66 that is switchable to receive inputs from different sets (columns) of synapses 25 and the OCG 64, the SNN 60 can alternatively be provided with a plurality of ADCs that can operate in parallel, each receiving inputs from different sets of synapses and the OCG. The plurality of ADCs results in a more complex and costly implementation form, but calibration measurements can be performed in parallel and thus more quickly.

[0100]

[0114] SNN60 may also include components for measuring the frequency of the neuron output signal 29 from each neuron 26, i.e., for measuring the frequency of the spikes generated by the neuron 26. The embodiment of FIG. 6 includes a frequency measurement circuit 68, which can be connected (e.g., via a switching connection or multiplexer or other circuit within the frequency measurement circuit 68 that selects the input) to receive the neuron output signal 29 from the neuron 26. The frequency measurement circuit 68 generates an output 69 indicating the frequency of the spike train generated by the neuron 26 being measured. The neuron output frequency can be measured by comparing the neuron output signal 29 to a reference clock using a counter or other circuit.

[0101]

[0115] FIG. 8 is a simplified schematic diagram of the overall structure of an SNN with an analog or mixed-signal SNN81, an analog-to-digital converter 66 with an external reference 83, a frequency measurement circuit 68, a processing element 84, and a memory 85. All of these components can be implemented on the same IC or SoC, or can be implemented in one or more separate ICs.

[0102]

[0116] The SNN81 of FIG. 8 can have the same structure and details as the SNN60 of the embodiment of FIG. 6 (except that the ADC66 and the frequency measurement circuit 68 are shown separately in FIG. 8). The ADC66 can be connected to receive analog signals as inputs from the SNN81 (in the same manner as described above for FIG. 6) and convert them to digital signals using an external reference signal 83. The digital signals output from the ADC66 are received by the processor 84, which executes software or firmware to perform calibration procedures and store the results in the memory 85. Two or more ADC66s can be used to speed up the calibration procedures, as described above.

[0103]

[0117] The frequency measurement circuit 68 is connectable to receive a neuron output signal as an input from the SNN81 (in the same manner as described above with respect to FIG. 6) and generate an output based on the input frequency. The output from the frequency measurement circuit 68 is also received by the processor 84. Two or more frequency measurement circuits 68 can be used to speed up the calibration procedure. Alternatively, the frequency measurement can be performed directly by the processor 84, which receives the neuron output signal 29 directly from the SNN81.

[0104]

[0118] The processor 84 can be implemented as software, a field programmable gate array (FPGA), an application specific circuit (ASIC), or a microprocessor that executes other types of processing elements suitable for executing the calibration procedures described herein. A single processor can be used to execute the calibration procedure, or multiple processors can be used and the calibration procedure can be divided among them.

[0105]

[0119] The calibration procedure comprises several stages where the relevant parameters are measured through the ADC66 and the network parameters are adjusted to cancel out PVT variations. Network elements (e.g., the input processing circuit 24, synapses 25, neurons 26, and OCG 64) can have adjustment mechanisms described herein that can be used to modify their relevant parameters to correct for element variations. These adjustments will generally be controlled through a digital interface. The values of the adjustments to be applied will then be stored in the digital memory 85. However, other methods are possible such as laser trimming where small cuts are made in the on-chip device or analog trimming where a voltage is accumulated on well-insulated capacitors.

[0106]

[0120] The calibration procedure described can be modified to account for manufacturing variations and differences in sensitivity to environmental factors, depending on the circuit in question. As an example, if the neuron design is largely based on the current mirror type of circuit, only a single calibration step may be required before the neuron is ready for use. If the neuron is based on a circuit that depends on the mutual conductance of transistors, it will require in-operation calibration. For each network component type, and thus for each step of the calibration procedure, a decision must be made as to whether to include it during calibration steps prior to first use, during the online calibration procedure, or both.

[0107]

[0121] All of the various steps of the calibration procedure have a certain cost in terms of chip area, power consumption, time, and memory requirements. These costs need to be taken into account when designing the various SNN components. For each component, a trade-off analysis needs to be performed to balance the cost incurred against the gain achieved.

[0108]

[0122] As an example, since synapses will generally occupy most of the area of a chip, adding tuning features to synapses will incur a significant chip area penalty for the entire neuromorphic array. Implementing gain and offset tuning features on each individual synapse may also become impractical from a wiring perspective, leading to a decision to minimize the number of tuning bits, or even to abolish this feature altogether by choosing to measure only the non-ideal behavior of each synapse and take it into account when programming the weights. For example, if it is found that one particular synapse has a gain error of -10%, this can be compensated for by increasing the weight value programmed into it by 10%.

[0109]

[0123] Another example of a trade-off in the adjustment circuit design is at the neuron threshold. In many cases, this will be a constant voltage, which will ultimately be derived from an on-chip voltage reference such as a bandgap reference circuit. Making such a voltage adjustable for each neuron may require implementing a digital-to-analog converter (DAC) for each individual neuron. High-resolution DACs generally require a large chip area and a large bias current to cancel out the effects of mismatches and noise. The number of DAC bits implemented, and thus the amount of additional chip area and power consumption added, must be considered against the expected gain in signal processing accuracy due to the additional threshold voltage accuracy.

[0110]

[0124] In contrast, making the on-chip ADC adjustable to high precision can be beneficial when there is only one ADC and thus no chip area penalty occurs multiple times across the chip. Additionally, since the ADC is only used during calibration and can then be turned off, the ADC can have relatively relaxed requirements for power consumption.

[0111]

[0125] When summing the effects of PVT variations for all circuits in a large array, the overall transfer function of the network becomes impossible to control precisely. Therefore, a modified crossbar circuit with calibration elements is proposed, along with procedures aimed at ensuring that the relevant parameters of each network element are controlled to a pre-determined level of accuracy.

[0112]

[0126] The overall procedure for calibrating an analog or mixed-signal SNN depends on the exact configuration of the network. As an example, the following calibration procedure is described for the network structures shown in FIGS. 6 and 8. The calibration procedure begins by calibrating the ADC 66 using an externally well-characterized signal source 83. The input / output transfer function of the ADC, once successfully characterized in this way, is stored and can be used to calibrate the components of the SNN 60 / 81.

[0113]

[0127] FIG. 9 is a flowchart showing the steps of the ADC calibration procedure 90. In step 91, the external reference 83 is connected to the ADC 66, which converts the external reference 83 received at the digital output. In step 92, the ADC 66 is adjusted so that the digital output is correct, and in step 93, the external reference 83 is disconnected. In step 94, the ADC 66 is used to calibrate the SNN 81 as described below.

[0114]

[0128] FIG. 10 is a flowchart showing the high-level stages of the calibration procedure for the SNN. First, the OCG 64 is calibrated (step 101). The neuron 26 is then calibrated, preferably using the previous calibration of the OCG 64 (i.e., using the transfer function determined prior to the OCG 64 or using the OCG 64 adjusted to reduce PVT variations) (step 102). Then, the input processing circuit 24 (step 103) and the synapse 25 (step 104) are calibrated, preferably using the previous calibrated OCG 64 and neuron 26 (i.e., using the transfer function determined prior to the OCG 64 and neuron 26 or using the OCG 64 and / or neuron 26 adjusted to reduce PVT variations).

[0115]

[0129] As described below, calibration at each stage involves measuring the transfer functions of the relevant elements of the SNN and may also involve adjusting the elements to correct or reduce the PVT variations of the elements if they are equipped with facilities for individual adjustment. As previously explained, the trade-off between reduced PVT variations and greater accuracy on the one hand and increased chip area and power consumption on the other may indicate that some elements of the SNN have limited or no facilities for individual adjustment.

[0116]

[0130] FIG. 11 is a flowchart of a procedure for calibrating the OCG64 of SNN81. In step 111, the OCG64 is set to output an offset current signal 65 equal to a predetermined reference output current for the calibration procedure. The ADC66 is then connected to receive the offset current signal 65 from each OCG64 and convert the received analog signal 65 into a digital signal. Due to PVT variations, these offset current signals 65 will be different from the nominal values designed to be generated by the OCG. The digital signal generated by the ADC66 is received by the processor 84, which can calculate the transfer function of each OCG64 and determine adjustment signals such as a gain adjustment signal 74 and an offset adjustment signal 75 to correct for any PVT variations of the relevant OCG64.

[0117]

[0131] After these measurements, the OCG64 can be adjusted as necessary so that their values are closer to the nominal values. The ultimate limit on the accuracy of the offset current signal 65 generated by the OCG64 is the accuracy and resolution of the ADC66 and the range and resolution of the adjustment mechanism of the OCG64. The transfer function and / or adjustment signals can be stored in the data storage memory 85.

[0118]

[0132] An additional benefit of calibrating the OCG in this way is that this step will simultaneously cancel out any leakage current flowing through the synapse array. The synapses 25 will be inactive and can be turned off during this step, so they should not conduct any current, but in reality, the synapses will draw leakage current. Since this leakage current cannot be distinguished from any signal current, it must be canceled to avoid interfering with the normal operation of the SNN60. Since the leakage synapses 25 are connected to the OCG64 and the ADC66 during the OCG calibration step, any leakage current present in the array is measured by the ADC66 and compensated when the OCG is adjusted.

[0119]

[0133] FIG. 12 is a flowchart of a procedure for calibrating neuron 26 of SNN81. If the offset current signal 65 generated by OCG64 is well characterized, it can be used as a reference input for calibrating neuron 26. In step 121, OCG64 (previously calibrated) is set to an output offset current 65 having a predetermined value suitable for the neuron calibration procedure. In step 122, the output frequency of neuron 26 (i.e., the frequency of the spikes generated by the neuron) is measured by frequency measurement circuit 68. Since a digital system generally includes an accurate clock that can be used for measurement, the neuron output frequency can usually be accurately measured. By measuring the output frequency of each neuron as a response to a known input signal (i.e., the calibrated offset current signal 65), the transfer function of each of neurons 26 can be accurately determined by processor 84.

[0120]

[0134] In step 123, the measured neuron output frequency is received by processor 84 for each neuron 26, and any deviation from the expected value can be detected, and thus, a deviation in the transfer function of neuron 26 caused by PVT variations can be determined. If neuron 26 is provided with adjustment mechanisms such as input gain adjustment 52, reference potential offset value 55, and / or spike threshold adjustment 56, processor 84 can calculate these adjustments and adjust the individual neurons to account for these PVT variations. The transfer function of neuron 26 and / or the adjustment signal for the neuron can be stored in data storage memory 85.

[0121]

[0135] FIG. 13 is a flowchart of a procedure for calibrating input processing circuit 24 and synapse 25 of SNN81. If OCG64 and neuron 26 are characterized, for example, by using the calibration procedure described above, this information can be used for calibrating synapse 25 and input processing circuit 24.

[0122]

[0136] In step 131, a predetermined input signal 61 (such as a spike train having known amplitude and frequency) is applied to one of the input processing circuits 24, and in step 132, the weights of the synapses 25 are set to predetermined values. In step 133, the frequency of the neuron output signal 29 generated by the neuron 26 is measured by the frequency measurement circuit 68, and these measured values are received by the processor 84.

[0123]

[0137] Since both the input signal 61 and the input / output transfer function of each neuron 26 are known, the combined transfer function of the input processing circuit 24 and the synapse 25 through which the input signal flows into each neuron 26 can be determined by the processor 84. This transfer function can then be corrected by adjusting the relevant input processing circuit 24, the relevant synapse 25, or both.

[0124]

[0138] Since both the input processing circuit 24 and the synapse 25 will be affected by PVT variations, both will contribute to the inaccuracies measured in step 133. The two contributions can be separated by measuring the average synapse output signal across the rows of synapses 25 connected to a single processing circuit 24. The PVT variations contributed by the synapses 25 are substantially averaged out of the measured row signals, leaving only the PVT variations contributed by the input processing circuit 24.

[0125]

[0139] The synapse output signals from the rows of synapses 25 will be received by the rows of neurons 26. The output frequencies of these neurons are measured in step 133 and can be averaged to substantially isolate the PVT variations contributed by the relevant input processing circuit 24. In step 134, the input processing circuit 24 can be adjusted, for example, by adjusting the input gain 34, the rise time 37A, the decay time 37B, the amplitude 37C, and / or the timing 37D so as to derive a transfer function within the desired tolerance.

[0126]

[0140] In step 135, after adjustment of the input processing circuit 24, the measured neuron output frequency for one of the neurons 26 can be used by the processor 84 to determine the transfer function of the relevant synapse 25 (i.e., the synapse 25 through which the input signal flows from the input processing circuit 24 to the measured neuron 26). In step 136, the transfer function of the relevant synapse 25 can be used to adjust the synapse 25, for example, by using the gain adjustment signal 43 and / or the offset adjustment signal 44, or by adjusting the weight 42 if the synapse has these facilities. The transfer functions of the input processing circuit 24 and the synapse 25 and / or the adjustment signals for the input processing circuit 24 and / or the synapse 25 can be stored in the data storage memory 85.

[0127]

[0141] An alternative for the calibration procedure of FIG. 13 is that instead of measuring the frequency of the neuron output signal 29 in steps 133 and 135, the output voltage or current of the relevant synapse 25 is routed to the ADC 66 and measured by the ADC in the same manner as used in the OCG calibration procedure. Thereby, the transfer function between the input processing circuit 24 and the synapse 25 can be directly measured by the ADC 66 instead of using the neuron 26 to generate a spike train that is then measured by the frequency measurement circuit 68. When using this alternative, the predetermined input signal 61 applied to the input processing circuit 64 can be a constant signal instead of a spike train.

[0128]

[0142] The calibration procedure described above can be modified to suit many other network architectures. As an example, another network topology can be designed with an encoder instead of an input shaping circuit. Such a topology would allow for a direct connection to the SNN without any intermediate signal processing. Calibrating such a topology follows a pattern similar to what has been described, but the calibration procedure of the input processing circuit is used to calibrate the encoder. The content of this step depends on the transfer function of the encoder and its specific sensitivity to processing and temperature variations.

[0129]

[0143] In the case of an encoder that generates a spike train with a frequency proportional to the amplitude of the input signal voltage, the calibration step of the encoder may consist of applying a well - defined static input signal to the encoder and measuring the resulting frequency of the current flowing through the synaptic train. The voltage - to - frequency conversion of the encoder can then be adjusted until it matches a pre - determined value.

[0130]

[0144] However, it is important to consider all possible non - ideal behaviors. For example, the encoder may be plagued by DC leakage current at all times when the signal is not being processed. In that case, the leakage of the encoder needs to be taken into account during the offset calibration step to ensure that the leakage does not interfere with any subsequent calibration steps.

[0131]

[0145] The calibration procedure cannot distinguish between variations caused by tolerances during the manufacturing process that are constant for each individual copy of the circuit and variations caused by changes in the supply voltage and temperature that change during the lifetime of the circuit. Therefore, the calibration procedure needs to be performed at least once to compensate for manufacturing variations and again, either fully or partially, when environmental factors change.

[0132]

[0146] Inaccuracies in manufacturing generally lead to variations in circuit performance that remain constant throughout the life of the product. A typical example is the offset in a matched pair of CMOS transistors commonly used in sub-circuits such as current mirrors. Designers attempt to have two transistors that are exactly equal, but manufacturing tolerances will cause the two transistors to be slightly different. Common reasons for such mismatches are the thickness of the lines drawn on the chip, the random scattering of implanted atoms, and the variation in the thickness of the oxide layer grown on the wafer.

[0133]

[0147] In a current mirror designed to accurately copy an incoming current to its output, the expected transfer function is I out =I in . Manufacturing tolerances will change this to I out =n*I in , where n is a number not equal to that determined by the specific manufacturing inaccuracies present in an individual copy of the circuit. Since the physical effects that influence this factor are constant and do not change, a single calibration step immediately after the circuit is manufactured will be sufficient to bring n closer to unity for the life of the circuit.

[0134]

[0148] Other circuit parameters are sensitive to environmental factors such as supply voltage and temperature. One example is the transconductance of a CMOS transistor, which is determined by the gate-source voltage, drain-source voltage, physical characteristics of the device, and temperature. Except for the physical characteristics, any of these parameters are likely to change at any moment the transistor is in use, so it is not possible to compensate for any variations in advance. Any calibration that is required will have to be performed while the device is operating.

[0135]

[0149] Depending on the expected variations of each parameter and the effect each parameter has on the overall performance of the chip, several steps are carried out at different instants during the chip's life cycle. The first test procedure, generally called "wafer sort", is carried out at the end of the wafer manufacturing process, and the essential functions of each chip are tested. Any chip determined at this point to have variations in important parameters that are greater than the range of its associated adjustment mechanism is discarded. The remaining chips are then packaged and measured again in the final test step. This step will include an initial calibration step aimed at canceling out as much as possible all manufacturing variations in circuit parameters that are expected to remain constant over the life of the chip. The values obtained are stored in memory that may be incorporated into the same chip package, or it is stored externally but may still be associated with the individual chips measured through management means such as serial number tracking.

[0136]

[0150] Accordingly, the present invention discloses a system and method for calibration of SNNs.

[0137]

[0151] Note that any features of any of the embodiments disclosed herein may be combined in a suitable manner.

Claims

1. Multiple input processing circuits are provided, each input processing circuit having an input for receiving a spiking neural network input signal and configured to apply a transfer function to the input signal to generate a processed input signal. Multiple offset current generators are configured to generate an offset current signal at a predetermined level. Multiple synapses are connected to each other to receive an input signal processed from one of the input processing circuits, and each synapse is configured to apply a predetermined weight to the processed input signal in order to generate a synaptic output signal. Multiple neurons are connected to receive synaptic output signals from a subset of synapses and offset current signals from one of the offset current generators, and each neuron is configured to generate a neuronal output signal in response to the received synaptic output signals and offset current signals. An analog-to-digital converter having an input, the input being connectable to receive an offset current signal from one of the offset current generators, and configured to convert the received offset current signal into a corresponding digital output signal. A spiking neural network equipped with [a specific feature].

2. Each of the offset current generators is individually adjustable to adjust the offset current signal. Preferably, the spiking neural network according to claim 1 further comprises a first processing circuit connected to receive the digital output signal generated by the analog-to-digital converter and configured to generate an adjustment signal for adjusting one of the corresponding offset current generators to generate an adjusted offset current signal.

3. The system further includes a frequency measurement circuit connected to receive a neuron output signal from one of the neurons and generate an output indicating the frequency of the received neuron output signal. Preferably, the neurons are individually tunable to adjust the input gain, spike threshold, and / or reference potential offset value of the neurons. More preferably, the spiking neural network according to claim 1 or 2 further comprises a second processing circuit connected to receive the output from the frequency measurement circuit and configured to generate an adjustment signal for adjusting one of the corresponding neurons.

4. Each of the synapses is adjustable to adjust the input gain and / or offset value of the synapse, and / or The spiking neural network according to claim 1 or 2, wherein each of the input processing circuits is adjustable to adjust the input gain, spike threshold, and / or timing values ​​of the input processing circuit.

5. The spiking neural network according to claim 1 or 2, wherein the analog-to-digital converter is connectable to receive a synaptic output signal from one of the synapses and is configured to convert the received synaptic output signal into a corresponding digital synaptic output signal.

6. The spiking neural network according to claim 1 or 2, further configured to provide a calibration mode in which one or more of the spiking neural network input signals are set to a predetermined value.

7. The spiking neural network according to claim 6, further comprising: a third processing circuit connected to a frequency measuring circuit to receive a signal from a frequency measuring circuit indicating the frequency of a neuronal output signal of one of the neurons during the calibration mode; the neuron being connected to receive a synaptic output signal from one of the synapses connected to receive an input signal processed from one of the input processing circuits, wherein the third processing circuit is configured to generate adjustment signals for adjusting the input processing circuits and / or the synapses.

8. The spiking neural network according to claim 6, further comprising a fourth processing circuit connected to receive the digital synapse output signal during the calibration mode and configured to generate adjustment signals for adjusting the input processing circuit and / or the synapse.

9. A method for calibrating a spiking neural network, the spiking neural network comprising: a plurality of offset current generators configured to generate offset current signals; a plurality of input processing circuits configured to generate processed input signals; a plurality of synapses configured to apply weights to the processed input signals to generate synaptic output signals; and a plurality of neurons connected to receive one of the offset current signals and synaptic output signals from the plurality of synapses and configured to generate neuron output signals, The aforementioned multiple offset current generators generate multiple offset current signals at predetermined levels, Converting the offset current signal to a corresponding digital output signal, A method for providing this.

10. Adjusting the offset current generators based on the digital output signal to generate adjusted offset current signals from each of the offset current generators. The method according to claim 9, further comprising:

11. Receiving the multiple adjusted offset current signals in the multiple neurons, To measure the frequency of the neuron output signal, Adjusting the input gain, spike threshold, and / or reference potential offset value of each neuron based on the adjusted offset current signal received by each neuron and the measured frequency of the neuron output signal generated by each neuron. Furthermore, Preferably, the method of claim 10, further comprising calculating a transfer function for the neurons, wherein the transfer function for each neuron is based on the adjusted offset current signal received by the neuron and the measured frequency of the neuronal output signal of the neuron.

12. The first of the input processing circuits is given a spiking neural network input signal set to a predetermined value, Setting the weight of one of the first synapses to a predetermined value, The first synapse among the plurality of synapses receives the input signal processed by the first input processing circuit, and generates a corresponding synaptic output signal in response. One of the neurons receives the synaptic output signal from the first synapse, and receives the adjusted offset current signal from one of the offset current generators. Measuring the frequency of the neuron output signal generated by the first neuron, Adjusting the first input processing circuit and / or the first synapse in order to adjust the transfer function between the first input processing circuit and the first synapse based on the spiking neural network input signal and the measured neuron output signal frequency. The method according to any one of claims 9 to 11, further comprising:

13. The first of the input processing circuits is given a spiking neural network input signal set to a predetermined value, Setting the weights of the first subset of synapses to predetermined values, The first subset of synapses receives the input signal processed by the first input processing circuit, and in response generates a corresponding synaptic output signal from each synapse of the first subset of synapses, The neuron receives the synaptic output signal from the first subset of synapses, The frequency of each neuronal output signal generated by the aforementioned neuron is measured, Adjusting the first input processing circuit to adjust the transfer function of the first input processing circuit based on the measured frequencies of the spiking neural network input signal and the neuron output signal. It further comprises, preferably, The first of the input processing circuits is given a spiking neural network input signal set to a predetermined value, The first of the synapses in the first subset receives the input signal processed by the first input processing circuit and generates a corresponding synaptic output signal in response, One of the neurons receives the synaptic output signal from the first synapse, and receives the adjusted offset current signal from one of the offset current generators. Measuring the frequency of the neuron output signal generated by the first neuron, Tuning the first synapse to adjust the transfer function of the first synapse based on the spiking neural network input signal and the measured neuron output signal frequency. The method according to any one of claims 9 to 11, further comprising:

14. The first of the input processing circuits is given a spiking neural network input signal set to a predetermined value, Setting the synaptic weight of one of the first synapses to a predetermined value, The first synapse receives the input signal processed by the first input processing circuit and generates a corresponding synaptic output signal in response, Converting the synaptic output signal to a corresponding digital output signal, The first input processing circuit and / or the first synapse are adjusted in order to adjust the transfer function between the first input processing circuit and the first synapse based on the spiking neural network input signal and the digital output signal. The method according to any one of claims 9 to 11, further comprising:

15. The first of the input processing circuits is given a spiking neural network input signal set to a predetermined value, Setting the weights of the first subset of synapses to predetermined values, The first subset of synapses receives the input signal processed by the first input processing circuit, and in response generates a corresponding synaptic output signal from each synapse of the first subset of synapses, Converting the synaptic output signal from the first subset of synapses to a corresponding digital output signal, To adjust the first input processing circuit in order to adjust the transfer function of the first input processing circuit based on the spiking neural network input signal and the digital output signal, It further comprises, preferably, The first of the input processing circuits is given a spiking neural network input signal set to a predetermined value, The first of the synapses in the first subset receives the input signal processed by the first input processing circuit and generates a corresponding synaptic output signal in response, Converting the synaptic output signal from the first synapse to a corresponding digital output signal, Adjusting the first synapse in order to adjust the transfer function of the first synapse based on the spiking neural network input signal and the digital output signal. The method according to any one of claims 9 to 11, further comprising:

Images