Systems and methods for efficient feature-centric analog two-spike encoders

Abstract

A signal processing circuit for a spiking neural network, the signal processing circuit comprising an interface for converting an analog input signal into a corresponding spike-time representation of the analog input signal, the interface comprising: an analog-to-information (A / I) converter configured to generate a modulated signal representative of one or more features of the analog input signal; a feature detector circuit configured to compare the modulated signal with a reference signal representative of the reference feature and configured to generate an error signal indicative of a difference between the modulated signal and the reference signal; a feature extractor circuit having an input for receiving the error signal and comprising a locked loop circuit configured to generate an output signal representative of one or more occurrences of the features represented by the modulated signal; and an encoder circuit configured to encode the output signal into a spike train for input to the spiking neural network.

Description

[Technical Field]

[0001] The present disclosure relates generally to automatic signal recognition techniques, and more particularly to systems and methods for energy-efficient feature-centric analog-to-spike encoders, and in particular, for feature-centric analog-to-spike encoders for spiking neural networks. [Background technology]

[0002] Sampling is the process of converting a signal (e.g., a continuous function of time or space) into a series of values (a discrete function of time or space). The sampling rate is the average number of samples taken during one second of the signal.

[0003]

[0003] A discrete signal can be considered sparse in a representation domain if the number of non-zero values in that domain is much shorter than the total signal length. A common situation in the case of real-world signals is that the number of significant coefficients is small compared to the number of other components. These coefficients can be ignored or set to zero. In other words, a sparse signal can be efficiently represented using a small number of non-zero elements.

[0004]

[0004] Sparse signals can be accurately represented at a lower sampling rate compared to non-sparse signals. The Nyquist-Shannon theorem states that the sampling rate for a non-sparse signal must be at least twice the highest frequency component, but for sparse signals, the sampling rate can be significantly lower due to the concentrated energy in a few coefficients.

[0005]

[0005] In many (natural) signal and sensor applications, for example, visual, acoustic, environmental, and medical monitoring systems, signals are not necessarily sparse due to corruption from noise, interfering signals, or circuit faults, and therefore standard sub-Nyquist sampling techniques may not be applicable. Sub-Nyquist sampling techniques, also known as compressive sensing or compressive sampling, are methods used to acquire and reconstruct signals at sampling rates significantly lower than those prescribed by the Nyquist-Shannon theorem.

[0006] Furthermore, in many of these applications (such as speech recognition applications, gesture detection systems, and heart rate monitors), neither the entire information content of the original signal nor its perfect reconstruction is generally of interest, and therefore processing power and data rate are wasted on processing uninteresting information.

[0007]

[0007] Signal conditioning, quantization, and power costs in wireless communications all scale linearly with data rate. If we want to design systems with ultra-low power requirements, a new approach to analog / digital system partitioning is required that aims to significantly reduce overall power consumption. Summary of the Invention

[0008] The inventors of the present invention have recognized that one is generally not interested in the entire information content of an original signal, nor in its complete reconstruction, but rather that for pattern recognition and signal classification applications, one typically requires only a specific subset of information extracted from the signal's waveform, referred to herein as the signal's features. Some examples of waveform features are the maximum signal level over a period of time, the number of zero crossings, etc. These features can provide information about other characteristics of the signal, such as its frequency content, phase, or timing characteristics.

[0009]

[0009] The present invention aims to sample a signal at its relevant information / feature rate by extracting a set of specific features embedded in the analog signal waveform through pulse modulation and feature-centric adaptive filter / locked loop / synchronization techniques before modulation-matched signal feature encoding and classification are performed. This allows signal processing to focus exclusively on the feature-bearing information and discard irrelevant information as early in the signal processing path as possible while filtering out / suppressing other irrelevant information or distorting interferers. By discarding irrelevant information early in the signal processing path, the energy efficiency of the overall system is significantly improved. This implies more intelligent analog signal processing (i.e., analog analytics) that emphasizes relevant features and reduces the dimensionality of the waveform through feature-preserving transformations intended to classify the features of the input signal waveform instead of reconstructing the original waveform.

[0010] To this end, the present invention, according to a first aspect, discloses a signal processing circuit for a spiking neural network, the signal processing circuit comprising an interface for converting an analog input signal into a corresponding spike-time representation of the analog input signal. The interface may comprise an analog-to-information (A / I) converter having an input for receiving the analog input signal and configured to generate a modulated signal representing one or more features of the analog input signal. The interface, in turn, may comprise an input for receiving the modulated signal and a feature detector circuit configured to compare the modulated signal with a reference signal representing the reference feature and to generate an error signal indicative of a difference between the modulated signal and the reference signal. The interface may further comprise a feature extractor circuit having an input for receiving the error signal and comprising a lock-loop circuit configured to generate an output signal representing one or more occurrences of the features represented by the modulated signal. The interface may also comprise an encoder circuit having an input for receiving the feature extractor output signal and configured to encode the output signal into a spike train for input to the spiking neural network.

[0011]

[0011] According to one embodiment of the first aspect, the signal processing circuit may further comprise a pre-processing circuit for pre-processing the analog input signal, preferably wherein the pre-processing circuit comprises a low-noise pre-amplifier, a band-pass filter, and / or a programmable gain (post-)amplifier.

[0012] According to one embodiment of the first aspect, the analog input signal may be in a voltage representation and may be used as a control voltage input to the A / D converter, thus determining the modulation of the analog input signal.

[0013]

[0013] According to one embodiment of the first aspect, the features may be one or more of: i) specific characteristics such as transient features, steady state features, etc.; ii) specific properties such as (non-)linearity features, statistical features, steady state features, transfer function features, energy quantity and / or energy base; iii) specific domain features such as time, delay, frequency, phase domain features, and preferably wherein the A / information converter comprises an analog-to-time converter which converts the analog input signal into a modulated signal representing some time domain features such as delay, frequency and / or phase.

[0014]

[0014] According to one embodiment of the first aspect, the A / information converter may form an analog-to-delay (A / delay) converter, an analog-to-frequency (A / frequency) converter, and / or an analog-to-phase (A / phase) converter.

[0015]

[0015] According to one embodiment of the first aspect, the A / information converter may comprise a voltage-controlled delay line, a voltage-controlled oscillator and / or a multi-phase voltage-controlled oscillator that performs at least part of the modulation of the analog input signal into a modulated signal.

[0016]

[0016] According to one embodiment of the first aspect, the A / delay converter may comprise a voltage-controlled delay line, the A / frequency converter may comprise a voltage-controlled oscillator, and the A / phase converter may comprise a multi-phase voltage-controlled oscillator.

[0017] According to one embodiment of the first aspect, the output signal of the feature extractor circuit may be fed back to the feature detector circuit to form a negative feedback loop.

[0018] According to one embodiment of the first aspect, the feature extractor circuit may comprise a voltage controlled oscillator driven by the error signal to produce the output signal of the feature extractor circuit and which is part of a locked loop.

[0019] According to an embodiment of the first aspect, the lock loop may be a delay lock loop, a frequency lock loop or a phase lock loop.

[0020]

[0020] According to one embodiment of the first aspect, the feature extractor circuit may further comprise a filter, preferably an adaptive filter, which stops the error signal if the portion is not within the locking range of the locking loop, but passes the portion of the error signal that is within the locking range of the locking loop.

[0021] According to one embodiment of the first aspect, the encoder circuit may use rate-based or temporal, spike-based encoding as the encoding scheme.

[0022]

[0022] According to one embodiment of the first aspect, the feature extractor circuit may be part of a set of parallel feature extractor circuits, and wherein the signal processing circuit further comprises a channel selector unit that may be adaptive in such a way that the most useful subset of features under particular operating conditions at a given moment is extracted, and thus one or more feature extractor circuits corresponding to the subset of features of interest may be adaptively selected, and information about those features is extracted from the signal from the A / information converter by the parallel feature extractor circuits.

[0023]

[0023] According to one embodiment of the first aspect, the feature extraction performed by the feature extractor circuit may be configurable through certain circuit parameters and / or system control parameters, preferably the configurable parameters include one or more of gain, linearity, bandwidth, feedforward and feedback, coarse / fine selectivity, ATC control, feedback control, calibration, timing, delay, sampling / sub-sampling mode selection.

[0024]

[0024] According to one embodiment of the first aspect, at runtime, the application / context / condition detection block may determine the current operating context and the optimal analog feature set to be extracted from the current operating context, preferably wherein a feedback loop dynamically adjusts the specifications of the feature extractor circuit within power or performance constraints, and wherein the runtime configuration manager then activates and configures only the relevant feature sets by activating the relevant feature detector circuits.

[0025] According to one embodiment of the first aspect, the analog input signal may comprise an analog differential voltage signal having positive and negative voltage inputs.

[0026]

[0026] According to one embodiment of the first aspect, there may be provided a plurality of feature signal paths for modulated signals each representing one or more particular features of the analog input signal, wherein at least one feature signal path comprises a programmable fabric preferably driven by a spiking neural network, which provides control feedback to at least one other feature signal path based on the output of the programmable fabric.

[0027] According to an embodiment of the first aspect, the signal processing circuit may further comprise a converter unit for converting the modulation signal or the error signal into a digital signal.

[0028] According to a second aspect of the present invention, a signal processing method is disclosed for converting an analog input signal into a corresponding spike-time representation of the analog input signal for a spiking neural network, the method comprising receiving the analog input signal, generating a modulated signal representing one or more features of the analog input signal, comparing the modulated signal with a reference signal representing the reference features to generate an error signal indicative of a difference between the modulated signal and the reference signal, generating an output signal representing one or more occurrences of the features represented by the modulated signal, and encoding the output signal into a spike train for input to the spiking neural network.

[0029]

[0029] According to one embodiment of the second aspect, the features may be one or more of: i) specific characteristics such as transient features, steady state features, etc.; ii) specific properties such as (non-)linearity features, statistical features, steady state features, transfer function features, energy quantity and / or energy base; iii) specific domain features such as time, delay, frequency, phase domain features, preferably wherein the modulated signal exhibits some time domain features such as delay, frequency and / or phase.

[0030]

[0030] Embodiments will now be described, by way of example only, with reference to the accompanying drawings in which corresponding reference symbols indicate corresponding parts, and in which: [Brief explanation of the drawings]

[0031] [Figure 1]

[0031] FIG. 1 illustrates a conventional circuit for implementing a typical processing path known in the art. [Figure 2]

[0032] 1 shows a signal processing system having an analog-to-information (quantity) converter, an adaptive feature extractor and selector, a spike encoder and an SNN classifier according to the present invention. [Figure 3]

[0033] Diagram outlining the general concept of utilizing modulation techniques and feature-centric locked loops / synchronization to convert analog input signals into corresponding spike-time representations. [Figure 4]

[0034] 1 is a schematic block diagram of one embodiment including analog-to-time conversion and control and coarse and fine filtering. [Figure 5A]

[0035] 10A-B illustrate the selection of local differences in label-specific data through a phase-locked data-driven feature extractor shown in the time-frequency domain. [Figure 5B] 10A-B illustrate the selection of local differences in label-specific data through a phase-locked data-driven feature extractor shown in the time-frequency domain. [Figure 5C] 10A-B illustrate the selection of local differences in label-specific data through a phase-locked data-driven feature extractor shown in the time-frequency domain. [Figure 6A]

[0036] FIG. 1 shows an example of the response of a spiking neural network to a parameterized set of input data after feature-centric preprocessing. [Figure 6B] FIG. 1 shows an example of the response of a spiking neural network to a parameterized set of input data after feature-centric preprocessing. [Figure 6C] FIG. 1 shows an example of the response of a spiking neural network to a parameterized set of input data after feature-centric preprocessing. DETAILED DESCRIPTION OF THE INVENTION

[0032]

[0037] In the following, some embodiments will be described in more detail. However, it should be appreciated that these embodiments should not be construed as limiting the scope of protection of the present disclosure.

[0033]

[0038] 1 shows a conventional circuit 10 for implementing a typical signal processing path known in the art. The signal processing path may form channels in a multi-channel sensing system. The maximum number of channels may be constrained by noise, area, bandwidth, power, and / or scalability and expandability of the sensing system.

[0034]

[0039] Without loss of generality, in conventional circuits, data acquired by the sensing elements is conditioned using analog circuitry in a front-end interface 10A. Each signal processing path may comprise, for example, a low noise pre-amplifier (LNA) 2A, a bandpass filter 3, a programmable gain (post-)amplifier (PGA) 2B, an analog-to-digital converter (ADC) 4, and / or a serial interface 5.

[0035]

[0040] The low noise preamplifier 2A is generally the first element in the signal processing path. Its primary purpose is to amplify weak input signals while introducing minimal additional noise. The significance of the LNA lies in its ability to boost the signal-to-noise ratio (SNR) of the input signal, ensuring that subsequent stages of the signal processing chain receive a sufficiently strong and clean signal for further processing.

[0036]

[0041] The bandpass filter 3 allows a specific frequency range, known as the passband, to pass while attenuating frequencies outside this range. The purpose of a bandpass filter is to remove unwanted noise or interference outside the desired frequency range. It helps to isolate the signal of interest, enhance its quality, and improve the overall SNR by reducing the amount of out-of-band noise or signal.

[0037]

[0042] The programmable gain amplifier 2B, also known as the post-amplifier, is responsible for adjusting the gain of the signal after it has been filtered. It allows the amplification of the filtered signal to an appropriate level for subsequent processing stages or to match the signal to the dynamic range of the following components. The significance of the PGA is that it provides control of the signal gain, allowing it to adapt to different input signal strengths or to adjust for different processing requirements.

[0038]

[0043] The analog-to-digital converter (A / D converter) 4 converts a continuous analog signal into a discrete digital representation. It samples the analog signal at a specified rate and quantizes the sampled values into a digital code. The significance of the A / D converter is that it converts a continuous-time signal into a digital format suitable for digital processing, storage, and transmission. It allows the application of various digital signal processing techniques to the captured data.

[0039]

[0044] The serial interface 5 is responsible for transmitting the digitized signals to other devices or systems. It facilitates communication between the signal processing path and other components or systems, for example, back-end signal processing units.

[0040]

[0045] The A / D converter 4 can be shared by a (post-)amplifier through time-division multiplexing. Alternatively, to lower the demand on the amplifier's drive capability and ease the power, noise, and crosstalk requirements, the PGA and A / D converter can be combined and embedded in every recording channel, thus simultaneously implementing both signal acquisition and amplification in addition to data conversion.

[0041]

[0046] The output of the A / D converter may be fed, for example, via a serial interface, to a back-end signal processing unit 10B, which may provide feature detection using a feature detector unit 6, which may use, for example, a threshold, and / or an extraction unit 7, which may use, for example, a TTL, and additionally or alternatively perform signal classification using a classification unit 8, for example, a spiking neural network. The extraction unit 7 and the classification unit 8 may require training 7A, 8A, respectively.

[0042]

[0047] External influences present a significant challenge in systems where a large number of sensing (sub)elements are required for accurate representation. External influences also present challenges for spatially wide analysis of a particular (sensed) activity, detection, or presence. External influences can be due, for example, to background activity of neighboring sensing / recording elements / circuits, slight perturbations in the position of the sensing / recording elements, external electrical interference, and / or external mechanical interference.

[0043]

[0048] Thus, the inventors of the present invention have recognized that the ability to distinguish between noisy signals and relevant signal information or to extract relevant signal information from noisy signals both depends on the discrepancy between the noise-free data obtained from each source and the signal-to-noise level (SNR) in the recording system.

[0044]

[0049] Depending on the SNR, for example, voltage thresholding can be used in relation to estimating noise amplitude in the signal, or more advanced techniques such as successive wavelet transforms can be used, or energy content based techniques such as examining the energy content of the waveform within a slicing window can be used to detect the time occurrence of transitions in the information or characteristics or feature set or data of the signal of interest.

[0045]

[0050] Voltage thresholding is a signal processing technique used to classify or identify specific events or features in a voltage waveform based on their amplitude or voltage level. A threshold (e.g., an estimate of noise amplitude in the signal) is set to distinguish between signal components above and below a certain voltage level. The voltage waveform is compared against this threshold, and any portion of the waveform that crosses or exceeds the threshold is considered to have met the detection criteria. The threshold is determined based on the characteristics of the signal and / or specific feature of interest. This threshold may be a fixed value or dynamically adjusted based on the signal properties or noise level. The voltage waveform is compared against the threshold at each sample point in the waveform; if the voltage amplitude, for example, exceeds the threshold, it is considered a positive detection, indicating the presence of the desired event or feature. Once the voltage waveform has been processed through thresholding, the identified event or feature can be marked, extracted, or used for further analysis.

[0046]

[0051] The continuous wavelet transform (CWT) is a mathematical technique used to analyze signals and data in the time-frequency domain. The CWT decomposes a signal into a set of wavelet functions, called wavelets. A wavelet is a wave-like oscillation that starts at zero, increases or decreases, and then returns to zero one or more times. Wavelets can be compared to short oscillations. If portions of a signal are similar, the wavelet correlates with the signal. A mother wavelet is used to generate different wavelets with which the signal is convolved (thus determining the correlation of that particular wavelet with the signal). By varying the scale and transformation parameters, the mother wavelet can be tuned to generate specific wavelets to analyze different features of the signal. The wavelet transform then decomposes the signal into a set of wavelet coefficients, capturing its time-varying frequency components. Thus, the CWT provides a time-frequency representation of the signal, where the magnitude of the CWT coefficients represents the strength of the corresponding frequency component at a particular time. By analyzing the CWT coefficients, various signal properties can be extracted, such as dominant frequencies, the time location of events, and the change in frequency components over time.

[0047]

[0052] Slicing windows, also known as sliding windows or analysis windows, are a technique used in signal processing and data analysis to analyze signals or data sequences in smaller, overlapping segments. It involves dividing a signal or data sequence into consecutive, partially overlapping windows of fixed length. A window is generally defined by its length or duration, which determines the size of the segment under analysis. The step size or overlap determines how much the window shifts between consecutive segments. Larger windows capture more information but provide lower time resolution, while smaller windows provide higher time resolution but capture less contextual information. By analyzing the amount of energy within a slicing window, specific events or features in a signal can be identified and detected. For example, a slicing window can be used to capture short segments of a signal's waveform, and the amount of energy within each window can be evaluated. If the energy exceeds a certain threshold, it indicates the presence of an event. The amount of energy within a slicing window can be used to extract relevant features from the signal. Different aspects of a signal can be characterized by calculating statistical measures such as the mean, variance, or higher-order moments of the energy.

[0048]

[0053] Without loss of generality, feature extraction (a step performed to simplify and / or optimize the classification process) may be based on explicit details of the input signal, such as i) specific characteristics, e.g., transient features, steady-state features; ii) specific properties, e.g., (non-)linearity features, statistical features, stationary features, transfer function features, energy content and / or energy-based; iii) specific domain features, e.g., time, delay, frequency, phase domain features. The completed processing includes several steps: i) preprocessing (including, e.g., signal detection, energy normalization, and noise removal); ii) feature domain analysis; iii) subrange selection, which specifies the selection of different variable / parameter ranges to attach to different subsets of features of interest; iv) final feature calculation; and v) classification.

[0049]

[0054] Transient features are transient or short-duration phenomena occurring during the initial or transitional period of a signal or system. They represent abrupt changes or disturbances in a signal. Thus, transient features generally exhibit characteristics such as rapid changes, high-frequency content, and short duration. They often contain rich spectral content and have distinct temporal profiles. For example, analyzing transient features can require high temporal resolution and the ability to track rapid changes in a signal. Techniques such as time-domain analysis, waveform analysis, or high-resolution time-frequency analysis (such as the continuous wavelet transform introduced above) are commonly used to extract transient features. Steady-state features, on the other hand, represent the long-term or fixed behavior of a signal or system after the initial transient period has passed. They correspond to the stable or cyclical portion of a signal that persists for an extended duration when the system reaches a steady state. Steady-state features exhibit characteristics such as relatively constant amplitude, stable spectral content, and regular temporal patterns. Analyzing steady-state features generally involves examining properties such as mean amplitude, frequency content, statistical measures, harmonic structure, or period. Techniques such as Fourier analysis, power spectral density estimation, or statistical analysis may be used to analyze steady-state features.

[0050]

[0055] Analyzing a signal based on specific properties involves applying specific techniques or methods to extract relevant information for each feature.

[0051]

[0056] For example, for (non-)linear features, one can perform a phase space reconstruction (to determine the underlying dynamics of the signal, which can reveal non-linear behavior and give insight into the complexity of the system), or determine the Lyapunov exponent (which quantifies the degree of chaos or predictability in the signal, larger exponents indicating chaotic or non-linear behavior).

[0052]

[0057] For statistical features, statistical measures such as mean, variance, skewness, and kurtosis can be calculated to characterize the distribution and central tendency of the signal. Furthermore, autocorrelation functions can be calculated to analyze time-dependence or recurring patterns in the signal, for example.

[0053]

[0058] For fixed features, for example, Fourier transform or spectral analysis techniques can be applied to examine the frequency content and power spectrum of the signal, which helps identify dominant frequencies and spectral characteristics. Furthermore, wavelet analysis, such as the continuous wavelet transform (CWT) or discrete wavelet transform (DWT), can be performed to analyze the time-frequency properties of the signal and identify local features.

[0054]

[0059] The amount of energy can be calculated, for example, by calculating the energy or power of a signal within a specific frequency band or time window to identify dominant frequency components or assess the power distribution of the signal. Another option may be to obtain a time-frequency representation of the signal and apply a short-time Fourier transform to analyze the energy distribution over time and frequency. Another option may be to extract the envelope of the signal by using techniques such as the Hilbert transform or amplitude modulation analysis, and then analyze the energy variation of the envelope of the signal.

[0055]

[0060] Analyzing a signal based on specific domain features such as time, frequency, or phase involves applying domain-specific techniques to extract relevant information.

[0056]

[0061] For time-domain analysis, statistical measures, such as the mean, variance, and higher moments of a signal, can be calculated to characterize its amplitude distribution and fluctuations. Furthermore, techniques such as autocorrelation can be applied to identify repetitive patterns or periodic components in the signal.

[0057]

[0062] For frequency domain analysis, for example, Fourier transform or fast Fourier transform (FFT) techniques can be applied to convert a signal from the time domain to the frequency domain. This results in the frequency spectrum of the signal, whose magnitude and phase of its frequency components can be analyzed. The power spectral density can be determined, which can give insight into the distribution of power across different frequencies at which the signal is structured. A short-time Fourier transform (STFT) can be used to determine the time-varying frequency components of the signal.

[0058]

[0063] For phase domain analysis, the phase components of a signal can be extracted through the application of techniques such as the Hilbert transform or phase unwrapping. This can reveal the phase relationships between different frequency components or oscillations. Measures of phase coherence, such as the phase locking value (PLV), can be measured to assess the degree of synchronization or coupling between different oscillation components.

[0059]

[0064] Time-domain waveforms, frequency spectra, and phase relationships can be compared together to gain a more comprehensive understanding of a signal's characteristics. For example, time-domain analysis can be used to detect transient events or sudden changes in a signal, and then their frequency and phase characteristics can be examined. Furthermore, time-frequency dynamics can be investigated to analyze how a signal's frequency content changes over time.

[0060]

[0065] It may be beneficial to use a combination of techniques to obtain a comprehensive analysis of the signal.

[0061]

[0066] Based on these (extracted) features, the relevant data can be classified by k-means, expectation maximization (EM), template matching, Bayesian clustering, and artificial neural networks (ANN) or spiking neural networks (SNN) into m-dimensional clusters, each cluster corresponding to the activity of a single sensing element / signal channel.

[0062]

[0067] Thus, the result of the classification process is the formation of m-dimensional clusters. Clusters represent groups of data points with similar feature patterns or characteristics. Feature extraction involves converting raw data into a reduced set of meaningful features that capture relevant information. Clusters are formed based on the similarity or proximity of these extracted features. Data points within the same cluster are more similar in terms of their feature representation than data points in different clusters. The dimensionality of a particular cluster, "m," refers to the number of features or variables used to describe the data points in that cluster. Each cluster represents the activity or behavior of a single sensing element or signal channel. In other words, data points within each cluster are similar to each other based on their extracted features, indicating that they belong to the same sensing element or signal channel.

[0063]

[0068] The unsupervised learning algorithm k-means clustering groups data points into clusters based on similarity, where "k" represents the desired number of clusters.

[0064]

[0069] EM is a commonly used statistical algorithm for clustering and classification, especially in the presence of hidden or latent variables. It iteratively estimates the parameters of a statistical model to assign data points to appropriate clusters.

[0065]

[0070] Template matching is a technique in which a predefined template or pattern is compared to data, and a similarity measure is used to identify matches or correspondences. It is often used for pattern recognition and signal matching.

[0066]

[0071] Bayesian clustering applies Bayesian inference principles to cluster data points. It incorporates prior knowledge and updates probabilities to determine the most likely clusters for the data.

[0067]

[0072] An artificial neural network is a signal processing system composed of interconnected artificial neurons organized into one or more layers. Each artificial neuron receives input signals, combines them with specific weights, and applies an activation function to generate an output. Connections between neurons, called synapses, with adjustable weights, allow information to propagate through the network, enabling the network to learn and make predictions or decisions based on the input data. ANNs do not inherently consider the timing of events or the temporal dynamics of information; they are primarily concerned with the magnitude of the inputs and their propagation through the network.

[0068]

[0073] A spiking neural network (SNN) is a signal processing system whose design is inspired by biological neural networks. Information is encoded in patterns of spike signals distributed across a network of neurons and synapses, as described in WO 2022 / 090542 A1, filed by the applicant and incorporated herein by reference in its entirety. By mimicking the processing performed in biological brains, spiking neural networks can perform signal processing tasks commonly performed by human brains. Examples include image recognition, sound recognition, and event detection based on inputs from multiple sensors.

[0069]

[0074] Without loss of generality, the present invention implements feature processing / classification as a neuromorphic event-based neural network. Time-continuous SNN architectures provide a platform for minimizing size and power consumption in real-time behavioral systems by explicitly conforming to the computing principles underlying autonomous adaptive behavior. In time-continuous spiking architectures, state variables naturally evolve over time, bypassing the need for clocks and additional circuitry to manage the time representation; the circuitry is directly driven by input data; i.e., synapses receive input spikes and neurons generate output spikes at the rate of the incoming data. Thus, for applications where signals have sparse activity in space and time, most neurons will be silent all the time, thus minimizing system power consumption. Essentially, such (time-aware spiking) neural network implementations employ a hybrid analog-digital signal representation; i.e., trains of pulses / spikes transmit analog information with event timing, which are converted back to analog signals in the neuron's dendrites (inputs).

[0070]

[0075] In neural systems, numerous information coding schemes exist, particularly those utilized to efficiently represent specific forms of data originating from different senses (e.g., visual, acoustic, physical). In the case of neuronal computation, such coding schemes can be distinguished into two broadly defined competing approaches: rate-based coding and temporal spike-based coding. In rate-based coding, information representation and computation are based not on individual spikes but on spike firing rates; i.e., information is embedded in the instantaneous or average rate of spike generation of a single neuron or a group of neurons. Therefore, the schemes can be subdivided into count, density, and population ratio coding. Count coding can emphasize the absolute number of spikes; density coding can normalize counts by considering the time window over which the counts were taken; and population ratio coding can take into account the average firing rate of a group of neurons.

[0071]

[0076] When designing a real-time neuromorphic system, the final choice of a particular encoding scheme must be evaluated from different perspectives, i.e., algorithm level, training process, and hardware implementation, and their impact and performance indicators and parameters must be evaluated over a wide range, e.g., in terms of classification accuracy, processing latency, energy consumption, number of synaptic operations, hardware cost, network compression efficacy, noise resistance, and fault tolerance.

[0072]

[0077] For practical implementation of the proposed feature extraction mechanism, it may be important to include the details of the signal modulation and its inherent relationship to signal distortion caused by hardware imperfections, especially in the case of low SNR.

[0073]

[0078] The features can be activated, and feature-based feature extraction can be performed and configured at runtime along several parameters, such as region or window length in the time, delay, frequency, or phase domain, gain, bandwidth, etc. For example, the region / window length can be important when considering feature extraction based on the heterogeneous properties of a signal in the form of specific distribution subregions rather than defining it as uniformly distributed (along the entire original feature). Thus, the signal modulation aspect can be characterized by the heterogeneous properties of the signal in the form of specific distribution subregions. Therefore, for selection, the feature vector can be segmented into multiple overlapping subregions. The overlap can be considered as a sliding window with a width of w and a sliding step of s for filtering the entire feature, satisfying s = [(nw) / k], where n is the feature length and k is the number of windows. Without loss of generality, to highlight the performance / separability of a particular feature, the inter- and intra-class feature variability of each window is measured, and subregions with strong class separability are selected. To do this, for example, separability can be measured using a predefined metric.

[0074]

[0079] Inter-class feature variability refers to the distribution or spread of feature representations among different classes or categories in classification problems. It measures the degree of separation or dissimilarity between the feature representations of different classes. Inter-class feature variability can be important for classification tasks because well-separated classes make it easier for a classifier to distinguish between them. When classes have low inter-class feature variability, they may overlap in the feature space, resulting in higher classification errors or ambiguity. On the other hand, high inter-class feature variability facilitates better discrimination and classification accuracy.

[0075]

[0080] Intra-class feature variability refers to the distribution or spread of feature representations within the same class or category in classification problems. It measures the degree of change or dissimilarity between feature representations belonging to the same class. By minimizing intra-class feature variability, a classifier can better capture salient characteristics shared by instances within the same class. This leads to improved classification accuracy, as the classifier can make more confident and accurate predictions.

[0076]

[0081] Therefore, the separability and distinguishability of different classes can be enhanced by maximizing the inter-class feature variability while minimizing the intra-class feature variability.

[0077]

[0082] Representative features with distribution characteristics of different devices (e.g., integrated circuit and system process-specific sensor signals) can be obtained using statistical methods, such as evaluating the size of local differences between representative features (e.g., in a separable feature space) and determining the location of its concentrated distribution based, for example, on weighted entropy.

[0078]

[0083] FIG. 2 shows a signal processing system 20 having an analog-to-information (quantity) converter, an adaptive feature extractor and selector, a spike encoder and an SNN classifier according to the present invention.

[0079]

[0084] A physical input signal 21, which may or may not be preconditioned, enters the signal processing system. Conditioning can include, for example, the use of a low-noise preamplifier (LNA), a bandpass filter, or a programmable gain (post)amplifier (PGA). An analog-to-information (A(Qo)I) converter, or more generally, an analog-to-information converter 22, can take the (analog) physical input signal 21 and extract only the most relevant information from the analog signal 21 for a given application in preparation for its digitization. The output of the A(Qo)I converter 22 can be a modulation signal of the input signal, which will be further explained using the example of FIG. 3. In particular, the A(Qo)I converter 22 can obtain, for example, specific details of the physical input signal 21, such as: i) specific characteristics, e.g., transient features, steady-state features; ii) specific properties, e.g., (non-)linearity features, statistical features, steady-state features, transfer function features, energy quantity or energy-based features; and iii) specific domain features, e.g., time, frequency, and phase domain features, as described above.

[0080]

[0085] The modulated signal generated by the A(Qo)I converter 22 is then sent to the channel selector and extractor unit 23. As mentioned above, each signal channel that the channel selector and extractor unit 23 can select can correspond to a specific cluster, where each cluster represents the activity of the signal channel and is described by a specific base of features. Thus, data in the physical signal 21 can be classified into one or more clusters depending on which features are present in the signal. Note that if the A(Qo)I converter includes a voltage-controlled delay line or voltage-controlled oscillator, event-driven voltage-controlled delay lines and voltage-controlled oscillators can be used; therefore, if a signal is present at the input of the A(Qo)I 22, the signal will then be present at the input of the subsequent signal channel. The task of the adaptive extractor / filter is to pass only signal information relevant to the SNN inference and control machine 25.

[0081]

[0086] The extractor unit 23 may comprise a set of parallel feature extractors 23A, 23B, etc., each functioning as a channel selector, with each channel representing a (sub)region of the feature domain to which it relates. For example, a channel may be defined by a (sub)region of the delay, frequency or phase domain of the input signal. The channel selector unit may be adaptive in such a way that the most useful subset of features under specific operating conditions at a given moment is activated; thus, one or more signal channels corresponding to the subset of features of interest may be adaptively selected, and information about those features may be extracted from the signal from the A(Qo)I converter by the parallel feature extractors comprised in the extractor units.

[0082]

[0087] Therefore, this approach may require a set of configurable feature extractors. In one embodiment, feature extraction is performed by the feature extractor through a feature-centric adaptive filter / locked loop / synchronization technique. Feature extraction is generally circuit / system specific in the way it is configurable, i.e., through specific circuit parameters (e.g., gain, linearity, bandwidth) and system (feedforward and feedback) control parameters. For example, in the embodiment of FIG. 4, coarse / fine selectivity, ATC control, feedback control, calibration, timing, delay, sampling / subsampling mode selection are all configurable / programmable.

[0083]

[0088] The features generated by the extraction unit 23 are fed to a classifier or information extractor that performs application-specific information estimation, for example speech recognition.

[0084]

[0089] For example, features coming from extraction unit 23 are supplied to spike coding stage 24, e.g., via a multiplexer. The features are encoded by spike coding stage 24 into spike trains, which are sequences of spikes (action potentials) that represent information in a temporal, event-based manner. As mentioned above, rate-based or temporal, spike-based coding may be used for the encoding scheme. In this way, the feature information extracted from the physical signal is converted into spike timing, rate, or pattern, which are then supplied to SNN processor or classifier 25.

[0085]

[0090] The SNN outputs information and / or inference metrics 26, which are quantitative measures used to assess the SNN's performance in classifying physical signals. The SNN inference machine can output two signals: (i) an inference result, which is a designation of the particular class to which the input signal belongs, and (ii) an inference metric, e.g., inference accuracy, as a quantitative measure for assessing the SNN's performance in classifying physical signals.

[0086]

[0091] As mentioned above, the SNN 26 can perform inference / classification. Additionally, the SNN can also perform control functions, albeit as a separate implementation. The application / context / condition detection block 27 represents such a control machine, which controls circuit and system parameters, e.g., adaptive channel selector and extractor. The application / context / condition detection block receives two inputs: one from the SNN inference machine (e.g., inference accuracy) and a second from the selector, extractor, and encoding stages that need to be controlled or calibrated. See, e.g., FIG. 4 for more explicit details.

[0087]

[0092] The reliability of the obtained estimation strongly depends on environmental conditions such as signal dynamics or signal interference. At runtime, the application / context / condition detection block 27 determines the current operating context and, therefore, the optimal analog feature set. A feedback loop dynamically adjusts the specifications of the feature extractor unit within power or performance constraints. The runtime configuration manager of the selector unit and extractor unit then activates and configures only the relevant feature sets.

[0088]

[0093] The parameter block 28 specifies parameters that the selector / control unit operates on to optimize only a particular figure of merit, which may be an efficiency figure (e.g., energy efficiency, energy-throughput efficiency, energy-throughput-accuracy efficiency), or a particular task (e.g., noise filtering, variation limiting / bounding / calibration, channel synchronization, or signal alignment). Training and test data across processing tasks 29A may be required as input to know which (performance parameter) settings to select for a particular processing task. Additionally, parameterized performance settings 29B may be used as input to the parameter block.

[0089]

[0094] It should be noted that feature extraction incurs certain costs, e.g., computational costs (e.g., searching for a particular pattern versus counting the number of occurrences) or memory costs (e.g., for storing the (effective) values of the features). Thus, for pattern recognition (or classification), when the cost of feature extraction (or the available budget in terms of power / performance / area (PPA)) is included in consideration, the performance of the classifier depends on the cost of the features used. The (feature) memory cost depends, among other things, on the number of features and the feature vector size / precision. What kind of memory device (e.g., flip-flop-based, MOS capacitor-based, or floating gate-based) is utilized depends on the circuit implementation. The effective values of the features may be used in the feature extractor.

[0090]

[0095] Figure 3 shows an overview of a system 30 for converting an analog input signal 31 into a corresponding spike-time representation using modulation techniques and feature-centric locked loop / synchronization. The system of Figure 3 illustrates various optional circuits for processing the input signal in the delay, frequency, and / or phase domains that may be implemented in the circuitry shown in Figure 2. For example, the A(Qo)I converter 32 may be implemented as the A(Qo)I converter 22, the detector circuit 33 and feature extractor 34 may be implemented as the channel selector and extractor 23, and the spike encoding stage and classifier 35 may be implemented as the spike encoder 24 and classifier 25.

[0091]

[0096] The physical signal 31, which may be preconditioned, enters the A(Qo)I converter 32, which may be considered part of the front-end interface 30A from which the modulated signal is obtained. Conditioning may include, for example, the use of a low-noise preamplifier (LNA), a band-pass filter, and / or a programmable gain (post-)amplifier (PGA). The A(Qo)I converter may include an analog-to-delay (A / delay) converter 32A, an analog-to-frequency (A / frequency) converter 32B, and / or an analog-to-phase (A / phase) converter 32C. The input signal may also be converted to other parameters other than delay, frequency, or phase. The physical signal may be in voltage representation and used as a control voltage input to each of one or more converters. The A / delay converter 32A may include a voltage-controlled delay line (VCDL). The A / frequency converter 32B may include a voltage-controlled oscillator (VCO). The A / phase converter 32C may include a multi-phase voltage-controlled oscillator (multi-phase VCO). Thus, a physical signal may be used as a control voltage input to each of the converters to determine the amount of delay, frequency, and / or phase of the modulated signal. For example, for A / F converter 32B, a physical signal may be used as a control voltage input to the VCO, with higher voltage inputs creating higher frequencies of the modulated output signal.

[0092]

[0097] The modulated signal may then be provided to a detector circuit 33, which may include a delay detector circuit 33A, a frequency detector circuit 33B, and / or a phase detector circuit 33C, where, for example, the delay, frequency, and / or phase are compared to respective reference delay, frequency, and / or phase signals. The references may be predetermined or dynamically determined. Thus, the detector 33 may generate an error signal 38 proportional to the difference between the modulated signal and the reference signal, for example, with respect to delay, frequency, and / or phase. The error signal may then be filtered and used to drive the feature extractor 34.

[0093]

[0098] The feature extractor 34 may include VCOs 34A, 34B, and / or 34C that create output delay, frequency, and / or phase signals. Additionally, an adaptive filter may be present in the feature extractor 34. The filter may shut off the error signal 38 when it is not within the locking range of the VCOs 34A, 34B, and 34C, but may pass it when it is within the locking range. The output signal may represent one or more occurrences of the features represented by the modulated signal in time, but may also represent the likelihood (e.g., a specified probability) that the feature is present in the signal at a particular time. The output signal may be fed back to the input of the system, creating a negative feedback loop. If the output delay, frequency, and / or phase drifts, the error signal will increase, driving the VCO's delay, frequency, and / or phase in the opposite direction, thereby reducing the error. Thus, the output delay, frequency, and / or phase are locked to the respective delay, frequency, and / or phase of the modulated signal or the error signal.

[0094]

[0099] The detector circuit 33 and feature extractor circuit 34 may be implemented using a set of parallel circuits (e.g., as shown in the feature extractor / channel selectors 23A, 23B of the embodiment of FIG. 2) that perform detection and feature extraction for each signal channel, each channel representing a subregion of a relevant delay, frequency, or phase domain of the input signal.

[0095]

[0100] The output signal of the feature extractor 34 is provided to a spike encoding stage 35, which encodes the output signal into a spike train. As mentioned above, rate-based or temporal spike-based encoding may be used for the encoding scheme. In this manner, the feature information extracted from the physical signal 31 is converted into spike timing, rate, or pattern, which are then provided to an SNN processor or classifier, also referenced by numeral 35. The spike encoding stage and classifier may be implemented together as one circuit or separately. The output of the spike encoding stage and classifier 35 may be, for example, a classification metric 36. Training 37 of the feature extraction circuit 34 and the encoding circuit 35 may also be required for the classifier.

[0096]

[0101] The type of encoding used in the encoding circuit 35 may vary depending on the types of parameters used in the transformer 32, detector 33, and feature extractor 34. When focusing on delay parameters, time-to-first-spike (TTFS), inter-spike interval (ISI), burst, or delay-synchronous encoding may be used. When focusing on frequency parameters, rate- or frequency-synchronous encoding may be used. When focusing on phase parameters, phase- or phase-synchronous encoding may be used.

[0097]

[0102] Time-to-first-spike coding is a method used in spiking neural networks (SNNs) to represent information based on the timing of the first spike generated by a neuron in response to a stimulus or input. In this coding scheme, information is represented by the time it takes a neuron to fire its first action potential (spike) after receiving a particular input. In time-to-first-spike coding, neurons are configured in such a way that they respond quickly to specific features or patterns in the input, and the first spike encodes the presence of those features. For example, if a neuron is sensitive to a certain visual pattern, it will quickly fire its first spike when it detects that pattern in the input.

[0098]

[0103] Interspike interval (ISI) coding is a method used in spiking neural networks (SNNs) to represent and process information based on the timing of spikes generated by neurons. In ISI coding, information is represented by the time interval between successive spikes emitted by a neuron. Each spike may be time-stamped, and the duration between two successive spikes is measured as the interspike interval. The pattern of the interspike intervals carries information about the input being processed.

[0099]

[0104] Burst coding in spiking neural networks is a method used to represent and process information by encoding patterns of spikes in short bursts, or bursts of action potentials. It involves groups of spikes generated closely in time, often accompanied by high-frequency firing, followed by a period of relative inactivity before another burst of spikes occurs. In traditional single-spiking neurons, information is generally encoded in individual spikes. However, in burst coding, information is represented in the temporal pattern of the bursts.

[0100]

[0105] In rate coding, information is represented by the frequency or rate at which neurons generate spikes over time. A higher firing rate generally indicates the presence or strength of a particular feature in the input data, while a lower firing rate may indicate the absence or weaker representation of that feature. The key idea behind rate coding is to exploit the temporal integration of spike trains. Rather than considering individual spikes, the network focuses on cumulative activity over a period of time. Rate coding can rely on the aggregate firing rate of spiking neurons over time or the average firing rate of neurons over a period of time.

[0101]

[0106] Phase coding is a method used to represent and process information by encoding it in the relative phase relationship between spikes emitted by different neurons. This encoding method utilizes the precise timing of spikes to convey information. In phase coding, neurons can be designed or tuned to fire their spikes at specific phases of a periodic cycle. For example, neurons can be configured to fire at the peak of an oscillatory signal or at a specific phase of a periodic input. The relative phase relationship between spikes of different neurons encodes information about the input. The main idea behind phase coding is that spike timing carries useful information, and by synchronizing spikes of multiple neurons to specific phases, a network can represent and process specific features or temporal patterns in the input data.

[0102]

[0107] Synchronous coding in spiking neural networks (e.g., by using either delay, frequency, or phase) is a method used to represent and process information by encoding patterns of synchronous spikes among different neurons. It involves neurons firing action potentials simultaneously or in close temporal proximity to convey specific information or encode some feature of the input. Synchronous coding utilizes precise timing in the form of synchronized spiking activity across multiple neurons to efficiently represent information. The simultaneous or near-simultaneous firing of neurons can convey information about correlations, patterns, or specific features in the input data. For example, the presence of a specific feature in the input may be represented by neurons firing simultaneously, while the absence of the feature may lead to desynchronized or uncorrelated activity.

[0103]

[0108] Without loss of generality, in the present invention, converting an analog input signal to a corresponding spike-time representation in the A(Qo)I converter 32 comprises first converting the signal information (quantity) during a transition (instead of instantaneous amplitude) of the input signal 31 by some form of modulation, i.e., representing and processing the signal by using (spike-time) properties as the signal's delay, frequency, and phase. The transition here can be between two well-defined values, e.g., between the ground and power supply voltage levels of the input signal. Correspondingly, this analog-to-information (quantity), e.g., analog-to-delay, analog-to-frequency, analog-to-phase, conversion allows for manipulation of the sampled information using delay difference, frequency difference, and phase difference variables as related events performed relative to a reference.

[0104]

[0109] As mentioned above, relevant information is extracted through feature-centric adaptive filter / locked loop / synchronization techniques in detector circuit 33 and feature extractor circuit 34 before encoding of modulation match signal features (e.g., in the case of analog-to-phase modulated signals, features are extracted through a phase-locked loop and represented using phase encoding) and classification / recognition is performed. In one embodiment of the present invention, time modulation (i.e., analog-to-time conversion) is performed using an (adaptive) voltage-controlled delay line (VCDL) circuit. In another embodiment, frequency modulation is completed using a voltage-controlled oscillator (VCO).

[0105]

[0110] VCO-based analog-to-frequency conversion offers high gain, short frequency, low power, high linearity, and small implementation size through different trade-off mechanisms. In another embodiment, phase modulation (i.e., analog-to-phase conversion) is implemented using a multi-phase VCO circuit. After completing pulse modulation, relevant information is extracted and selected through feature-centric locking loop / synchronization techniques (e.g., delay-locked loop (DLL), frequency-locked loop (FLL), and phase-locked loop (PLL)) corresponding to delay modulation, frequency modulation, and phase modulation, respectively. A DLL is a circuit similar to a phase-locked loop, with the main difference being the absence of an internal VCO, which is replaced by a delay line.

[0106]

[0111] In one embodiment, the delay / frequency / phase detector circuit 33 uses the output delay / frequency / phase difference to provide up / down feedback signals to close the feedback loop. When the up / down signals are combined and applied to a discriminator (e.g., feature selector 34), the response of the feature of interest is extracted, i.e., matching the delay / frequency / phase of the input in the detector circuit 33 with the delay / frequency / phase of the reference.

[0107]

[0112] Thus, a signal processing circuit 30 for a spiking neural network is disclosed, which may include an interface for converting an analog input signal 31 into a corresponding spike-time representation of the analog signal. The interface may include a delay circuit 32A for converting the analog input signal into a corresponding delayed input signal having a value representing the delay of the analog signal, a frequency circuit 32B for converting the analog input signal into a corresponding frequency input signal having a value representing the frequency of the analog signal, and / or a phase circuit 32C for converting the analog input signal into a corresponding phase input signal having a value representing the phase of the analog signal. The signal processing circuit 30 may further include a feature extraction unit 34 comprising delay signal processing circuits 33A, 34A connected to receive the delayed input signal, frequency signal processing circuits 33B, 34B connected to receive the frequency input signal, and / or phase signal processing circuits 33C, 34C connected to receive the phase input signal.

[0108]

[0113] The delay circuit 32A may comprise a voltage-controlled delay line circuit having an input for receiving the analog input signal 31, wherein the voltage-controlled delay line circuit 32A is configured to generate a delayed input signal. The frequency circuit 32B may comprise a voltage-controlled oscillator circuit having an input for receiving the analog input signal 31, wherein the voltage-controlled oscillator circuit is configured to generate a frequency input signal. The phase circuit 32C may comprise a multi-phase voltage-controlled oscillator circuit having an input for receiving the analog input signal 31, wherein the multi-phase voltage-controlled oscillator circuit is configured to generate a phase input signal.

[0109]

[0114] The delay signal processing circuits 33A, 34A may comprise delay-locked loop circuits having a first input for receiving a delayed input signal and a second input for receiving a delayed reference signal, where the delay-locked loop circuit is configured to generate a delayed output signal based on a comparison of the delayed input signal with the delayed reference signal. The frequency signal processing circuits 33B, 34B may comprise frequency-locked loop circuits having a first input for receiving a frequency input signal and a second input for receiving a frequency reference signal, where the frequency-locked loop circuit is configured to generate a frequency output signal based on a comparison of the frequency input signal with the frequency reference signal. The phase signal processing circuits 33C, 34C may comprise phase-locked loop circuits having an input for receiving a phase input signal and a second input for receiving a phase reference signal, where the phase-locked loop circuit is configured to generate a phase output signal based on a comparison of the phase input signal with the phase reference signal.

[0110]

[0115] FIG. 4 shows a schematic block diagram of one embodiment 40 including analog-to-time conversion and control as well as coarse and fine filtering.

[0111]

[0116] Without loss of generality, in this disclosure, a time-based feature-centric locked loop / synchronization processing circuit allows for the manipulation of sampled analog information by using a time difference variable as the amount of time between events occurring relative to a reference time or event.

[0112]

[0117] Time encoding consists of representing an analog signal with a modulated square wave, where the signal information is encoded in the transitions instead of the instantaneous amplitude; i.e., the input signal is converted into a time-difference variable. The time signal is then processed by various circuits to produce a time-difference output, where a pulse modulator encodes the analog input signal information in the pulse width, frequency, or position of a signal (or set of signals) with only two levels. This two-level signal, still analog, is then sampled at a sampling frequency. Here, the sampler can be a simple D-type edge-triggered flip-flop-based one, since the signal has only two values. From the sampled square wave, a multi-bit approximation of the analog input signal can be reconstructed using a digital low-pass filter. Conversion of the time interval between two clock edges into a digital number is performed using a time-to-digital converter (TDC). As with all sampling processes, the continuous time signal needs to be band-limited to prevent aliasing effects. In one embodiment, a PLL is utilized to band-limit the phase-modulated input signal. The TDC can be realized using a time comparator (D flip-flop) and other digital blocks, such as a single counter, a flash TDC, a vernier oscillator, a cyclic pulse reduction TDC.

[0113]

[0118] The circuit includes a programmable analog-to-time (A / T) converter, a coarse-tuner, and an extended-range fine-tuner for in-loop filter characteristics, supported by a background calibration mechanism to reduce the effects of nonidealities. It combines the advantages of sampling and sub-sampling loops to achieve robust time, phase, and frequency acquisition under frequency disturbances and low-power operation. During the switching process from the sampling loop to the sub-sampling loop, the phase remains locked without any discontinuity.

[0114]

[0119] In one embodiment, the circuit first performs analog-to-time conversion (ATC), i.e., when an analog voltage is sensed, a conversion is performed during a time pulse, embedding analog information in the time period between the rising and falling edges. The analog signal and its differential signal (phase-shifted by 180°) are provided to two voltage-controlled delay line circuits, and their output signals are therefore time pulses that depend on the amplitude of the input signal. To obtain sufficient dynamic range, a control circuit is added to combine the outputs of the voltage-controlled delay line circuits, and then create a single time pulse that represents the analog input signal by the duration of the pulse width. The control circuit also averages a programmable number of samples to obtain a targeted oversampling ratio.

[0115]

[0120] To improve phase lock time, a coarse T / D converter is inserted after the phase detector to quantize the coarse phase error and assist in coarse phase and frequency locking. The phase detector and frequency detector work in conjunction with switching feedback, which monitors frequency disturbances and ensures that the digitally controlled oscillator (DCO) is always locked to the correct frequency. However, directly switching from sampling mode to sub-sampling mode in the feedback path would result in phase discontinuity due to delays in the feedback path. Therefore, an additional delay is inserted after the ATC to mitigate the phase discontinuity. When sampling mode is engaged to acquire the time / frequency / phase of the input signal, the (SNN) controller generates relevant information for controlling the feedback loop and the A / T converter. If the PLL is locked to an incorrect frequency, the frequency detector can be used to switch the feedback from sub-sampling mode to sampling mode for frequency locking.

[0116]

[0121] In further detail of this embodiment, an input signal enters an analog-to-time converter. The input signal may be a differential signal with positive and negative input voltages to obtain a better dynamic range. The positive and negative input voltages may each be passed through a voltage-controlled delay line before entering the analog-to-time converter. The input signal may be pre-conditioned, and the conditioning may comprise, for example, the use of a low-noise pre-amplifier (LNA), a band-pass filter, or a programmable gain (post-)amplifier (PGA).

[0117]

[0122] An analog-to-time converter is a converter that converts an analog input (voltage) signal, which may be pre-processed or may comprise several input voltages (as for differential signals), into a time parameter. The time parameter may be, for example, delay, frequency, and / or phase, as detailed in relation to FIG. 3. In this embodiment, the analog-to-time converter converts the analog input signal into a delay-, frequency-, and phase-modulated signal, although more or less conversions may be performed.

[0118]

[0123] Then there are essentially four signal paths: a frequency signal path, a delay signal path, a coarse phase signal path, and a fine phase signal path.

[0119]

[0124] The coarse phase signal path may include, for example, a phase detector 44, a time-to-digital converter 45, and a coarse filter 46. The fine phase signal path may include, for example, a delay block, a time-to-digital converter 45, and a fine filter 47. Both the coarse phase signal path and the fine phase signal path are connected to a digitally controlled oscillator 48, each forming a phase-locked loop as disclosed above. As mentioned above, the phase lock time of the fine phase-locked loop may be improved by using the coarse phase-locked loop to quantize the coarse phase error and assist the coarse phase locking. The output signal 49 of the digitally controlled oscillator 48 may be provided to a spike encoding stage, as described above, which encodes the output signal into a spike train. As mentioned above, rate-based or temporal spike-based encoding may be used for the encoding scheme. In this way, feature information extracted from the physical signal is converted into spike timing, rate, or pattern, which may then be provided to an SNN processor or classifier.

[0120]

[0125] The frequency signal path may comprise, for example, a timer control 50, a time-to-digital converter 45, and a digital frequency detector 51. The timer control 50 may ensure that the different signal paths are synchronized. The delay signal path may comprise, for example, a phase detector 44 and delay control logic 52. In this embodiment, both the frequency signal path and the delay signal path are connected to a programmable fabric 53, for example, a spiking neural network.

[0121]

[0126] Programmable fabric 53 and / or digitally controlled oscillator 48 connected to the frequency and delay signal paths may provide output signals to control feedback 54, which may provide feedback to one or more of the different signal paths and / or analog-to-time converter 43. In this example, the control feedback provides feedback to the coarse and fine phase signal paths and analog-to-time converter 43.

[0122]

[0127] The control feedback 54 may be further controlled by a control signal coming from the control block 42, which may be driven by a spiking neural network. The control block 42 may control the analog-to-time converter 43 together with the calibration block 44, for example through programmable logic. The calibration block 44 may be responsible for calibrating certain parts of the overall signal processing module and may take as input some system settings or calibration data.

[0123]

[0128] It should be noted that although one analog-to-time converter 43 is shown, multiple may be present in the signal processing path. Furthermore, in general, the analog-to-time converter may be an A-to-information converter or an analog-to-information content converter that extracts not only time parameters of the signal, such as frequency, delay, and / or phase, but also other parameters of the signal that may be of interest for a particular application.

[0124]

[0129] The different feature signal paths (e.g., frequency signal path, delay signal path, and phase signal path) may be phase-locked by connection to a digitally controlled oscillator or may be connected, for example, to a programmable fabric. One or more of the feature signal paths may be omitted or added. For example, a frequency-locked loop signal path and / or a delay-locked loop signal path may be added. Instead of fine and coarse phase signal paths, only one phase signal path may be used, which may or may not be a phase-locked loop signal path. There may be multiple signal paths for the same feature (such as, for example, fine and coarse phase or frequency control and frequency-locked loop).

[0125]

[0130] In other words, the feature signal path has as input the feature modulated signal coming from the A(Qo)I converter and may be, for example, a feature locked loop, a non-feature locked loop, an adaptive filter loop, a synchronization loop, and / or may be used as control (feedback) for other parts of the signal processing module (e.g., another feature signal path).

[0126]

[0131] By utilizing a feature signal path that is a feature-locked loop, relevant feature values equal to the reference signal can be selected, and the resulting output signal can be fed to the encoder and classifier. In this way, the original non-sparse input signal can be optionally made sparse by filtering out irrelevant information from the signal. Furthermore, rectification and calibration can be performed on the signal input before or after the signal processing module.

[0127]

[0132] Although time-to-digital converters were used in some of the feature signal paths above, the feature signal paths can remain fully analog. Additional timer or delay blocks can be added to the different feature signal paths to synchronize them in the correct manner.

[0128]

[0133] Note that coarse / fine selectivity, ATC control, feedback control, calibration, timing, delay, sampling / sub-sampling mode selection are all configurable / programmable and feature extraction can be made configurable in this way.

[0129]

[0134] FIG. 5 illustrates an example of phase-locked feature extraction. In particular, it illustrates an example of selecting local differences in label-specific data (e.g., through assessment of the phase angle of voltage shifts) through a phase-locked data-driven feature extractor shown in the time-frequency domain. FIG. 5A illustrates a spectrogram of an input signal. FIG. 5B illustrates a spectrogram of the same signal, but showing those components of the signal that have a different phase from the applicable phase reference signal used to extract the relevant feature from the input signal. FIG. 5C illustrates a spectrogram of the same signal, but showing those components extracted by the feature extraction circuit that have a similar phase to the applicable phase reference signal. Each spectrogram shows time in microseconds on the horizontal axis and frequency in kilohertz on the vertical axis. As can be clearly seen, through phase locking, features of interest can be successfully selected from the entire input signal.

[0130]

[0135] 6A-6C show an example of the response of a spiking neural network to a (parameterized) input signal after feature-centric preprocessing. FIG. 6A shows a segment of an analog input signal varying over time, with time plotted on the horizontal axis and signal amplitude plotted on the vertical axis. FIG. 6B shows the input signal of FIG. 6A after preprocessing and feature extraction. The horizontal axis represents time. The vertical axis represents the amplitude of the post-processed (rectified, calibrated, and sparsified) input to a spiking neural network with, for example, 100 neurons.

[0131]

[0136] Figure 6C shows the spike response after applying feature extraction, i.e., the spike response of a spiking neural network to which the post-processed input signal was input. The horizontal axis represents time, while the vertical axis represents each neuron in the spiking neural network. As can be seen, the spike response of the spiking neural network is sparse and more distributed around the apparent feature of interest after feature-centric preprocessing than without the analog-to-information converter and feature filtering. By sampling the input signal at its relevant information / feature rate through the extraction of a specific set of features, signal processing focuses exclusively or nearly exclusively on feature-bearing information while filtering out / suppressing other irrelevant information or distorting interferers, discarding irrelevant information as early in the signal processing pathway as possible. By discarding irrelevant information early in the signal processing pathway, the energy efficiency of the overall system is significantly improved because neurons do not spike during the processing of irrelevant information.

[0132]

[0137] It should be noted that the features of any of the embodiments disclosed herein may be combined in any suitable manner.

Claims

1. A signal processing circuit for a spiking neural network (25), comprising an interface for converting an analog input signal (21) into a corresponding spike-time representation of the analog input signal, wherein the interface is An analog-to-information converter (22, 32) is provided with an input for receiving the aforementioned analog input signal and is configured to generate a modulated signal representing one or more characteristics of the aforementioned analog input signal, A feature detector circuit (23, 33) is provided with an input for receiving the modulated signal, configured to compare the modulated signal with a reference signal representing a reference feature, and configured to generate an error signal indicating the difference between the modulated signal and the reference signal, A feature extractor circuit (23, 34) having an input for receiving the error signal and a locked-loop circuit configured to generate an output signal representing the occurrence of one or more of the features represented by the modulated signal, An encoder circuit (24) having an input for receiving the output signal of the feature extractor and configured to encode the output signal into a spike train for input to the spiking neural network, A signal processing circuit equipped with the following features.

2. The signal processing circuit according to claim 1, further comprising a preprocessing circuit configured to preprocess the analog input signal.

3. The signal processing circuit according to claim 1, wherein the analog input signal is in voltage form and is used as a control voltage input to the analog-to-information converter, thereby determining the modulation of the analog input signal.

4. The features are information extracted from the waveform of the signal, and the features are based on one or more of the following: i) specific characteristics such as transient features and steady-state features; ii) specific properties such as (non)linearity features, statistical features, steady-state features, transfer function features, and energy quantities; and / or iii) specific domain features such as time, delay, frequency, and phase domain features. The signal processing circuit according to claim 1.

5. The analog-to-information converter forms an analog-to-delay (A / delay) converter, an analog-to-frequency (A / frequency) converter, and / or an analog-to-phase (A / phase) converter. The signal processing circuit according to claim 1.

6. The signal processing circuit according to claim 1, wherein the output signal of the feature extractor circuit is fed back to the feature detector circuit to form a negative feedback loop.

7. The signal processing circuit according to claim 1, wherein the feature extractor circuit is driven by the error signal, generates the output signal of the feature extractor circuit, and comprises a voltage-controlled oscillator which is part of the locked-loop circuit.

8. The signal processing circuit according to claim 1, wherein the locking loop circuit is a delay-locked loop, a frequency-locked loop, or a phase-locked loop.

9. The signal processing circuit according to claim 1, wherein the feature extractor circuit further comprises a filter that stops the portion of the error signal if it is not within the locking range of the lock loop circuit, but allows the portion of the error signal that is within the locking range of the lock loop circuit to pass through.

10. The signal processing circuit according to claim 1, wherein the encoder circuit uses rate-based or temporal, spike-based coding as the coding method.

11. The signal processing circuit according to claim 1, wherein the feature extractor circuit is part of a set of parallel feature extractor circuits, and the signal processing circuit further comprises a channel selector unit which is adaptive in such a way that a subset of the most useful features is extracted under specific operating conditions at a given moment, and so one or more feature extractor circuits corresponding to the subset of features of interest can be adaptively selected, and information regarding those features is extracted by the parallel feature extractor circuits from the signal from the analog-to-information converter.

12. The signal processing circuit according to claim 1, wherein the feature extraction performed by the feature extractor circuit can be configured through specific circuit parameters and / or system control parameters.

13. The signal processing circuit according to claim 1, wherein, at runtime, the application / context / condition detection block is configured to determine the current operating context, and from the current operating context, it is configured to determine the optimal set of analog features to be extracted.

14. The signal processing circuit according to claim 1, wherein the analog input signal comprises an analog differential voltage signal having positive and negative voltage inputs.

15. The signal processing circuit according to claim 1, comprising a plurality of feature signal paths for modulated signals, each representing one or more specific features of the analog input signal, wherein at least one feature signal path comprises a programmable fabric which provides control feedback to at least one other feature signal path based on the output of the programmable fabric.

16. The signal processing circuit according to claim 1, further comprising a converter unit configured to convert the modulated signal or the error signal into a digital signal.

17. A signal processing method for converting an analog input signal into a corresponding spike-time representation of the analog input signal for a spiking neural network, Receiving the aforementioned analog input signal, To generate a modulated signal that represents one or more characteristics of the aforementioned analog input signal, The modulated signal is compared with a reference signal representing a reference characteristic, To generate an error signal indicating the difference between the modulated signal and the reference signal, To generate an output signal that represents the occurrence of one or more of the features represented by the modulated signal, Encoding the output signal into a spike train for input to the aforementioned spiking neural network A signal processing method comprising:

18. The features are information extracted from the waveform of the signal, and the features are based on one or more of the following: i) specific characteristics such as transient features and steady-state features; ii) specific properties such as (non)linearity features, statistical features, steady-state features, transfer function features, and energy quantities; and / or iii) specific domain features such as time, delay, frequency, and phase domain features. The signal processing method according to claim 17.

Images