Systems and methods for efficient feature cented analog-pulse encoder

Abstract

A signal processing circuit for a spiking neural network includes an interface for converting an analog input signal to a corresponding pulse-time representation of the analog input signal. The interface includes: an analog-to-information (A / information) converter configured to: generate a modulated signal representative of one or more characteristics of an analog input signal; a feature detector circuit configured to: compare the modulated signal to a reference signal representative of a reference feature, and configured to: generate an error signal indicative of a difference between the modulated signal and the reference signal; a feature extractor circuit comprising a locked loop circuit having an input for receiving the error signal and configured to: generate an output signal representative of an occurrence of one or more features represented by the modulated signal; and an encoder circuit configured to encode the output signal into a pulse sequence for input to the spiking neural network.

Description

Technical Field

[0001] The present disclosure relates generally to automatic signal recognition techniques, and more particularly to systems and methods for energy-efficient feature-centered analog-to-spike encoders, particularly feature-centered analog-to-spike encoders for spiking neural networks. Background Art

[0002] Sampling is the process of converting a signal (e.g., a function of continuous time or space) into a sequence of values ​​(a function of discrete time or space). The sampling rate is the average number of samples taken of the signal in one second.

[0003] A discrete signal can be considered sparse in a certain representation domain if the number of non-zero values ​​in that domain is much smaller than the total signal length. In the case of real-world signals, it is a common case that the number of significant coefficients is small compared to the number of other components. These coefficients can be ignored or set to zero. In other words, a sparse signal can be efficiently represented using a small number of non-zero elements.

[0004] Sparse signals can be accurately represented with lower sampling rates than non-sparse signals. The Nyquist-Shannon theorem states that the sampling rate of a non-sparse signal should be at least twice the highest frequency component, while for sparse signals, the sampling rate can be significantly lower because the energy is concentrated in a few coefficients.

[0005] In many (natural) signal and sensor applications (e.g., vision, acoustics, environmental, medical monitoring systems), the signals are not necessarily sparse due to noise, interfering signals or corruption due to circuit impairments, so standard sub-Nyquist sampling techniques may not be applicable. Sub-Nyquist sampling techniques, also known as compressed sensing or compressed sampling, are methods for acquiring and reconstructing signals at sampling rates significantly lower than those dictated by the Nyquist-Shannon theorem.

[0006] Furthermore, in many of these applications (e.g., speech recognition applications, gesture detection systems, heart rate monitors, etc.), the entire information content of the original signal and its complete reconstruction are usually not of interest. Therefore, processing power and data rate are wasted on processing uninteresting information.

[0007] The power cost in signal conditioning, quantization, and wireless communications all scales with the data rate. If you want to design a system with ultra-low power requirements, you need a new approach to analog / digital system partitioning with the goal of significantly reducing overall power consumption. Summary of the invention

[0008] The inventors of the present invention have recognized that the entire information content of the original signal is usually of interest for its complete reconstruction, but for pattern recognition and signal classification applications, only a specific subset of information extracted from the signal waveform (referred to herein as features of the signal) is usually required. Some examples of waveform features are the maximum signal level over a period of time, the number of zero crossings, etc. These features can provide information about other features, such as the frequency content, phase or timing characteristics of the signal.

[0009] The present invention aims to sample the signal at the relevant information / feature rate of the signal by extracting a set of specific features embedded in the analog signal waveform with the help of pulse modulation and feature-centric adaptive filter / locked loop / synchronization technology before performing modulation matching signal feature encoding and classification. This allows signal processing to focus on feature-bearing information and discard irrelevant information as early as possible in the signal processing path, while filtering out / suppressing other irrelevant information or distortion interference sources. By discarding irrelevant information early in the signal processing path, the energy efficiency of the entire system is significantly improved. This means that smarter analog signal processing (i.e., analog analysis) emphasizes relevant features and reduces the dimensionality of the waveform through feature-preserving transformations, with the intention of classifying the features of the input signal waveform rather than reconstructing the original waveform.

[0010] To achieve this purpose, the present invention discloses, according to a first aspect, a signal processing circuit for a spiking neural network, comprising an interface for converting an analog input signal into a corresponding pulse-time representation of the analog input signal. The interface may include an analog-to-information (A / information) converter, which includes an input for receiving the analog input signal, and may be configured to: generate a modulated signal representing one or more features of the analog input signal. Secondly, the interface may include a feature detection circuit, which may include an input for receiving the modulated signal, and may be configured to: compare the modulated signal with a reference signal representing a reference feature, and is configured to: generate an error signal indicating the difference between the modulated signal and the reference signal. In addition, the interface may include a feature extractor circuit, which includes a locked loop circuit having an input for receiving the error signal, and may be configured to: generate an output signal representing the occurrence of one or more features represented by the modulated signal. In addition, the interface may include an encoder circuit, which may have an input for receiving the output signal of the feature extractor, and is configured to: encode the output signal into a pulse sequence for input to the spiking neural network.

[0011] According to an embodiment of the first aspect, the signal processing circuit may further include a preprocessing circuit which preprocesses the analog input signal, preferably wherein the preprocessing circuit includes a low noise preamplifier, a bandpass filter and / or a programmable gain (post) amplifier.

[0012] According to an embodiment of the first aspect, the analog input signal may be represented as a voltage and may be used as a control voltage input of an A / information converter, thereby determining a modulation of the analog input signal.

[0013] According to an embodiment of the first aspect, the feature may be one or more of the following: i) specific characteristics, such as transient features, steady-state features, ii) specific properties, such as (non-) linear features, statistical features, stationary features, transfer function features, energy content and / or based on iii) specific domain features, such as time domain, delay domain, frequency domain, phase domain features, preferably wherein the A / information converter comprises an analog-to-time converter, which converts an analog input signal into a modulated signal representing specific time domain features (e.g., delay, frequency and / or phase).

[0014] According to an embodiment of the first aspect, the A / information converter may form an analog-delay (A / delay) converter, an analog-frequency (A / frequency) converter and / or an analog-phase (A / phase) converter.

[0015] According to embodiments of the first aspect, the A / information converter may include a voltage controlled delay line, a voltage controlled oscillator and / or a multi-phase voltage controlled oscillator, which performs at least a part of the modulation of the analog input signal into the modulation signal.

[0016] According to an embodiment of the first aspect, the A / delay converter may include a voltage-controlled delay line, the A / frequency converter may include a voltage-controlled oscillator, and the A / phase converter may include a multi-phase voltage-controlled oscillator.

[0017] According to an embodiment of the first aspect, the output signal of the feature extractor circuit may be fed back to the feature detector circuit, thereby forming a negative feedback loop.

[0018] According to an embodiment of the first aspect, the feature extractor circuit may comprise a voltage controlled oscillator which is driven by the error signal and generates an output signal of the feature extractor circuit and is part of a locked loop.

[0019] According to an embodiment of the first aspect, the locked loop may be a delay locked loop, a frequency locked loop or a phase locked loop.

[0020] According to an embodiment of the first aspect, the feature extractor circuit may further comprise a filter, preferably an adaptive filter, which blocks parts of the error signal that are not within the locking range of the locked loop, but allows parts of the error signal that are within the locking range of the locked loop to pass.

[0021] According to embodiments of the first aspect, the encoder circuit may use rate-based encoding or time-based, pulse-based encoding as encoding scheme.

[0022] According to an embodiment of the first aspect, the feature extractor circuit can be part of a group of parallel feature extractor circuits, and wherein the signal processing circuit also includes a channel selector unit, which can be adaptive in such a way that the most beneficial feature subset is extracted at any given moment under specific operating conditions, so that one or more feature extractor circuits corresponding to the feature subset of interest can be adaptively selected, and the parallel feature extractor circuits extract information about these features from the signal from the A / information converter.

[0023] According to an embodiment of the first aspect, the feature extraction performed by the feature extractor circuit can be configured by specific circuit parameters and / or system control parameters. Preferably, the configurable parameters include one or more of gain, linearity, bandwidth, feedforward and feedback, coarse / fine selectivity, ATC control, feedback control, calibration, timing, delay, and sampling / sub-sampling mode selection.

[0024] According to an embodiment of the first aspect, at runtime, the application / context / condition detection block can determine the current operating context and determine the optimal set of analog features to extract from the current operating context, preferably wherein a feedback loop dynamically tunes the specifications of the feature extractor circuit within power or performance constraints, and wherein the runtime configuration manager then activates and configures the relevant feature sets simply by activating the relevant feature detector circuits.

[0025] According to an embodiment of the first aspect, the analog input signal may include an analog differential voltage signal having a positive voltage input and a negative voltage input.

[0026] According to an embodiment of the first aspect, the signal processing circuit may include multiple characteristic signal paths, each characteristic signal path being used to represent a modulated signal of one or more specific characteristics of an analog input signal, wherein at least one characteristic signal path includes a programmable structure, which is preferably driven by a pulse neural network and provides control feedback to at least one other characteristic signal path based on the output of the programmable structure.

[0027] According to an embodiment of the first aspect, the signal processing circuit may further include a converter unit that converts the modulation signal or the error signal into a digital signal.

[0028] According to a second aspect of the present invention, a signal processing method for converting an analog input signal into a corresponding pulse-time representation of the analog input signal for use in a spiking neural network is disclosed. The method comprises: receiving the analog input signal; generating a modulated signal representing one or more features of the analog input signal; comparing the modulated signal with a reference signal representing the reference features; generating an error signal indicating the difference between the modulated signal and the reference signal; generating an output signal representing the occurrence of one or more features represented by the modulated signal; and encoding the output signal into a pulse train for input to the spiking neural network.

[0029] According to an embodiment of the second aspect, the feature may be one or more of the following: i) specific characteristics, such as transient characteristics, steady-state characteristics; ii) specific properties, such as (non-)linear characteristics, statistical characteristics, stationary characteristics, transfer function characteristics, energy content; and / or based on iii) specific domain characteristics, such as time domain, delay domain, frequency domain, phase domain characteristics, preferably, wherein the modulated signal represents specific time domain characteristics, such as delay, frequency and / or phase. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Embodiments will now be described, by way of example only, with reference to the accompanying drawings, in which corresponding reference characters indicate corresponding parts and in which:

[0031] Figure 1 Conventional circuitry for implementing a typical processing path known in the art is shown;

[0032] Figure 2 A signal processing system having an analog-to-information (quantity) converter, an adaptive feature extractor and selector, a pulse encoder and an SNN classifier according to the present invention is shown;

[0033] Figure 3 An overview of the general concepts of converting an analog input signal to a corresponding pulse-time representation using modulation techniques and feature-centric locked loop / synchronization is shown;

[0034] Figure 4 A schematic block diagram of an embodiment including analog-to-time conversion and control as well as coarse and fine filtering is shown;

[0035] Figures 5A-5C shows the selection of local differences in label-specific data (shown in the time-frequency domain) by phase-locking the data-driven feature extractor;

[0036] Figures 6A-6C An example of a spiking neural network response to a parameterized input dataset after feature-centric preprocessing is shown. DETAILED DESCRIPTION

[0037] Some embodiments will be described in more detail below. However, it should be understood that these embodiments should not be interpreted as limiting the scope of protection of the present disclosure.

[0038] Figure 1 A conventional circuit 10 for implementing a typical signal processing path known in the art is shown. The signal processing paths may form channels in a multi-channel sensing system. The maximum number of channels may be constrained by noise, area, bandwidth, power, and / or scalability and expandability of the sensing system.

[0039] Without loss of generality, in conventional circuits, analog circuits in the front-end interface 10A are used to condition the data acquired by the sensing element. Each signal processing path may include, for example, a low noise preamplifier (LNA) 2A, a bandpass filter 3, a programmable gain (post) amplifier (PGA) 2B, an A / D converter (ADC) 4 and / or a serial interface 5.

[0040] The low noise preamplifier 2A is usually the first element in the signal processing path. Its main purpose is to amplify weak input signals while introducing minimal additional noise. The significance of the LNA is that it can improve the signal-to-noise ratio (SNR) of the input signal, thus ensuring that the subsequent stages of the signal processing chain receive a sufficiently strong and clean signal for further processing.

[0041] A bandpass filter 3 allows a specific range of frequencies (called the passband) to pass, while attenuating frequencies outside of that range. The significance of a bandpass filter is to remove unwanted noise or interference outside the desired frequency range. It helps to isolate the signal of interest, improving its quality, and improves the overall SNR by reducing the amount of out-of-band noise or signal.

[0042] The programmable gain amplifier 2B, also called a post-amplifier, is responsible for adjusting the gain of a signal after it has been filtered. It allows the filtered signal to be amplified to an appropriate level for subsequent processing stages, or to match the signal to the dynamic range of the following components. The significance of the PGA is to provide control over the signal gain, adapting to different input signal strengths or adjusting for different processing requirements.

[0043] The analog-to-digital converter (A / D converter) 4 converts a continuous analog signal into a discrete digital representation. It samples the analog signal at a specified rate and quantizes the sampled values ​​into digital codes. The significance of the A / D converter is to convert the continuous time signal into a digital format suitable for digital processing, storage and transmission. It enables the application of various digital signal processing techniques to the captured data.

[0044] The serial interface 5 is responsible for sending the digitized signal to other devices or systems. It facilitates the communication between the signal processing path and other components or systems (e.g., back-end signal processing units).

[0045] The A / D converter 4 can be shared by the (post) amplifier by time multiplexing. Alternatively, in order to reduce the requirements on the amplifier drive capability and to relax the power, noise and crosstalk requirements, the PGA and A / D converter can be combined and embedded in each recording channel. Thus, both signal acquisition and amplification and data conversion are achieved simultaneously.

[0046] The A / D converter output may be fed to a backend signal processing unit 10B (e.g., via a serial interface). The backend signal processing unit 10B may use a feature detector unit 6 (which may, for example, use a threshold) and / or an extraction unit 7 (e.g., using TTL) to provide feature detection, and may additionally or alternatively use a classification unit 8 (e.g., a spiking neural network) to perform signal classification. The extraction unit 7 and the classification unit 8 may each require training 7A, 8A.

[0047] External influences present significant challenges in systems that require a large number of sensing (sub)elements for accurate representation. External influences also present challenges for spatially extensive analysis of specific (sensed) activities, detections or presence. External influences can be attributed, for example, to background activity of neighboring sensing / recording elements / circuits, slight perturbations of sensing / recording element positions, external electrical disturbances, and / or external mechanical disturbances.

[0048] Thus, the inventors of the present invention have recognized that the ability to distinguish or extract relevant signal information from a noisy signal depends on both the difference between the noise-free data acquired from each source and the signal-to-noise ratio (SNR) in the recording system.

[0049] To detect the temporal occurrence of relevant signal information or characteristics or feature sets or data transitions based on the SNR, voltage thresholding can be used (e.g., relative to an estimate of the noise amplitude in the signal), or more advanced techniques (e.g., continuous wavelet transforms) can be used, or energy content-based techniques can be used (e.g., examining the energy content of the waveform within a slice window).

[0050] Voltage thresholding is a signal processing technique used to classify or identify specific events or features in a voltage waveform based on their amplitude or voltage level. A threshold (e.g., an estimate of the noise amplitude in the signal) is set to distinguish between signal components above and below a certain voltage level. The voltage waveform is compared to the threshold, and any portion of the waveform that crosses or exceeds the threshold is considered to meet the detection criteria. The threshold is determined based on the characteristics of the signal and / or the specific features of interest. The threshold can be a fixed value or can be dynamically adjusted based on the nature of the signal or the noise level. The voltage waveform is compared to the threshold, and at each sample point in the waveform, if the voltage amplitude is, for example, above the threshold, it is considered a positive detection, indicating the presence of the desired event or feature. Once the voltage waveform has been thresholded, the identified events or features can be marked, extracted, or used for further analysis.

[0051] The Continuous Wavelet Transform (CWT) is a mathematical technique used to analyze signals and data in the time-frequency domain. The CWT decomposes a signal into a set of small wavelet functions, called wavelets. A wavelet is a wave-like oscillation whose amplitude starts at zero, increases or decreases, and then returns to zero one or more times. Wavelets can be compared to short-lived oscillations. If a portion of the signal is similar, the wavelet is related to the signal. A mother wavelet is used to generate a different wavelet that is convolved with the signal (thus determining the relevance of that particular wavelet to the signal). By varying the scale and translation parameters, the mother wavelet can be adjusted to generate a specific wavelet in order to analyze different features of the signal. The wavelet transform then decomposes the signal into a set of wavelet coefficients that capture its time-varying frequency content. Therefore, the CWT provides a time-frequency representation of the signal, where the amplitude of the CWT coefficient represents the strength of the corresponding frequency component at a specific time. By analyzing the CWT coefficients, various signal properties can be extracted, such as the dominant frequency, the temporal localization of events, and the variation of the frequency content over time.

[0052] Slicing windows (also called sliding windows or analysis windows) are a technique used in signal processing and data analysis to analyze a signal or data sequence in smaller overlapping segments. It involves dividing the signal or data into consecutive and partially overlapping fixed-length windows. A window is usually defined by its length or duration, which determines the size of the segments to be analyzed. The step size or overlap determines how much the window moves between consecutive segments. Larger windows capture more information but provide lower temporal resolution, while smaller windows provide higher temporal resolution but capture less contextual information. By analyzing the energy content within the slicing windows, specific events or features in the signal can be identified and detected. For example, short segments of a signal waveform can be captured using slicing windows, and the energy content within each window can be evaluated. If the energy exceeds a certain threshold, it indicates the presence of an event. The energy content within the slicing windows can be used to extract relevant features from the signal. By calculating statistical measures of the energy (e.g., mean, variance, or higher-order moments), different aspects of the signal can be characterized.

[0053] Without loss of generality, feature extraction (a step performed to simplify and / or optimize the classification process) can be based on distinct details of the input signal: i) specific characteristics, such as transient features, steady-state features, ii) specific properties, such as (non-)linear features, statistical features, stationary features, transfer function features, energy content and / or based on iii) specific domain features, such as time domain, delay domain, frequency domain, phase domain features. The complete process includes several steps, namely i) preprocessing (including, for example, signal detection, energy normalization and denoising), ii) feature domain analysis, iii) sub-range selection, specifying the selection of ranges of different variables / parameters suitable for different feature subsets of interest, iv) final feature calculation, and v) classification.

[0054] Transient features are temporary or short-term phenomena that occur during the initial or transition period of a signal or system. They represent sudden changes or disturbances in a signal. Therefore, transient features usually exhibit characteristics such as rapid changes, high-frequency components, and short durations. They often contain rich spectral content and have a unique time profile. The analysis of transient features may require, for example, high time resolution and the ability to track rapid changes in signals. Techniques such as time domain analysis, waveform analysis, or high-resolution time-frequency analysis (e.g., continuous wavelet transform introduced above) are usually used for transient feature extraction. On the other hand, steady-state features represent the long-term or stable behavior of a signal or system after the initial transient period has passed. They correspond to the stable or repetitive part of the signal that continues to extend when the system reaches a stable state. Steady-state features exhibit characteristics such as relatively constant amplitude, stable spectral components, and regular time patterns. Analyzing steady-state features usually involves examining properties such as average amplitude, frequency content, statistical measurements, harmonic structure, or period. Steady-state features can be analyzed using techniques such as Fourier analysis, power spectral density estimation, or statistical analysis.

[0055] Analyzing a signal based on specific properties involves applying specific techniques or methods to extract relevant information about each feature.

[0056] For example, for (non-)linear features, a phase space reconstruction can be performed (in order to determine the underlying dynamics of the signal, which can reveal nonlinear behavior and provide insights into the complexity of the system), or the Lyapunov exponent can be determined (which quantifies how chaotic or predictable a signal is; a higher exponent indicates chaotic or nonlinear behavior).

[0057] For statistical features, statistical measures such as mean, variance, skewness and kurtosis can be calculated to characterize the distribution and central tendency of the signal. In addition, autocorrelation functions can be calculated to analyze time dependence or repetitive patterns in the signal.

[0058] For static features, you can, for example, apply Fourier transform techniques or spectral analysis techniques to examine the frequency content and power spectrum of the signal. This helps identify the main frequencies and spectral characteristics. In addition, you can perform wavelet analysis, such as continuous wavelet transform (CWT) or discrete wavelet transform (DWT), to analyze the time-frequency characteristics of the signal and identify local features.

[0059] For energy content, one can, for example, calculate the energy or power of a signal within a specific frequency band or time window to identify the main frequency components or evaluate the power distribution of the signal. Another option could be to apply a short-time Fourier transform to obtain a time-frequency representation of the signal and analyze the distribution of energy over time and frequency. Another option could be to extract the envelope of the signal using techniques such as the Hilbert transform or amplitude modulation analysis and then analyze the energy variation of the signal envelope.

[0060] Analyzing a signal based on specific domain features (eg, time, frequency, or phase) may involve applying domain-specific techniques to extract relevant information.

[0061] For time domain analysis, statistical measures such as the mean, variance, and higher-order moments of a signal can be calculated to characterize its amplitude distribution and variability. Additionally, techniques such as autocorrelation can be applied to identify repeating patterns or periodic components in the signal.

[0062] For frequency domain analysis, techniques such as the Fourier transform or fast Fourier transform (FFT) can be applied to convert the signal from the time domain to the frequency domain. This produces a spectrum of the signal, from which the amplitude and phase of its frequency components can be analyzed. The power spectral density can be determined, which can provide insight into the distribution of power over the different frequencies from which the signal is constructed. The short-time Fourier transform (STFT) can be used to determine the time-varying frequency content of the signal.

[0063] For phase domain analysis, the phase component of the signal can be extracted by applying techniques such as Hilbert transform or phase unwrapping. This can reveal the phase relationship between different frequency components or oscillations. Metrics of phase coherence, such as the phase locking value (PLV), can be measured to assess the degree of synchronization or coupling between different oscillating components.

[0064] Time domain waveforms, spectra, and phase relationships can be compared together to gain a more complete understanding of a signal's characteristics. For example, you can use time domain analysis to detect transient events or sudden changes in a signal and then examine their frequency and phase characteristics. Additionally, you can study time-frequency dynamics to analyze how the frequency content of a signal changes over time.

[0065] It can be valuable to use a combination of techniques to obtain a comprehensive analysis of your signal.

[0066] Based on these (extracted) features, the relevant data can be classified into m-dimensional clusters, each cluster corresponding to the activity of a single sensing element / signaling channel, via k-means, expectation maximization (EM), template matching, Bayesian clustering, and artificial neural networks (ANN) or spiking neural networks (SNN).

[0067] Therefore, the result of the classification process is the formation of m-dimensional clusters. A cluster represents a group of data points with similar characteristic patterns or characteristics. The extraction of features involves converting the raw data into a simplified set of meaningful features that capture relevant information. Clusters are formed based on the similarity or proximity of these extracted features. Data points within the same cluster are more similar in their feature representation than data points in different clusters. The dimension "m" of a particular cluster refers to the number of features or variables used to describe the data points within the cluster. Each cluster represents the activity or behavior of a single sensing element or signal channel. In other words, based on the extracted features, the data points within each cluster are similar to each other, indicating that they belong to the same sensing element or signal channel.

[0068] The unsupervised learning algorithm k-means clustering groups data points into clusters based on similarity.

[0069] “k” represents the desired number of clusters.

[0070] EM is a statistical algorithm commonly used for clustering and classification, especially in the presence of hidden or latent variables. It iteratively estimates the parameters of a statistical model to assign data points to appropriate clusters.

[0071] Template matching is a technique that compares a predefined template or pattern to data and uses a similarity measure to identify matches or correspondences. It is often used in pattern recognition and signal matching.

[0072] Bayesian clustering applies the principles of Bayesian inference to cluster data points. It incorporates prior knowledge and updates the probabilities to determine the most likely clustering of the data.

[0073] An artificial neural network is a signal processing system consisting of interconnected artificial neurons organized into one or more layers. Each artificial neuron receives an input signal, combines it with a specific weight, and applies an activation function to produce an output. The connections between neurons (called synapses and have adjustable weights) allow information to propagate through the network, enabling the network to learn and make predictions or decisions based on the input data. ANNs themselves do not consider the timing of events or the temporal dynamics of information. They focus primarily on the magnitude of the input and its propagation through the network.

[0074] Spiking neural networks (SNNs) are signal processing systems whose design is inspired by biological neural networks. Information is encoded as a pattern of spike signals distributed across a network of neurons and synapses, as described in WO2022 / 090542A1 filed by the present applicant, which is incorporated herein by reference in its entirety. By mimicking the processes that occur in biological brains, spiking neural networks can perform signal processing tasks that are typically performed by the human brain. Examples include image recognition, sound recognition, and event detection based on input from multiple sensors.

[0075] Without loss of generality, the present invention implements feature processing / classification as a neuromorphic event-based neural network. The time-continuous SNN architecture provides a platform that minimizes the size and power consumption of real-time behavioral systems by explicitly matching the computational principles behind autonomous adaptive behavior. In a time-continuous pulse architecture, state variables evolve naturally over time, eliminating the need for clocks and additional circuits to manage the representation of time; the circuit is driven directly by the input data, i.e., synapses receive input pulses, and neurons generate output pulses at the rate of input data. Therefore, in the case of applications where signals have sparse activity in space and time, most neurons will remain silent at any time, minimizing system power consumption. In essence, this (time-aware pulse) neural network implementation uses a mixed analog digital signal representation, i.e., pulse / spike sequences send analog information at the timing of events, which are converted back to analog signals in the dendrites (inputs) of neurons.

[0076] In the nervous system, there are a variety of information encoding schemes that are specialized for efficiently representing specific data modalities originating from different senses (e.g., visual, auditory, somatic). For neural computation, such coding schemes fall into two broadly defined competing approaches, namely rate-based coding and temporal spike-based coding. In rate-based coding, information representation and computation are not based on individual spikes, but on spike firing rates, i.e., information is embedded in the instantaneous or average spike generation rate of a single or a group of neurons, so this scheme can be subdivided into count, density, and population rate coding. Count coding can emphasize the absolute number of spikes, density coding can normalize the counts by considering the time window in which the counts are made, and population rate coding can consider the average firing rate of a group of neurons.

[0077] When designing a real-time neuromorphic system, the final choice of a specific encoding scheme should be evaluated from different perspectives (i.e., algorithm level, training process, and hardware implementation perspective) and their impact and performance metrics and parameters should be evaluated over a wide range (e.g., in terms of classification accuracy, processing latency, energy consumption, number of synaptic operations, hardware cost, network compression efficiency, noise resilience, and fault tolerance).

[0078] For practical implementation of the proposed feature extraction mechanism, it may be important to include the signal modulation details and its inherent relationship with signal distortion caused by hardware imperfections (especially for low SNR).

[0079] These features can be activated, for example, feature-based feature extraction can be performed and configured at runtime based on multiple parameters, such as region or window length in time, delay, frequency domain or phase domain, gain, bandwidth, etc. The region / window length may be important when considering feature extraction based on the non-uniform nature of the signal (in the form of a specific distribution sub-region) rather than defining it as a uniform distribution (along the entire original feature). Therefore, the modulation aspect of the signal can be characterized by the non-uniform nature of the signal (in the form of a specific distribution sub-region). Therefore, for selection, the feature vector can be split into multiple overlapping sub-regions. The overlap can be regarded as a sliding window of width w, and the sliding step size used to filter the entire feature is s, satisfying s = [(nw) / k], where n is the feature length and k is the number of windows. Without loss of generality, in order to emphasize specific feature performance / separability, the inter-class and intra-class feature dispersion is measured for each window, and sub-regions with strong class separability are selected. To do this, for example, a predefined metric can be used to measure separability.

[0080] Inter-class feature dispersion refers to the distribution or spread of feature representations between different classes or categories in a classification problem. It measures the separation or difference between feature representations of different classes. Inter-class feature dispersion is important for classification tasks because well-separated classes make it easier for the classifier to distinguish between them. When classes have low inter-class feature dispersion, they may overlap in the feature space, resulting in higher classification errors or ambiguity. On the other hand, high inter-class feature dispersion is conducive to better discrimination and classification accuracy.

[0081] Intra-class feature dispersion refers to the distribution or spread of feature representations within the same class or category in a classification problem. It measures the variation or difference between feature representations belonging to the same class. By minimizing the intra-class feature dispersion, the classifier can better capture the unique characteristics shared by instances within the same class. This can improve classification accuracy because the classifier can make more confident and accurate predictions.

[0082] Therefore, the separability and distinguishability of different classes can be enhanced by maximizing the inter-class feature dispersion while minimizing the intra-class feature dispersion.

[0083] Statistical methods can be used to obtain representative features of the distribution characteristics of different devices (e.g., integrated circuits and systems that process specific sensor signals), such as evaluating the size of local differences between representative features (e.g., in a separable feature space) and determining the location of their concentrated distribution based on weighted entropy.

[0084] Figure 2 A signal processing system 20 having an analog-to-information converter, an adaptive feature extractor and selector, a pulse encoder, and an SNN classifier according to the present invention is shown.

[0085] A physical input signal 21 (which may or may not have been previously conditioned) enters the signal processing system. Conditioning may include the use of, for example, a low noise preamplifier (LNA), a bandpass filter, a programmable gain (post) amplifier (PGA). An analog-to-information (quantity) (A(Qo)I) converter (more generally, an analog-to-information converter 22) takes the (analog) physical input signal 21 and may extract from the analog signal 21 the information most relevant to a given application to digitize it. The output of the A(Qo)I converter 22 may be a modulated signal of the input signal, using Figure 3 Specifically, the A(Qo)I converter 22 obtains significant details of the physical input signal 21, such as: i) specific characteristics, such as transient characteristics, steady-state characteristics, ii) specific properties, such as (non-) linear characteristics, statistical characteristics, stationary characteristics, transfer function characteristics, energy content or based on iii) specific domain characteristics, such as time domain, frequency domain, phase domain characteristics.

[0086] Next, the modulated signal generated by the A(Qo)I converter 22 is sent to the channel selector unit and extractor unit 23. As described above, each signal channel that can be selected by the channel selector unit and extractor unit 23 can correspond to a specific cluster, where each cluster represents the activity of the signal channel and is described by a set of specific features. Therefore, the data in the physical signal 21 can be classified into one or more clusters based on which features are present in the signal. Note that if the A(Qo)I converter includes a voltage-controlled delay line or a voltage-controlled oscillator, event-driven voltage-controlled delay lines and voltage-controlled oscillators can be used, so if a signal appears at the input of the A(Qo)I22, the signal will subsequently appear at the input of the subsequent signal channel. The task of the adaptive extractor / filter is to pass only the relevant signal information to the SNN reasoning and control engine 25.

[0087] The extractor unit 23 may comprise a set of parallel feature extractors 23A, 23B, etc., each acting as a channel selector, each channel representing a (sub)region of the relevant feature domain. For example, a channel may be defined by a certain (sub)region of the delay domain, frequency domain or phase domain of the input signal. The channel selector unit may be adaptive in such a way that the most beneficial subset of features under certain operating conditions is activated at any given moment, so that one or more signal channels corresponding to the subset of features of interest may be adaptively selected and information about these features extracted from the signal from the A(Qo)I converter by means of the parallel feature extractors comprised in the extractor unit.

[0088] Therefore, this approach may require a set of configurable feature extractors. In one embodiment, feature extraction is performed by the feature extractor through feature-centric adaptive filter / locked loop / synchronization techniques. Feature extraction is usually circuit / system specific because it is configurable, i.e., through specific circuit parameters (e.g., gain, linearity, bandwidth) and system (feedforward and feedback) control parameters. For example, in Figure 4 In an embodiment, coarse / fine selectivity, ATC control, feedback control, calibration, timing, delay, sampling / sub-sampling mode selection are all configurable / programmable.

[0089] The features generated by the extraction unit 23 are fed to a classifier or information extractor which performs application specific information estimation, such as speech recognition.

[0090] For example, features from the extraction unit 23 are fed to a pulse encoding stage 24, for example, via a multiplexer. The pulse encoding stage 24 encodes the features into a pulse train, which is a sequence of pulses (action potentials) that represent information in a time and event-based manner. As described above, rate-based or time-based, pulse-based encoding can be used for the encoding scheme. In this way, the feature information extracted from the physical signal is converted into the timing, rate or pattern of pulses, which are then fed into the SNN processor or classifier 25.

[0091] The SNN outputs information and / or inference metrics 26, which are quantitative metrics used to evaluate the performance of the SNN in classifying physical signals. The SNN inference engine can output two signals: (i) an inference result, i.e., a designation of a particular class to which the input signal belongs, and (ii) an inference metric, such as inference accuracy, which is a quantitative metric for evaluating the performance of classifying physical signals.

[0092] As mentioned above, the SNN 26 can perform inference / classification. In addition, the SNN can also perform control functions, although as a separate implementation. The application / context / condition detection block 27 represents such a control engine that controls circuits and system parameters, such as adaptive channel selectors and extractors. The application / context / condition detection block receives two inputs, one from the SNN inference engine (e.g., inference accuracy) and a second from the selectors, extractors, and encoding stages that need to be controlled or calibrated. For example, see Figure 4 for more detailed information.

[0093] The confidence of the resulting estimate depends strongly on environmental conditions, such as signal dynamics or signal interference. At runtime, the application / context / condition detection block 27 determines the current operating context and, therefore, the best simulation feature set. A feedback loop dynamically tunes the specifications of the feature extraction units within power or performance constraints. The runtime configuration manager of the selector unit and the extractor unit then activates and configures only the relevant feature sets.

[0094] The parameter block 28 specifies the parameters of the selector / control unit operation to optimize a specific figure of merit, i.e., an efficiency factor (e.g., energy efficiency, energy-throughput efficiency, energy-throughput-precision efficiency) or just a specific task (e.g., noise filtering, variability limits / bounds / calibration, channel synchronization, or signal alignment). Training and test data across processing tasks 29A may be required as input to understand which (performance parameter) settings to select for a specific processing task. In addition, parameterized performance settings 29B may be used as input to the parameter block.

[0095] It should be noted that the extraction of features incurs certain costs, such as computational costs (e.g., searching for a specific pattern vs. counting the number of occurrences) or memory costs (e.g., storing the (running) value of the feature). Therefore, if the cost of feature extraction (or the available budget in terms of power / performance / area (PPA)) is taken into account for pattern recognition (or classification), the performance of the classifier depends on the cost of the features used. The (feature) memory cost depends on, among other things, the number of features and the feature vector size / accuracy. Which type of memory device is used (e.g., flip-flop based, MOS capacitor based, or floating gate based) depends on the circuit implementation. The running value of the feature can be used in the feature extractor.

[0096] Figure 3 An overview of a system 30 is shown that converts an analog input signal 31 to a corresponding pulse-time representation utilizing modulation techniques and feature-centric locked loop / synchronization. Figure 3 The system of FIG. 1 shows various optional circuits for processing input signals in the delay domain, frequency domain, and / or phase domain, which can be Figure 2 For example, the A(Qo)I converter 32 may be implemented as the A(Qo)I converter 22, the detector circuit 33 and the feature extractor 34 may be implemented as the channel selector and the extractor 23, and the pulse encoding stage and the classifier 35 may be implemented as the pulse encoder 24 and the classifier 25.

[0097] The physical signal 31 (which may be pre-conditioned) enters the A(Qo)I converter 32, which may be considered as part of the front-end interface 30A, from which the modulated signal is obtained. Conditioning may include the use of, for example, a low noise preamplifier (LNA), a bandpass filter, and / or a programmable gain (post) amplifier (PGA). The A(Qo)I converter may include an analog-delay (A / delay) converter 32A, an analog-frequency (A / frequency) converter 32B, and / or an analog-phase (A / phase) converter 32C. The input signal may also be converted to other parameters besides delay, frequency, or phase. The physical signal may be represented by a voltage and may be used as a control voltage input for each of the one or more converters. The A / delay converter 32A may include a voltage controlled delay line (VCDL). The A / frequency converter 32B may include a voltage controlled oscillator (VCO). The A / phase converter 32C may include a multi-phase voltage controlled oscillator (multi-phase VCO). The physical signal that may be used as the control voltage input for each converter thereby determines the amount of delay, frequency, and / or phase of the modulated signal. For example, for A / frequency converter 32B, the physical signal may be used as a control voltage input to a VCO, and a higher voltage input produces a higher frequency of the modulated output signal.

[0098] Next, the modulated signal can be fed to a detector circuit 33, which can include a delay detector circuit 33A, a frequency detector circuit 33B and / or a phase detector circuit 33C, in which, for example, the delay, frequency and / or phase are compared with corresponding reference delay, frequency and / or phase signals. The reference can be predetermined or dynamically determined. Thereby, the detector 33 can generate an error signal 38, which is proportional to the difference between the modulated signal and the reference signal with respect to, for example, delay, frequency and / or phase. The error signal can then be filtered and used to drive the feature extractor 34.

[0099] The feature extractor 34 may include a VCO 34A, 34B, and / or 34C that generates an output delay, frequency, and / or phase signal. In addition, an adaptive filter may be present in the feature extractor 34. If the error signal 38 is not within the locking range of the VCO 34A, 34B, 34C, the filter may block the error signal 38, but if the error signal 38 is within the locking range, it may pass. The output signal may represent the occurrence in time of one or more features represented by the modulated signal, but may also represent the possibility (e.g., a specified probability) of the feature being present in the signal at a particular time. The output signal may be fed back to the input of the system, thereby generating a negative feedback loop. If the output delay, frequency, and / or phase drifts, the error signal will increase, thereby driving the delay, frequency, and / or phase of the VCO in the opposite direction to reduce the error. Therefore, the output delay, frequency, and / or phase is locked to the corresponding delay, frequency, and / or phase of the modulated signal, or is locked to the error signal.

[0100] The detector circuit 33 and the feature extractor circuit 34 may be implemented by a set of parallel circuits (e.g., Figure 2 The feature extractor / channel selector 23A, 23B in the embodiment is implemented as shown, which performs detection and feature extraction on the corresponding signal channels, each channel representing a sub-region of the relevant delay domain, frequency domain or phase domain of the input signal.

[0101] The output signal of the feature extractor 34 is fed to a pulse coding stage 35, which encodes the output signal into a pulse train. As described above, the coding scheme can use rate-based or time-based, pulse-based coding. In this way, the feature information extracted from the physical signal 31 is converted into the timing, rate or pattern of pulses, which are then fed into an SNN processor or classifier (also represented by the number 35). The pulse coding stage and the classifier can be implemented together as one circuit, or separately. The output of the pulse coding stage and the classifier 35 can be, for example, a classification indicator 36. It may be necessary to train 37 the feature extraction circuit 34 and the encoding circuit 35, as well as train the classifier.

[0102] The type of encoding used in encoding circuit 35 may vary depending on the type of parameters used in converter 32, detector 33, and feature extractor 34. When looking at delay parameters, time to first pulse (TTFS), inter-pulse interval (ISI), burst or delay synchronization encoding may be used. When looking at frequency parameters, rate or frequency synchronization encoding may be used. When looking at phase parameters, phase or phase synchronization encoding may be used.

[0103] First spike time coding is a method used in spiking neural networks (SNNs) to represent information based on the timing of the first spike generated by a neuron in response to a stimulus or input. In this coding scheme, information is represented by the time it takes a neuron to fire its first action potential (spike) after receiving a particular input. In first spike time coding, neurons are configured in such a way that they respond quickly to specific features or patterns in the input, and the first spike encodes the presence of these features. For example, if a neuron is sensitive to a certain visual pattern, it will quickly fire its first spike when it detects that pattern in the input.

[0104] Inter-spike interval (ISI) coding is a method used in spiking neural networks (SNNs) to represent and process information based on the timing of spikes generated by neurons. In ISI coding, information is represented by the time intervals between consecutive pulses fired by a neuron. Each pulse can be timestamped, and the duration between two consecutive pulses is measured as the inter-spike interval. The pattern of inter-spike intervals carries information about the input being processed.

[0105] Burst coding in spiking neural networks is a method used to represent and process information by encoding spike patterns in short bursts or bursts of action potentials. It involves a group of spikes (often with high-frequency firing) generated closely in time, followed by a period of relative inactivity, after which another burst of spikes occurs. In traditional single-spiking neurons, information is usually encoded in individual spikes. However, in burst coding, information is represented in a temporal pattern of bursts.

[0106] In rate coding, information is represented by the frequency or rate at which a neuron generates spikes over time. A higher firing rate generally indicates the presence or strength of a particular feature in the input data, while a lower firing rate can indicate the absence or weaker representation of that feature. The key idea behind rate coding is to exploit the time integration of spike trains. Instead of considering individual spikes, the network looks at the accumulated activity over a period of time. Rate coding can rely on the overall firing rate of spiking neurons over time, or the average firing rate of neurons over a period of time.

[0107] Phase encoding is a method of representing and processing information by encoding it in the relative phase relationship between spikes fired by different neurons. This encoding scheme uses the precise timing of spikes to convey information. In phase encoding, neurons can be designed or tuned to fire their spikes at specific phases of a periodic cycle. For example, neurons can be configured to fire at the peak of an oscillating signal or at specific phases of a rhythmic input. The relative phase relationship between the spikes of different neurons encodes information about the input. The key idea behind phase encoding is that the timing of spikes carries valuable information, and by synchronizing the spikes of multiple neurons to specific phases, the network can represent and process specific features or temporal patterns in the input data.

[0108] Synchronous coding (e.g., using delay, frequency, or phase) in spiking neural networks is a method used to represent and process information by encoding patterns of synchronized spiking between different neurons. It involves neurons firing action potentials simultaneously or close in time to convey specific information or encode some feature of the input. Synchronous coding exploits the precise timing of synchronized spiking activity on multiple neurons to efficiently represent information. The simultaneous or nearly simultaneous firing of neurons can convey information about correlations, patterns, or specific features in the input data. For example, the presence of a specific feature in the input can be represented by the simultaneous firing of neurons, while the absence of that feature can result in unsynchronized or unrelated activity.

[0109] Without loss of generality, in the present invention, the conversion of the analog input signal into a corresponding pulse-time representation in the A(Qo)I converter 32 includes: first converting the signal information (quantity) in the transition of the input signal 31 (rather than the instantaneous amplitude) by some form of modulation, that is, by representing and processing the signal using (pulse-time) properties as the delay, frequency and phase of the signal. The transition here can be between two well-defined values, such as between the ground and power supply voltage levels of the input signal. Correspondingly, this conversion of analog-information (quantity) (e.g., analog-delay, analog-frequency, analog-phase) enables the manipulation of the sampled information using delay difference, frequency difference, phase difference variables as related events occurring relative to a reference.

[0110] As described above, in the detector circuit 33 and the feature extractor circuit 34, the relevant information is extracted by a feature-centric adaptive filter / locked loop / synchronization technique, and then modulation matching signal feature encoding is performed (for example, for analog-phase modulated signals, features are extracted by a phase-locked loop and represented using phase encoding), and classification / recognition is performed. In one embodiment of the present invention, time modulation (i.e., analog-to-time conversion) is performed using a (adaptive) voltage-controlled delay line (VCDL) circuit. In another embodiment, frequency modulation is performed using a voltage-controlled oscillator (VCO).

[0111] VCO-based analog-to-frequency conversion provides high gain, high frequency, low power, high linearity, and small implementation through different trade-off mechanisms. In yet another embodiment, phase modulation (i.e., analog-to-phase conversion) is performed with a multi-phase VCO circuit. After pulse modulation is completed, relevant information is extracted and selected by feature-centric locked loop / synchronization techniques (e.g., delay locked loop (DLL), frequency locked loop (FLL), and phase locked loop (PLL)), corresponding to delay, frequency, and phase modulation, respectively. A DLL is a circuit similar to a phase-locked loop, the main difference being that there is no internal VCO, instead a delay line is used.

[0112] In one embodiment, the delay / frequency / phase detector circuit 33 uses the output delay / frequency / phase difference to close the feedback loop, thereby providing an up / down feedback signal. When the up / down signals are combined and added to the discriminator (e.g., feature selector 34), the response of the feature of interest is extracted, that is, the input delay / frequency / phase is matched to the reference delay / frequency / phase in the detector circuit 33.

[0113] Thus disclosed is a signal processing circuit 30 for a spiking neural network, which may include an interface for converting an analog input signal 31 into a corresponding pulse-time representation of the analog signal. The interface may include: a delay circuit 32A for converting the analog input signal into a corresponding delayed input signal having a value representing the delay of the analog signal; a frequency circuit 32B for converting the analog input signal into a corresponding frequency input signal having a value representing the frequency of the analog signal; and / or a phase circuit 32C for converting the analog input signal into a corresponding phase input signal having a value representing the phase of the analog signal. The signal processing circuit 30 may also include a feature extraction unit 34, which includes a delay signal processing circuit 33A, 34A connected to receive the delay input signal, a frequency signal processing circuit 33B, 34B connected to receive the frequency input signal, and / or a phase signal processing circuit 33C, 34C connected to receive the phase input signal.

[0114] The delay circuit 32A may include a voltage-controlled delay line circuit having an input for receiving the analog input signal 31, wherein the voltage-controlled delay line circuit 32A is configured to generate a delayed input signal. The frequency circuit 32B may include a voltage-controlled oscillator circuit having an input for receiving the analog input signal 31, wherein the voltage-controlled oscillator circuit is configured to generate a frequency input signal. The phase circuit 32C may include a multi-phase voltage-controlled oscillator circuit having an input for receiving the analog input signal 31, wherein the multi-phase voltage-controlled oscillator circuit is configured to generate a phase input signal.

[0115] The delay signal processing circuit 33A, 34A may include a delay phase-locked loop circuit having a first input for receiving a delayed input signal and a second input for receiving a delayed reference signal, wherein the delay phase-locked loop circuit is configured to generate a delayed output signal based on a comparison of the delayed input signal and the delayed reference signal. The frequency signal processing circuit 33B, 34B may include a frequency-locked loop circuit having a first input for receiving a frequency input signal and a second input for receiving a frequency reference signal, wherein the frequency-locked loop circuit is configured to generate a frequency output signal based on a comparison of the frequency input signal and the frequency reference signal. The phase signal processing circuit 33C, 34C may include a phase-locked loop circuit having an input for receiving a phase input signal and a second input for receiving a phase reference signal, wherein the phase-locked loop circuit is configured to generate a phase output signal based on a comparison of the phase input signal and the phase reference signal.

[0116] Figure 4 A schematic block diagram of an embodiment 40 is shown, including analog-to-time conversion and control as well as coarse and fine filtering.

[0117] Without loss of generality, in the present disclosure, a time-based feature-centric locked loop / synchronous processing circuit enables manipulation of sampled analog information using a time difference variable as the amount of time between events relative to a reference time or occurrence of the event.

[0118] Time encoding consists of representing analog signals with modulated square waves, where signal information is encoded in transitions rather than instantaneous amplitudes, i.e., the input signal is converted into a time difference variable. Therefore, the time signal is then processed by various circuits to produce a time difference output, where the pulse modulator encodes the analog input signal information in the pulse width, frequency, or position of the signal (or signal set) at only two levels. The two-level signal is still analog and is then sampled at the sampling frequency. Since the signal has only two values, the sampler can now be a simple D-type edge-triggered trigger. From the sampled square wave, a multi-bit approximation of the analog input signal can be reconstructed with a digital low-pass filter. The time interval between two clock edges is converted to digital using a time-to-digital converter (TDC). As with all sampling processes, continuous time signals must be band-limited to prevent aliasing effects. In one embodiment, a PLL is used to band-limit the phase modulated input signal. TDC can be implemented with a time comparator (D flip-flop) and other digital blocks (e.g., a single counter, a flash TDC, a Vernier oscillator, a cyclic pulse contraction TDC).

[0119] The circuit includes a programmable analog-to-time (A / T) converter, a coarse tuner, and an enhanced range fine tuner with an in-loop filter characteristic, supplemented by a background calibration mechanism to reduce the effects of non-idealities. It combines the advantages of a sampling loop and a subsampling loop to achieve robust time, phase, and frequency acquisition in the presence of frequency interference and low-power operation. During the switch from the sampling loop to the subsampling loop, the phase remains locked without any discontinuity.

[0120] In one embodiment, the circuit first performs analog-to-time conversion (ATC), that is, when an analog voltage is sensed, a time pulse conversion is performed to embed analog information in the time period between the rising edge and the falling edge. The analog signal and its differential signal (180° phase shift) are provided to two voltage-controlled delay line circuits, so that their output signals are time pulses that depend on the amplitude of the input signal. In order to obtain sufficient dynamic range, a control circuit is added to combine the outputs of the voltage-controlled delay line circuits and then generate a single time pulse that represents the analog input signal by the duration of the pulse width. The control circuit also averages a programmable number of samples to obtain the target oversampling rate.

[0121] To improve the phase lock time, a coarse T / D converter is inserted after the phase detector to quantify the coarse phase error and assist in coarse phase and frequency lock. The phase detector and frequency detector work in conjunction with switching feedback, which monitors frequency disturbances and ensures that the digitally controlled oscillator (DCO) is always locked to the correct frequency. However, switching directly from sampling mode to subsampling mode in the feedback path will cause phase discontinuity due to the delay in the feedback path. Therefore, an additional delay is inserted after the ATC to mitigate the phase discontinuity. When the sampling mode is used for input signal time / frequency / phase acquisition, the (SNN) controller generates relevant information for controlling the feedback loop and A / T conversion. If the PLL is locked to an incorrect frequency, the frequency detector can be used to switch the feedback from subsampling mode to sampling mode for frequency lock.

[0122] In more detail of this embodiment, the input signal enters the analog-to-time converter. The input signal can be a differential signal, including a positive input voltage and a negative input voltage, to obtain a better dynamic range. The positive input voltage and the negative input voltage can pass through a voltage-controlled delay line before entering the analog-to-time converter. The input signal can be pre-conditioned, and the conditioning can include the use of, for example, a low noise preamplifier (LNA), a bandpass filter, and a programmable gain (post) amplifier (PGA).

[0123] An analog-to-time converter is a converter that converts an analog input (voltage) signal (which may be pre-processed, or may include several input voltages (for differential signals)) into a time parameter. The time parameter may be, for example, delay, frequency and / or phase, as described in relation to Figure 3 In this embodiment, the analog-to-time converter converts the analog input signal into a delayed, frequency and phase modulated signal, but more or fewer conversions may be performed.

[0124] Next, there are essentially four signal paths: a frequency signal path, a delay signal path, a coarse phase signal path, and a fine phase signal path.

[0125] The coarse phase signal path may include, for example, a phase detector 44, a time-to-digital converter 45, and a coarse filter 46. The fine phase signal path may include, for example, a delay block, a time-to-digital converter 45, and a fine filter 47. The coarse phase signal path and the fine phase signal path are both connected to a digitally controlled oscillator 48, and each forms a phase-locked loop as described above. As described above, the phase-locked time of the fine phase-locked loop can be improved by using a coarse phase-locked loop to quantize the coarse phase error and assist in coarse phase locking. The output signal 49 of the digitally controlled oscillator 48 can be fed to a pulse encoding stage as described above, which encodes the output signal into a pulse sequence. As described above, the encoding scheme can use rate-based or time-based, pulse-based encoding. In this way, the characteristic information extracted from the physical signal is converted into the timing, rate, or pattern of pulses, which can then be fed into an SNN processor or classifier.

[0126] The frequency signal path may include, for example, a timer control 50, a time-to-digital converter 45, and a digital frequency detector 51. The timer control 50 may ensure synchronization of the different signal paths. The delay signal path may include, for example, a phase detector 44 and a delay control logic 52. In this embodiment, both the frequency signal path and the delay signal path are connected to a programmable structure 53, such as a spiking neural network.

[0127] The programmable fabric 53 connected to the frequency and delay signal path and / or the numerically controlled oscillator 48 can provide an output signal to a control feedback 54, which can provide feedback to one or more different signal paths and / or the analog-to-time converter 43. In an example, the control feedback provides feedback to the coarse phase signal path and the fine phase signal path and the analog-to-time converter 43.

[0128] The control feedback 54 may also be controlled by a control signal from the control block 42. The control block 42 may be driven by a spiking neural network. The control block 42 together with the calibration block 44 may control the analog-to-time converter 43 through, for example, programmable logic. The calibration block 44 may be responsible for calibrating a specific portion of the overall signal processing module and may take certain system settings or calibration data as input.

[0129] Note that although one analog-to-time converter 43 is shown, multiple analog-to-time converters may be present in the signal processing path. Furthermore, typically, the analog-to-time converter may be an A-to-Information converter or an analog-to-information converter that extracts not only the time parameters of the signal (e.g., frequency, delay, and / or phase), but also other parameters of the signal that may be of interest to a particular application.

[0130] Different characteristic signal paths (e.g., frequency signal path, delay signal path, and phase signal path) may be phase locked by connection to a numerically controlled oscillator, or may be connected to a programmable fabric. One or more characteristic signal paths may be omitted or added. For example, a frequency locked loop signal path and / or a delay phase locked loop signal path may be added. Instead of a fine phase signal path and a coarse phase signal path, only one phase signal path may be used, which may or may not be a phase locked loop signal path. Multiple signal paths may exist for the same characteristic (e.g., fine phase and coarse phase, or frequency control and frequency locked loop).

[0131] In other words, the characteristic signal path has a characteristic modulation signal from the A(Qo)I converter as input and can be, for example, a characteristic locked loop, a non-characteristic locked loop, an adaptive filter loop, a synchronization loop and / or can be used as control (feedback) of other parts of a signal processing module (e.g., another characteristic signal path).

[0132] By utilizing the eigensignal path as a feature locked loop, relevant eigenvalues ​​equal to the reference signal can be selected and the resulting output signal can be fed into the encoder and classifier. In this way, in some cases, the original non-sparse input signal can be made sparse by filtering out irrelevant information from the signal. Further rectification and calibration can be performed on the input of the signal before or after the signal processing module.

[0133] Although the above-mentioned time-to-digital converters are used in some characteristic signal paths, the characteristic signal paths can remain completely analog. Additional timer blocks or delay blocks can be added to different characteristic paths in order to synchronize the different characteristic signal paths in a correct manner.

[0134] Note that coarse / fine selectivity, ATC control, feedback control, calibration, timing, delays, sampling / subsampling mode selection are all configurable / programmable and feature extraction can be configured in this way.

[0135] An example of phase-locked feature extraction is shown in Figure 5. Specifically, it shows an example of a data-driven feature extractor selecting local differences in tag-specific data by phase-locking (e.g., by evaluating the phase angle of voltage offset), which is shown in the time-frequency domain. Figure 5A shows the spectrum of the input signal. Figure 5B is a spectrogram of the same signal but showing those elements of the signal that have a different phase from an applicable phase reference signal used to extract relevant features from the input signal. Figure 5C are spectrograms of the same signal, but showing those elements of the signal extracted by the feature extraction circuit that have a similar phase to a suitable phase reference signal. Each spectrogram shows time in microseconds on the horizontal axis and frequency in kilohertz on the vertical axis. It can be clearly seen that the feature of interest can be successfully selected from the entire input signal by phase locking.

[0136] Figures 6A-6C An example of a spiking neural network response to a (parameterized) input signal after feature-centric preprocessing is shown. Fig. 6A A segment of an analog input signal is shown versus time, with time plotted on the horizontal axis and signal amplitude plotted on the vertical axis. Figure 6B shows the image after preprocessing and feature extraction Fig. 6A The horizontal axis shows time. The vertical axis shows the amplitude of the post-processed (corrected, calibrated and sparsified) input to a spiking neural network (comprising, for example, 100 neurons).

[0137] Figure 6C The impulse response after applying feature extraction, i.e., the impulse response of a spiking neural network with the post-processed input signal as input, is shown. The horizontal axis shows time, while the vertical axis represents each individual neuron of the spiking neural network. It can be seen that the impulse response of the spiking neural network is sparse and distributed around features of obvious interest, more sparse than in the absence of analog-to-information (quantity) converters and feature filtering (i.e., after feature-centric preprocessing). By extracting a specific set of features and sampling the input signal at the relevant information / feature rate, the signal processing will be completely or almost completely focused on the feature-bearing information and discarding irrelevant information as early as possible in the signal processing path, while filtering out / suppressing other irrelevant information or distortion interference sources. By discarding irrelevant information early in the signal processing path, the energy efficiency of the entire system is significantly improved because neurons will not generate pulses due to processing irrelevant information.

[0138] Note that the features of any of the embodiments disclosed herein may be combined in any suitable manner.

Claims

1. A signal processing circuit for a spiking neural network, comprising an interface for converting an analog input signal into a corresponding pulse-time representation of the analog input signal, the interface comprising: an analog-to-information (A / information) converter, comprising an input for receiving the analog input signal, and configured to: generate a modulated signal representative of one or more characteristics of the analog input signal; a feature detector circuit comprising an input for receiving the modulated signal and configured to: compare the modulated signal with a reference signal representative of a reference feature, and configured to: generate an error signal indicative of a difference between the modulated signal and the reference signal; a feature extractor circuit including a locked loop circuit having an input for receiving the error signal and configured to: generate an output signal representative of the presence of one or more features represented by the modulated signal; and An encoder circuit has an input for receiving an output signal of the feature extractor and is configured to encode the output signal into a pulse train for input to a pulse neural network.

2. The signal processing circuit according to claim 1, further comprising: A preprocessing circuit is used to preprocess the analog input signal. Preferably, the pre-processing circuit comprises a low noise preamplifier, a bandpass filter and / or a programmable gain (post) amplifier.

3. The signal processing circuit according to claim 1 or 2, wherein: The analog input signal is represented by a voltage and is used as a control voltage input for the A / information converter, thereby determining the modulation of the analog input signal.

4. The signal processing circuit according to any one of the preceding claims, wherein: The feature is one or more of the following: i) specific characteristics, such as transient characteristics, steady-state characteristics; ii) specific properties, such as (non-) linear characteristics, statistical characteristics, stationary characteristics, transfer function characteristics, energy content and / or based on iii) specific domain characteristics, such as time domain, delay domain, frequency domain, phase domain characteristics, Preferably, the A / information converter comprises an analog-to-time converter, which converts the analog input signal into a modulated signal representing specific time domain characteristics, such as delay, frequency and / or phase.

5. The signal processing circuit according to any one of the preceding claims, wherein: The A / information converter forms an analog-delay (A / delay) converter, an analog-frequency (A / frequency) converter and / or an analog-phase (A / phase) converter; Preferably, wherein the A / information converter comprises a voltage-controlled delay line, a voltage-controlled oscillator and / or a multi-phase voltage-controlled oscillator, which performs at least a part of the modulation of the analog input signal into the modulation signal; More preferably, the A / delay converter includes the voltage-controlled delay line, the A / frequency converter includes the voltage-controlled oscillator, and the A / phase converter includes the multi-phase voltage-controlled oscillator.

6. A signal processing circuit according to any one of the preceding claims, wherein: The output signal of the feature extractor circuit is fed back to the feature detector circuit to form a negative feedback loop.

7. A signal processing circuit according to any one of the preceding claims, wherein: The feature extractor circuit includes a voltage controlled oscillator, which is driven by the error signal and generates an output signal of the feature extractor circuit and is part of the locked loop.

8. A signal processing circuit according to any one of the preceding claims, wherein: The locked loop is a delay locked loop, a frequency locked loop or a phase locked loop.

9. The signal processing circuit according to any one of the preceding claims, wherein: The feature extractor circuit further comprises a filter, preferably an adaptive filter, which blocks portions of the error signal that are not within a lock range of the lock loop, but allows portions of the error signal that are within a lock range of the lock loop to pass.

10. The signal processing circuit according to any one of the preceding claims, wherein: The encoder circuit uses rate-based or time-based, pulse-based encoding as the encoding scheme.

11. The signal processing circuit according to any one of the preceding claims, wherein: The feature extractor circuit may be part of a set of parallel feature extractor circuits, and The signal processing circuit further comprises a channel selector unit, which is adaptive in such a way that the most beneficial feature subset is extracted at any given moment under specific operating conditions, so that one or more feature extractor circuits corresponding to the feature subset of interest can be adaptively selected, and the parallel feature extractor circuits extract information about these features from the signal from the A / information converter.

12. A signal processing circuit according to any one of the preceding claims, wherein: The feature extraction performed by the feature extractor circuit can be configured by specific circuit parameters and / or system control parameters. Preferably, the configurable parameters include one or more of gain, linearity, bandwidth, feedforward and feedback, coarse / fine selectivity, ATC control, feedback control, calibration, timing, delay, and sampling / sub-sampling mode selection.

13. A signal processing circuit according to any one of the preceding claims, wherein: At runtime, the application / context / condition detection block determines the current operating context and from said current operating context determines the best set of simulation features to extract, Preferably, wherein a feedback loop dynamically tunes the specifications of the feature extractor circuit within power or performance constraints, and Therein, the runtime configuration manager then activates and configures the relevant feature sets by activating only the relevant feature detector circuits.

14. A signal processing circuit according to any one of the preceding claims, wherein: The analog input signal includes an analog differential voltage signal having a positive voltage input and a negative voltage input.

15. A signal processing circuit according to any one of the preceding claims, comprising a plurality of characteristic signal paths, each characteristic signal path being used for a modulated signal representing a specific characteristic or characteristics of the analog input signal, in, At least one characteristic signal path includes a programmable structure that is preferably driven by a spiking neural network and provides control feedback to at least one other characteristic signal path based on the output of the programmable structure.

16. The signal processing circuit according to any one of the preceding claims, further comprising: A converter unit converts the modulation signal or the error signal into a digital signal.

17. A signal processing method for converting an analog input signal into a corresponding pulse-time representation of the analog input signal for use in a spiking neural network, the method comprising: receiving the analog input signal; generating a modulated signal representative of one or more characteristics of the analog input signal; comparing the modulated signal to a reference signal representative of a reference characteristic; generating an error signal indicative of a difference between the modulated signal and the reference signal; generating an output signal representative of the presence of one or more features represented by the modulated signal; as well as The output signal is encoded into a pulse train for input into the spiking neural network.

18. The signal processing method according to claim 17, wherein: The feature is one of the following One or more of: i) specific characteristics, such as transient characteristics, steady-state characteristics, ii) specific properties, such as (non-) linear characteristics, statistical characteristics, stationary characteristics, transfer function characteristics, energy content and / or based on iii) specific domain characteristics, such as time domain, delay domain, frequency domain, phase domain characteristics, Preferably, the modulated signal represents specific time domain characteristics, such as delay, frequency and / or phase.

Images