ADAPTATION OF SNNs THROUGH TRANSIENT SYNCHRONY
Patent Information
- Application Number
- TW114148747
- Authority / Receiving Office
- TW · TW
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-10-05
- Filing Date
- 2021-10-05
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2041-10-04
AI Technical Summary
Conventional spike neural networks (SNNs) lack a practical method for controlling and synchronizing instantaneous dynamics, limiting their ability to process specific frequency ranges of input signals effectively, which is crucial for applications like radar, light, and sound signal processing.
A method to configure SNNs by setting synaptic element weights and spike neuron behaviors to achieve a periodic steady state when the input signal includes a predetermined oscillation frequency, allowing certain local clusters to prioritize processing specific frequency ranges.
Enables SNNs to efficiently handle specific frequency ranges within the frequency domain, improving adaptability, scalability, and reliability by ensuring network subgroups synchronize and process signals effectively.
Smart Images

Figure TWG2TB001920973_001 
Figure TWG2TB001920973_002 
Figure TWG2TB001920973_003
Abstract
Description
Technical Field
[0001] This disclosure is generally related to automatic signal recognition technology, and more specifically to time control and adaptation that allow for instantaneous synchronization mechanisms and constant regulation in networks of spike neurons. Prior Technology
[0002] In biological neural network models, individual neurons communicate asynchronously and via sparse events or spikes. In this event-based spike neural network (SNN), only neurons that change state generate spikes that can trigger signal processing in subsequent layers, thus saving computational resources. Spike neural networks (SNNs) are a promising approach for many different applications to achieve automatic signal recognition (ASR), identifying signals by discerning their constituent features.
[0003] SNNs encode information as one or more precisely timed (voltage) spikes, rather than as integer or real-valued vectors. Inference computation (i.e., inferring the existence of a feature in the input signal) can be performed efficiently in both the analog and temporal domains. Therefore, SNNs are typically implemented in hardware as fully custom mixed-signal integrated circuits, which allows them to perform inference functions with several orders of magnitude less power consumption than their deep neural network (DNN) counterparts, in addition to having a smaller network size.
[0004] SNNs consist of a network of spike neurons interconnected by synapses, which specify the strength of the connections between neurons. This strength is represented by weights, which mitigate the influence of the output of presynaptic neurons on the input of postsynaptic neurons. Typically, these weights are set during training, which involves exposing the network to a large amount of labeled input data, and the synaptic weights are gradually adjusted until the desired network output is achieved.
[0005] SNN can be directly applied to pattern recognition and sensor data fusion. It relies on the principle that the amplitude, time and frequency domain features of the input signal can be encoded into a unique spatial and temporal encoded spike sequence.
[0006] Neuromorphic SNN simulators, such as systems containing electronic analog / mixed-signal circuits that mimic the neurobiological architecture present in the nervous system, form distributed (in the non-Van Neumann sense, i.e., computing elements and memory are co-located, resulting in memory storage and complex nonlinear operations being executed simultaneously by neurons in the network), parallel and event-driven systems that provide capabilities such as adaptation (including adaptation of physical characteristics, trigger frequency, constant (behavioral) regulation, self-organization and learning.
[0007] The unique characteristic of the output of a robust neuromorphic system is the tunable excitability of designated neurons, combined with circuit-level interactions that maintain the correct (i.e., within the limits or definitions imposed by the learning rules or operating frequencies used) temporal coordination / phasing of these neurons. An example of circuit-level interactions is circuits that mimic / model the brain's biochemical (Na-, K-, Ca-) mechanisms and influence neuronal triggering and frequency adaptation.
[0008] The complexity and computational power of SNNs are assessed by their dynamics, specifically by the spatial localization and temporal variation of their activity. The dynamics (and architectural configuration) of SNNs can be fitted to known (ANN) connection models, such as multilayer feedforward networks or repetitive networks, through temporal encoding.
[0009] However, since SNNs have inherent characteristics that are completely different from those of conventional networks, they do not have to adhere to the rigid schemes associated with such conventional networks.
[0010] However, there is currently no practical mapping method to provide users of SNNs with an adapted and trained SNN or a portion thereof that can be used for classification or further processing within a specific predetermined frequency range of interest to the user. Conventional SNN mappings do not provide an effective mechanism for controlling / synchronizing the instantaneous dynamics of the network, and therefore limit the network's ability to acquire selected frequencies of the input signal, be sensitive to selected frequencies of the input signal, or filter out selected frequencies of the input signal, thereby effectively preventing the network from fully utilizing the frequency content within the spatiotemporal spikes.
[0011] For example, when processing radar signals, light signals, image signals, or sound signals, or any other application where signals can be represented in the frequency domain, the signals consist of different frequency ranges encoding specific information. Using SNNs to process specific frequency ranges within the frequency domain, and achieving a high level of network modularity and sophistication through the synchronized behavior of network subgroups, will greatly improve the practical feasibility of SNNs in terms of cost-effectiveness, adaptability, scalability, and reliability. Summary of the Invention
[0012] To address the shortcomings of the prior art discussed above, a method for configuring a spike neural network is proposed according to a first aspect of this disclosure. The spike neural network includes a plurality of spike neurons and a plurality of synaptic elements, which interconnect the spike neurons to form a network at least partially implemented in hardware. Each synaptic element is adapted to receive a synaptic input signal and weight the synaptic input signal to generate a synaptic output signal. The synaptic elements can be configured to adjust the weights applied by each synaptic element, and each spike neuron is adapted to receive synaptic signals from one or more of the synaptic elements. One or more of the synaptic output signals are received, and a spatiotemporal spike output signal is generated in response to one or more received synaptic output signals. The response local cluster within the network includes a set of spike neurons and a plurality of synaptic elements that interconnect the set of spike neurons. The method includes setting the weights of the synaptic elements in the response local cluster and the spike behavior of the spike neurons, such that when the input signal of the response local cluster is represented in the frequency domain, including a predetermined oscillation frequency, the network state in the response local cluster is in a periodic steady state, such that the network state in the response local cluster is periodic with a predetermined oscillation frequency.
[0013] In this way, a spike neural network can be configured such that when the input signal to a local cluster includes a predetermined oscillation frequency, certain local clusters within the spike neural network can be in a periodic steady state. This means that the portion of the input signal that is close to and includes the predetermined oscillation frequency will have a greater impact on the local cluster than other portions of the input signal. This allows a spike neural network configured in this way to handle a specific (predetermined) frequency range within the frequency domain.
[0014] In an embodiment, setting the weights of the synaptic elements responding to the local cluster and the peak behavior of the peak neurons involves iteratively training the local cluster by optimizing the weights of the synaptic elements and the peak behavior of the peak neurons to achieve the desired periodic steady-state behavior.
[0015] In this way, the response of a local cluster to a specific input signal can be configured effectively and the desired behavior can be achieved reliably.
[0016] In another embodiment, when the input signal of the response local cluster includes a predetermined oscillation frequency, the random distribution activity or statistical parameters of the set of neurons within the response local cluster are cyclically stationary at the predetermined oscillation frequency.
[0017] In yet another embodiment, the periodic steady state is the solution to the following equation: ;
[0018] Where T is the predetermined period, and Φ(t,τ) is the synaptic drive of all neurons in the local cluster in response. The state transition matrix, F(t), is given by the following equation for the synaptic drive. Deterministic function of the random part: ;
[0019] And K(t) is a synaptic driver The autocorrelation function, where These are the initial conditions.
[0020] In this way, an effective, scalable and reliable method is obtained to configure spike neural networks, and specifically local clusters, to achieve periodic steady state.
[0021] In another embodiment, the spike neural network includes a driving local cluster comprising a collection of spike neurons and a plurality of synaptic elements interconnecting the collection of spike neurons, such that the output signal of the driving local cluster serves as the input signal of the responding local cluster to couple the driving local cluster and the responding local cluster with a specific coupling strength, wherein the method further includes setting the network state within the responding local cluster to have a steady-state and / or time-varying state when the input signal from the responding local cluster from the driving local cluster does not include a predetermined oscillation frequency when represented in the frequency domain or when the specific coupling strength is less than the predetermined coupling strength.
[0022] In another embodiment, setting the weights of synaptic elements and the peak behavior of peak neurons in the response local cluster includes iteratively training the response local cluster by optimizing the weights of synaptic elements and the peak behavior of peak neurons to achieve the desired steady-state behavior and / or time-varying behavior.
[0023] In this way, the response of a local cluster to a specific input signal can be configured effectively and the desired behavior can be achieved reliably.
[0024] In another embodiment, when the response local cluster self-driving local cluster receives an input signal that does not include a predetermined oscillation frequency when represented in the frequency domain, or when a certain coupling strength is less than a predetermined coupling strength, the random distribution activity or statistical parameters of the set of neurons within the response local cluster are stationary or non-stationary.
[0025] In yet another embodiment, the steady state is the solution to the following equation: ;
[0026] in Synaptic drive The steady-state value of the autocorrelation function K(t), F(t), is given by the following equation for the synaptic drive. Deterministic function of the random part: ,
[0027] Where Φ(t,τ) represents the synaptic drive of all neurons within the local cluster. The state transition matrix, and It is an infinitesimal random variation, and E(t) is derived from... Deterministic functions by definition
[0028] Furthermore, the time-varying state is the solution to the following matrix equation:
[0029] It is the differential Lyapunov matrix equation. The continuous-time algebraic Lyapunov matrix equations are given, where Pr and Qr are discretized versions of E and F, and tr represents the time point of numerical integration.
[0030] In this way, an effective, scalable and reliable method is obtained to configure spike neural networks, and specifically local clusters, to obtain steady-state and / or time-varying states.
[0031] In another embodiment, the structural dimension of the response local cluster is increased by ensuring generalized external synchronization between the driving local cluster and the response local cluster, wherein the generalized external synchronization is achieved by coupling the driving local cluster to the response local cluster by means of a specific coupling strength equal to or greater than a predetermined coupling strength.
[0032] In this way, for example, we can obtain high-level network modularity and granularity with synchronous behavior of network subgroups, which improves the practical implementation possibilities of SNN in terms of adaptability and scalability.
[0033] In another embodiment, generalized external synchronization is ensured based on the average autocorrelation function 1 / N˟{Σ E[Γ(t+τ / 2)Γ(t-τ / 2)T]} of the synaptic driver Γ(t), where τ is the delay and N is the number of neurons in the local response cluster, where the average value exceeds the number of neurons.
[0034] In another embodiment, steady-state numerical solutions, time-varying numerical solutions, and / or periodic steady-state solutions are obtained by using feedback connections between neurons in a local cluster of responses, which cause synchronization of neuronal activity.
[0035] According to a second aspect of this disclosure, a spike neural network for processing input signals that can be represented in the frequency domain is disclosed. The spike neural network includes a plurality of spike neurons and a plurality of synaptic elements, which interconnect the spike neurons to form a network at least partially implemented in hardware. Each synaptic element is adapted to receive a synaptic input signal and weight the synaptic input signal to generate a synaptic output signal. The synaptic elements can be configured to adjust the weights applied by each synaptic element. Each spike neuron is adapted to receive one or more synaptic output signals from one or more synaptic elements and generates a spatiotemporal spike output signal in response to receiving one or more synaptic input signals. A local response cluster within the network includes a set of spike neurons and a plurality of synaptic elements interconnecting the set of neurons. When the input signal of the local response cluster, when represented in the frequency domain, includes a predetermined first oscillation frequency, the random distribution activity or statistical parameters of the set of neurons within the local cluster are cyclically stationary at the predetermined first oscillation frequency.
[0036] In this way, a spike neural network can be configured such that when the input signal to a local cluster includes a predetermined oscillation frequency, certain local clusters within the spike neural network can be in a periodic steady state. This means that the portion of the input signal that is close to and includes the predetermined oscillation frequency will have a greater impact on the local cluster than other portions of the input signal. This allows a spike neural network configured in this way to handle a specific (predetermined) frequency range within the frequency domain.
[0037] In this embodiment, the cyclically stationary random distribution activity or statistical parameters of the neuron set within a local cluster are described by the periodic steady-state solution of the following equation: ;
[0038] Where T is the predetermined period, and Φ(t,τ) is the synaptic drive of all neurons in the local cluster in response. The state transition matrix, F(t), is given by the following equation for the synaptic drive. Deterministic function of the random part: ;
[0039] And K(t) is a synaptic driver The autocorrelation function, where These are the initial conditions.
[0040] In another embodiment, the spike neural network includes a driving local cluster comprising a collection of spike neurons and a plurality of synaptic elements interconnecting the collection of spike neurons, such that the output signal of the driving local cluster acts as an input signal to the response local cluster, such that the driving local cluster and the response local cluster are coupled at a specific coupling strength, wherein when the response local cluster receives an input signal from the driving local cluster that does not include a predetermined oscillation frequency when represented in the frequency domain, or when the specific coupling strength is less than the predetermined coupling strength, the random distribution activity or statistical parameters of the neuron collection within the response local cluster are stationary or non-stationary.
[0041] In another embodiment, the stable random distribution activity or statistical parameters of the neuron set within a local cluster are described by the steady-state numerical solution of the following equation: ;
[0042] in Synaptic drive The steady-state value of the autocorrelation function K(t), F(t), is given by the following equation for the synaptic drive. Deterministic function of the random part: ,
[0043] Where Φ(t,τ) represents the synaptic drive of all neurons within the local cluster. The state transition matrix, and It is an infinitesimal random variation, and E(t) is derived from... A deterministic function is defined whereby the non-stationary random distribution of activity or statistical parameters of a set of neurons within a local cluster is described by a time-varying numerical solution of the following matrix equation:
[0044] It is the differential Lyapunov matrix equation. The continuous-time algebraic Lyapunov matrix equations are given, where Pr and Qr are discretized versions of E and F, and tr represents the time point of numerical integration.
[0045] In another embodiment, the driving local cluster is the input encoder of the spike neural network, which transforms the sampled input signal into a spatiotemporal spike train that is subsequently processed by the response local cluster.
[0046] According to the third embodiment of this disclosure, a method for processing a specific frequency portion of an input signal that can be represented in the frequency domain using a spike neural network is disclosed, comprising: using a spike neural network as described in the second embodiment of this disclosure or an embodiment thereof, or a spike neural network obtained via the first embodiment of this disclosure or an embodiment thereof; supplying an input signal in the form of a spatiotemporal spike array to a response local cluster of the spike neural network, wherein the input signal includes one or more frequency portions, and processing the input signal using the response local cluster such that a specific frequency portion of the input signal including a predetermined oscillation frequency has a greater influence on the neurons of the response local cluster than other frequency portions of the input signal.
[0047] In this way, when processing any application, the input signal of the spike neural network can be represented in the frequency domain. The signal consists of different frequency ranges that encode certain information, and specific frequency ranges can be processed in a specific way. Therefore, a high level of network modularity and precision with synchronous behavior of network subgroups is achieved, and the practical implementation possibility of SNN is improved in terms of cost-effectiveness, adaptability, scalability and reliability.
[0048] According to the fourth aspect of this disclosure, a physical signal inference processor for adaptively processing physical signals is disclosed, comprising: a selector and an extractor for selecting and extracting specific signal features from the physical signal; a spike neural network for classifying and processing the physical signal based on the specific signal features extracted from the physical signal; characterized in that: the processor further comprises: an operation block for establishing the current operation state and the optimal feature set; the processor includes a feedback loop to the selector and the extractor, the selector and the extractor being adaptive, meaning that they can select and extract different signal features based on a specific processing task.
[0049] In this way, the physical signal inference processor can select the optimal feature set in each operating situation, thereby improving the adaptability, cost-effectiveness and optimization of the use of spike neural networks.
[0050] According to the fifth specification of this disclosure, a method for adaptively processing physical signals is disclosed, comprising receiving physical signals in a physical signal inference processor as in the fourth specification of this disclosure, selecting and capturing specific signal features using a selector and a capturer, processing the specific signal features using a spike neural network, determining the current operating condition and the optimal feature set using an operation block, and sending feedback signals to the selector and capturer to adaptively change the selected and captured signal features as necessary.
[0051] In this way, the optimal feature set can be selected, thus adaptively processing physical signals and improving the adaptability, cost-effectiveness, and optimization of the use of spike neural networks. Simple Explanation of the Diagram
[0052] Embodiments will now be described with reference to the accompanying schematic drawings, which are merely examples, in which corresponding reference numerals indicate corresponding parts, and in which:
[0053] Figure 1 shows an exemplary neural network composed of neurons and synaptic elements;
[0054] Figure 2 schematically illustrates the spike neural network within the microcontroller integrated circuit;
[0055] Figure 3 schematically illustrates an exemplary runtime physical signal inference signal processor with an adaptive spike neural network;
[0056] Figure 4 shows the nominal values of the peak adaptation mechanism in the leftmost two charts, the reset and non-reaction period mechanisms used to model the activation and deactivation of sodium and potassium conductivity in the middle two charts, and the variability of the insertion during peak generation in the rightmost chart.
[0057] Figure 5 illustrates networks with different synchronized cell components;
[0058] Figure 6 shows a neuron-like voltage trajectory in the top chart, synaptic activation in the middle, and synchronization and constant regulation at the bottom;
[0059] Figure 7 shows the non-regulated state at the top and the regulated state at the bottom. The two charts on the left show the activity of individual neurons over time, and the two charts on the right show the network activity of the (non)regulated network.
[0060] The figures are intended for illustrative purposes only and do not constitute a limitation on the scope or protection listed in the claims. Implementation
[0061] Certain embodiments will be described in more detail below. However, it should be understood that these embodiments are merely illustrative and should not be construed as limiting the scope of this disclosure.
[0062] Figure 1 is a simplified diagram of neural network 10. Neurons 1 are interconnected via synaptic elements 2. To avoid cluttering the diagram, only a few neurons and synaptic elements are shown (and only some have reference numerals attached to them). The connection structure shown in Figure 1, i.e., the way synaptic elements 2 and neurons 1 are interconnected, is merely an example and many other topologies can be used. Each synaptic element 2 transmits a signal to the input of a neuron 1, and each neuron 1 receiving the signal can process the signal and subsequently generate an output, which is transmitted to other neurons 1 via other synaptic elements 2. Each synaptic element 2 has a weight assigned to it, which is applied to each synaptic input signal received and transmitted by the synaptic element to generate a weighted synaptic output signal. Therefore, the weight of a synaptic element is a measure of the type of causal relationship between two neurons 1 connected by a synaptic element 2. The relationship can be causal (positive weight), anti-causal (negative weight), or nonexistent (zero weight).
[0063] Neuron 1 and synaptic element 2 can be implemented in hardware, for example, using analog circuits or circuit elements, digital wired logic circuits or circuit elements, or a combination thereof. For example, each neuron and each synaptic element can be implemented as a hardware circuit or circuit element. This type of hardware implementation can also include functionality executed using software, such that the neuron and synaptic element are partially implemented in hardware and partially in software; that is, the hardware circuit executes software to perform the functions of individual neurons and synaptic elements. This contrasts with designs that use large processors where the software mimics individual neurons and synaptic elements to execute software. Such (partial) hardware implementations enable faster processing of input signals, for example, enabling faster pattern recognition and event-driven processing, in which blocks of neurons and synaptic elements are activated only when needed.
[0064] Neural network 10 may be a spike neural network. Neuron 1 is then a spike neuron, which generates a neuronal output signal in the form of one or more spikes or events generated by the neuron. Spike neuron 1 may be configured to be triggered (i.e., generate an output spike) only when the membrane potential within the neuron (e.g., energy potential or voltage or current level) reaches a predetermined threshold. The membrane potential of the spike neuron changes due to the received input signal, i.e., the synaptic output signal received by the neuron from the synaptic element is accumulated, integrated, or otherwise processed to change the membrane potential. When the weight of synaptic element 2 is positive, the synaptic output signal received from the synaptic element excites the receiving spike neuron 1, thereby increasing its membrane potential. When the weight of synaptic element 2 is negative, the synaptic output signal received from the synaptic element inhibits the receiving spike neuron 1, thereby decreasing its membrane potential. When the weight of synaptic element 2 is zero, the synaptic output signal received from the synaptic element has no effect on the membrane potential of the receiving spike neuron 1.
[0065] When the membrane potential of spiking neuron 1 reaches a threshold value, the neuron is triggered, generating a spike upon triggering, and the membrane potential decreases due to the trigger. If the membrane potential subsequently reaches the threshold value again, the neuron will be triggered again, generating a second spike. Each spiking neuron 1 is thus configured to generate one or more spikes in response to input signals received from the self-connecting synaptic element 2, forming a spatiotemporal spike sequence. Since spiking neuron 1 is only triggered when its membrane potential reaches a predetermined threshold value, the encoding and processing of temporal information are incorporated into neural network 10. In this way, a spatiotemporal spike sequence is generated in spiking neural network 10, which is a temporal sequence of spikes generated by spiking neurons 1 of network 10.
[0066] The temporal characteristics of spikes encode the amplitude and frequency characteristics of the input signal. These temporal characteristics include: the latency between the onset of a stimulus (e.g., an input signal from a synaptic element) and the generation of a spike at the output of the neuron; the latency between consecutive spikes from the same neuron; and the number of spikes triggered by the neuron during the duration of the applied input stimulus.
[0067] Synaptic element 2 can be configurable, allowing for variations, such as the corresponding weights of the synaptic element, for example, by training neural network 10. Neuron 1 can be configured in the manner in which it responds to signals from synaptic elements. For example, in the case of a spike neural network, neuron 1 can be configured to: increase or decrease the membrane potential by a signal, the time taken for the membrane potential to naturally decay toward the resting potential, the value of the resting potential, and / or the threshold value that triggers the spike of neuron 1. The configuration of neuron 1 can, for example, remain constant during training, or can be varied and set by training neural network 10 using a specific training set.
[0068] The input signal 11 can be, for example, multiple, different, sampled input signals or a spatiotemporal spike series. The input can be an analog-to-digital conversion value of a signal sample, or, for example, a digital value of a sample in the case of an analog or digital integrator, or an analog value of a sample in the case of an analog integrator.
[0069] The output signal 12 of the neural network 10 is, for example, a spatiotemporal spike train, which can be read from the output neuron 1 and further classified and transformed by the output transformation stage into a set of digital values corresponding to the output code type selected by the user.
[0070] Biological SNNs are characterized by sparse and irregular connections between neurons, low average activity, and only active neurons participating in information processing. Examples of sparse connections between neurons include local clusters of neurons, short or long path loops, and the formation of synchronous cellular components.
[0071] Local clusters, also known as cellular components, refer to a group of neurons and the synaptic elements connecting these neurons that share a strong common excitatory input. When a sufficient subset of neurons is stimulated, the neurons in this local cluster tend to be activated as a whole group. In this way, local clusters can be viewed as processing units. The connections between neurons in a local cluster, i.e., the possible mechanisms defining the cluster, can take many forms.
[0072] A synchronized response of a group of neurons is an instance defining a local cluster of neurons. An instance of interest is the time-varying (cyclic) periodic activation of neurons in a local cluster responding to a stimulus. Other possible mechanisms for defining local clusters include built-in redundancy (i.e., additional circuitry / neurons to ensure fail-safe operation), averaging (i.e., the final result is the average of multiple neurons rather than a single neuron), layer definition (layers can be defined based on network concepts, layer size, and network depth, among others), etc. SNNs exhibit complex systems defined by intricate dynamic interactions between neurons, such as communication between neurons within a local cluster or between different local clusters.
[0073] If synchronous activation of neurons serves as a mechanism for the formation of local clusters, then short-term memory (STM) can thus be expressed as the continuous activity of neurons within a local cluster maintained by reverberation (i.e., periodic activation of neurons), while long-term memory (LTM) corresponds to the formation of new local clusters, for example, through plasticity mechanisms or other learning methods. Within this framework, the synchronization of the triggering times of neuronal groups is examined as a series of synchronous triggers of a specific subset of neurons, detected as collective synchronization of a specific subset of neurons providing spatiotemporal integration, and detected as an extended concept of multiplexing within a subset of neurons.
[0074] Synchronized neuronal responses can be defined here as the temporal correlation of two or more events associated with various patterns of neuronal activity. This synchronization can take the form of synchronized triggering rates and / or triggering times of subsets of neurons at or around a specific frequency. Multiple subsets of neurons can be synchronized at different frequencies, reflecting the role of these subsets in learning / memory formation, or some other predefined signal processing function, such as filtering. Local clusters can be formed through multiplexing, i.e., groups of neurons in an SNN self-organize into local clusters, which exhibit reproducible, temporally locked spike activity patterns due to synaptic enhancement.
[0075] In one embodiment, neurons within a local cluster have synchronous neuronal responses, causing neurons to fire synchronously (i.e., generate output spikes). This synchronous response can take different forms. Neurons in a local cluster can fire periodically at the same time and rate in a repetitive cycle at a certain oscillation frequency.
[0076] The timing and rate of output spikes can vary over time; for example, the oscillation frequency may be substantially constant or decrease or increase over time. The timing and rate of output spikes within neurons of a local cluster can vary such that the triggering time of a particular neuron falls within a certain time window of the peak triggering frequency, i.e., the peak of the oscillation cycle when the maximum number of neurons are triggered. This distribution of the spike triggering times of neurons within a local cluster can follow a Gaussian curve or an equivalent statistical distribution (e.g., log-normal, Poisson, exponential, etc.), where the maximum number of output spikes occurs at the oscillation frequency and the number of spikes decreases with increasing distance from the peak. A specified time window can be considered as the time range within the oscillation cycle where the input signal has the greatest impact on the local cluster oscillating at a predetermined frequency.
[0077] Local clusters can exhibit, as a whole, synchronous periodic neuronal firing as described herein. This can be attributed to the periodic firing of all or fewer neurons in the local cluster, but with average peak activity of the local cluster following, for example, the synchronous periodic behavior described herein. Here, average peak activity is the average activity of the local cluster over a specific time range. Average peak activity can increase or decrease over multiple predetermined oscillation cycles. This is an example of frequency- or rate-based encoding. It is assumed that most (if not all) of the information about the stimulus is contained in the firing frequency of the neurons. Because the sequence of action potentials generated by a given stimulus varies from trial to trial, neuronal responses are typically handled statistically or probabilistically.
[0078] When local clusters do not oscillate synchronously, but instead wait for input signals, for example, if these input signals arrive sequentially at a predetermined frequency that the local cluster is sensitive to, the neurons within the local cluster will be more responsive to the input signals.
[0079] Other suitable examples of encoding are time or latency or first spike time encoding, and alternatively, rank-order encoding. The former concerns the relative timing of spikes, interaural delay times, or timing differences between spikes. The latter does not examine precise timing information, but only the order in which spikes arrive.
[0080] Generally, we can focus on, for example, statistical parameters or randomly distributed activity related to neurons within a local cluster. Thus, one example of a statistical parameter is the mean peak activity. In the case of a probability distribution, for example, when neurons in a local cluster exhibit peak activity, we can examine, for example, the first moment (expectation), the second central moment (variance), the third standardized moment (skewness), and the fourth standardized moment (kurtosis).
[0081] Synaptic delay, that is, the time required for a signal to propagate across synaptic elements, is a fundamental quantity for the oscillation / instantaneous dynamics of SNNs and thus for coherent and uncorrelated state transitions. Noise, in the form of random input or inherent non-uniformity of triggering frequencies, reduces the coherence of oscillations between neurons. Here, oscillation refers to the rhythmic or repetitive pattern of neural activity in an SNN. Neurons can generate oscillatory activity in many ways, driven by mechanisms within individual neurons or by interactions between neurons. In individual neurons, oscillations can occur as oscillations in membrane potentials or rhythmic patterns of action potentials, which then generate oscillatory activation of postsynaptic neurons. At the level of local clusters, the synchronized activity of a large number of neurons can cause oscillations within local clusters. Oscillatory activity in groups of neurons can be caused by feedback connections between neurons, which cause synchronization of neuronal triggering patterns. Oscillations and associated oscillatory activity are among the concepts used to analyze the properties of spikes, such as rate, spike overlap, and other temporal patterns.
[0082] The effectiveness of synchronization depends on the oscillation frequency. For example, synchronization with lower-frequency oscillations (used at the local level) can establish instantaneous networks that implement specific processes (learning, adaptation, STM / LTM, etc.), while synchronization with higher-frequency oscillations (used at the global level) can convey information about the process outcome, such as power or phase-locked loop cross-frequency modulation. In this cross-frequency modulation, the first cell modulates the second cell using a modulation method, such as cross-coupling two oscillators via a feedback loop, modulating the high-frequency amplitude with a low-frequency phase, etc. The total coupling strength defines whether the network will operate in a stable or chaotic state. In the absence of significant coordination at peak times, stronger non-homogeneity habitually leads to states that are very different from synchronous and sometimes completely asynchronous. Higher coupling strength generally leads to greater synchronization. Synchronized information transmission and signal integration provide peak neural networks with high levels of cost-effectiveness, adaptability, scalability, and reliability.
[0083] Neurons in a network primarily communicate via rapid all-or-nothing events, i.e., spikes in the neuron's membrane potential. The relative spike triggering times within a neuronal population are considered information codes carried by the network, and synchronization among neuronal populations is considered a signal for encoding and decoding information. Neuronal characteristics and associated variables describe neuronal states (e.g., the time course describing the membrane potential). The information to be encoded can be represented, for example, by these neuronal states. When a neuron or neuronal population oscillates and synchronizes, each of these variables can also oscillate at the same frequency and return to the same value once per oscillation cycle, one of which (e.g., the membrane potential) corresponds to a spike and the subsequent interval between spikes.
[0084] Next, an implementation of a mapping method will be described, which optimizes the temporal dynamics of random cell components and allows for enhanced synchronization of cell components with information transmission in spike neural networks.
[0085] A (potentially discontinuous) function fj(.) (this is a nonlinear transformation; for mathematical convenience, it is modeled as φ(x) = tanh(x), although other choices of φ are possible) is used to specify the input-output transfer function, which transforms the excitation ( / inhibition) voltage vjE(.) (vjI(.)) across the membrane of the j-th neuron into the firing rate of the j-th neuron (in Hz). The superscripts E and I signify excitatory and inhibitory, respectively, and are associated with excitatory and inhibitory neurons, respectively. Therefore, the voltage vi(t) in the i-th neuron, either received or postsynaptic, is given by:
[0086] in and The exponents are used to indicate the triggering of excitatory and inhibitory (i.e., presynaptic) neurons at triggering times tj and tj', respectively. The number of excitatory and inhibitory neurons are denoted as nE and nI, respectively, and αjE(.) and αjI(.) are functions describing the evolution of excitatory and inhibitory postsynaptic potentials, respectively. Furthermore, vth,iE(.) and vth,iE(.) are continuous threshold input voltages, specifying constant inputs, i.e., external contributions to the membrane potential, which can be external stimulus voltages. The threshold voltage is the voltage at which the membrane potential needs to reach to produce a spike in the corresponding neuron; when the threshold is met, the neuron is triggered in the case of an excitatory neuron, or the generation of action potentials in the postsynaptic (inhibitory) neuron cell is inhibited. Each term in the neuronal connectivity matrix wij represents the coupling strength of the j-th neuron to the i-th neuron.
[0087] Excitation and inhibitory group interactions—through the cross terms in equations (1) and (2) above—can lead to a state of complete or partial synchronization, which can be enhanced by increasing inhibitory dynamics. In heterogeneous inhibitory networks with sparse random connectivity, inhibitory dynamics provide duality between inhibiting group activity and generating neural reactivation. When the network is saturated, it will be unable to properly process incoming signals. Through neural reactivation, the network will be able to respond to incoming signals again. The concept of inhibition, among other things, requires the interruption or blocking of activity and the restriction of activity patterns in both space and time within the network. However, inhibitory interneurons not only provide a stopping signal for excitation; the operational dynamics in the neuronal network can only be maintained when the excitation force is counteracted by the effective inhibitory force.
[0088] When the input from an excitatory neuron to an adjacent inhibitory neuron is sufficiently strong, the synchronicity of spike transmission within the circuit occurs. The coupling strength is controlled by the inhibitory neuron: when inhibitory input is present, i.e., when there is intermediate coupling, the neuron spikes with a minimal rhythm / synchronous spike. The inhibitory spike-time dependent plasticity (STDP) function of the following equation (6) is symmetric and helps to reduce synaptic conductance; in contrast, the excitatory STDP function is antisymmetric and biased towards enhancement.
[0089] In the above equation, the function τ(t) is a bounded differential function of time t, and this function satisfies the following condition: ,
[0090] Where r and h are positive constants. Assume ,in For gain—which modulates the exponential decay of the synaptic voltage and scales in a time-dependent manner to mimic the peak of the input conductance—we can define the (excitatory or inhibitory) synaptic drive of the i-th postsynaptic neuron by the following formula.
[0091] Synaptic drive essentially indicates synaptic strength. Synaptic drive of neuron i It can be considered as the total transfer function of all synaptic elements driving neuron i, which can be a function of, for example, the exponential decay value, weight, and gain of the synaptic elements driving neuron i, etc. Based on (1), (2), and (4), we obtain...
[0092] If we consider the case that includes peak temporal dependent plasticity (STDP) learning and long-term plasticity (LTP), we obtain the following limits and changes: ; ; like ;
[0093] Here, η+ and η- are soft limits, meaning that for larger weights, the enhancement of synaptic inhibition ratio is dominant.
[0094] Noise occurs in both the generation of spikes and the transmission of synaptic signals in neurons. This noise originates from quantum releases of neural transmitters, random opening of ion channels, and coupling with background neural activity. Subsequently, this noise induces neuronal variability, increases neuronal sensitivity to environmental stimuli, affects synchronization between neurons, and promotes probabilistic reasoning.
[0095] The elements wij of the connectivity matrix are taken from a Gaussian distribution with correlation [wijwji]J = ρ, where square brackets [·]J specify the average value of random connections. The parameter ρ quantifies the degree of symmetry of the connections; that is, for ρ = 0, the elements wij and wji are independent and the connectivity matrix is completely asymmetric; for ρ = 1, the connectivity matrix is completely symmetric; for ρ = -1, the connectivity matrix is completely antisymmetric. The degree of symmetry can be used to observe the evolution of a particular network configuration, that is, how likely the configuration is a random outcome.
[0096] Subsequently, the definition of generalized external synchronization of stochastic neural networks with time-varying delays can be extended by the following equation.
[0097] Here, x and y represent unique cellular components or local clusters, i.e., the x network forms the driving network, and the y network forms the response network. The driving network x thus drives the response in the response network. Cellular component y experiences perturbations. Cellular components can be positioned directly adjacent to each other, i.e., at least one output neuron of the driving network x can be directly connected to at least one input neuron of the response network y. Cellular components x and y can, for example, be located in different layers or the same layer of the network, or they can be separated by hardware chips coupling two spike neural networks to effectively form a single spike neural network. The time-dependent function ω(t) describes the noise in the input voltage and is represented by Brownian motion, i.e., the n-dimensional standard Wiener process. We assume that the noise intensity function σi(x,y,t) satisfies the Lipschitz condition and that there exist positive constants p and q, therefore the trajectory (σi Tσi) ≤ px Tx + qy Ty. By selectively stimulating neurons, we can target certain neural network properties, such as the excitation / inhibition balance, network dynamics, and the computations performed. Here, the excitation / inhibition balance refers to the fact that neurons can operate optimally in a (sparse) detection state, i.e., most synapses are excitable and the trigger rate is low; and in a balanced state, there is a roughly balanced excitation and inhibition, and a higher output trigger rate.
[0098] When a perturbation occurs, the neuron's computational elements generate spikes based on the sign, intensity, and phase of the perturbation; that is, the coupled cluster of neurons (cellular components) maintains the phase difference generated by the cluster-related inputs. We can observe that if the frequency response changes, each neuron adjusts the phase of its spikes. The adaptive controller ui(t) of node i is expressed as...
[0099] Where J(Bi (.)) is the Jordan matrix of the vector function Bi (Γ x,i ), εi =yi (t)-Bi (Γ x,i (t)) is the synchronization error, and k is a sufficiently large positive constant. Therefore, the adaptive controller is used to control the synaptic drives of neurons within the response network y, thereby reducing the synchronization error between the driving network x and the response network y. In this way, synchronization occurs between the driving network x and the response network y. As mentioned above, the total coupling strength u defines whether the network will operate in a stable or chaotic state of activity. In the absence of significant coordination at peak times, stronger non-homogeneity, such as cell components x and y not oscillating at the same frequency, habitually leads to states that are very different from synchronous and sometimes completely asynchronous. Higher coupling strength generally leads to greater synchronization.
[0100] In this manner, a method is described for increasing the structural dimension of cellular component y' (i.e., increasing the number of synchronized neurons in cellular component y) by ensuring generalized external synchronization of a stochastic neural network based on the average autocorrelation function. This specification provides a robust mechanism against heterogeneity, connectivity sparsity, and noise.
[0101] Next, we can transform formula (8) into a system of linear stochastic differential equations in standard form, as shown below.
[0102] The solution to (10) is given by the following equation.
[0103] Where Φ(t,τ) is the state transition matrix, which describes the time evolution of the dynamic system between time τ and t. It is determined as a function of t as the solution of dΦ(t,τ) / dt=E(t)Φ(t,τ) and Φ(τ,τ)=Im. Solving equation (11) involves stochastic Itô integrals. If the deterministic matrix functions E(t) and F(t) are bounded in the time interval of interest, then each initial value vector Γ(t0) has a unique solution.
[0104] The timing of neuronal spikes, such as the simultaneous spikes of neurons with well-defined time lags in addition to the temporal coordination of neuronal outputs, conveys information about sensory input. Therefore, where the internal states of overlapping spikes in a network can be correlated, specifically, synchronous groups of neuronal subpopulations represent the perceptual content that forms coherent entities, such as visual objects in a scene.
[0105] Network activity can be described based on the average autocorrelation function of Γ(t), which is the correlation between the signal and its own delayed replica as a function of the delay, 1 / N˟{Σ E[Γ(t+τ / 2)Γ(t-τ / 2)T]}, where τ is the delay, and the average value exceeds the neuron population. Define K(t) as the autocorrelation matrix of Γ, i.e., E[Γ(t+τ / 2)Γ(t-τ / 2)T], and using Itô's theorem on stochastic differentials, and then taking the weighted average of both sides, we obtain the differential Lyapunov matrix equation:
[0106] The solution to this equation has the following form.
[0107] Under steady state, equation (12) is expressed as follows:
[0108] Subscript Specify the steady-state values, and let E and F be deterministic matrix functions E(t) and F(t). Steady-state represents the state of a network exhibiting stable behavior, meaning the most recently observed behavior of the system will continue into the future. In a stochastic system in steady state, the probability of repeating various states remains constant. For example, steady state only occurs when the network is idle and only direct current is used. If t ≤ t', the steady-state covariance function is obtained as follows:
[0109] And if t>t' as follows:
[0110] To obtain a numerical solution, equation (12) must be discretized in time using a suitable scheme, such as any linear multistep method (e.g., the trapezoidal method, the inverse differential formula, or the Runge-Kutta method). Using the backward Euler (first-order inverse differential formula), the differential Lyapunov matrix equation (12) can be written in a specific form called the continuous-time algebraic Lyapunov matrix equation.
[0111] Here, tr = t(r-1) + hr, where hr is the time step of the numerical integration. At each time point tr, the numerical method can compute an approximation of the exact solution. Furthermore, matrices Qr and Pr are the discretized (in time) versions (numerical integration method) of the deterministic matrix functions E and F, respectively.
[0112] If Pr is sparse and large-scale, and if Qr is low-rank, we solve equation (17) using a Krylov-type method. We solve equation (17) by determining... and The low-rank solution is calculated such that K(tr) = UOUT, where U is an orthogonal matrix. Starting from the following equation...
[0113] It reduces the Lyapunov equation by using the orthogonality of U:
[0114] We calculate an approximation by applying the Galerkin condition. That is, given U, we calculate O by applying the Galerkin orthogonality condition UTR(UOU T)U=0, where R(UOU T) := Pr (UOU T)+(UOU T)Pr T+QrQr T is the residual associated with UOU T.
[0115] This is a small-scale dense Lyapunov equation, which can be solved directly, for example, by the Bartels-Stewart method. For dense Pr, we solve equation (17) by a low-rank version of an iterative method related to rational matrix functions. The assumed iteration of Lyapunov equation (17) at i = 1,2,… is given by K(0) = 0, and
[0116] Where γi is the iterative shift parameter. For effective implementation, we can replace the iteration with its Cholesky factor, i.e., Ki = LiLi H, and re-formulate it according to the factor Li.
[0117] Therefore, a mathematical description of the time-varying solution of equation (12) is given. For example, such networks do not exhibit stable behavior, and the activity of neurons changes over time.
[0118] For a periodic steady-state solution (if it exists here), for all t and And for some time intervals T>0, E(t+kT)=E(t) and F(t+kT)=F(t). The initial condition K0 of equation (12) satisfies equation (13) by setting K(T)=K0, which is expressed as
[0119] in
[0120] It can be calculated using numerical integration equation (12) and the initial condition K(0) = 0. Equation (21) is a system of algebraic linear equations for the terms of matrix K0, and this type of equation is usually called the discrete-time algebraic Lyapunov matrix equation because it originates from the Lyapunov stability theory of discrete-time systems. Equation (21) can be numerically solved using the Bartel-Stuwart algorithm for continuous-time algebraic Lyapunov matrix equations.
[0121] Equations (14) to (16) thus describe the steady-state numerical solution, equations (17) to (20) describe the time-varying numerical solution, and finally equations (21) to (22) describe the periodic steady-state numerical solution. These solutions describe the spike neural network according to its network activity, i.e., the linear solution, describing the synaptic drive of each neuron in the cellular components or other local clusters of the spike neural network. Local clusters may follow steady-state, time-varying numerical solutions, and / or periodic steady-state numerical solutions depending on the current state of the network.
[0122] Therefore, in order to achieve synchronous oscillation within a spike neural network or a subset of neurons within a spike neural network (e.g., cellular components or other local clusters), the network drive of different neurons exhibiting synchronous oscillation behavior must follow equations (21) to (22). The synaptic drive of a specific neuron in a spike neural network can be tuned by changing, for example, the corresponding weights of the synaptic elements connected to the neuron, by configuring a signal to increase or decrease the neuron's membrane potential, by configuring the time taken for the neuron's membrane potential to naturally decay to its resting potential, by changing the value of the neuron's resting potential, and / or by changing the threshold value of the spike that triggers the neuron.
[0123] Through an iterative process, using, for example, software, the parameters defining the values of synaptic drives at a specific time instance can be optimized. This can be accomplished through hardware simulation, or directly through hardware implementation and analysis of the output. Thus, a network can be trained to make specific cellular components follow equations describing synchronous oscillatory behavior.
[0124] In other words, when the driving network x oscillates at a predetermined frequency and the response network y has been mapped such that it is possible for the response network y to follow equations (21) to (22) that govern the periodic steady-state solution, and the connection strength between the driving network x and the response network y is sufficiently strong, the response network y will oscillate in the manner governed by equations (21) to (22). The oscillation period is then a predetermined oscillation period T. This means that the random distribution activity or statistical parameters of the set of neurons in the response network y are cyclically stationary at the frequency of the oscillation of the driving network x.
[0125] If the coupling strength is not strong enough, the random distribution of activity or statistical parameters of the set of neurons in the response network y may be stationary or non-stationary. The value of the coupling strength required to make the coupling between the driving network x and the response network y sufficiently strong can be a predetermined value, resulting in a specific response to the input signal. This can be determined on a case-by-case basis depending on the application.
[0126] Generally, the driver network x will perceive the input signal from the sensor or other device that acts as the input to the spike neural network as closer than the response network y. This is typically because the spike is processed by the driver network x in order to drive the response network y, before being processed by the response network y.
[0127] The input encoder transforms the sampled input signal into a spatiotemporal spike train. Each output neuron of the input encoder forms the output signal of the spatiotemporal spike train. The input encoder is configured by a specific configuration, such as setting the weights of the synaptic elements of the input encoder. Multiple input encoders can be used, implemented across multiple frequency ranges in the frequency domain of the input signal.
[0128] The driver network x can be an input encoder. The driver network x can be connected to the input encoder such that the spatiotemporal spikes of the output signal of the input encoder are processed by the driver network and induce spatiotemporal spikes in the driver network x. Since these spatiotemporal spikes are encoded to specific frequencies in the sampled input signal of the spike neural network, the spatiotemporal spikes will occur at specific frequencies in specific neurons of the driver network x. This means that the driver network x can also oscillate at this specific frequency, but this is not necessary.
[0129] Since the driving network x and the response network y are coupled, the spatiotemporal spikes processed by the driving network x will then be processed by the response network y. The response network y can therefore become stationary, non-stationary, or cyclically stationary, that is, the random distribution activity or statistical parameters of the set of neurons in the response network y are stationary, non-stationary, or cyclically stationary.
[0130] The first two can be the result of weak coupling strength between the driving network x and the response network y, or the driving network x not providing an input signal in the form of a spatiotemporal spike at a predetermined frequency.
[0131] The latter result can thus provide a stronger and more sufficient coupling strength between the driving network x and the response network y, and provide the driving network with an input signal in the form of a spatiotemporal spike at a predetermined frequency.
[0132] The oscillation frequency of the synchronous oscillating cell component is a positive input constant and can be selected based on which frequency the cell component needs to be sensitive to. In principle, any frequency at which a local cluster can oscillate can be a predetermined frequency.
[0133] All neurons in a local cluster or cellular assembly do not need to spike at exactly the same time, i.e., they do not need to be perfectly synchronized. The probability of spikes can be rhythmically modulated, making it more likely that neurons will fire simultaneously, which will cause oscillations in their mean motion. Therefore, the frequency of large-scale oscillations within a local cluster or cellular assembly does not need to match the firing patterns of each individual neuron within it.
[0134] Neurons in synchronous and oscillating local clusters or cellular components may also spike within a certain time range, for example, all neurons may spike within the following time ranges: within 10 percent of the oscillation period, or within 1 percent of the oscillation period, or within 0.1 percent of the oscillation period, or within 0.01 percent of the oscillation period.
[0135] Generally, spike neural networks are first modified to meet their timing requirements. Local clusters can operate within a frequency range centered on a predetermined frequency, where the distance from the center of the range can be a three-sigma variation. Other distances from the center of the range can be one or two sigma variations. Synchronization within a given time range can then be defined by the probability density function described above. Choosing a specific sigma can then result in a given time range.
[0136] Within this time frame, the number of neurons at the peak of a local cluster may be equal to or exceed 50 percent of the total number of neurons in the local cluster, more preferably equal to or exceed 75 percent, more preferably equal to or exceed 90 percent, more preferably equal to or exceed 95 percent, more preferably equal to or exceed 97.5 percent, and more preferably equal to or exceed 99 percent.
[0137] Figure 2 illustrates one embodiment of a high-level architecture of a microcontroller integrated circuit 100 including a learning spike neural network 110. In this context, the microcontroller is an economical method for data collection, sensing, pattern recognition, and actuating physical signals. In a spike neural network, individual neurons communicate asynchronously via sparse events or spikes. A spike neural network system is implemented as a spike integrator array containing a neural-synaptic core. One possible implementation of a learning spike neural network is described in WO2020 / 099680 A2.
[0138] Spike neural network 110 is connected to one or more streaming input ports 111, which provide input to the spike neural network 110 for conversion into a spatiotemporal spike sequence. Spike neural network 110 is connected to one or more output ports 112. Memory mapping control and configuration interface 113 controls the configuration parameters of spike neural network 110, such as synaptic weights and / or neuron configuration, and may further include peripheral devices (e.g., A / D converters, D / A converters, bandgap, PLLs) and control and adaptation circuitry for neurons, synapses, and plasticity (learning) circuitry. Interface 113 reads from memory device 102 storing settings for spike neural network 110 and sends signals to spike neural network 110 to configure the hardware accordingly. Interface 113 may send analog signals to spike neural network 110. The configuration may include the configuration parameters or network topology of each neuron 1 or synaptic element 2 of the spike neural network 110.
[0139] Each neuron 1 may have a set of configuration parameters that control the precise triggering behavior of other neurons 1. For example, a neuron may be designed with a trigger threshold, which represents the threshold value of voltage, energy, or other variables accumulated in the neuron due to received input, and wherein the neuron produces an output spike (such as a voltage, current, or energy spike) when the accumulated variable meets or exceeds the trigger threshold. The neuron may implement an integral function that integrates inputs to the neuron to determine the adjustment of the accumulated variable. In addition, the neuron may also be designed with: (a) a leakage rate, which represents the rate at which the accumulated variable in the neuron decays over time; (b) a resting value of the accumulated variable, which represents the value to which the accumulated variable will decay over time in the absence of any input signal to the neuron; (c) an integration time constant, which represents the time it takes for the input signal to be integrated to determine any increase in the accumulated variable in the neuron; (d) a non-responsive level, which represents the value of the variable accumulated in the neuron immediately after the neuron is triggered; and (e) a non-responsive period, which represents the time required for the accumulated variable in the neuron to rise to the resting value after the neuron is triggered. For each neuron, these parameters can be predetermined and / or configurable and / or adjustable. By adjusting, for example, the neuron's trigger threshold, leakage rate, integral time constant, and non-response period to match the energy content of key input signal features, neuron 1 will generate one or more precisely timed spikes when stimulated by an input signal containing those features.
[0140] The configuration parameters of synaptic element 2 include its weights and gain. The weights of synaptic element 2 are typically used to adjust synaptic element 2, while the gain of synaptic element 2 is used to amplify the signal in hardware and typically involves low-pass filter implementation. Typically, the gain is fixed during the initialization of network 110, while the weights can be changed based on the evolution / training of the spike neural network 110.
[0141] The microcontroller integrated circuit 100 further includes a microprocessor core 101 to perform calculations and control of the integrated circuit 100. For example, the microprocessor core 101 can manage memory mapping control and communication between the configuration interface 113 and the memory device 102.
[0142] Memory device 102 may be any computer-readable storage medium. Memory device 102 may be a non-transitory storage medium. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media on which information can be permanently stored (e.g., read-only memory devices within a computer, such as CD-ROM disks, ROM chips, or any type of solid-state non-volatile semiconductor memory that can be read by a CD-ROM drive); and (ii) writable storage media on which variable information can be stored, such as hard disk drives or any type of solid-state random access semiconductor memory, flash memory.
[0143] An external bus 104 is connected to one or more sensors or other data sources 103. The microcontroller integrated circuit 100 can also be directly attached to a sensor 105. The sensor may first pass through an analog-to-digital converter 106. One or more serial input / output ports 107 and general purpose input / output ports 108 may be present on the microcontroller integrated circuit 100. Direct memory access (DMA) 109 can be configured to access the memory of the microcontroller integrated circuit 100.
[0144] The core of an SNN can employ a hybrid analog-digital computing platform, where spikes incorporate analog information into the timing of events and then transform it back into an analog representation at the input of the synaptic matrix.
[0145] The neural-synaptic core disclosed in this invention can be organized as a repeating array of synaptic circuits and neuronal units, each of which can form a cellular component. The system incorporates the presence of electronic synapses at the connection points of the array. The periphery of the array includes a synaptic circuit array that mimics the somatic tissue and axonal protuberances of a biological neuron. Furthermore, each neural-synaptic core in the array can have a local router that communicates with routers in other cores within a dedicated, real-time reconfigurable chip network.
[0146] The classifier in this invention may have a set of output neurons (one output neuron for each classification category), wherein each neuron triggers events (spikes) according to its trigger probability distribution. Neural synaptic computational elements can generate complex spatiotemporal dynamics and can be extended to specific features that can assist in target signal processing functions.
[0147] The spike properties of neurons are controlled by a specific set of parameters. Neurons act as (frequency) bandpass devices, where phase-locking conditions occur at frequencies associated with the time constants of both the neuron and the spike-frequency adaptive dynamics. Here, the bandpass device can be viewed as a bandpass filter, processing only signals falling within a specified (designed) frequency band (range). The spike-frequency adaptive mechanism leads to specific consequences in the processing of sensory signals, such as amplifying changes in stimulus parameters, preventing spike frequency saturation, or tuning the frequency response to specific stimulus characteristics. The topological structure of neural synaptic elements controls the regularity of spontaneous neuronal spikes, such as the trigger rate scaling factor and the intensity of intrinsic noise, thereby generating coherence / frequency generation.
[0148] This invention covers the application of instantaneous synchronization, constant regulation, heterogeneous learning rules, weight storage types, and communication protocols to the synaptic matrix in the neural-synaptic core.
[0149] In one embodiment, the neural-synaptic core can be organized as a single core implementing homogeneous learning rules. In another embodiment, different regions of the synaptic matrix can be configured by heterogeneous learning rules based on instantaneous synchronization. The size of this region of the synaptic matrix can be configured at design time to generate dedicated circuitry, or can be dynamically configured at runtime, in which time the array will have heterogeneous clusters of synaptic circuitry, each cluster implementing, for example, different learning rules.
[0150] This heterogeneous cluster may require synchronous mapping and / or synthesis algorithms to map each neuron unit to the cluster determined by the synaptic matrix based on instantaneous synchronization. Without loss of generality, adaptive spike neural networks can implement runtime physical signal inference signal processors, such as those used for radar, lidar, and biophysical signal inference.
[0151] Figure 3 schematically illustrates an exemplary runtime physical signal inference signal processor 200 with an adaptive spike neural network. As illustrated, system 200 may include a set of parallel adaptive channel selectors and extractors 203, wherein each channel or subset of channels can be activated at any given time based on a specific signal processing task.
[0152] Physical signal 201 enters processor 200, and based on a specific signal processing task, selector and extractor 203 select and extract specific signal features from the physical signal 201, which serves as input to processor 200. The adaptability of the feature selection and extraction module 202, including the selector and extractor, means that different signal features can be selected and extracted based on a specific processing task. This is important because the system becomes more versatile in this way. Different applications often require completely different signal features; for example, image analysis is performed differently than electrocardiogram analysis, and even images can be analyzed differently based on the information people want to obtain from them. Because processor 200 is adaptive, it can be used in many applications.
[0153] Next, the different signals from the feature selection and acquisition module 202 can pass through the multiplexing and preprocessing platform 204. Here, the feature set is preprocessed (e.g., filtered, amplified, etc.) and transferred on a single output line. In some cases, signal preprocessing may not be necessary.
[0154] The feature set from the adaptive channel selector and extractor is provided to the SNN information processor 205, which performs preprocessing signal classification and processing, and performs application-specific signal information estimation that eliminates the need for a complete reconstruction of the sensor waveform, i.e., the sensor-specific signal. This provides a significant improvement in energy efficiency. Features can be activated and configured at runtime along several parameters, such as gain, bandwidth, noise filtering, and variability delimitation. The information or inference metric 206 output from the SNN can then be used by the specific application.
[0155] Inference accuracy depends on environmental conditions such as signal dynamics or signal interference. During runtime, the operation settings and conditions block 207 establishes the current operating condition and thus the optimal feature set. The feedback loop dynamically adapts to channel specifications within the power-performance design space. The runtime configuration execution unit then activates only the relevant settings and configures them.
[0156] Based on the training and testing data from cross-processing tasks 209 and parameterized performance settings 210, a subset 208 of learning and signal processing tasks can be selected, such as noise filtering, variability delimitation, and channel synchronization.
[0157] Depending on the conditions of excitatory / inhibitory network interactions, the heterogeneity of input patterns, and the spatiotemporal dynamics of neural synaptic elements, neuromorphic systems exhibit a wide range of patterned activities, such as fully synchronous, clustered, or asynchronous states, see Figures 4 and 5.
[0158] Figure 4 shows the nominal values of the spike adaptation mechanism in the leftmost two charts, illustrates the variability of the signal generation and non-response period mechanism of modeled sodium (Na) and potassium (K) conductance activation and deactivation in the middle two charts, and shows the spikes of neurons with similar variability levels inserted in the rightmost chart.
[0159] Figure 5 illustrates a network with different synchronizing cellular components 501 to 512. At different locations within the network, local clusters 501 to 512 are shown, which depict synchronized behavior: these neurons oscillate synchronously and spike at, for example, the same frequency. The behavior of different local clusters 501 to 512 can differ depending on the requirements of each local cluster 501 to 512 within the network. The SNN can be adaptive, thus the local clusters 501 to 512 can change, for example, their oscillation frequency, or even the neurons included within a single local cluster can be altered.
[0160] Neurons generate action potentials with a wide range of peak dynamics. Figures 6 and 7 illustrate synchronization for reconfigurable networks over a broad time span.
[0161] Figure 6 illustrates the increase in oscillatory synchronicity of spike neurons within a cellular component over a specific time range. As seen in the top chart, spikes become increasingly localized over time. In the bottom chart, an increasing number of neuronal spikes are observed within the same time window of the oscillation cycle. As seen in the middle chart, although the cellular component starts with close to zero synchronicity, by the end of the specific time range, synchronicity is close to approximately 80%. Depending on the application, the amount of synchronicity can be further increased.
[0162] Figure 7 shows the non-regulated state at the top and the regulated state at the bottom. The two charts on the left show the activity of individual neurons over time, and the two charts on the right show the network activity of the (non)regulated network.
[0163] The regulated network exhibits synchronous oscillatory behavior. Therefore, network activity is more localized; essentially, all neurons spike within specific time windows of the oscillation cycle. This contrasts with the unregulated network state in the upper figure, where network activity is almost random.
[0164] Duration-selective curve asymmetry is employed to address peak latency and non-homogeneity in the resulting latency curve. Phase-locking strength at the target frequency is controlled by altering its detuning parameters, such as changing the initial frequency. The network is guided to a stable dynamic state via an initial set of instantaneous dynamical assemblies through synaptic coupling. With increasing coupling, a shift occurs from the collective of individually triggered neurons to a coherent synchronization network.
[0165] In addition to information transmission, the extent to which instantaneous synchronization captures the (transient) phase relationship between (controlled) target frequencies is also detected. Neurons generate time-locked patterns; due to the interaction between conductance delay and plasticity rules, the network can form a cluster of neurons with reproducible and precisely triggered sequences conditioned on activation patterns. As conductance increases (thus resulting in net firing of the network), the firing rate can increase and become more uniform with a lower difference coefficient. The entrainment of low-frequency synchronization behavior includes phase reorganization so that the optimal (i.e., most excitable) phase is consistent with the temporal characteristics of events in the ongoing input stimulus. Due to phase locking, the phase difference between the signal from the characteristics of the input signal and the firing of local clusters is thus eliminated. The sequence of synaptic currents, i.e., reducing the time jumps in the generation of action potentials in individual neurons outward and inward, and thus producing a network with increased controllability of synchronous activity and constant regulation.
[0166] In this way, by using the synchronous oscillation of the cell components within the spike neural network, the network becomes sensitive to specific frequencies within the (converted) input signal.
[0167] This invention relates to an implementation of a mapping method that optimizes the temporal dynamics of random cell components and allows for enhanced synchronization of cell components in information transmission within a spike neural network.
[0168] The present invention further relates to a method for synchronizing the trigger frequency of a random synaptic driver in an excitatory and inhibitory spike network having system time delay and random input disturbance.
[0169] The present invention further increases the dimensionality of cell component structures by ensuring generalized external synchronization of random neural networks based on average autocorrelation function, and provides a robust mechanism to combat heterogeneity, sparsity of connectivity and noise.
[0170] The present invention further implements a synchronization mechanism that is compatible with hedonistic approaches such as Hebbian learning, i.e., the relevant input tends to reinforce (spike-temporal dependent plasticity).
[0171] The present invention further implements a spike neural network information processor system that performs application-specific signal information estimation that eliminates the need for complete reconstruction of sensor waveforms.
[0172] The present invention further relates to a system-inherent transfer procedure for transitioning from spatiotemporally chaotic activity patterns to impulse activity, wherein the system-inherent transfer represents structural or sensory invariants in stimuli, or the content forming coherent entities, such as visual objects in a scene, and the procedure allows neural circuits to retrieve relevant information from real sensory stimuli in the presence of interference.
[0173] The present invention further implements a computing element capable of generating complex spatiotemporal dynamics, which can be extended to specific characteristics, such as changes in frequency response and the resulting phase adjustment of spikes, to perform time-dependent calculations and phase-locked loops for network activity.
[0174] One or more embodiments may be implemented as a computer program product for use with a computer system. The program of the program product may define the functionality of the embodiments (including the methods described herein) and may be contained on various computer-readable storage media. Computer-readable storage media may be non-transitory storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media on which information can be permanently stored (e.g., read-only memory devices within a computer, such as CD-ROM disks readable by a CD-ROM drive, ROM chips, or any type of solid-state non-volatile semiconductor memory); and (ii) writable storage media on which variable information can be stored, such as hard disk drives or any type of solid-state random access semiconductor memory, flash memory.
[0175] Two or more of the above embodiments can be combined in any suitable manner.
[0176] 1: Neuron 2: Synaptic elements 10: Neural Networks / Peak Neural Networks / Networks 11: Input signal 12: Output signal 100: Microcontroller integrated circuit 101: Microprocessor core 102: Memory Device 103: Source 104: External Bus 105: Sensor 106: Analog / Digital Converter 107: One or more serial input / output ports 108: General Purpose Input / Output Port 109: Direct Memory Access 110: Spiral Neural Network 111: One or more stream input ports 112: One or more output ports 113: Interface 200: Processor 201: Physical Signal 202: Feature Selection and Extraction Module 203: Selectors and Fetchers 204: Multiplexing and Preprocessing Platform 205: SNN Information Processor 206: Information or reasoning measurement 207: Operation Settings and Conditions Block 208: A subset of learning and signal processing tasks 209: Task Processing 210: Parametric Performance Settings 501: Cellular Components / Local Clusters 502: Cellular Components / Local Clusters 503: Cellular Components / Local Clusters 504: Cellular Components / Local Clusters 505: Cellular Components / Local Clusters 506: Cellular Components / Local Clusters 507: Cellular Components / Local Clusters 508: Cellular Components / Local Clusters 509: Cellular Components / Local Clusters 510: Cellular Components / Local Clusters 511: Cellular Components / Local Clusters 512: Cellular Components / Local Clusters
Claims
1. A computer-implemented method for configuring a spike neural network, wherein the spike neural network includes a plurality of spike neurons and a plurality of synaptic elements, the plurality of synaptic elements interconnecting the spike neurons to form a network at least partially implemented in hardware, wherein each synaptic element is adapted to receive a synaptic input signal and apply a weight to the synaptic input signal to generate a synaptic output signal, the synaptic elements being configurable to adjust the weight applied by each synaptic element, and wherein each of the spike neurons is adapted to receive one or more of the synaptic output signals from one or more of the synaptic elements, and generates a spatiotemporal spike output signal in response to the received one or more synaptic output signals, wherein a responsive local cluster within the network includes a set of the spike neurons and a plurality of synaptic elements interconnecting the set of the spike neurons, wherein the computer-implemented method includes: The weights of the synaptic elements and the peak behaviors of the peak neurons in the response local cluster are set such that: when one of the random distribution activities or statistical parameters of the set of neurons in the response local cluster is stationary, the network state in the response local cluster is a steady-state numerical solution, which represents a state of stable behavior of the network, wherein the most recently observed behavior of the system will continue into the future; and / or when a certain coupling strength is less than a predetermined coupling strength, the network state in the response local cluster is a time-varying numerical solution, which is a solution that does not exhibit stable behavior and is for the activity of its peak neurons changing over time.
2. The computer implementation method for configuring a spike neural network as claimed in claim 1, wherein the setting of the weights of the synaptic elements and the spike behavior of the spike neurons in the response local cluster includes iteratively training the response local cluster by optimizing the weights of the synaptic elements and the spike behavior of the spike neurons to achieve a desired steady-state behavior or a desired time-varying behavior.
3. The computer implementation method for configuring a spike neural network as described in claim 1, wherein the signal to be processed includes a radar signal, a light signal, an image signal, or an audio signal.
4. A computer implementation method for configuring a spike neural network as described in claim 1, wherein the steady-state numerical solution is a solution to one of the following equations: , where is the steady-state value of the autocorrelation function of the synaptic drive, F(t) is a deterministic function of the random part of the synaptic drive given by the following equation: , where Φ(t,τ) is the state transition matrix of the synaptic drive for all neurons in the local response cluster, and is an infinitesimally random variation, and E(t) is a deterministic function defined by ; and wherein the time-varying numerical solution is a solution to one of the following equations: which is a continuous-time algebraic Lyapunov matrix equation of the differential Lyapunov matrix equation, where is the autocorrelation function of the synaptic drive, Pr and Qr are discretized versions of E and F, tr represents the numerical integration time point, and F(t) is the deterministic function of the random part of the synaptic drive given by the following equation: , where Φ(t,τ) The state transition matrix is the synaptic-driven state transition matrix of all neurons within the local cluster of the response, and is an infinitesimal random variation, and E(t) is the deterministic function defined.
5. A computer implementation method for configuring a spike neural network as claimed in claim 1, wherein the spike neural network includes a driving local cluster, the driving local cluster including a set of spike neurons and a plurality of synaptic elements interconnecting the set of spike neurons, such that an output signal of one of the driving local clusters serves as an input signal of one of the responding local clusters to couple the driving local clusters to the responding local clusters with a specific coupling strength, wherein the computer implementation method further includes: When a specific coupling strength is less than the predetermined coupling strength, the network state within the response local cluster is set to a steady state, or the network state within the response local cluster is set to a time-varying state.
6. The computer implementation method for configuring a spike neural network as claimed in claim 5, wherein the setting of the weights of the synaptic elements and the spike behavior of the spike neurons in the response local cluster includes iteratively training the response local cluster by optimizing the weights of the synaptic elements and the spike behavior of the spike neurons to achieve a desired steady-state behavior and / or time-varying behavior.
7. The computer implementation method for configuring a spike neural network as claimed in claim 5, wherein when the response local cluster receives an input signal from the driving local cluster that does not include the predetermined oscillation frequency when represented in the frequency domain, or when the specific coupling strength is less than the predetermined coupling strength, the random distribution activity or statistical parameters of the set of neurons in the response local cluster are stationary or non-stationary.
8. The computer implementation method for configuring a spike neural network as claimed in claim 1, wherein the SNN is included in a neuromorphic SNN simulator, the neuromorphic SNN simulator preferably being a system containing electronic analog / mixed-signal circuits that mimic the neurobiological architecture present in the nervous system; and / or wherein the neurons and the synaptic elements are implemented in hardware using analog circuits or circuit elements or digital wired logic circuits or circuit elements, or a combination thereof, preferably wherein each neuron and each synaptic element is implemented as a hardware circuit or circuit element.
9. The computer implementation method for configuring a spike neural network as claimed in claim 5, wherein an increase in the structural dimension of the response local cluster is achieved by ensuring generalized external synchronization between the driver local cluster and the response local cluster, wherein the generalized external synchronization couples the driver local cluster to the response local cluster by means of a specific coupling strength equal to or greater than the predetermined coupling strength.
10. A computer implementation method for configuring a spike neural network as claimed in claim 9, wherein the generalized external synchronization is ensured based on an average autocorrelation function 1 / N˟{Σ E[Γ(t+τ / 2) Γ(t-τ / 2)T]} of a synaptic driver Γ(t), where τ is a delay and N is one of the number of neurons in the local cluster of the response, wherein an average value exceeds one number of neurons.
11. The computer implementation method for configuring a spike neural network as claimed in claim 1, wherein a steady-state numerical solution and / or a time-varying numerical solution is obtained by using feedback connections between neurons in a local response cluster, the feedback connections causing synchronization of neuronal activity of the neurons.
12. A spike neural network for processing input signals representable in a frequency domain, the spike neural network comprising a plurality of spike neurons and a plurality of synaptic elements, the plurality of synaptic elements interconnecting the spike neurons to form the network at least partially implemented in hardware, wherein each synaptic element is adapted to receive a synaptic input signal and apply a weight to the synaptic input signal to generate a synaptic output signal, the synaptic elements being configurable to adjust the weight applied by each synaptic element, and wherein each of the spike neurons is adapted to receive one or more of the synaptic output signals from one or more of the synaptic elements and generate a spatiotemporal spike output signal in response to the received one or more synaptic input signals, wherein a responsive local cluster within the network comprises a set of the spike neurons and a plurality of synaptic elements interconnecting the set of neurons, wherein: When the random distribution activity or statistical parameters of the set of neurons within the response local cluster are stationary, the network state within the response local cluster is a steady-state numerical solution, which represents a state of stable behavior of the network, wherein the most recently observed behavior of the system will continue into the future; and / or when a certain coupling strength is less than a predetermined coupling strength, the network state within the response local cluster is a time-varying numerical solution, which is a solution that does not exhibit stable behavior and is for the time-varying activity of its peak neurons.
13. The spike neural network of request item 12, wherein the steady-state numerical solution is a solution to one of the following equations: , where is the steady-state value of the autocorrelation function of the synaptic drive, F(t) is a deterministic function of the random part of the synaptic drive given by the following equation: , where Φ(t,τ) is the state transition matrix of the synaptic drive for all neurons in the response local cluster, and is an infinitesimal random variation, and E(t) is a deterministic function defined by ; and wherein the time-varying numerical solution is a solution to one of the following equations: which is a continuous-time algebraic Lyapunov matrix equation of the differential Lyapunov matrix equation, where is the autocorrelation function of the synaptic drive, Pr and Qr are discretized versions of E and F, tr represents the numerical integration time point, and F(t) is the deterministic function of the random part of the synaptic drive given by the following equation: , where Φ(t,τ) The state transition matrix is the synaptic-driven state transition matrix of all neurons within the local cluster of the response, and is an infinitesimal random variation, and E(t) is the deterministic function defined.
14. The spike neural network of claim 12, wherein the spike neural network includes a driving local cluster, the driving local cluster including a set of spike neurons and a plurality of synaptic elements interconnecting the set of spike neurons, such that an output signal of the driving local cluster acts as an input signal of a response local cluster, such that the driving local cluster and the response local cluster are coupled at a specific coupling strength, wherein when the response local cluster receives an input signal from the driving local cluster that does not include a predetermined first oscillation frequency in the frequency domain, or when the specific coupling strength is less than a predetermined coupling strength, the random distribution activity or statistical parameters of the set of neurons in the response local cluster are stationary or non-stationary.
15. The spike neural network of request item 14, wherein the signal to be processed includes a radar signal, a photoelectric signal, an image signal, or an audio signal.
16. The spike neural network of claim 12, wherein the driving local cluster is an input encoder of the spike neural network, the input encoder transforming a sampled input signal into a spatiotemporal spike train which is subsequently processed by the response local cluster.
17. A computer-implemented method for processing a specific frequency portion of an input signal that can be represented in the frequency domain using a spike neural network as described in claims 1 to 11 or the spike neural network as described in claims 12 to 16, comprising: Receive the signal to be processed; use an encoder to transform the signal to be processed into one or more spatiotemporal spikes; and use the configured spike neural network to process the one or more spatiotemporal spikes to obtain a spike neural network output signal.
Citation Information
Patent Citations
Artificial neural network and perceptron learning using spiking neurons
TW201541372A
Resilient neural network
WO2020099680A2
