Spiking Neural Networks

Abstract

A spiking neural network for classifying an input pattern signal comprises a plurality of spiking neurons implemented in hardware or a combination of hardware and software, and a plurality of synaptic elements interconnecting the spiking neurons to form a network. Each synaptic element is adapted to receive a synaptic input signal and apply a weight to the synaptic input signal to generate a synaptic output signal, the synaptic element being configured to adjust the weight applied by each synaptic element, and each of the spiking neurons is adapted to receive one or more of the synaptic output signals from one or more of the synaptic elements, and to generate a spatiotemporal pulse train output signal in response to the received one or more synaptic output signals.

Description

Technical Field

[0001] The present disclosure generally relates to composing spiking neural networks. The present disclosure more particularly, but not exclusively, relates to composing pattern recognizers constructed from spiking neurons, and a system and method for compositionally constructing spiking neural networks using unique response methods. Background Art

[0002] Automatic signal recognition (ASR) refers to the recognition of a signal by identifying its constituent features. ASR is used in a range of applications, such as recognizing a speaker's voice and spoken words in speech / voice recognition systems, identifying arrhythmias in an electrocardiogram (ECG), determining the shape of gestures in motion control systems, etc. ASR is usually performed by characterizing the patterns present in short samples of the input signal, and therefore accurate pattern recognition capabilities are the basis for an effective ASR system.

[0003] Measuring some physical quantity to derive an input signal for an ASR system may require fusing data from multiple types of sensors. For example, recognizing gestures using a handheld input device may require fusing data from an accelerometer to measure motion, and data from a gyroscope to measure orientation. Combining the data from these two sensors enables the detection of gestures in three-dimensional space.

[0004] In general, data fusion refers to the integration of data collected from different and possibly heterogeneous sources in order to reduce the uncertainty in the interpretation of the data from those individual sources. It is important that during the fusion process, the essential features in the different input signals are fully represented in the fused signal.

[0005] Pattern recognition and fusion are usually performed using microprocessors and / or digital signal processors, both of which implement a stored program architecture. This architecture is inherently inefficient for analyzing streaming data. On a single processor, pattern extraction and recognition are performed sequentially. This is because pattern extraction and recognition are implemented in accordance with a common simple instruction set (such as a RISC or CISC instruction set), resulting in a lengthy execution order for each pattern in the signal sample. Complex patterns in the input signal require the use of more complex signal processing algorithms, which further requires the use of higher clock frequencies for the processor in systems that require real-time responses from pattern recognition engines. This is not feasible in power-constrained devices (such as portable electronic devices, wearable devices, etc.). In addition, due to the sequential execution paradigm, the delay and power consumption of pattern recognition operations on microprocessors increase greatly with the complexity and list of patterns. The presence of noise in the input signal further increases the complexity of the analysis and has an adverse effect on performance and efficiency.

[0006] Data fusion is a non-native operation of microprocessors. This means that before an input signal or data stream can be fused, its information content must be extracted and thereafter combined with corresponding content from other streams. Therefore, in microprocessor implementations, each input signal / data source for fusion is processed separately by an independent pattern recognition pipeline, and then the different pattern recognition results are combined by a rule-based framework. This approach requires multiple calls to the pattern recognition infrastructure for each input signal or data stream, resulting in increased power consumption. In addition, the limitations of load-store architecture microprocessors and digital signal processors (DSPs) for pattern recognition mean that as the complexity of patterns increases, the power and latency costs of recognizing these patterns also increase. Although the latency costs can be alleviated by increasing the clock frequency, this is achieved at the expense of further increasing power consumption. Similarly, the quality of fusion is limited by the complexity of the processing performed on the input signals and the ASR, the number of signals being fused, and the computational power of the microprocessor or DSP. The sequential nature of the processing reduces the throughput of fusion-based ASR, and therefore, as the complexity of the patterns in the signal increases, the number of input signals that can be fused using a microprocessor or DSP decreases.

[0007] Artificial neural networks have been proposed as an alternative to microprocessor implementations. Spiking neural networks (SNNs) are a promising means of implementing ASR for many different applications. SNNs encode information in the form of one or more precisely timed (voltage) pulses, rather than in the form of integers or real-valued vectors. The computations for pattern classification are performed efficiently in both the analog and time domains. For this reason, SNNs are typically implemented in hardware as full-custom mixed-signal integrated circuits. This enables them to perform pattern classification with energy consumption several orders of magnitude lower than their artificial neural network counterparts, in addition to having smaller network sizes.

[0008] SNNs consist of a network of spiking neurons interconnected by synapses, which determine the strength of the connection between spiking neurons. This strength is represented as a weight, which regulates the influence of the output of the presynaptic neuron on the input of the postsynaptic neuron. Typically, these weights are set during a training process that involves exposing the network to a large amount of labeled input data and gradually adjusting the weights of the synapses until the desired network output is achieved. In practice, however, this large amount of labeled data may not exist at all.

[0009] Furthermore, training large multi-layer networks in a monolithic fashion is time-consuming, as the size of the network leads to a complex optimization problem whose solution is computationally expensive due to the need to back-propagate errors through several layers of the network.

[0010] For the purpose of determining the error, spiking neural networks typically rely on a learning rule based on the relative firing time of two neurons connected by a synapse. A common learning rule is called spike timing dependent plasticity or STDP. Although this approach enables the SNN to potentially set the weights of synapses without supervision, it is inherently unstable. This is because STDP targets weight adjustments at individual synapses and lacks a mechanism to balance synaptic weights at the network level. Therefore, the STDP process can be unstable and produce divergent synaptic weight configurations. Although this problem can be alleviated by balancing the magnitude and distribution of excitatory and inhibitory connections formed by synapses in the network, the process is difficult to implement and requires an exhaustive search to produce a stable, convergent weight configuration. Spiking neural networks can also be trained using traditional back-propagation methods. However, due to the complex causal relationships between neurons across different layers of the network, these techniques are computationally expensive when applied to deep, multi-layer networks. Summary of the invention

[0011] In order to address the shortcomings of the prior art discussed above, according to one aspect of the present disclosure, a spiking neural network for classifying an input pattern signal is proposed, comprising a plurality of spiking neurons implemented in hardware or a combination of hardware and software, and a plurality of synaptic elements interconnecting the spiking neurons to form a network. Each synaptic element is adapted to receive a synaptic input signal and apply a weight to the synaptic input signal to generate a synaptic output signal, the synaptic element being configured to adjust the weight applied by each synaptic element, and each of the spiking neurons is adapted to receive one or more of the synaptic output signals from one or more of the synaptic elements, and generate a spatiotemporal pulse train output signal in response to the received one or more synaptic output signals.

[0012] The spiking neural network comprises a first sub-network comprising a first subset of spiking neurons connected to receive synaptic output signals from a first subset of synaptic elements. The first sub-network is adapted to generate a sub-network output pattern signal from the first subset of spiking neurons in response to a sub-network input pattern signal applied to the first subset of synaptic elements, and to configure the weights of the first subset of synaptic elements by training the sub-network on a training set of the sub-network input pattern signals such that the sub-network output pattern signal is different for each unique sub-network input pattern signal of the training set.

[0013] In one embodiment, the distance between each different sub-network output pattern signal is greater than a predetermined threshold, the distance being measured by an output pattern metric. The spiking neurons and synaptic elements can be configured such that: a corresponding distance between two output pattern signals of the sub-networks measured by the output pattern metric is maximized for all sub-network input pattern signals of the training set, the two output pattern signals of the sub-networks being generated in response to two corresponding different sub-network input pattern signals. Each corresponding distance can be maximized until the output pattern signal at least meets a first minimum sensitivity threshold required to distinguish features of the input pattern signal.

[0014] A feature is any characteristic of an input pattern signal that enables identification of the signal and / or a pattern of interest within the signal. A pattern may consist of one or more features. For example, relevant features in an ECG input signal may be various time-varying amplitudes that characterize different phases of a heartbeat, for an audio input signal, relevant features may be the frequency content of the signal within each discrete time interval at which the audio signal is sampled, or for an image input signal, relevant features may be the basic shapes and lines present in the image. Additional examples are provided herein.

[0015] In a spiking neural network, each of the spiking neurons is configured to adjust the response of the neuron to one or more synaptic output signals received. Spikes may be generated by the spiking neurons at one or more firing times, and a subset of synaptic elements and / or a subset of neurons are configured such that: a union of two sets of firing times of the subset of neurons that are fired for two different sub-network input pattern signals is minimized for all sub-network input pattern signals of the training set. Each union may be minimized until the output pattern signal at least satisfies a second minimum sensitivity threshold required to distinguish features of the input pattern signal.

[0016] A spiking neural network may include a second subnetwork comprising a second subset of spiking neurons connected to receive synaptic outputs from a second subset of synaptic elements, wherein the second subnetwork is adapted to receive a second subnetwork input pattern signal applied to the second subset of synaptic elements and to generate a corresponding second subnetwork output pattern signal from the second subset of neurons, and wherein the configuration of the second subset of synaptic elements is adjusted such that the second subnetwork output pattern signal is different for each unique feature in the second subnetwork input pattern signal, wherein the network includes a third subnetwork comprising a third subset of spiking neurons connected to receive synaptic outputs from the third subset of synaptic elements, wherein the first subnetwork output pattern signal and the second subnetwork output pattern signal are input pattern signals to the third subnetwork, and wherein the configuration of the third subset of synaptic elements is adjusted such that the third subnetwork output pattern signal is different for each unique feature and unique combination of the input pattern signals from both the first subnetwork and the second subnetwork, such that features present in the input pattern signals from both the first subnetwork and the second subnetwork are encoded by the third subnetwork.

[0017] The synaptic elements of the third subnetwork may be configured such that the input pattern signals from the first subnetwork and the second subnetwork are weighted according to the importance of a particular input pattern. The network may include multiple subnetworks of synaptic elements and spiking neurons, for which the subnetwork output pattern signals are different for each unique feature in the subnetwork input pattern signals, wherein the network may be divided into multiple layers having a specific sequential order in the network, and wherein the multiple subnetworks are instantiated in the specific sequential order of the multiple layers to which each respective subnetwork belongs.

[0018] In spiking neural networks, synaptic elements can be arranged as a configurable switch matrix.

[0019] Configuration information of neurons, synaptic elements, interconnection topology of neurons and synaptic elements and / or output configuration of the neural network can be saved on a configuration memory, and the configuration information can be loaded from the configuration memory when the neural network is brought to an initialized state.

[0020] A group of neurons may be arranged in one or more neuron template networks, wherein the neurons of a particular neuron template network form a subnetwork of the neural network, wherein the subnetwork output pattern signal is different for each unique feature in the subnetwork input pattern signal, and wherein each template network in the neural network is instantiated in a pre-trained manner.

[0021] The neural network can be configured to take one or more sampled input signals as input and convert the input signals into a set of representative neural network input pattern signals. The output pattern signals of the neural network can be classified into one or more output categories.

[0022] According to one aspect of the present invention, a method for classifying an input pattern signal using a spiking neural network is proposed, the spiking neural network comprising a plurality of spiking neurons implemented in hardware or a combination of hardware and software, and a plurality of synaptic elements interconnecting the spiking neurons to form a network, wherein each of the synaptic elements is suitable for receiving a synaptic input signal and applying weights to the synaptic input signal to generate a synaptic output signal, the synaptic element being configured to adjust the weights applied by each synaptic element, and wherein each of the spiking neurons is suitable for receiving one or more of the synaptic output signals from one or more of the synaptic elements, and generating a spatiotemporal pulse train output signal in response to the received one or more synaptic output signals. The method comprises the following steps: defining a first subnetwork of a spiking neural network, the first subnetwork comprising a first subset of spiking neurons connected to receive synaptic output signals from a first subset of synaptic elements; configuring weights of the first subset of synaptic elements by training the subnetwork on a training set of subnetwork input pattern signals such that the subnetwork output pattern signal is different for each unique subnetwork input pattern signal of the training set; applying the subnetwork input pattern signal to the first subnetwork of the spiking neural network, the first subnetwork comprising a first subset of spiking neurons connected to receive synaptic output signals from the first subset of synaptic elements; and receiving a subnetwork output pattern signal generated by the first subset of spiking neurons in response to the subnetwork input pattern signal, wherein the output pattern signal identifies one or more features of the input pattern signal.

[0023] According to another aspect of the present invention, a method for classifying predetermined features in an input signal is provided. The method comprises establishing and training a neural network as described above, submitting at least one sampled input signal to the neural network, and classifying the features in the at least one sampled input signal into one or more output categories of the neural network.

[0024] According to another aspect of the present invention, there is provided a template library comprising one or more template networks of spiking neurons used as subnetworks of a spiking neural network or information about the configuration thereof, wherein each template network comprises a group of spiking neurons connected to receive synaptic outputs from a group of synaptic elements. Each template network is adapted to receive a template network input pattern signal applied to the group of synaptic elements and to generate a corresponding template network output pattern signal from the group of neurons. The configuration of the group of synaptic elements is adjusted by training the template network on a training set of template network input pattern signals, so that the template network output pattern signal is different for each unique template network input pattern signal of the training set, wherein the training set is used to train the template network to perform a specific task, so that each template network can be used as a subnetwork of a spiking neural network in a pre-trained manner to perform a specific task, or so that a subnetwork of a spiking neural network can be instantiated in a pre-trained manner based on information about the configuration of the template network to perform a specific task.

[0025] According to another aspect of the present invention, a method for composing a spiking neural network is proposed, the method comprising obtaining one or more template networks or information about the configuration of one or more template networks from the template library as described above, and instantiating the one or more template networks as sub-networks of the spiking neural network, so that the sub-network of the spiking neural network can perform the specific tasks that the template network is pre-trained to perform.

[0026] The present invention consists of a method for implementing a pattern classifier in a bottom-up manner using smaller network levels that are trained separately. Each level of the pattern classifier is trained for its specific function using a common method that is applied together with different training objectives specific to the function being implemented. The training process is assisted by a method that takes into account the long-range causal influence of synapses when adapting weights, thereby overcoming the instability problems faced by STDP. The three methods can be used independently of each other. The method of the present invention enables the implementation of a low-latency pattern recognizer that can be applied to different signal types and can recognize and associate patterns across multiple signals. The method of the present invention simplifies the training and deployment time of the pattern recognizer, facilitates the reuse of previously trained pattern recognition networks, while concomitantly ensuring that the implementation remains flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Embodiments will now be described, by way of example only, with reference to the accompanying schematic drawings, in which corresponding reference numerals indicate corresponding parts, and in which:

[0028] Figure 1 An exemplary neural network consisting of neurons and synaptic elements is shown;

[0029] Figure 2shows a pattern recognizer composed by integrating a data converter stage, an input encoder stage, and a pattern classifier stage, each stage being trained separately and instantiated in subsequent stages;

[0030] Figure 3A Schematically shows how the input encoder transforms the sampled input signal into a spatiotemporal pulse train;

[0031] Figure 3B Examples of expected impulse responses for three input patterns described in the form of impulse tuples are shown;

[0032] Figure 4 The principle of causal chain STDP is shown to identify deep causal, anti-causal, and non-causal relationships between postsynaptic neurons and neurons in subsequent network layers;

[0033] Figure 5A shows a pattern recognizer for sensor fusion formed by fusing two separate pulse coded data streams, and subsequent pattern classification using the fused pulse train;

[0034] Figure 5B schematically illustrates how to fuse multiple temporal pulse trains into a single fused pulse train;

[0035] Figure 6 A system for fusing multiple data streams is shown, and the system can use a network of spiking neurons to automatically identify signals on the fused data streams;

[0036] Fig. 7A An example showing how to interface input signals to neurons;

[0037] Figure 7B An example showing how to interface input signals to neurons;

[0038] Figure 8 shows an example of how to generate a reset pulse;

[0039] Fig. 9 A method for fusing multiple data streams using a network of spiking neurons and performing automatic signal recognition on the resulting fused data stream is shown;

[0040] Fig.10 shows an array of neurons and synaptic elements interconnected by a configurable switch matrix;

[0041] Fig.11 An example of a configuration array is shown, illustrating an example mapping of ASR pipeline stages onto neuronal and synaptic elements.

[0042] The drawings are intended to be illustrative only and not to limit the scope or protection provided by the claims. DETAILED DESCRIPTION

[0043] Certain embodiments will be described in further detail below. However, it should be understood that these embodiments cannot be interpreted as limiting the scope of protection of the present disclosure.

[0044] Figure 1 is a simplified diagram of a neural network 100. Neurons 1 are connected to each other via synaptic elements 2. In order not to clutter the image, only a small number of neurons and synaptic elements are shown (and only some have reference numerals attached to them). Figure 1 The connection topologies shown in, i.e. the way in which synaptic elements 2 connect neurons to each other 1, are only examples, and many other topologies may be adopted. Each synaptic element 2 may transmit a signal to the input of a neuron 1, and each neuron 1 receiving a signal may process the signal and may subsequently generate an output, which is transmitted to other neurons 1 via further synaptic elements 2. Each synaptic element 2 has a certain weight assigned to it, which is applied to each synaptic input signal received and transmitted by the synaptic element to produce a weighted synaptic output signal. The weight of a synaptic element is therefore a measure of the kind of causal relationship between two neurons 1 connected by the synaptic element 2. The relationship may be causal (positive weight), anti-causal (negative weight) or non-existent (zero weight).

[0045] Neurons 1 and synaptic elements 2 may be implemented in hardware, for example using analog circuit elements or digital hard-wired logic circuits. They may also be implemented partially in hardware and partially in software, or entirely in software. Implementing in hardware is preferred in order to achieve faster processing, for example enabling much faster pattern recognition, and event-driven processing (where blocks of neurons and synaptic elements are activated only when needed).

[0046] The neural network 100 may be a spiking neural network. Then, neuron 1 is a spiking neuron that generates a neuron output signal in the form of one or more pulses. The spiking neuron 1 may be configured to excite (i.e., generate an output pulse) only when the membrane potential (e.g., energy potential, or voltage or current level) within the neuron reaches a predetermined threshold. Since the received input signal, i.e., the synaptic output signal received by the neuron from the synaptic element is accumulated, integrated, or otherwise processed to change the membrane potential, the membrane potential of the spiking neuron changes. When the weight of the synaptic element 2 is positive, the synaptic output signal received from the synaptic element excites the spiking neuron 1 that receives the signal, increasing its membrane potential. When the weight of the synaptic element 2 is negative, the synaptic output signal received from the synaptic element inhibits the spiking neuron 1 that receives the signal, reducing its membrane potential. When the weight of the synaptic element 2 is zero, the synaptic output signal received from the synaptic element has no effect on the spiking neuron 1 that receives the signal.

[0047] When the membrane potential of a spiking neuron 1 reaches a threshold value, the neuron fires, generating a pulse upon firing, and the membrane potential decreases as a result of the firing. If the membrane potential subsequently reaches the threshold value again, the neuron will fire again, generating a second pulse. Thus, each spiking neuron 1 is configured to generate one or more pulses in response to an input signal received from a connected synaptic element 2, the pulses forming a spatiotemporal pulse train. Since a spiking neuron 1 fires only when its membrane potential reaches a predetermined threshold value, the encoding and processing of temporal information is incorporated into the neural network 100. In this way, a spatiotemporal pulse train is generated in the spiking neural network 100, which is a temporal order of pulses generated by the spiking neurons 1 of the network 100.

[0048] The temporal properties of the pulse train encode the amplitude and frequency characteristics of the input signal. The temporal properties include: the delay between the onset of a stimulus (e.g., an input signal from a synaptic element) and the generation of a pulse at the output of the neuron; the delay between consecutive pulses from the same neuron; and the number of pulses that the neuron fires during the duration of the applied input stimulus.

[0049] The synaptic element 2 may be configurable, such that, for example, the corresponding weight of the synaptic element may be changed, for example by training the neural network 100. The neuron 1 may be configured in the way that it responds to a signal from the synaptic element. For example, in the case of a spiking neural network, the neuron 1 may be configured in the following way: a certain signal raises or lowers the membrane potential, the time required for the membrane potential to decay naturally to the resting potential, the value of the resting potential, the threshold that triggers a spike of the spiking neuron 1. The configuration of the neuron 1 may, for example, remain constant during training, or be variable and set for a specific training set in the training of the neural network 100.

[0050] The input signal 11 is, for example, a plurality of different, sampled input signals, or a temporal pulse train. The input may be the analog-to-digital converted value of a signal sample, or the digital value of a sample in the case of an analog or digital integrator, or the analog value of a sample in the case of an analog integrator.

[0051] The output signal 12 of the neural network 100 is, for example, a spatiotemporal pulse train, which can be read out from the output neuron 1 and further classified and transformed into a set of digital values ​​corresponding to the type of output code selected by the user through the output transformation stage.

[0052] Operations such as pattern recognition can be accomplished in multiple steps (e.g., data conversion, feature encoding, classification). The present embodiment enables these steps to be implemented using standardized sub-networks of synaptic elements and neurons (units), and configured using flexible training methods for each application domain. The goal of the training method is to configure each level or sub-network of the neural network to produce a unique response to each unique pattern or feature in the input signal.

[0053] Figure 2 The composition of a neural network is shown, formed by the integration of multiple constituent stages, in this example including a data converter stage 201, an input encoder stage 202, and a pattern classifier stage 203. Each stage can be trained individually and instantiated in the network for use in training subsequent stages. The term "instantiation" refers to a process in which a portion of a network, such as a stage or unit of a network, forming a sub-network has been previously configured, typically through training, and the configuration data of the sub-network is used to configure the sub-network in the entire network.

[0054] The data converter stage 201 receives an input pattern signal for processing by a neural network. In this example, the data converter stage 201 is shown as a cell 301, the data converter stage 202 is shown as a part of the cell 302, and the pattern classifier stage 203 is shown as a part of the cell 303. Each cell 301, 302, 303 includes a sub-network formed by a subset of the synaptic elements 2 of the network and a subset of the neurons 1, which are connected in a topological structure that implements the function that the cell is intended to perform. Each cell 301, 302, 303 is parameterized, that is, it can be configured to have a different number of synaptic elements 2, neurons 1, input connections, output connections, hidden layer connections (i.e., connections between neuron layers that are not visible to the input of the output), etc.

[0055] The weights of the synaptic elements 2 in each of the stages 201, 202, 203 may be configured by training. In this example, the data converter stage 201 is first trained in the unit 301. After training the data converter stage 201, the configuration settings of the trained data converter stage 201 and the unit 301 may be saved. These saved settings may then be used to configure the unit 302, which includes both the data converter stage 201 and the input encoder stage 203.

[0056] Unit 302 may then be trained, and the configuration settings of the trained encoder stage 203 and unit 302 may be saved. These saved settings may then be used to configure unit 303, which includes data converter stage 201, input encoder stage 203, and pattern classifier stage 203. Unit 303 may then be trained, and the resulting configuration settings of the trained pattern classifier stage 203 and unit 303 may be saved.

[0057] Training can be accomplished using a unique response algorithm. The unique response algorithm implements the weight configuration of the synaptic elements of each unit, which causes unique features in the input signal to be transcoded into a unique response of the neuron 1 of the unit. In the case of a spiking neuron, this can be a unique spiking response. The unique response algorithm is further described below. The uniqueness of the above-mentioned unit response indicates that the unit has the ability to separate different features in the input pattern signal 11 into different output signals 12A, 12B, 12C of the unit. In the case of a spiking neuron, these different output signals 12A, 12B, 12C can be different output pulse trains, for example, a set of unique spatiotemporal pulse train outputs from the output neurons of the unit. The uniqueness of a set of spatiotemporal pulse train outputs can be determined by the number of pulses generated by each output neuron of the unit and / or the timing of the pulses. Therefore, during the training of the network, after the input is presented to the network and converted into pulses, the uniqueness of each response is tested, and the weights of the synaptic elements of the trained unit are adjusted using the unique response method until the output responses 12A, 12B, 12C of each unit 301, 302, 303 meet the uniqueness criteria. Teacher input 13A, 13B, 13C may be used to obtain a desired unique response for each unit 301, 302, 303, as described below.

[0058] To implement higher level functions, previously trained units 301, 302, 303 may be instantiated and interconnected in a bottom-up manner, i.e., the lowest level unit 301 may be trained first and then instantiated in a higher level unit 302 containing a subsequent level 202. Then, a mid-level unit 302 may be instantiated in a higher level unit 303 containing a subsequent level 203. After each creation of a higher level unit, a unique response algorithm is called.

[0059] Units 301, 302, 303 can be trained in this way for specific functions in different application areas, and the configuration settings of the trained units can be saved in a template library, from where they can be reused in different networks. For example, the input encoder stage 202 of the spoken language recognition network can be reused in the hot word trigger network without any changes. This reuse helps reduce overall training time and complexity.

[0060] Figure 3A It is schematically shown how an input encoder 22 transforms a sampled input signal 20 into a spatiotemporal pulse train 24. Each output neuron of the input encoder 22 forms an output signal 23, which is a spatiotemporal pulse train. The input encoder 22 is configured by means of a configuration 21, wherein for example the weights of the synaptic elements of the input encoder are set.

[0061] Figure 3B Examples of three expected impulse responses 14A, 14B, 14C to three different input patterns of a unit of a spiking neural network are shown. Each response in this example takes the form of a set of one or more spatiotemporal pulse trains generated by the output neurons N1, N2, N3, N4, N5 of the unit. Each response 14A, 14B, 14C is described for a discrete time bin 15A, 15B, 15C that can be selected for each network. The time bin 15A, 15B, 15C can be at least as large as the time between the start of the first input stimulus of the unit and the last response from the output neuron of the unit. The time bin can be a multiple of the fall delay of the unit, which is equal to the typical number of pulses generated by each output neuron of the unit due to the input stimulus.

[0062] During each invocation of a unique response method, the units 301, 302, 303 may be exposed to input data 11 derived from the input pattern to be processed. The input data 11 is derived from the input pattern p presented to the neural network for processing.

[0063] The input pattern p may include, for example, an audio signal for speech recognition applications, a still image or video image for image recognition applications, or many other types of patterns suitable for pattern recognition systems. The input data 11 may consist of a single item of time-varying data (e.g., a single data stream derived from an audio signal, such as an analog or digital signal output by a microphone), multiple items of data (e.g., a set of input data representing individual pixels of a still image output by a CCD array), multiple items of time-varying data (e.g., a set of input data streams representing individual pixels of a video image), or multiple enhanced versions generated from a single item of data (e.g., combinations of input signals).

[0064] The weights of the synaptic elements 2 of the units 301, 302, 303 can be adapted using Causal Chain Spike Timing Dependent Plasticity (CC-STDP), which is a learning rule that takes into account long-range causality when adapting the synaptic weights. In this sense, it differs from the conventional Spike Timing Dependent Plasticity (STDP) learning rule, which only takes into account short-range causality when adapting the weights, making the weight updates using some training sets rather unstable and chaotic, and mainly the result of local behavior. CC-STDP is further proposed below. Once a unit is trained, the configuration settings for the trained unit can be stored into a template unit library, from which the configuration settings can be subsequently called and reused.

[0065] The expected impulse response 14A, 14B, 14C for each input pattern p can be described as a tuple that specifies the impulse response in each time bin where input data 11 derived from the input pattern p is presented. For example, the tuple may specify the output neuron population (A) of the unit that fires in response to the input data 11. p ), and their precise excitation time (B p The distinctiveness of a response is determined by calculating its distance from the unit’s responses to other input patterns using a distance function or metric (such as the Victor Purpura or Van Rossum distance functions), which is typically A p and B p function.

[0066] In a preferred embodiment, for each input pattern p, the set A p can describe a disjoint set containing a single output neuron 1, and set B p The time bins corresponding to when this output neuron 1 responded can be described. Thus, in this embodiment, each input pattern p can be identified by the single neuron 1 that responds earliest. Later responses from the same or other neurons can be suppressed, for example using a winner-takes-all network, or ignored. For additional robustness, the tuple can be modified with an additional field (C p ), which describes the strength of the neuron’s response (expressed as the number of spike events generated by a single earliest spiking neuron during the application of input data 11, C p ).

[0067] In an alternative embodiment, for each pattern p, the set A p can be a disjoint set containing multiple output neurons 1, and set B p can be a collection of the same size containing the response times of those neurons. Using the causal chain STDP rule proposed below, the responses of the units can be directed towards different A p and B p values ​​in order to achieve the desired pulse tuple.

[0068] B can be reduced by increasing the weight in the synaptic element that contributes to the firing of the corresponding neuron 1. p value, and B can be increased by reducing those weights p This can be achieved by artificially promoting causal firing of output neuron 1, or by promoting anti-causal firing by inducing an early spike accompanying the application of input data 11. This results in positive or negative synaptic weight adjustments of the contributing pathways, respectively.

[0069] exist Figure 3B, the expected impulse responses 14A, 14B, 14C for three input patterns p are shown. The response for the first input pattern is generated during a first time bin 15A, the response for the second pattern is generated during a second time bin 15B, and the response for the third pattern is generated during a third time bin 15C. The first pattern (p 1 ) has a tuple and The second mode (p 2 ) has a tuple and Finally, the third mode (p 3 ) has a tuple and

[0070] The goal of the unique response method can be to generate a unique set of spatiotemporal pulse trains for each specific input pattern. These spatiotemporal pulse trains can be composed as a series of precisely timed electrical pulses. The uniqueness of the response can be described as follows.

[0071] Let A represent a set of neurons that generate output spike trains. Let p be a given known input pattern. p is a subset of A, representing the set of neurons that fire for a given input pattern p. s ={A p |p∈P} is the set of all group neurons that fire for each input pattern of all known input patterns P in the group.

[0072] In addition, let F i p is the set of precise firing times of neuron i for a specific input pattern p. For all output neurons 1,2,...,N O The set of exact firing times for a particular input pattern p. Denote B s = {B p |p∈P} is a set of multiple groups containing the exact firing times of all input patterns p for all known input patterns P in the set.

[0073] In one embodiment, the network's response to input data 11 may be called unique if, within each given sampling window of, for example, a time-varying input, one or more of the following rules apply. As a first rule, for all unique input patterns p, q in P (so that p≠q), Δ(A p ,A q )≠0. As a second rule, for all unique input patterns p, q in P, we should assume that Δ(B p ,B q )≠0.

[0074] Distance Δ(Ap ,A q ) and Δ(B p ,B q ) can be calculated together using an appropriate pulse distance metric Δ(·;·), such as Victor-Purpura, Van Rossum, ISI, or pulse-distance.

[0075] These rules can be applied as goals for synaptic weight configuration, where they can be restated as follows. The first weight configuration goal can be: s All A's p , maximize A s Each A q The distance Δ(A p ,A q ), where q is a specific different input mode in P. The second weight configuration target can be: for B s All B p , maximize B s Each B q The distance A(B p ,B q ), and minimize B s Each B q The union of p ∪B q , where q is a specific different input pattern in P.

[0076] Desired pulse tuples can be generated based on these weight configuration targets, and, for example, the input encoder stage 202 can be trained using CC-STDP as proposed below. The goal of this exercise is to arrive at a configuration of the input encoder stage 202 that produces a unique response that subsequent stages can use to distinguish features in the input pattern.

[0077] Although the configuration objective is intended to produce the best result, this can be computationally expensive. A simpler approach is to constrain the objective so that A(A p ,A q )、A(B p ,B q ) increases, and B p ∪B q The weights are then reduced until the output of the input encoder stage at least meets the minimum sensitivity threshold required to distinguish features in subsequent processing stages. This allows the runtime of the weight configuration training process to be reduced without compromising the robustness and quality of the encoding.

[0078] Once the complete set of spike tuples is described or generated for all the patterns of interest, the weights of the synaptic elements 2 connected to the neurons 1 in the units 301, 302, 303 can be trained so that they exhibit the desired unique response to the proposed input pattern. In the supervised learning regime, this training is performed with supervised methods (e.g. autoencoder methods, exhaustive search, gradient descent, using backpropagation, etc.). However, this is computationally expensive. In the following, we describe a semi-supervised training method that relies on a new STDP weight promotion scheme. Semi-supervised may mean that one does not say what the errors produced are, but, for example, one says what the expected or required spikes are in order to obtain a unique response to a certain input pattern.

[0079] In conventional STDP, the weight change of a synaptic element depends on the relative firing times of two neurons 1 connected to each other by this particular synaptic element 2. However, the resulting weight change is primarily the result of local behavior. Causal chain STDP (CC-STDP) instead projects the change of weights on a longer causal chain of presynaptic neurons, i.e., the weights are adjusted only if the firing event contributes to the firing of neurons in subsequent layers of the network.

[0080] Figure 4 The principle of CC-STDP learning rule is shown, which can identify 1,2 The deep causal, anti-causal, and non-causal relationships between the presynaptic neuron N1 and postsynaptic neuron N2 of a specific synaptic element 2 and the neurons N3, N4, and N5 in subsequent layers of the network.

[0081] The CC-STDP rule can be applied to the semi-supervised training of a unit to obtain a specific output impulse response. For example, suppose n is a specific output neuron where at a specific time t spike n During training, the expected spike is spike n Artificially induced spikes can establish the nature of the relationship between neurons in the early layers of the network and neuron n. These relationships can be causal (neurons in the early layers cause spikes at the output neurons), anti-causal (neurons in the early layers do not cause spikes at the output neurons, but the firing of the two neurons is synchronized, with the firing of the early neurons following that of the output neurons), or non-causal (the firing behavior of the two neurons is not synchronized). Thus, CC-STDP enables the identification of causal relationships between neurons in the previous layers of the network and the desired output neurons, and enables it to find causal relationships between output neurons in the network, and enables the weights of intervening synaptic elements along the causal path to be adjusted.

[0082] At neurons where a spike might not be expected, an anti-causal relationship can be simulated by inducing a spike along with the input sample presented to the unit, i.e., in the same time interval. This can cause the output neuron to fire before neurons in all previous layers, thus creating an anti-causal relationship, and in turn causing local inhibition of intervening synaptic elements along the path.

[0083] Therefore, by spike n By inducing spikes at the desired output neuron(s), and inducing spikes at other neurons at times concomitant with the application of input stimuli, the network is configured to respond with a desired spike response pattern.

[0084] For example, the weight w of the synaptic element located between neuron i and neuron j is i,j Can vary according to the following rules:

[0085]

[0086] Here in formula (1), Δw i,j is the effective weight change of the synaptic element between neurons i and j, w j,k is the weight of the synaptic element between neuron j and neuron k in the subsequent layer. spike i ,t spike j and t spike k It refers to the precise time when the pulses of neurons i, j and k appear respectively. i,j is the change in weight of the synaptic element between neurons i and j due to their relative spike timing and can be calculated using the STDP learning rule.

[0087] Therefore, according to the learning rule of formula (1), if the weight change of the synaptic element between neurons i and j should be positive, and if the weight between neurons j and k is non-zero, and if the pulse of neuron k is later than that of both neurons i and j, then Δw i,j =δw i,j Finally, the weight change will be zero if neuron k does not spike, if neuron k spikes immediately, if neuron k spikes before neurons i or j, or if the weight between neurons j and k is zero. In the latter case, there appears to be no causal relationship between the firing of neuron i and the firing of neuron k.

[0088] Alternatively, a learning rule may also be used, in which case the pulse time parameter in the above equation may be replaced by a basic parameter of the chosen learning rule (eg, pulse rate).

[0089] This principle can be referred to Figure 4For example, in a particular output neuron, a spike is induced at a particular time (e.g., in neuron N3 at spike time t spike N3 = 4 ms, in neuron N4 at the pulse time t spike N4 =3 ms, and no induced spike at neuron N5). In this way, a desired specific spatiotemporal spike output pattern is produced. This desired output pattern belongs to a specific spatiotemporal input pattern of the spiking neural network (e.g., neuron N1 at t spike N1 =1ms). For example, due to the input pattern and the specific synaptic weight w 1,2 , neuron N2 at t spike N2 =3ms. The effective weight change of the synaptic element is Δw 1,2 This can be calculated using, for example, the above formula (1). 2,3 ≠0, and if w 1,2 > 0, then the contribution from the firing of neuron N3 at that particular time point is δw 1,2 . Due to w 2,4 =0, so the resulting change in the synaptic weight from the firing of neuron N4 is zero. Since neuron N5 has no spikes, the resulting change from neuron N5 is zero.

[0090] In embodiments where CC-STDP is used to train a spiking neural network, it may be desirable to be able to induce and / or inhibit the generation of spikes at a particular neuron 1 at a particular time. As examples, this may be accomplished in the following ways (this list is not exhaustive). To induce a spike, one approach is to drive the membrane of the corresponding neuron to a voltage exceeding the excitation threshold by means of a bias voltage input, thereby inducing the generation of a spike. Another approach may be to inject an artificial spike into the output of the neuron at the desired time. To prevent a spike, one approach is to drive the membrane of the corresponding neuron to an anti-excitation voltage or the lowest possible voltage, thereby preventing the generation of a spike. Alternatively, one may inhibit the generation of a spike at a neuron by, for example, disconnecting the neuron input from an integrator, thereby, for example, disconnecting the pulse generating circuitry from the output. These signals for inducing or inhibiting the generation of a spike in a neuron are shown as teacher inputs 13A, 13B, 13C, which may be used to obtain a desired unique response for each unit 301, 302, 303.

[0091] Figure 5A The composition of a sensor fusion pattern recognizer is shown by fusing two separate data streams 11A, 11B, and subsequently using the fused stream 12C for pattern classification in a pattern classifier stage 204. The output 12D of the pattern classifier stage 204 may be, for example, a set of spatiotemporal spike trains 12D generated by output neurons of the pattern classifier stage 204, wherein each spike train of the set 12D is generated by a single specific output neuron in the pattern classifier stage 204.

[0092] That is, in the case of a layer in a stage having multiple concurrent units (e.g., multiple input encoders 202A, 202B forming the input encoder in a fusion system), each of these units 202A, 202B can be configured first before being instantiated in a unit with a subsequent stage 203. All stages of the network can be trained using the teacher inputs 13A1, 13A2, 13B1, 13B2, 13C, 13D. In the case where all concurrent units perform the same function on the same data (e.g., input encoding of different receiving antennas in a radar transceiver, or input encoding of different microphones pointing in different directions), it may not be necessary to run a separate training sequence for each instance. Instead, a single training unit can be instantiated multiple times. Therefore, the trained unit or its configuration can be implemented in different spiking neural networks to perform the same function. Therefore, during the composition of the spiking neural network, if the same function is required, the already trained unit can be implemented without any new training. Therefore, the composition of the spiking neural network will take less time and will be less complex.

[0093] The memory device may be configured to serve as a template library of trained units. On such a memory device, software and / or hardware settings of one or more trained units of a spiking neural network may be stored. Configuration settings of a synaptic element 2 of such a unit may be, for example, the respective weights of the synaptic element. Configuration settings of a neuron 1 of such a unit may be, for example, a firing threshold, a leakage rate, a resting value, an anti-firing level, an anti-firing period or other parameters of a neuron.

[0094] Each unit is trained to perform a specific function. When such a new unit with that specific function is needed in a specific spiking neural network, the unit does not have to be trained from scratch, which saves time and greatly improves the efficiency of implementing spiking neural networks. Instead, the configuration settings from a pre-trained unit designed for that function can be obtained from a memory device configured as a template library. By instantiating a new unit using the pre-trained unit configuration settings, the new unit will be able to operate in the same manner as the pre-trained unit.

[0095] The memory device may be any computer-readable storage medium. The memory device may be a non-transitory storage medium. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., a read-only memory device within a computer, such as a CD-ROM disc readable by a CD-ROM drive, a ROM chip, or any type of solid-state non-volatile semiconductor memory), on which information can be permanently stored; and (ii) writable storage media, such as a hard drive or any type of solid-state random access semiconductor memory, flash memory, on which modifiable information can be stored.

[0096] The template library can also be a collection of hardware spiking neural network units that are pre-trained to each perform a specific function. These hardware units can be directly used in a spiking neural network that requires a unit with that specific function.

[0097] exist Figure 5A , two data converter stages 201A, 201B may be trained with input pattern signals 11A, 11B in units 301A, 301B, respectively, to generate impulse response outputs 12A1, 12AB. The configurations of units 301A, 301B may be saved and instantiated in units 302A, 302B, each of which also includes a subsequent stage, namely, input encoder stage 202A, 202B. Units 302A, 302B may be trained with input pattern signals 11A, 11B, respectively, which causes outputs 12A1, 12B1 of data converter stages 201A, 201B to become input data of input encoder stages 202A, 202B. The configurations of units 302A, 302B may be similarly saved and instantiated in unit 303, which also includes a subsequent stage, namely, data fusion stage 203. Unit 303 can be trained using both input pattern signals 11A and 11B, which causes the outputs 12A2 and 12B2 of input encoder stages 202A and 202B to become input data for data fusion stage 203. The spatiotemporal pulse trains of the outputs 12A2 and 12B2 of input encoder stages 202A and 202B are then overlapped on one or more neurons 1 of data fusion stage 203. The output 12C of data fusion stage 203 is a temporal pulse train (if data fusion stage 203 includes only one output neuron 1) or a set of spatiotemporal pulse trains (if data fusion stage 203 includes multiple output neurons 1) generated in response to the joint stimulation of multiple pulse trains (outputs 12A2 and 12B2 from input encoder stages 202A, 202B). The resulting set of (multiple) fused pulse trains 12C represents a combination of the feature spaces of all input pattern signals 11A, 11B that have been fused. Therefore, the fused pulse train encodes all features present in the input pattern signals in the current time window.

[0098] For example, when two sets of input pulse trains 12A2, 12B2 containing features e and f of input patterns 11A, 11B, respectively, are fused, the resulting (multiple) fused pulse trains 12C contain corresponding unique representations of both features e and f. The purpose of configuring the data fusion stage 203 is to achieve this unique mapping of the complete feature space of each input pattern signal 11A, 11B to the feature subspace of the fused output signal 12C.

[0099] Consider two input encoders α and β, which have their corresponding input pattern signals M α and M βThe features are encoded in and two sets of pulse trains P are generated α and P β .

[0100] One wants to encode the signal M α The set of u known features in is given by Φ(α)={φ 1 ,φ 2 ,...,φ u}, which is therefore the signal M α For i in {1,...,u}, each feature φ i is encoded as a set of spatiotemporal pulse sequences. α ={A 1 ,A 2 ,...,A u} is the feature φ i Multiple groups of output neurons A are stimulated i The set of multiple sets of exact firing times of all u neurons with known characteristics is B α = {B 1 ,B 2 ,...,B u}, where B α Each B i is used for the feature φ i The precise firing times of the output neurons.

[0101] In order to make data fusion lossless, the feature space of the (multiple) fused pulse trains Φ(γ) can cover the individual feature spaces of different input signals:

[0102] Φ(γ)=Φ(α)·Φ(β)+Φ(α)+Φ(β). (2)

[0103] The spatiotemporal pulse trains corresponding to the feature space Φ(γ) can obey rules 1 and 2 as stated for input encoding, that is, so that they can be sufficiently unique to enable subsequent processing stages to distinguish the pulse encoding features. Therefore, the generation of pulse tuples can use the unique response method described above, where each unique combination of features in the input pulse train produces a unique set of output spatiotemporal pulse trains in the fusion block.

[0104] Figure 5BSchematically illustrates how multiple temporal pulse trains 30A, 30B are fused into a single fused pulse train 35 in a working example. The temporal pulse trains 30A, 30B are each transmitted through synaptic elements 31A, 31B and enter a neuron 34. The neuron 34 may have a recurrent connection 32 connected to the neuron 34 via one or more synaptic elements 33. The fused pulse train 35 represents a combination of the feature spaces of both the first temporal pulse train 30A and the second temporal pulse train 30B. Subsequent processing stages can distinguish the pulse encoding features of the first temporal pulse train 30A and the second temporal pulse train 30B because the representation of these features in the fused pulse train 35 is sufficiently unique.

[0105] In a signal recognition system using data fusion, the importance of data sources can vary based on their impact on the final result. Certain features may be primary to the signal recognition process, while others are only used to enhance the quality of the recognition result. Therefore, A γ (It represents A α and A β The size of the fused feature space of the set of pulse sequences can be reduced in size by downscaling the effects of less important features. This can be done in at least three ways.

[0106] First, by introducing anti-causal spikes (which are accompanied by less important features of a particular input spike train) in the output neurons of the data fusion stage 203. This method is applicable to situations where none of the input patterns contain the desired features.

[0107] Second, less desirable features may also be suppressed by introducing a weight bias in the input synaptic element of the data fusion stage 203, which receives the output from each input encoder stage 202A and 202B corresponding to the priority of each stage in the fusion process. Therefore, the pulse trains from the input encoder stages 202A, 202B may be weighted according to their expected impact on the output of the data fusion stage 203 before being fused. This facilitates controlling the energy transfer between the input signal and the resulting fused representation. Increasing the priority bias of a signal may increase the impact of that signal on the fused representation. Maximizing the bias for a signal may cause the fused representation to replicate the input signal. Conversely, minimizing the priority bias weight for a signal may reduce its impact on the fused representation, while canceling the weight altogether completely eliminates the impact of that signal on the fusion result. During fusion, the spatial pulse distribution A α and A β The transfer function f implemented by the configuration of the data fusion stage 203 is transformed into the fused spatial distribution A γ Thus, in a priority-based embodiment, the fused distribution takes the form

[0108] Aγ=f((Aα ,w α ),(A β ,w β )) and B γ =f((B α ,w α ),(B β ,w β )). (3)

[0109] In formula (3), w α and w β is the set of priority bias weights that controls the pulse training set P α and P β Impact on the data fusion stage 203.

[0110] The third approach, arguably the simplest, involves not suppressing or reducing any feature, but including its influence in the fused representation. In the subsequent classification stage 204, the representation corresponding to that feature can simply be ignored, i.e., assigned to a class of no interest. This approach helps improve accuracy because it explicitly classifies undesirable feature combinations, thereby enabling more accurate identification of desired features.

[0111] Next, the application of the above principles in fusion and pattern recognition systems is described.

[0112] Figure 6 is a simplified block diagram of an embodiment of a system 700 for data fusion and automatic signal recognition based on a spiking neural network 400 interfaced with a microprocessor or hardware logic system 600.

[0113] The spiking neural network 400 includes one or more input encoder stages 401. 1 ,...,401 N ; Input encoder stage 401 1 ,...,401 N Each of the may be configured by configuration data 717. Input encoder stage 401 1 ,...,401 N Can receive input mode signal 711 1 ,...,711 N , and convert these input mode signals into each input mode signal 711 1 ,...,711 N The feature space encodes the spatiotemporal pulse sequence P 1 ,...,P N The group of input mode signals 711 1 ,...,711 NThe method may be used to derive the input pattern data, for example, by sampling the input pattern data to generate the set of signals, for example, by using a CCD array image sensor to record an image to generate a set of input pattern signals 711 1 ,...,711 N , each of which represents image data recorded by one or more pixels of the image sensor.

[0114] Each pulse train P 1 ,...,P N may include a series of precisely timed voltage pulses which are input by the encoder stage 401 1 ,...,401 N One or more neurons respond to each input pattern signal 711 1 ,...,711 N And generated.

[0115] If multiple neurons are used to encode a single input pattern signal 711 1 ,...,711 N , then the output from each of the plurality of neurons may be a pulse train comprising one or more pulses having one or more of the above-mentioned temporal characteristics. 1 ,...,401 N The output neuron generates a cumulative spike train P 1 ,...,P N The input mode signal 711 is represented by 1 ,...,711 N The encoded spatiotemporal impulse response of .

[0116] Fig. 7A 8 shows how neuron 1 interfaces with an input signal 801, such as an output signal from synaptic element 2. Neuron 1 has a certain configuration that can change the way neuron 1 reacts to input signal 1, where the neuron configuration can be changed using configuration signal C N Depending on the input signal 801 and the neuron configuration, neuron 1 will output a certain spatiotemporal pulse train S. Figure 7B In the example, the input signal 802 is from a plurality of synaptic elements 2 A , 2 B Each synaptic element is configured in a certain manner, wherein the configuration of each synaptic element can be determined using a configuration signal C SB , C SB Settings. Synaptic Component 2 A , 2 B It may have as input an input signal 803 or a recurrent input from another neuron 804 .

[0117] Each neuron 1 may have a set of configuration parameters C that control the precise firing behavior of the neuron 1. For example, a neuron may be designed to have a firing threshold that represents a threshold of voltage, energy, or other variable that is accumulated in the neuron as a result of receiving input, and wherein the neuron generates an output pulse (such as a voltage, current, or energy pulse) when the accumulated variable reaches or exceeds the firing threshold. The neuron may implement an integral function that integrates the input to the neuron to determine an adjustment to the accumulated variable. In addition, the neuron may also be designed to have: (a) a leakage rate that represents the rate at which the accumulated variable in the neuron decays over time; (b) a resting value of the accumulated variable that represents the value that the accumulated variable will decay over time in the absence of any input signal to the neuron; (c) an integration time constant that represents the time over which the input signal is integrated to determine any increase or decrease in the accumulated variable in the neuron; (d) an anti-excitation level that represents the value of the accumulated variable in the neuron immediately after the neuron fires; (e) an anti-excitation period that represents the period of time required for the accumulated variable in the neuron to rise to the resting value after the neuron fires. These parameters may be predetermined and / or configurable and / or adjustable for each neuron. For example, by adjusting the neuron's firing threshold, leak rate, integration time constant, and anti-firing period to match the energy content of a key input signal feature, neuron 1 will generate one or more precisely timed pulses when stimulated with an input signal containing that feature.

[0118] Back to Figure 6 , each input encoder stage 401 1 ,...,401 N A plurality of spiking neurons 1 may be included. Input encoder stage 401 1 ,...,401 N The input to can be an analog value or a digital value (e.g., an analog-to-digital conversion of the input pattern data). When using a neuron with a digital integrator, the digital value of each input signal to the neuron is provided as a direct input to the integrator of the neuron. The bit width (B) of the input signal s ) and the bit width of the integrator (B i ) are matched by one or a combination of the following exemplary methods.

[0119] As a first method, when B s >B i When the end of the digital input (B s -B i ) bits are discarded. As a second method, when B i s When using (B s -B i ) to fill the digital input with trailing "0" bits. As a third method, use B n ​Trailing "0" bits to fill the digital input, or B n The tail part is discarded, where B n Can be a configurable parameter that allows the amplitude of the digital input to be shifted within the voltage range of the membrane. This shifting can also be achieved by using a combination of padding and truncating sample values ​​when the input signal and integrator bit width match.

[0120] When using neuron 1 with an analog integrator, a digital value can be provided as a presynaptic input to B. s Synaptic element 2, whose post-synaptic output terminal is connected to the input encoder stage 401 1 ,...,401 N The amplitude shift can be achieved by padding and / or truncating the input signal vector (comprising a sequence of bits) B n bits, as previously described for digital neurons. In another embodiment, the analog value of the input signal can be provided directly as an input to an analog integrator. The analog value can be presented to the integrator over a period of time, such as to cause the integrator to accumulate charge. By selecting an appropriate threshold voltage, the precise firing time of the neuron can be varied inversely with the amplitude of the analog input signal value (i.e., the higher the amplitude, the earlier the neuron fires). In addition, by utilizing multiple neurons 1 with appropriately selected sensitivity thresholds, complex multi-stage analog to pulse converters can be implemented.

[0121] In such an embodiment, the data converter stage 201 may be included. 1 ,...,201 N , which will input mode signal 711 1 ,...,711 N The proposed analog voltage values ​​are converted into unique, equivalent pulse representations, for example, where the distance between the input values ​​and the corresponding distance between the output pulse responses are proportional. Then, the input encoder stage 401 1 ,...,401 N From the data converter stage 201 1 ,...,201 N Features identified in the transcoded impulse response are encoded into a unique representation.

[0122] Input mode signal 711 provided to system 700 1 ,...,711 N may have different dynamic ranges: some low, some high. Although the absolute dynamic range itself may not be important, the ability to distinguish features at different amplitudes may be important. Therefore, it may be necessary to ensure that the input encoder stage 401 1 ,...,401 NThe configuration causes all known features of interest to be encoded as unique pulse representations regardless of the amplitude at which they occur, i.e., the intention may be to obtain specific pulse characteristics at more than one amplitude within the range of interest.

[0123] Therefore, the input encoder stage 401 1 ,...,401 N It can include a group of neurons 1 that connect any number of inputs and produce any number of spike train outputs P 1 ,...,P N , these output pairs are present in the input mode signal 711 1 ,...,711 N The features in are encoded, where the input signal 711 1 ,...,711 N It can be defined as, for example, a single sampled data point of the input pattern signal, or a spatial or temporal or space-time series of sampled data points of the input pattern signal.

[0124] The feature or pattern to be detected in the input pattern signal can be, for example, the specific content of a single sampled input data point or a group of sampled input data points, such as the amplitude of the input signal at a specific point in time, the presence of an "ON" pixel at a certain position in a binary image frame, the intensity of a specific input signal attribute at a specific point in time.

[0125] The features or patterns to be detected may also be in the spatial order of content in a series of sampled data points forming the input signal, such as "ON" pixels corresponding to the edges of objects in a binary image.

[0126] The feature or pattern to be detected may also be in the temporal sequence of the contents across multiple samples of a single data point forming the input signal, or in the temporal sequence of the contents across multiple spatial sequences of sampled data points. For example, a rising sequence of amplitudes followed by a falling sequence of amplitudes characterizes a triangular wave pattern in the input signal, or a displacement of an object edge across multiple frames of a video image forming the input signal corresponds to movement of the object over time.

[0127] Back to Figure 6 In the embodiment shown, the input encoder stage 401 1 ,...,401 N The generated pulse train output P 1 ,...,P N First, they are selectively weighted according to the priority biasing scheme in the priority biasing module 402 as explained above, and then Figure 6The resulting spatiotemporal spike train 712 is then received by a classifier stage 404 that classifies the spatiotemporal spike train 712. The classifier stage 404 can identify the strongest spatiotemporal pattern present in the transcoded representation and associate it with, for example, a specific output neuron.

[0128] The input to the classifier can be, for example, a set of pulse trains P from the input encoder stage 1 ,...,P N , for example, with only one set of input signals 711 1 ,...,711 N , or in the case of a system with multiple sets of input signals 711, a set of pulse trains 712 from the data fusion stage 403 1 ,...,711 N In the case of a system, the multiple sets of input signals are encoded and then fused into a set of representative pulse trains 712. Figure 6 , the classifier stage 404 obtains its input 712 from the data fusion stage 403 .

[0129] The output 713 of the classifier can be a set of classified spatiotemporal pulse trains 713 on output port R, where each pulse train 713 can be generated by a single specific output neuron in the classifier 404, or by multiple output neurons. In a classifier embodiment that conforms to the unique response principle, a pulse can be generated by a separate output neuron for each unique input feature or unique combination of input features in the input pattern signal. In this embodiment, the pulse tuple for each pattern of interest is generated by a single neuron (i.e., A). p is a set of cells of pattern p) that have pulses occurring within a certain time interval and at least one pulse. Therefore, the output port can contain at least as many pulse trains as the patterns of interest.

[0130] To restate this in terms of the weight configuration objective described earlier, consider A to be a generic set of classifier output neurons that generate output spike trains 713 in response to known input patterns, and let A p is a subset of A containing neurons that fire for a given known input pattern p, and for all p in the set of known input patterns P, let A s ={A p |p∈P} is all A p superset of, then for the above situation, the classifier level 404 is: A s Each A in p will be a unit set, and for every q in P where p≠q, then A p ≠A q Established.

[0131] Next, look at Figure 6 , the logic unit 500 including the output transformation stage 501 can transform the output 713 from the classifier stage 404 (e.g., a set of spatiotemporal pulse sequences 713) into a digital code 714 that can be stored and retrieved from, for example, the dictionary 601.

[0132] Thus, the output of the classifier stage 404 may be a set of spatiotemporal pulse trains 713, wherein one or more specific pulse trains 713 exhibit specific characteristics, thereby allowing, for example, the input pattern 711 to be 1 ,...,711 N The output transformation stage 501 can facilitate this identification process by transforming the pulse train characteristics into a digital code 714 that can be stored and retrieved from the dictionary 601 .

[0133] The characteristic information in the pulse train can be encoded in several forms, for example: population code, in which the output neuron 1 of the classifier stage 404 fires within the elapsed time period in which the input stimulus is present; population time code, in which the output neuron 1 of the classifier stage 404 first fires within a specific time period since the input stimulus was present; population rate code, which describes how many pulses the output neuron 1 of the classifier 404 generates within the elapsed time period in which the input stimulus is present; time code, which describes the precise time at which a specific output neuron 1 of the classifier stage 404 fires; and finally rate code, which describes the firing rate of a specific output neuron 1 of the classifier stage 404.

[0134] The combination of spatial and temporal pulse information can provide a greater degree of detail than the output of a purely spatial classifier 404. For example, consider the input pattern signal 711 1 ,...,711 N Contains two dominant features X and Y, one stronger than the other. The relative strengths of the two features can be determined by measuring the firing rates of their corresponding classification level outputs 713. Alternatively, the stronger feature can be identified based on the temporal encoding of the corresponding output neurons 1, specifically, by determining which output neuron 1 fired earlier.

[0135] The output transformation stage 501 may contain circuitry that analyzes the spiking activity on the classifier output port. For example, the circuitry is activated according to a user configuration 717, based on the type of analysis the user wishes to perform on the spiking behavior of the neurons 1 of the array: spiking populations, precise spiking timing, firing rates, and combinations thereof.

[0136] For the coding type population, the transformation method consists of the following: each output neuron of the classifier stage 404 has a register to indicate whether the neuron has pulsed in the current time interval after the input pattern signal is applied. For the coding type population temporality, the transformation method consists of the following: each output neuron of the classifier stage 404 has a register to indicate which neurons have pulsed first in the current time interval after the input pattern signal is applied. After the earliest pulse event is registered, the other registers are disabled. For the coding type population rate, the transformation method consists of the following: each output neuron of the classifier stage 404 has a counter to count the number of pulses generated by the output neuron in the current time interval after the input pattern signal is applied. A counter with a non-zero pulse count value also indicates a population code. For the coding type time, the transformation method consists of the following: each output neuron of the classifier stage 404 has a time-to-digital converter to accurately measure the time elapsed from the application of the input pattern signal to the output neuron pulse. Finally, for the rate coding type, the transformation method consists of the following: Each output neuron of the classifier stage 404 has a counter to count the number of pulses generated by the output neuron in the current time interval after the input pattern signal is applied.

[0137] The input to the output transformation stage 501 may be a vector 713 of pulses from all output neurons 1 of the classifier stage 404, and the output 714 from the circuit 501 may be a set of digital values ​​corresponding to the output code type selected by the user. The input 713 to the output transformation stage 501 may be present for the duration of a time window in which the signal sample is valid. The output 714 from the output transformation stage 501 may be valid at the end of the time window, and may be reset before the next time window begins. The classification output 713 may be transformed into a single hash code, or a set of hash codes, which may be used as the input mode signal(s) 711. 1 ,...,711 N The unique fingerprint of the pattern in .

[0138] Next, look at Figure 6 , a microprocessor or logic circuit 600 including a hash table or dictionary lookup function 601 takes the digital code 714 and may then perform a hash table or dictionary lookup operation to determine a user space identifier corresponding to the identifying feature.

[0139] This operation allows the output of the classifier stage 404 to be decoded into a user space, i.e., user defined, input pattern identifier. During the configuration phase, the pattern recognizer 400 may be trained with sample features and patterns, and the generated hash codes may be recorded. The sample patterns may be assigned a user defined identifier, which may be stored, for example, in a hash table or dictionary 601 along with the generated hash codes. At runtime, when the recognizer 400 is used to recognize features in a signal, the generated hash codes are used in a hash table or dictionary lookup operation to determine the user space identifier corresponding to the recognized feature.

[0140] Figure 6 The reset generator and sequencer 718 shown can reset the contents of, for example, the data fusion stage 403, the classifier stage 404, and the output transformation stage 501, for example, at the beginning of a new sampling window. This operation removes residual pulses from the network 400 and allows the next sampling window to be evaluated without being affected by the previous window.

[0141] The sampling window can be implemented as an input buffer with a depth equal to the window size. Alternatively, a configurable virtual sampling window can be implemented by using a periodic reset signal that stops the pulsing behavior in the network and resets the output stage. The sampling window in this approach can have the same period as the reset signal. When configuring the sampling window, the period should be long enough to fully accommodate the longest known feature, but small enough to minimize the feature content of each window, thereby improving the accuracy of feature identification. The width of the sampling window may affect the granularity of features that can be detected by the system 700. In general, it can be said that the sampling window duration must be greater than the time scale on which the features exist in the input signal, where the sampling rate is greater than or equal to the Nyquist rate. Features at different time scales can be identified by using multiple input encoder stages 401. 1 ,...,401 N , or a single input encoder stage 401 1 ,...,401 N The length of the sampling window may also have an impact on the width of the counter used in the output conversion stage 501.

[0142] Figure 8 An example of generating a reset pulse is shown. The clock information 904 modifies the value of the up-counter 901. Together with the information about the sampling period 905, the value of the up-counter 901 is submitted to the comparator 902. The comparator 902 checks whether the value of the up-counter 901 is still within the sampling period 905. If this is no longer the case, the comparator 902 sends a signal to the pulse generator 903, which sends a reset signal 906 as Figure 6 The signal 716 in is sent to both the up-counter 901 and the corresponding module within the system 700.

[0143] Similarly, for the reset signal, the period between two consecutive reset pulses may follow the following formula, for example:

[0144] t reset ≥max(t feature )+max(t fallthrough )+t output +max(t lookup ). (4)

[0145] In formula (4), t feature is the duration of the feature, t fallthrough is the fall delay of the spike train propagating from the input encoder stage to the output of the classifier stage, t output is the delay of the output transform stage to generate a hash or a set of hashes, and t lookup is the worst-case latency of a hash table / dictionary lookup operation.

[0146] The duration of the reset pulse must be sufficient to trigger the reset circuitry of the appropriate components. During reset, the states of all neurons 1 in the network 400 are reset, i.e., for example, membrane voltages are brought back to a quiescent state, interface signals are reset to their initialization states (i.e., their states after the system was programmed), synaptic readout circuitry is reset (weights are preserved), and inflight synaptic updates are abandoned. In the output stage 500, counters and registers that track populations and firing rates in neurons 1 may also be reset. The reset does not affect the input encoder stage 401. 1 ,...,401 N , thereby ensuring that the input mode signal 711 1 ,...,711 N Continuous encoding is pulsed, even during the transition between sampling windows.

[0147] Fig. 9 A method 1000 is shown that fuses a plurality of different input pattern signals into a single set of representative pulse trains and then performs automatic signal recognition on the pulse trains to detect and identify features / characteristics.The method 1000 begins at a starting point 1000S.

[0148] In a first step 1001, new signal samples are input into one or more encoder stages 401. 1 ,...,401 N The output of one or more output encoders is a space-time pulse sequence P 1 ,...,P N .

[0149] In a second step 1002, the pulse sequence P 1,...,P N The priority biases are weighted according to their importance in the fusion. The output of the priority bias module 402 is a weighted temporally distinct pulse train.

[0150] In the third step 1003, the different spatiotemporal pulse sequences are unified into a set of representative pulse sequences 712 covering each characteristic subspace by means of the data fusion stage 403. The output of the data fusion stage is the fused spatiotemporal pulse sequence 712.

[0151] In a fourth step 1004, the classifier stage 404 classifies the spatial, temporal and rate features in the fused pulse train 712. Thus, the temporal pulse train is classified.

[0152] In a fifth step 1005 , the output transformation stage 501 converts the classified pulse sequence 713 into a classification code 714 .

[0153] In a sixth step 1006, the method may wait until a new sampling window begins.

[0154] In a seventh step, a lookup and output of the pattern identifier 700 corresponding to the classification code 714 is performed in the hash table or dictionary 601 .

[0155] Finally, in an eighth step 1008, the data fusion stage 403, the classifier stage 404 and the output transformation stage 501 are reset.

[0156] The method can then be repeated starting again at the first step 1001 .

[0157] In general, the systems and methods described in the present invention use a network of spiking neurons 1 as a means of generating unique spatiotemporal spike trains in response to unique stimuli, where the uniqueness of the response can be controlled by the operating parameters of the spiking neurons 1 and the interconnected network. The system is composed of an array of spiking neurons 1, which have configurable parameters, interconnected by synaptic elements 2 with configurable parameters. By virtually partitioning the array into multiple different networks, the system can simultaneously perform multiple functions.

[0158] For example, a network can be implemented to convert multiple different sampled input signals into a set of representative spatiotemporal spike trains that fully encode the feature space of the input signals.

[0159] In a second configuration, the present invention may enable conversion of a plurality of different sampled input signals into a single set of representative spatiotemporal pulse trains having a feature space that is a superset of the individual feature spaces of all input signals.

[0160] In a third configuration, the present invention can achieve classification of spatiotemporal pulse trains into one or more output categories by appropriately configuring the uniqueness of the network.

[0161] The configurability of the neurons, synaptic elements and interconnection networks of the present invention facilitates the simultaneous realization of all three configurations in different combinations on separate neuron populations in the same array.The synaptic elements act as a configurable switch matrix.

[0162] According to the embodiment, the pulse neural network 1100 Fig.10 , where neurons N0, . . . , N7 are arranged in a neuron array 1101 , interconnected by a configurable switch matrix consisting of an array 1102 of synaptic elements 2 configurable via configurable switches 1103 .

[0163] According to the weights of the synaptic elements 2, the input signal 1104 can be transmitted to the neurons N0, ..., N7 via the synaptic element array 1102. In addition, according to the weights of the synaptic elements 2 and the connection topology of the configurable switches 1103, two neurons N0, ..., N7 can be interconnected via the synaptic element array 1102 and the configurable switches 1103. Positive weights in the synaptic elements 2 connecting the input signal 1104 to the neurons N0, ..., N7, or interconnecting a pair of neurons N0, ..., N7, cause excitation of the postsynaptic neurons N0, ..., N7. Negative weights in these connecting synaptic elements 2 cause inhibition of the postsynaptic neurons N0, ..., N7, and zero weights in these connecting synaptic elements 2 cause no connection to the postsynaptic neurons N0, ..., N7.

[0164] Configuration parameters of neurons N0, ..., N7 and synaptic element 2, interconnection topology and output stage configuration parameters may be loaded from the configuration memory each time the spiking neural network 1100 is set to an initialized state. This operation causes neurons N0, ..., N7 to be interconnected according to the loaded interconnection topology and synaptic weights, neuron and synaptic element parameters to be initialized according to the associated loaded configuration parameters, system parameters to be initialized according to the associated loaded configuration parameters, and the output dictionary or hash table to be populated with entries stored in the configuration memory.

[0165] The input signal 1104 is submitted to a subset of synaptic elements 2, and the network 1100 produces some output 1105 that can be sent to an output decoder.

[0166] The spiking neural network 1100 can be Fig.11 As shown in the implementation, the Fig.11 An example of a configured synapse array 1202 is shown, illustrating an example mapping of the described neural network levels to neurons N0, ..., N7 and synaptic elements 2.

[0167] Input encoder stages 1211A, 1211B, data fusion stage 1212, and classifier stage 1213 may be implemented as shown in the spiking neural network 1200. Output transformations may be implemented using digital logic circuits, and hash table / dictionary lookup operations may be implemented as software code executed on a microprocessor accompanying the spiking neural network in one embodiment, and in another embodiment, using digital logic circuits.

[0168] The basic principle of each of the network stages 1211A, 1211B, 1212, 1213 is the same, and each stage 1211A, 1211B, 1212, 1213 generates a unique response when subjected to an input stimulus according to its configuration parameters. The goal of the input encoder stage 1211A, 1211B can be to generate a unique response for each unique feature present in the input pattern signal 1204A, 1204B. The goal of the data fusion stage 1212 can be to generate a unique response for each unique combination of input pulse trains from the input encoder stage 1211A, 1211B. The goal of the classifier stage 1213 can be to generate a unique classification response for each unique feature or feature combination.

[0169] One or more embodiments may be implemented as a computer program product for use with a computer system. The program(s) of the program product may define the functionality of the embodiments (including the methods described herein) and may be contained on various computer-readable storage media. Computer-readable storage media may be non-transitory storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., a read-only memory device within a computer, such as a CD-ROM disc readable by a CD-ROM drive, a ROM chip, or any type of solid-state non-volatile semiconductor memory), on which information may be permanently stored; and (ii) writable storage media, such as a hard drive or any type of solid-state random access semiconductor memory, flash memory, on which modifiable information may be stored.

[0170] Two or more of the above embodiments may be combined in any suitable manner.

Claims

1. A spiking neural network for classifying an input pattern signal, comprising a plurality of spiking neurons implemented in hardware or a combination of hardware and software, and a plurality of synaptic elements interconnecting the spiking neurons to form the network, wherein the spiking neurons and the synaptic elements are both implemented using configurable analog circuit elements and / or digital hard-wired logic circuits; wherein each synaptic element is adapted to receive a synaptic input signal and apply a weight to the synaptic input signal to generate a synaptic output signal, the synaptic element being configured to adjust the weight applied by each synaptic element, and wherein each of the spiking neurons is adapted to receive one or more of the synaptic output signals from one or more of the synaptic elements and to generate a spatiotemporal pulse train output signal in response to the received one or more synaptic output signals, wherein the spiking neural network comprises a first subnetwork comprising a first subset of the spiking neurons, the first subset of the spiking neurons being connected to receive synaptic output signals from the first subset of the synaptic elements, wherein the first sub-network is adapted to generate a sub-network output pattern signal from a first subset of the spiking neurons in response to a sub-network input pattern signal applied to a first subset of the synaptic elements, in, configuring the weights of the first subset of the synaptic elements by training the subnetwork on a training set of the subnetwork input pattern signals so that the subnetwork output pattern signal is unique for each unique subnetwork input pattern signal of the training set, wherein the degree of uniqueness can be controlled by operating parameters of the first subset of spiking neurons and the synaptic elements; wherein the network comprises a second sub-network comprising a second subset of the spiking neurons, the second subset of the spiking neurons being connected to receive synaptic outputs from a second subset of the synaptic elements, wherein the second sub-network is adapted to receive a second sub-network input pattern signal applied to a second subset of the synaptic elements and to generate a corresponding second sub-network output pattern signal from a second subset of the neurons, and wherein the configuration of the second subset of synaptic elements is adjusted so that the second sub-network output pattern signal is unique for each unique feature in the second sub-network input pattern signal, wherein the network comprises a third sub-network comprising a third subset of the spiking neurons, the third subset of the spiking neurons being connected to receive synaptic outputs from a third subset of the synaptic elements, wherein the first sub-network output mode signal and the second sub-network output mode signal are input mode signals of the third sub-network, and wherein the configuration of the third subset of the synaptic elements is adjusted so that the third sub-network output pattern signal is unique for each unique feature in the input pattern signals from both the first sub-network and the second sub-network and their unique combination, so that the features present in the input pattern signals from both the first sub-network and the second sub-network are encoded by the third sub-network.

2. A spiking neural network according to claim 1, wherein the corresponding distance between each unique sub-network output pattern signal is greater than a predetermined threshold, and the distance is measured by an output pattern metric.

3. A spiking neural network according to claim 1, wherein the spiking neurons and synaptic elements are configured so that: the corresponding distance between two output pattern signals of the sub-network measured by the output pattern metric is maximized for all sub-network input pattern signals of the training set, and the two output pattern signals of the sub-network are generated in response to two corresponding different sub-network input pattern signals.

4. A spiking neural network according to claim 3, wherein each corresponding distance is maximized until the output pattern signal at least meets a first minimum sensitivity threshold required to distinguish the features of the input pattern signal.

5. The spiking neural network of claim 1 , wherein the weights of the first subset of synaptic elements are configured by training the subnetwork on a training set of subnetwork input pattern signals using a semi-supervised training method, the subnetwork being configured to respond with the desired subnetwork output pattern signal by inducing spikes at desired output neurons at predetermined times and at other neurons at times accompanying application of the subnetwork input pattern signal.

6. A spiking neural network according to claim 5, wherein desired spikes are artificially induced at predetermined times to obtain a unique response to a certain input pattern.

7. A spiking neural network according to claim 5 or 6, wherein by artificially inducing a pulse at a neuron n of the sub-network at a desired time, the nature of the relationship between the neurons in the previous layer of the sub-network and the neuron n is established as causal, anti-causal or non-causal.

8. A spiking neural network according to claim 1, wherein the weights of the first subset of synaptic elements are configured using a Causal Chain Spike Timing Dependent Plasticity (CC-STDP) learning rule, which enables the identification of a causal relationship between a desired output neuron and a neuron in a previous layer of the subnetwork and causes the weights of intervening synaptic elements along the causal path to be adjusted.

9. A spiking neural network according to claim 8, wherein using the CC-STDP learning rule, the sub-network output pattern signal can be directed toward different populations of output neurons of the sub-network, and the different populations of output neurons of the sub-network are fired in response to predetermined sub-network input pattern signals and their precise firing time so as to achieve the predetermined sub-network output pattern signal.

10. A spiking neural network according to claim 8 or 9, wherein the CC-STDP learning rule adjusts the weights of the first subset of synaptic elements based on a firing event only if the firing event contributes to the firing of neurons in a subsequent layer of the subnetwork.

11. The spiking neural network of claim 8, wherein the training using the CC-STDP learning rule comprises inducing and / or inhibiting spike generation at neurons of the first subset of the spiking neurons at predetermined times.

12. A spiking neural network according to claim 11, wherein inducing pulse generation in a neuron comprises driving the membrane of the neuron to a voltage exceeding the excitation threshold of the membrane by means of a bias voltage input, thereby inducing the pulse generation, or injecting an artificial pulse into the output of the neuron at the predetermined time.

13. A spiking neural network according to claim 11 or 12, wherein inhibiting pulse generation in a neuron comprises driving the membrane of the neuron to its anti-excitation voltage or lowest possible voltage, thereby preventing pulse generation, or inhibiting pulse generation at the neuron.

14. The spiking neural network according to claim 8, wherein the causal chain STDP learning rule changes the weight w of the synaptic element located between neuron i and neuron j according to the following rule: i,j : Where Δw i,j is the effective weight change of the synaptic element between neurons i and j; w j,k is the weight of the synaptic element between neuron j and neuron k of the subsequent layer; t spikei ,t spikej and t spikek are the basic parameters of the second learning rule for neurons i, j, and k respectively; and δw i,j is the weight change of the synaptic element between neurons i and j due to their relative spike timing and is calculated using the second learning rule.

15. The spiking neural network of claim 14, wherein the second learning rule is the STDP learning rule, and t spikei ,t spikej and t spikek It is the precise time when a spike occurs in neurons i, j, and k respectively.

16. The spiking neural network of claim 1, wherein each of the spiking neurons is configured to adjust the response of the neuron to the received one or more synaptic output signals.

17. A spiking neural network according to claim 1, wherein spikes are generated by the spiking neurons at one or more firing times, and wherein the subset of synaptic elements and / or the subset of neurons are configured so that: the union of two sets of firing times of the subset of neurons that are fired for two unique sub-network input pattern signals is minimized for all sub-network input pattern signals of the training set.

18. The spiking neural network of claim 17, wherein each union is minimized until the output pattern signal at least satisfies a second minimum sensitivity threshold required to distinguish features of the input pattern signal.

19. The spiking neural network of claim 1, wherein the synaptic elements of the third sub-network are configured such that: the input pattern signals from the first sub-network and the second sub-network are weighted according to the importance of the unique features in the input pattern signal.

20. The spiking neural network of claim 1, wherein the network comprises a plurality of sub-networks of synaptic elements and spiking neurons, for which the sub-network output pattern signals are unique for each unique feature in the sub-network input pattern signals, wherein the network can be divided into a plurality of layers having a predetermined sequential order in the network, and wherein the plurality of sub-networks are instantiated in the predetermined sequential order of the plurality of layers to which each respective sub-network belongs.

21. The spiking neural network of claim 1, wherein the synaptic elements are arranged as a configurable switch matrix.

22. The spiking neural network according to claim 1, wherein configuration information of the neurons, the synaptic elements, the interconnection topology of the neurons and the synaptic elements and / or the output configuration of the neural network is stored on a configuration memory, and Wherein when the neural network is brought to an initialization state, the configuration information is loaded from the configuration memory.

23. The spiking neural network according to claim 1, wherein a group of neurons are arranged in one or more neuron template networks, the neurons of a predetermined neuron template network forming a sub-network of the neural network, wherein the sub-network output pattern signal is unique for each unique feature in the sub-network input pattern signal, and Each template network in the neural network is instantiated as a pre-trained sub-network configured according to claim 17.

24. The spiking neural network of claim 1, wherein the neural network is configured to take as input one or more sampled analog or digital input signals and convert the input signals into a set of representative neural network input pattern signals.

25. The spiking neural network of claim 1, wherein output pattern signals of the neural network are classified into one or more output categories.

26. A method for classifying an input pattern signal using a spiking neural network, the spiking neural network comprising a plurality of spiking neurons implemented in hardware or a combination of hardware and software, and a plurality of synaptic elements interconnecting the spiking neurons to form the network, wherein the spiking neurons and synaptic elements are each implemented using configurable analog circuit elements and / or digital hard-wired logic circuits; wherein each of the synaptic elements is adapted to receive a synaptic input signal and apply a weight to the synaptic input signal to generate a synaptic output signal, the synaptic element being configured to adjust the weight applied by each synaptic element, and wherein each of the spiking neurons is adapted to receive one or more of the synaptic output signals from one or more of the synaptic elements and to generate a spatiotemporal pulse train output signal in response to the received one or more synaptic output signals, The method comprises: defining a first subnetwork of the spiking neural network, the first subnetwork comprising a first subset of the spiking neurons, the first subset of the spiking neurons being connected to receive synaptic output signals from a first subset of the synaptic elements; configuring the weights of the first subset of synaptic elements by training the subnetwork on a training set of subnetwork input pattern signals such that the subnetwork output pattern signal is unique for each unique subnetwork input pattern signal of the training set, wherein the degree of uniqueness is controlled by operating parameters of the first subset of spiking neurons and the synaptic elements; applying a sub-network input pattern signal to a first sub-network of the spiking neural network, the first sub-network comprising a first subset of the spiking neurons connected to receive synaptic output signals from a first subset of the synaptic elements; and receiving a sub-network output pattern signal generated by the first subset of spiking neurons in response to the sub-network input pattern signal, wherein the output pattern signal identifies one or more features of the input pattern signal; defining a second sub-network comprising a second subset of the spiking neurons, the second subset of the spiking neurons connected to receive synaptic outputs from a second subset of the synaptic elements; adapting the second sub-network to receive second sub-network input pattern signals applied to a second subset of the synaptic elements and to generate corresponding second sub-network output pattern signals from a second subset of the neurons; adjusting the configuration of the second subset of synaptic elements so that the second sub-network output pattern signal is unique for each unique feature in the second sub-network input pattern signal; defining a third sub-network comprising a third subset of the spiking neurons, the third subset of the spiking neurons connected to receive synaptic outputs from a third subset of the synaptic elements; The configuration of the third subset of the synaptic elements is adjusted so that the third sub-network output pattern signal is unique for each unique feature and unique combination of the input pattern signals from both the first sub-network and the second sub-network, so that the features present in the input pattern signals from both the first sub-network and the second sub-network are encoded by the third sub-network.

27. A method for classifying a predetermined feature in an input signal, the method include: Constructing and training a neural network according to claim 26; submitting at least one sampled input signal to the neural network; as well as The features in at least one sampled input signal are classified into one or more output classes of the neural network.

28. A template library comprising information about the configuration of one or more template networks of spiking neurons used as subnetworks of a spiking neural network, wherein each template network comprises a set of said spiking neurons, the set of said spiking neurons being implemented in hardware or a combination of hardware and software, connected to receive synaptic outputs from a set of synaptic elements, wherein said spiking neurons and synaptic elements are both implemented using configurable analog circuit elements and / or digital hard-wired logic circuits; and wherein each template network is adapted to receive a template network input pattern signal applied to the set of synaptic elements and to generate a corresponding template network output pattern signal from the set of neurons, and wherein the configuration of the set of synaptic elements is adjusted by training the template network on a training set of template network input pattern signals such that the template network output pattern signal is unique for each unique template network input pattern signal of the training set, wherein the degree of uniqueness is controlled by operating parameters of the first subset of spiking neurons and the synaptic elements, The training set is used to train the template network to perform a predetermined task. enabling the sub-network of the spiking neural network to be instantiated in a pre-trained manner based on the information about the configuration of the template network to perform the predetermined task, wherein the network comprises a second sub-network comprising a second subset of the spiking neurons, the second subset of the spiking neurons being connected to receive synaptic outputs from a second subset of the synaptic elements, wherein the second sub-network is adapted to receive a second sub-network input pattern signal applied to a second subset of the synaptic elements and to generate a corresponding second sub-network output pattern signal from a second subset of the neurons, and wherein the configuration of the second subset of synaptic elements is adjusted so that the second sub-network output pattern signal is unique for each unique feature in the second sub-network input pattern signal, wherein the network comprises a third sub-network comprising a third subset of the spiking neurons, the third subset of the spiking neurons being connected to receive synaptic outputs from a third subset of the synaptic elements, wherein the first sub-network output mode signal and the second sub-network output mode signal are input mode signals of the third sub-network, and wherein the configuration of the third subset of the synaptic elements is adjusted so that the third sub-network output pattern signal is unique for each unique feature in the input pattern signals from both the first sub-network and the second sub-network and their unique combination, so that the features present in the input pattern signals from both the first sub-network and the second sub-network are encoded by the third sub-network.

29. A method for forming a spiking neural network, the method include: obtaining one or more template networks or information about the configuration of one or more template networks from a template library according to claim 28, and The one or more template networks are instantiated as sub-networks of a spiking neural network, so that the sub-networks of the spiking neural network can perform the predetermined tasks that the template networks are pre-trained to perform.

Images