Discovering and exploiting informative loop signals in a pulsed neural network with temporal encoders

Abstract

Computer network with paths, where the network includes: a plurality of units configured to communicate via the paths, wherein the network is configured to identify informative loop signals in loops formed from a plurality of network paths connecting a first of the plurality of units to a second of the plurality of units, wherein the network is further configured to apply spike timing dependent plasticity (STDP)-dependent inhibitory gating to the plurality of network paths, wherein the network is further configured to open and close gates in the loop in a phase-shifted manner by applying STDP functions to open gate outputs and closed gate outputs, and wherein the network is further configured to make at least a rate or a direction of the phase shift dependent on a regulation signal, wherein the regulatory signal is based at least in part on a change in STDP-dependent inhibitory gating.

Description

background

[0001] The present invention generally relates to the use of computer systems for modeling biological neural systems. More specifically, the present invention relates to the modeling of biological neural systems using computer systems that implement a pulsed neural network having temporal encoders and informative loop signals.

[0002] An animal's perception and behavioral response to dynamic sensory stimuli are continuously evolving and are ultimately computed from information contained in relatively short segments of so-called neuron-generated spike trains. These spike trains, also known as pulses or action potentials, are voltage spikes that can travel along a nerve fiber. A typical neuron-generated spike train contains spatiotemporal patterns that include localized events with a distinct spatial and temporal structure. Although these events occur with great variance across spatial and temporal scales, a biological neural system, such as the brain, can integrate them and extract the relevant pieces of information.Therefore, biological neural systems impose order on the information provided by noisy spatiotemporal sensory inputs.

[0003] The neuron is a central element of biological neural systems. The majority of the brain consists of structural cells known as glial cells and astrocytes. Between these structural cells lie neurons, which conduct electrical impulses along fibers. It is estimated that the average human brain contains about 10 billion neurons, and on average, each neuron is connected to approximately 1,000 other neurons. These neurons and their connections form vast and complex biological neural networks that are the core of the brain's processing power. With their ability to rapidly transmit signals such as spike trains or action potentials over long distances, neurons are remarkable among the body's cells.So-called sensory neurons change their activity by firing sequences of action potentials in different temporal patterns in response to the presence of external stimuli, such as light, sound, taste, smell, and touch. Information about a stimulus is encoded in these patterns of action potentials and transmitted in and around the brain.

[0004] In biological neuronal systems, the point of contact between the action of a first neuron and the dendrite of another neuron is called a synapse, and with respect to the synapse, the two neurons are called presynaptic and postsynaptic, respectively. Neurons mimic spikes when they are activated by sufficient input received across a synapse, with the spikes being delivered to those synapses to which the neuron is presynaptic. Neurons can be either "excitatory" or "inhibitory." Synaptic conductance is a measure of the amount of influence a synapse has on its postsynaptic target when the synapse has been activated by a presynaptic spike. Synaptic conductance can change over time as a function of the relative spike timing of presynaptic and postsynaptic neurons (spike-timing dependent plasticity (STDP)).According to the standard STDP rule, the conductance efficiency of a synapse increases when its postsynaptic neuron fires after its presynaptic neuron, and the conductance efficiency decreases if the firing order is reversed. The mental possession of a person's individual experiences is stored in the conductance of signals across the trillions of synapses within the brain.

[0005] The computations underlying all aspects of the operation of a biological neural system are performed within the context of a neural code, which represents the relevant information in the spike trains. Determining an information coding scheme used within a biological neural system is useful not only because of intrinsic interest in the nature of the neural code itself, but also because the conditions for knowledge of the code can be used to model the mechanism underlying a biological neural computation.

[0006] Evolution has endowed the human brain with many desirable characteristics not present in contemporary computing systems, including, but not limited to, massive parallelism, distributed representation and computation, learning capacity, generalization capacity, adaptability, inherent contextual information processing, fault tolerance, and low energy consumption. Contemporary digital computers surpass humans in areas of numerical computation and related symbol manipulation. Yet, humans can effortlessly solve complex perceptual problems (recognizing a familiar person in a crowd by a fleeting glance at the person's face) at a speed and to a degree that dwarfs the fastest computers.One reason for such a remarkable difference in the performance of biological neural systems and computers is that biological neural system architecture is completely different from typical computer system architecture. This difference significantly influences the types of functions each computational model can best perform.

[0007] Artificial neural networks (ANNs), also known as neuromorphic and synaptonic systems, are computational systems that essentially allow electronic systems to function analogously to biological neural systems, such as the human brain. In the present invention, any reference to an ANN is used in the broadest possible sense to cover the widest range of computers and electronic systems that model biological neural networks, including, but not limited to, pulsed neural networks (PNNs). ANNs generally do not use the traditional digital model of manipulating ones and zeros. Instead, a typical ANN creates connections between processing elements that function similarly to neurons in a biological brain.

[0008] ANNs are often implemented as relatively large, parallel processing systems comprising many simple processors with many connections. Modeling biological nervous systems using ANNs (or PNNs) can contribute to a better understanding of biological functions. Many ANN models attempt to use "organizing" principles believed to be used by the human brain. Contemporary computer hardware technology (such as VLSI or optical) has made such modeling possible. ANNs draw on knowledge from various disciplines, including neurophysiology, cognitive science / psychology, physics (statistical mechanics), control theory, computer science, artificial intelligence, statistics / mathematics, pattern recognition, computer vision, parallel processing, and hardware (e.g., digital / analog / VLSI / optical).

[0009] A contemporary ANN comprises a network of artificial neurons (also known as "nodes"). These nodes are interconnected, and the strength of their connection is assigned a value. For example, connections can be identified as inhibitory (maximum inhibition is -1.0) or excitatory (maximum excitation is +1.0). A high connection value indicates a strong connection. A transfer function is built into the design of each node. In a typical ANN, there are three types of neurons: input nodes, hidden nodes, and output nodes.

[0010] Input nodes receive information that can be expressed numerically. This information is represented as an activation value, with each node assigned a number, and a node with a higher number causing greater activation. This information is then passed through the network. Based on the connection strength or effectiveness (weights), inhibition or excitation, and the transfer function, the activation value is passed from one node to another. Each node sums the activation values ​​it receives. Each node then modifies the value based on its transfer function. The activation flows through the network, through hidden layers, until it reaches the output nodes. The output nodes reflect the input signal (input for short) to the outside world in a meaningful way.

[0011] There are many types of neural networks, but the two dominant categories are feedforward and feedback networks. A feedforward network is a non-repeating network with inputs, outputs, and hidden layers. Signals can only travel in one direction. Input data is passed to a layer of processing elements that perform computations. Each processing element performs its calculations based on a weighted sum of its inputs. The newly computed values ​​then become new inputs for the next layer. This process continues until it has passed through all layers and the output value is determined. Sometimes a threshold transfer function is used to quantify the output of a neuron in the output layer.

[0012] A feedback network includes feedback paths, meaning signals can travel in two directions using loops. All possible connections between neurons are allowed. Because loops exist in this type of network, under certain circumstances these networks can become a nonlinear dynamic system, continuously changing until they reach an equilibrium state. Feedback networks are often used for associative memory and optimization problems, where the network searches for the best arrangement of the connected factors.

[0013] In an ANN, a spike-generating function can be modeled by a component known as a temporal encoder or spike generator. Therefore, the ability to gain greater knowledge about the previously described biological neural systems depends on the development of ANNs that model how neural information is encoded in repetitive networks of spiking temporal encoders. Temporal encoders provide an alternative to digital and analog encoding of information by asynchronously integrating received signals and producing subsequent sets of asynchronous pulsed signals. Certain configurations of temporal encoder networks allow the extraction of information about sets of signals input to the network from sets of pulses, or spikes, output from the network.

[0014] The scientific publication "Configurable neural phase shifter with spike-timing-dependent plasticity" by L. Zhang et al., published in IEEE Electron Device Letters, 2010, Vol. 31, No. 7, pp. 716-718, describes a phase shifter designed for use in neuromorphic circuits. It can be tuned to a phase change in the range of 9.0 to 24.5 ms using a temporally correlated pair of presynaptic and postsynaptic pulses and exhibits STDP behavior. The design of the phase shifter is briefly explained, and experimental observations of its properties are described. Furthermore, a physical rationale for the observed behavior of the phase shifter is provided, and a comparison is made to previous devices with similar functions based on memristors and floating-gate transistors.

[0015] The scientific publication "Delay-and coupling-induced firing patterns in oscillatory neural loops" by O. Popovych et al., published in Physical review letters, 2011, Vol. 107, No. 22, p. 228-102, describes a feedforward loop of oscillatory Hodgkin-Huxley neurons interacting via excitatory chemical synapses. It shows that carefully chosen communication delays and synaptic weights can encode a variety of spatiotemporal periodic firing patterns, thus contributing to the concept of temporal coding by spikes. These patterns can be achieved by modulating the multiple, simultaneously existing stable, phase-synchronous states or by traveling waves propagating along or against the coupling direction. Explicit conditions for the network parameters are derived to achieve a desired pattern.It is observed that the delays directly influence the time differences between the spikes of interacting neurons, while the synaptic weights control the phase differences. Summary

[0016] Embodiments are directed to a computer network having paths. The network includes a plurality of units configured to communicate over the paths. The network is configured to identify informative loop signals in loops formed from a plurality of network paths connecting a first plurality of the units to a second plurality of the units. The network is further configured to apply spike timing dependent plasticity (STDP)-dependent inhibitory gating to the plurality of network paths. The network is further configured to open and close gates in the loop out of phase by applying STDP functions to open gate outputs and closed gate outputs.The network is further configured to make at least a rate or a direction of the phase shift dependent on a regulatory signal, wherein the regulatory signal is based at least in part on a change in STDP-dependent inhibitory gating.

[0017] Embodiments are further directed to a computer-implemented method for implementing a network with paths. The method includes communicating between a plurality of units through the paths. The method further includes identifying informative loop signals in loops formed from a plurality of network paths connecting a first of the plurality of units to a second plurality of units. The method further includes applying spike-timing-dependent plasticity (STDP)-dependent inhibitory gating to the plurality of network paths. The method further includes phase-shifting opening of gates and closing of gates in the loop by applying STDP functions to open gate outputs and closed gate outputs.The method further comprises making at least one of a rate and a direction of the phase shift dependent on a regulatory signal, wherein the regulatory signal is based at least in part on a change in STDP-dependent inhibitory gating.

[0018] Embodiments are further directed to a computer program product for implementing a network with paths. The computer program product comprises a computer-readable storage medium having program instructions stored thereon, wherein the computer-readable storage medium is not a transmitting signal per se. The program instructions are readable by a processor circuit and cause the processor circuit to execute a method. The method comprises communicating between a plurality of units via the paths. The method further comprises identifying informative loop signals in loops formed by a plurality of network paths connecting a first unit of a plurality of units to a second unit of the plurality of units. The method further comprises applying spike-timing-dependent plasticity (STDP)-dependent inhibitory gating to the plurality of network paths.The method further comprises phase-shifting the opening and closing of gates in the loop by applying STDP functions to open gate outputs and closed gate outputs. The method further comprises making at least one of a rate and a direction of the phase shift dependent on the regulation signal, wherein the regulation signal is based at least in part on a change in the STDP-inhibiting gating.

[0019] Embodiments are further directed to a computer-implemented method for processing loop signals in a computer network having paths. The method includes identifying, by a processor component of the network, informative loop signals of the computer network, wherein the informative loop signals comprise spikes with characteristics, and wherein the characteristics of the spikes comprise states, an order in which the spikes cycle through the states, and the usability of spikes and spike patterns to cause a change of states through which the spikes cycle sequentially. The method further includes a network that uses informative loop signals to control a forward driver gate to either open or close a forward path.

[0020] Embodiments are further directed to a method for computer networks with paths. The method comprises providing input data to an information-maximizing network by a processor of the network. The method further comprises generating spikes in a spike generator as a function of outputs from the information-maximizing network. The method further comprises applying a first spike timing dependent plasticity (STDP) function to the generated spikes to affect the loop topologies of the generated spikes and their paths. The method further comprises calculating spike responses to a forward driver gate to the spike generators and disabling modulators based on the responses of the forward driver gate to the spike generators.

[0021] Additional features and advantages are realized by techniques described herein. Other embodiments and aspects are described in detail herein. For a better understanding, reference is made to the description and drawings. Brief description of the different views in the drawing

[0022] The objects considered as embodiments are specifically pointed out and clearly pointed out in the claims of the summary of the specification. The foregoing and other features and advantages of the embodiments will be apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: Fig. 1 shows a sketch of a biological neuron. Fig. 2 a block diagram showing the structure of part of the human brain. Fig. 3 shows an exemplary computer system capable of implementing one or more embodiments of the present invention. Fig. 4 shows a block diagram illustrating a configuration for information maximizing regions that may be used for one or more embodiments. Fig. 5 shows details of a block diagram of a behavioral area that may be used in one or more embodiments of the present invention. Fig. 6A is a block diagram illustrating a configuration for information maximizing regions and behavior regions that may be used by one or more embodiments of the present invention. Fig. 6B is a block diagram illustrating another configuration for information maximizing regions and behavior regions that may be used by one or more embodiments of the present invention. Fig. 7A shows a block diagram illustrating a configuration of behavior regions and a forward driver gate that may be used by one or more embodiments of the present invention. Fig. 7B shows a block diagram illustrating another configuration of behavior regions and a forward driver gate that may be used by one or more embodiments of the present invention. Fig. 8 shows a flowchart illustrating a method according to one or more embodiments of the present invention. Fig. 9 shows a flowchart illustrating another method according to one or more embodiments of the present invention. Fig. 10 shows a flowchart illustrating another method according to one or more embodiments of the present invention. Fig. 11 shows a flowchart illustrating another method according to one or more embodiments of the present invention. Fig. 12 shows an exemplary implementation of an information-maximizing network consisting of an information-maximizing region and input measurements, which may be used by one or more embodiments of the present invention. Fig. 13 shows an exemplary configuration of an information maximizing network and behavioral network that may be used by one or more embodiments of the present invention. Fig. 14A-D are diagrams illustrating various portions of a loop-regulating STDP function that may be used by one or more embodiments of the present invention. Fig. 15 is a diagram illustrating various aspects of a large loop covering the sensory, limbic, and motor areas of the human brain that may be used in conjunction with one or more embodiments of the present invention. Fig. 16 is a diagram further illustrating an exemplary configuration for a forward driver gate that may be used by one or more embodiments of the present invention. Fig. 17 is a diagram showing additional details of how STDP functionality may be implemented in a forward driver gate in accordance with one or more embodiments of the present invention. Fig. 18 shows a pie chart of time illustrating how a large loop may be repeatedly traversed in accordance with one or more embodiments of the present invention. Fig. 19 shows a pie chart of time illustrating how a large loop may be repeatedly traversed in accordance with one or more embodiments of the present invention. Fig. 20 shows a pie chart of time illustrating how a large loop may be repeatedly traversed in accordance with one or more embodiments of the present invention. Fig. 21 is a diagram illustrating the source of a modulatory input in forward-gating STDP in accordance with one or more embodiments of the present invention. Fig. 22 shows another diagram illustrating the source of a modulatory input in forward gate STDP in accordance with one or more embodiments of the present invention. Fig. 23 shows a computer program product in accordance with one or more embodiments of the present invention. Detailed description

[0023] It is understood in advance that, although this disclosure relates to modeling biological neural networks, with a particular emphasis on modeling brain structures and functions, the implementation of the teachings mentioned herein is not limited to modeling a specific environment. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of environment, for example, beta patterns, any data collected from the Internet, etc., as long as the various inputs to the environment can be transformed into a vector.

[0024] Although the present disclosure is directed to a computer system architecture, for ease of reference and explanation, various aspects of the disclosed computer system architecture are described using neurological terminology, such as neurons, synapses, DOPA, and GABA. It should be understood that for any discussion or illustration of a computer system architecture herein, the use of neurological terminology or neurological shorthand notations is for ease of reference and is intended to cover the pulsed neural network equivalents of the described neurological functions or neurological components.

[0025] As mentioned above, contemporary temporal encoders provide an alternative to digital and analog information encoders by asynchronously integrating received signals and producing subsequent sets of asynchronous, pulsed signals. Certain configurations of temporal encoder networks allow the extraction of information about sets of signals input to a network from sets of pulses or "spikes" output by networks. This information can be extracted from the spike rates (spikes per second) of a temporal encoder, the correlation of spiking rates with the population of temporal encoders, the time intervals between pulses produced by a temporal encoder's outputs, or the correlation of time pattern intervals between pulses produced by the population of temporal encoder outputs.However, known temporal encoder networks do not disclose how repeating networks can be configured to identify unique loop signals corresponding to useful relationships between the features encoded by the network, which can be selected for persistence, used for a variety of additional network operations, such as transferring states within the network, and then terminated. Furthermore, known temporal encoders do not demonstrate the operation of repeating networks in which the selection of output actions is based on input data and the selection of loop signals.Furthermore, known temporal encoder networks do not determine whether the encoding information in repetitive networks, through the selection of loop signals to drive specific output actions based on input data, is useful for solving challenging machine learning problems, such as behavioral selection in novel environments. Furthermore, known temporal encoder networks do not exhibit a preliminary phase of repetitive network learning, in which the selection of loop signals serves to transition states of the network while maintaining a variety of subsequent loop signals and their paths through the network.Furthermore, known temporal encoder networks do not demonstrate the use of external sensing to amplify internal modulatory signals that measure appropriate transitions of a network's states while maintaining a diversity of subsequent loop signals and their paths through the network. Furthermore, known temporal encoder networks do not demonstrate the use of network categorizations of recurring network states to limit paths for loop signals and their selection to those paths that are maximally informative about combined sensing of external and internal states.

[0026] An overview of the present invention will now be provided, with one or more embodiments directed to modeling biological neural systems using computer systems implementing a PNN with temporal encoders and informative loop signals. A static network configuration and a set of reconfiguration mechanisms capable of discovering, maintaining, and determining informative and useful loop signals within a network of temporal encoders are disclosed.The informative and useful loop signals can be used to maintain short-term memory, influence orderly informative transitions between network states, and provide drivers to influence coordinated and useful behavior in response to information input and encoded by a network configuration, referred to herein as an information-based exchange network (IBEX). The static configuration and reconfiguration mechanisms work together as a generic repository for learned transitions and are associated with useful behavior in response to specific internal states and input data sets.

[0027] Additionally, highly informative loop signals, once identified according to a network mechanism of the present invention, can be used for operations such as providing short-term memory for information in temporally stable network states, learning useful orderings of transitions between network states, and providing motor drivers to influence coordinated useful behavior in response to the information encoded in transitional network states. Furthermore, persistent loop signals that traverse informative network features, which appear as persistent activations of network components, enable the relationship information to be quickly represented and utilized by the network.This relationship information, represented in the unique and ordered passage of persistent network states by a loop signal, such as a spike, allows information encoded by these states to be exploited and enhanced in a context-specific manner. In the present disclosure, this process is referred to as information-based exchange, and the term information-based exchange network (IBEX) is used herein. Spikes that traverse unique states in a unique order iteratively accumulate this state and relationship information, as well as information about internal network configuration "decisions," so that upon arrival (for example, at a set of actuators or at a gate), spikes carry a large amount of information. Such spikes are referred to herein as informative loop signals.

[0028] In one or more embodiments of the present disclosure, at least one feedforward information-maximizing network of regions is provided. Each region of the information-maximizing network comprises a set of units that maximize information about input data comprising an external data source, feedforward inputs from an upstream region, or both. These regions may be connected sequentially, in a splitting, fanning / merging network topology, or in a complex network topology including multiple overlapping feedforward paths. Each information-maximizing unit in a region controls a spike generator that feeds spikes into a behavioral network of temporal encoders.

[0029] The spike generator generates spikes at a rate that is a function of the response level of an information-maximizing unit. The timing of the spikes can be random, can encode information from the input data, or, if available, can be drawn or relayed from a set of input spike times that combine the gated forward driver spikes of an upstream area spike generator and ungated feedback spikes of a downstream area spike generator. If more than one feedforward network is in the system, they can be concatenated in the feedforward direction by having connections that resemble ungated feedback connections, or they can be concatenated in the feedforward direction by having specific forward driver gates. The probability of drawing a spike time from this set is weighted by a relay probability specific to each spike generator contributing to the set.Spikes from a spike generator are then forwarded to feedback connections, a forward driver gate, or both. When opened, the forward driver gate adds the spike times to the set from which its downstream spike generator determines its own spike times and additionally merges data about these spikes (such as the rate) with the data input to the corresponding information-maximizing unit. To do this, this unit receives dual feedforward inputs from upstream regions, namely direct information-maximizing unit response stages, and the information about their spiking outputs is conditionally forwarded through the forward driver gate.

[0030] Spike generators implement functions to modify the propagation probability of each input spike generator. This function operates on temporal input spike / output spike sequences and intervals and is known as Spike Timing Dependent Plasticity (STDP). Through this function, the network of spike generators, formed by positive feedback connections and positive forward driver inputs, favors the elimination of short loops and the maintenance of long loops whose spike propagation period exceeds the STDP functional time window. One loop of particular interest is the grand sensory, limbic, and motor loop (SLM loop), which interconnects a set of feedforward networks that receive either sensory, limbic, or behavioral inputs.In the human brain, this large loop is completed by a light projection from primary motor areas to primary somatosensory areas of the cortex.

[0031] One or more embodiments also include a set of functions implemented by the forward driver gate. Here, the function is also STDP, but without loop regulation, since connections through the gate are negative, thus precluding sustained recurring loop signals. Instead, STDP serves to alternate the phase of a negative (inhibitory) gating spike input to the gate relative to a periodic input pattern of positive spike arrival times from various spike generators in the network attempting to pass the gate.These phase shifts are controlled by two types of negative gating spikes, one type (direct / dSPN / D1) producing a pause in a tonic closed signal from a GPi (open gate) and the other type (indirect / iSPN / D2) producing an increase of the tonic closed signal from the GPi (by means of a pause in a tonic negative signal in GPe, e.g., closed gate).

[0032] The independent control of open and closed negative gating phases is controlled by the two pathways described above. Subject to an additional modulator (e.g., DOPA) and local competition (e.g., GABA), phase shifts are established for "local winner" units in the presence of the modulator. Note that competition is determined by lateral inhibition between units and is well known in the art, for example, as the Kohonen network. Phase shifts propagate for dSPN units and are constant for iSPN units, effectively separating the open-gate and closed-gate time windows and ensuring that the gating window remains open when the gate is activated. When the modulator is withdrawn, the dSPN phase remains constant, while the iSPN phase advances.As a result, the "gate closed" window advances to the closed "gate open" window, and the loop signal is terminated. For "losers" of the local contention, the phase shifts for dSPN and iSPN are delayed in the presence of a modulator. In the absence of the modulator, the phase shifts for all "losers" are set to zero. Note that the concepts of "competition," "winner," and "loser" are treated herein as an embodiment of the disclosed network. Other "winnerless" spiking networks, as described in the prior art, may also be used, with the relative level of inhibition still being provided by the necessary modulation categories for determining phase shifts and their directions. Furthermore, lateral connection patterns do not have to follow the patterns of Kohonen networks and can, for example, be smaller.

[0033] Finally, the source of the modular input to the forward-gating STDP is revealed. This modular input is positively driven and rarely driven by external measurements. In the closed-loop system of the present disclosure, the drivers of large populations of dSPN and iSPN spikes are negative. When this inhibition is released, the modulator signal is present and strong (e.g., rebound spiking). This influences the system's gating function, which maximizes the entropy of dSPN and iSPN spiking activity over time (e.g., rapid orthogonal shifts in their gating vectors) and the entropy of transitions between large-loop dSPN and iSPN controls via their gating functions. The controls on the response to inhibition (e.g., integration times of activation, inactivation, unit compartmentalization, etc.)) further influences the system to gating functions that change their gating vectors at specific intervals (e.g., a few hundred milliseconds) according to the temporal constraints imposed by these controls. Therefore, the present disclosure is directed to the modeling of biological neural systems using computer systems implementing a type of pulsed artificial neural network (ANN), referred to in the present disclosure as information-based exchange (IBEX) networks with temporal encoders and informative loop signals.

[0034] Accordingly, the present disclosure provides technical advantages, including, but not limited to, one or more embodiments of the present disclosure not requiring imposed value functions for closed-loop functions. In addition, relatively fast learning results are achieved through loop continuation and the discovery of gating functions that maximize entropy and the diversity of all passes through the information-maximizing regions. Behavioral and network dynamics are deployed as a coding plan and as a means of measuring code usefulness. Categories emerge through unsupervised learning of high-entropy loop passes that terminate other loops (e.g., threat elimination). Open-loop control problems are specified by an internal set of closed-loop selection mechanisms.State planning components are more robust and undergo rapid reorganization when the conditions for loop selections change. Multiple latencies for procedure initiation and termination are available due to extremely large numbers of loop states and the availability of certain states that do not induce any external behavior but serve only to provide state information about specific behavioral outcomes and nothing else.

[0035] As context for one or more embodiments disclosed herein, an overview of biological neural networks is now provided. Fig. Figure 1 shows a sketch of a biological neuron 100. The neuron 100, also known as a nerve cell, is a specialized biological cell that processes information. As shown, it consists of a cell body, or soma, and two types of outward-extending tree-like branches, namely the axon and the dendrite. The cell body has a nucleus, which contains information about the important properties, and a cytosol, which contains the molecular building blocks for producing material needed by the neuron. A neuron receives signals (impulses) from other neurons through its dendrites (receivers) and transmits signals generated by its cell body along the axon (transmitter), which may divide into strands and substrands. At the ends of these strands are the synapses.A synapse is a basic structure and functional unit between two neurons (an axon strand of one neuron and a dendrite of another). When impulses reach the end of the synapse, certain chemicals called neurotransmitters are released. The neurotransmitters spread across the synaptic cleft to widen or inhibit the receptor neuron's predisposition to emit electrical impulses, depending on the type of synapse. The effectiveness of a synapse can be adjusted by signals traveling across the synapse, allowing the synapse to learn through the course of activities in which it participates. This dependence on the course functions like a storage device, which may be responsible for human memory.

[0036] Fig. Figure 2 shows a block diagram representing a portion of the structure of a human brain 200. The brain 200 includes a cerebral cortex 202, a thalamus 204, a subthalamic nucleus 206, a striatum 208, and a globus pallidus 210, which are configured and arranged as shown. A more detailed complete description of Fig. 2 is provided by the publication by Sebastian Helie, Srinivasa Chakravarthy and Ahmed A. Moustafa, entitled “Exploring The Cognitive and Motor Functions of The Basal Ganglia: An Integrative Review of Computational Cognitive Neuroscience Models,” which was published in the journal Frontiers In Computational Neuroscience in December 2013. For the purpose of this description, the cerebral cortex 202 in humans is a large, flat area of ​​neurons, approximately 3 to 4 millimeters thick and with a surface area of ​​about 2200 cm 2which is approximately twice the area of ​​a standard computer keyboard. The cerebral cortex 202 comprises approximately 10 11 Neurons, which is roughly equivalent to the number of stars in the Milky Way. Neurons are massively interconnected, and the networks they form are comparatively much more complex and dense than telephone networks. Each neuron is connected to 10 3 up to 10 4 other neurons. In total, the human brain comprises 200 approximately 10 14 up to 10 15 Connections (not shown). Neurons (for example, neuron 100 shown in Fig. 1) Within the cerebral cortex 202, neurons communicate via very short sequences of pulses, typically lasting a few milliseconds. The message is modulated at the pulse transmission frequency. This frequency can vary between a few and several hundred hertz, which is comparatively a million times slower than the fastest switching speed in electronic circuits.

[0037] Yet complex perceptual decisions, such as facial recognition, are performed by humans in a few hundred milliseconds. These decisions are performed by a network of neurons whose operating speed is only a few milliseconds. This implies that the computations can occupy no more than 100 serial steps. In other words, the human brain processes 200 parallel programs, requiring approximately 100 steps for such perceptual tasks. This is known as the 100-step rule. The same temporal consideration shows that the amount of information sent from one neuron to another must be very small (a few bits). This implies that critical information is not transmitted directly, but is captured and distributed through the connections.Further details about biological neurons and biological neural networks are available in the publication by S. Brunak and B. Lautrup entitled Neural Networks, Computers with Intuition, published in World Scientific Singapore 1990.

[0038] As additional context for one or more embodiments disclosed herein, an overview of ANNs is now provided. ANNs can be viewed as weighted and directed graphs, in which artificial neurons are nodes and directed edges (with weights) are connections between neuron outputs and neuron inputs. Based on the connection pattern (e.g., architecture), ANNs can be divided into two categories: feedforward networks, in which the graphs have no loops, and feedback or recurrent networks, in which loops exist due to feedback connections. In the most common family of feedforward networks, commonly known as multilayer perceptron networks, neurons are organized into layers that have one-way connections between them.

[0039] Different connections in ANNs cause different network behaviors. In general, feedforward networks are static. In other words, they produce only one set of output values ​​from a given input, rather than a series of values. Feedforward network dynamics are memoryless in the sense that their response to an input is independent of the previous network state (the weights may still include history-dependent states). In contrast, feedback or recurrent networks are dynamic systems. When a new input pattern is presented, neuron outputs are computed. Due to the feedback pathways, the inputs of each neuron are modified, causing the network to reach a new state.

[0040] Different network architectures require appropriate learning algorithms. The ability to learn is a fundamental property of intelligence. However, a precise definition of learning is difficult to formulate; a learning process in the ANN context can be viewed as the task of updating a network architecture and the connection weights so that a network can perform a specific task. The network typically needs to learn the connection weights from an available training sample. Performance improves over time by iteratively updating the weights in the network. The ability of an ANN to automatically learn from examples makes it an attractive design option. Instead of following a set of rules established by human experts, an ANN appears to learn the underlying rules (such as input-output relationships) from a given collection of representative examples.This is one of the most important advantages of an ANN over traditional expert systems.

[0041] To understand or design a learning process, it is necessary to have a model of the environment in which the neural network operates. In other words, the information available to the network must be known. This model can be referred to as a learning pattern. In addition, it is necessary to understand how network weights are updated. In other words, the learning rules that govern the update process must be understood. A learning algorithm refers to a procedure in which the learning rules are used to adjust the weights. There are three main learning patterns: supervised, unsupervised, and hybrid. In supervised learning, or learning with a "teacher," the network is provided with a correct answer (output) for each input pattern. Weights are determined that allow the network to produce answers that are as close as possible to the known correct answers.Reinforcement learning is a variant of supervised learning in which the network is provided only with a critique of the correctness of the network's outputs, but not with the correct answers themselves. In contrast, unsupervised learning, or learning without a teacher, does not require correct answers to be associated with each input pattern of a training dataset. It explores the underlying structures of the data or correlations between patterns in the data and organizes patterns into categories from these correlations. Hybrid learning combines supervised and unsupervised learning. Parts of the weights are usually determined by supervised learning, while others are determined by unsupervised learning. Additional details for ANNs and learning rules are described in "Artificial Neural Networks: A Tutorial" by Anil K. Jain, Jianchang Mao, and K. M. Mohiuddin, IEEE, March 1996.

[0042] The drawings are now described in greater detail, with reference symbols identifying the corresponding elements. Fig. 3 shows a high-level block diagram illustrating an example of a computer-based information handling system 300 useful for implementing one or more embodiments of the present disclosure. Although an example computer system 300 is shown, computer system 300 includes a communication path 326 connecting computer system 300 to an additional system (not shown) and one or more wide area networks (WANs) and / or local area networks (LANs), such as the Internet, intranet(s), and / or wireless communication networks (WLANs). Computer system 300 and the additional system communicate, e.g., communicate data, via communication path 326.

[0043] Computer system 300 includes one or more processors, such as processor 302. Processor 302 is connected to a communications infrastructure 304 (e.g., a communications bus, crossover bar, or network). Computer system 300 may include a display interface 306 that forwards graphics, text, or other data from communications infrastructure 304 for display on a display device 308. Computer system 300 further includes a main memory 310, preferably a random access memory (RAM), and may also include a second memory 312. Second memory 312 may, for example, include a hard disk 314 and / or a removable storage drive 316, represented, for example, by a floppy disk drive, a magnetic tape, or an optical drive.The removable storage drive 316 reads from or writes to the removable storage medium 318 in a manner known in the art. The removable storage medium 318 may be, for example, a floppy disk, a CD, a magnetic tape, an optical disk, etc., which can be read from or written to by the removable storage drive 316. The removable storage medium 318 comprises a computer-readable medium having computer software and / or data stored thereon.

[0044] In alternative embodiments, secondary storage 312 may include other similar means for loading computer programs or instructions into a computer system. Such means may include, for example, a removable storage unit 320 and an interface 322. Examples of such means may include a program package and package interfaces (such as those found in video game devices), a removable memory chip (such as an EPROM or a PROM) and an associated socket, or other removable storage units 320 and interfaces 322 that allow the transport of software and data from the removable storage unit 320 to the computer system 300.

[0045] Computer system 300 may also include a communications interface 324. Communications interface 324 allows software and data to be transferred between the computer system and an external device. For example, such a communications interface 324 may include a modem, a network interface (such as an Ethernet card), a communications port, or a PCM-CIA slot and card, etc. Software and data are transferred from communication interface 324 in the form of signals, which may, for example, be electronic, electromagnetic, or optical, or include other signals suitable for reception by communication interface 324. These signals are provided to communication interface 324 via a communications path (such as a channel) 326.The communication path 326 carries signals and may be provided using cables, optical fibers, telephone cables, a cellular connection, a radio frequency connection, and / or other communication channels.

[0046] In the present disclosure, the terms "computer program medium," "computer-readable medium," and "computer-usable medium" generally refer to media such as main memory 310 and secondary storage 312, removable storage drives 316, or a hard disk installed in a hard disk drive 314. Computer programs (also called computer control logic) are stored in main memory 310 and / or secondary storage 312. Computer programs may also be received via communications interface 324. Such computer programs, when executed, may cause the computer system to perform the features of the present disclosure as discussed herein. In particular, the computer programs enable processor 302 to perform the features of the computer system when executed. Accordingly, such computer programs represent the controls of the computer system.

[0047] Now, one or more embodiments of the present disclosure will be described in more detail. Fig. 4 and Fig. 5 show how the Cortex 202 (shown in Fig. 2) a human brain 200 (also shown in Fig. 2) can be characterized by two different types of areas, namely the information-maximizing area (IMA) and the behavioral area (BA). Specifically Fig. 4 shows a block diagram showing the configuration of IMAs and Fig. Figure 5 shows an example of a BA. The IMAs of Fig. 4 and the BA of Fig. 5 are capable of being used by one or more embodiments of the present disclosure.

[0048] The block diagram from Fig. Figure 4 shows a first IMA 402, a second IMA 404, a third IMA 406, a first phenotypic reading 408, a second phenotypic reading 410, and a third phenotypic reading 412, configured and arranged as shown. Input vectors are provided to the IMAs 402, 404, and 406 via the phenotypic readings 408, 410, and 412. Phenotypic readings refer to measurements made by various sensory receptors of the body, including, for example, the retina, skin receptors, internal receptors of the stomach, skeletal extensor receptors in the muscles, receptors in the joints, taste buds, and temperature sensors in the skin. The IMAs 402, 404, and 406 operate according to the principles of information maximization. Information maximization takes a set of input signals and assigns them a useful set of output signals.Information maximization is also known as feature identification or basic neural network categorization. The use of the term "information maximization" herein generally refers to the tendency of an IMA to increase the information of an output vector ensemble over an input vector ensemble and does not imply that optimal maximum information is ever achieved at any particular point in the network's operation. The categories of input vectors are self-organizing and use unsupervised learning, with the goal that information maximization attempts to achieve being the achievement of maximum entropy, or maximum information. In other words, information maximization attempts to cover the largest possible output space with an output vector ensemble for a given input vector ensemble.For example, input vectors that are bound together in a region of the input space are distributed by the information maximization to the output space. This has the effect that the analysis of the input vectors can be performed in much more detail, since a larger region of the output space is encoded for the input vectors. Therefore, the IMAs 402, 404, and 406 correspond to a component or structure in a model or architecture of the present disclosure, referred to herein as a region. The number of inputs and the number of outputs of a region can be purely random, so that a region generally represents a plurality of units that cooperate to maximize information.

[0049] The block diagram from Fig. Figure 5 shows an exemplary BA 500 with spike generators 502, 504, 506 configured and arranged as shown. The spike generators 502, 504, 506 operate according to spike generation principles, whereby a scalar (e.g., from zero to one) is transformed into a series of temporal events. The time or frequency of an event is a reflection of the scalar value (e.g., a value has a magnitude but no direction). Accordingly, if an element "u" is a scalar with a value between zero and one, the spike generators 502, 504, 506 generate spikes at a rate that depends on a function of the element "u."

[0050] Because BA generates 500 spikes, it can be thought of as a temporal encoder. As previously described, contemporary temporal encoders provide an alternative to digital and analog coding of information by integrating asynchronously received signals and producing successive sets of asynchronously pulsed signals. Certain configurations of temporal encoder networks allow the extraction of information about sets of signals input to the network from sets of pulses or spikes output from a network. This information can be extracted from spiking rates (spikes per second) of a temporal encoder output, the correlation between spiking rates and the population of temporal encoders, the time intervals between pulses produced by a temporal encoder output, or the correlation of patterns of time intervals between pulses.However, known temporal encoder networks implemented without the benefits of the present disclosure do not disclose how to configure repetitive networks so that unique loop signals can be related to useful relationships among the features encoded by a network, identified, selected for persistence, used for a variety of different network operations, and terminated. Furthermore, known temporal encoder networks do not demonstrate repetitive network operation in which the selection of output actions is based on input data and the selection of loop signals.Additionally, known temporal encoder networks do not determine whether encoding information in repetitive networks by selecting loop signals to drive specific output actions based on input data is useful for solving difficult machine learning problems such as behavioral selection in new environments. Furthermore, known temporal encoder networks do not exhibit a preliminary phase of repetitive network learning during which the selection of loop signals is responsible for propagating network states while maintaining a diversity of successive loop signals and their paths through the network.Furthermore, known temporal encoder networks do not teach the use of external measurements of values ​​to amplify internal modulatory signals that measure appropriate transitions of the network's states while maintaining a diversity of successive loop signals and their paths through the network. Furthermore, known temporal encoder networks do not teach the use of network categorization of repeating network states for limiting paths for loop signals and the selection of those paths that are maximally informative with respect to combined measurements of external and internal states.

[0051] The Fig. 6A and Fig. 6B show block diagrams illustrating a configuration of IMAs and BAs that may be used by one or more embodiments of the present disclosure. In particular, Fig. 6A and Fig. 6B IMAs 402, 404, 406 and BAs 602, 604, 606, which are configured and arranged as shown. With reference to Fig. 6A, the IMAs 402, 404, 406 receive input vectors from the phenotypic measurements 408, 410, 410 (shown in Fig. 4). Furthermore, the IMAs 404, 406 receive input measurement values ​​from a preceding or ascending IMA. If the configuration of the Fig. 6A and Fig. 6B the human brain 200 Fig. 2, the IMAs 402, 404, and 406 are located in a part of cortex 202 (e.g., layer 2 / 3 of various Brodmann areas on the cortex). Deeper (e.g., 2 to 4 millimeters) in cortex 202 are the BAs 602, 604, and 606 (e.g., layer 5 of the corresponding Brodmann area of ​​the cortex).

[0052] The IMAs 402, 404, and 406 are cascaded in the feedforward direction, and the BAs are cascaded in the feedback direction. As previously mentioned herein, the feedforward connection is a non-recurring connection in which signals can travel in only one direction, identified as forward. Input data is passed to an area of ​​processing elements where computations are performed. Each processing element performs a computation, and the newly computed values ​​become new input values ​​provided to the next area. This process continues until it has passed through all layers and an output is determined. In contrast, feedback connections can travel in a direction identified as backward, which is different from the forward direction.Given constant input and a suitable network configuration, loops that exist in feedback connections can become nonlinear dynamic systems that change continuously until they reach a state of equilibrium.

[0053] In Fig. 6B, BAs 602, 604, 606 correspond one-to-one to IMAs 402, 404, 406. The correspondence of the IMAs and BAs is represented by directed arrows extending from the IMAs to the BAs and from the BAs to the IMAs. The correspondences are identified as "pulse rate setting" from the IMAs to the BAs and as "pulse rate measurement" from the BAs to the IMAs. The directed arrows between IMAs 402, 404, 406 and BAs 602, 604, 606 are assigned to the columns in cortex 202 (shown in Fig. 2) that extend between different parts (e.g., IMA 402 and BA 602) of cortex 202. Therefore, in a system architecture in one or more embodiments, the rate or specific timing of the spikes generated by the BA is modulated or set by the IMA. The IMA sets or modulates the pulse rate, and the IMA measures the pulse rate to develop a scalar representation of the pulse rate.

[0054] As previously described, BAs 602, 604, 606 are spike-generating (as shown by BA 500 in Fig. 5). With the configuration shown in the Fig. 6A and Fig. 6B, there is no path for the spikes generated in BAs 602, 604, 606 to propagate in the feedforward direction. Accordingly, Fig. 7A is a block diagram in which BAs 602, 604, 606 and a forward driver gate (FDG) 702, configured and arranged as shown, provide a path for spikes generated in BAs 602, 604, 606 for propagation in the feedforward direction through FDG 702, according to one or more embodiments of the present disclosure. FDG 702 can drive either a many-to-one, a one-to-one, a many-to-many, or one-to-many relationship between ascending (e.g., BA 602 or BA 604) and descending (e.g., BA 606) BAs. In the example shown in Fig. 7A and in the example shown in Fig. 7B, FDG 702 controls a many-to-one relationship between BAs 602, 604, and BA 606 in the feedforward direction. FDG 702 receives inputs from BA 602 and BA 604 and decides whether to open or not and whether to allow these inputs to flow to BA 606.

[0055] Fig. 7B is a diagram with more detail of the configuration used in Fig. 7A, where FDG 702 includes a gate 706 and a large loop STDP gating function 704, with a local STDP function 708 applied to the spike generators (SGs) 502A, 504A, 506A and the feedback connections between the BAs 602, 604, 606. Indeed, the local STDP gating function 708 and the large loop gating function 704 both apply STDP functionality, but each STDP application is used to obtain different results. The local STDP function 708 is applied within and between the BAs 602, 604, 606 to eliminate local (e.g., relatively short) loops within and between the BAs 602, 604, 606. Since STDP depends on pulse timing and since FDG 702 uses inputs that are timed and pulsed, the large loop STDP function 704 between the inputs of FDG 702 and gate 706 is used to determine whether or not FDG 702 should close or open.Thus, units in BAs 602, 604, 606 are connected only by large loop paths and not by local loop paths, since the local loop paths are effectively inhibited by the application of the local STDP function 708. Note that large loop paths may involve signals that can be propagated through feedback and feedforward connections between separate units in BAs 602, 604, 606, but that local loop signals between the units are largely inhibited. Accordingly, there are two inputs 710, 712 from the BA 602 into the FDG 702, as well as two inputs 714, 716 from the BA 604 into the FDG 702. The inputs 710, 714 attempt to access the large loop STDP gating function 704, which learns which paths to open and close through gate 706 based on any given set of inputs.Once Gate 706 is opened or closed, inputs 712, 716 from either BA 602 or 604 directly into Gate 706 are affected with respect to their ability to pass through Gate 706 and reach BA 606.

[0056] Therefore, FDG 702 uses the STDP gating function 704 to open or close the gate 706 through learning. The large loop STDP gating function 704 shifts the phase of negative (inhibitory) gating spike inputs to 706 relative to a periodic input pattern of positive spike arrival times from various spike generators 502A, 504A, 506A, 502B, 504B, 506B attempting to pass through gate 706. The phase shifts are controlled by two types of negative gating spikes; a first type (direct / dSPN / D1) produces a pause in a strengthened closed signal from GPi (gate open), and a second type (indirect / iSPN / D2) increases an increment to the strengthened closed signal from GPi (via a pause in a strengthened negative signal in GPe; gate closed). Independent control of open and closed negative gating phases is controlled by these two paths, as described in the next paragraph.

[0057] Depending on an additional modulator (e.g., DOPA) and local competition (e.g., GABA), phase shifts are described for "local winner" units in the presence of the modulator. Note that competition is determined by lateral inhibition between units and is well known in the art, for example, as the Kohonen network. Phase shifts advance for dSPN units and are constant for iSPN units, effectively separating the gate-open and gate-closed windows in time, ensuring that the gating windows remain open once activated. When the modulator is withdrawn, the dSPN phase remains constant, while the iSPN phase advances. In this way, the "gate-closed" window advances until it obscures the "gate-open" window and the loop signal terminates.For the "losers" of the local competition in the presence of the modulator, the phase shifts for each dSPN and iSPN are delayed. In the absence of the modulator, all phase shifts are zero for all losers. Note that the concepts of "competition," "winners," and "losers" are used here as one embodiment of the network. Other "winnerless" spiking networks described in the prior art can also be used, where the relative level of inhibition can still provide the necessary modulation categories for determining phase shifts and their directions. Furthermore, patterns of lateral connections do not have to follow these Kohonen networks and can, for example, be short.

[0058] The modulatory input for forward-gating STDP is positively driven and rarely driven by external sensing. In the closed-loop system of the present disclosure, the drivers are negatively driven by large numbers of dSPN and iSPN spikes. When inhibition is released, the modulator signal is present and strong (rebound spiking). This influences the system's gating function, which maximizes the entropy of dSPN and iSPN spiking activity over time (e.g., rapid orthogonal shifts in their gating vectors) and the entropy of transitions between large-loop dSPN and iSPN controls via their gating functions. The controls on the response to inhibition (e.g., integration times of activation, inactivation, unit compartmentalization, etc.)) further influences the system to gating functions that change their gating vectors at specific intervals (for example, a few hundred milliseconds) according to the temporal constraints caused by these controls.

[0059] The Fig. 8-11 show flowcharts illustrating method steps 800, 900, 1000, 1100 according to one or more embodiments of the present disclosure. It should be noted that although steps 800, 900, 1000, 1100 are depicted in order, the blocks comprising steps 800, 900, 1000, 1100 need not necessarily be performed in that order.

[0060] Fig. Figure 8 shows a method step 800 illustrating a general application of a network architecture in accordance with the present disclosure. As shown in Fig. 8, method step 800 begins with block 802, where input data is provided to an IMA. In block 804, spikes are generated as a function of the network response layers. Block 806 subjects spikes to a local STDP function to influence local topologies. Block 808 calculates FDG spike responses to spike generators. In block 810, inhibition and modulation based on FDG responses to spike generation are implemented. In block 812, the phase for gate-open and gate-closed signals of the FDG is shifted based on a large loop STDP function. In block 814, data from forward-driving spikes is changed to data input to the IMA. Block 816 provides a response modulation input for a large loop STDP function of an FDG. In block 818, the behavior can now be selectively driven based on couplings between the spike generators and phenotypic effectors.It should be noted that phenotypic effectors, although external to the network described herein, may themselves be located in a part of the human brain, such as a stem neuromodulatory nucleus.

[0061] The Fig. 9 - 11 break down the general process steps of Fig. 8 into three parts to provide greater detail. In particular, method step 900 shows how in Fig. 9 illustrates method steps as implemented by feedforward, feedback, and gating connections in network regions, as well as in one or more embodiments of the present disclosure. In general, method 900 describes the operation of a network incorporating the present disclosure in terms of what can be described as a wave of activity propagating through feedforward and feedback directions. Method 900 begins at block 902 by collecting a first set of phenotypic measurements. At block 904, features (e.g., output vectors) of a first IMA are modified by the collected first phenotypic measurements. At block 906, the features (e.g., pulse rate settings) of the first IMA modify spiking in a first BA. At block 908, the spikes in the first BA excite either a disinhibitory (D1) unit or an inhibitory (D2) unit of an FDG.Block 910 refers to input behavioral spikes (e.g., through spike timing correlation analysis) and D1 or D2 unit spikes analyzed by a large loop STDP gating functionality. In block 912, the connection between a BA and a D1 or D2 unit is modified according to a large loop STDP gating function correlation analysis (e.g., spike timing correlation analysis). In other words, block 912 reads the large loop STDP gating functions and modifies the connection between a BA and a D1 or D2 unit according to the correlation between pre- and post-spike times in accordance with the STDP functionality. In block 914, a D1 unit decreases the inhibition of the gating unit, or a D2 unit increases the inhibition of the gating unit.In block 916, spikes in the first BA pass through the gate if inhibition is low enough to assist other spikes forward through a downward BA. In block 918, spikes in a second BA directly evoke spikes in the first BA through feedback connections. Block 920 correlates (e.g., by spike timing correlation analysis) the spikes of a first BA and the spikes of a second BA analyzed by a large loop STDP gating functionality. In block 922, the connections between the first BA and the second BA are modified according to the local STDP functionality to eliminate local loops. In block 924, the IMA measures features of feedforward IMA features and feedforward BA behavioral spikes. Therefore, in essence, block 924 convolves the BA spike measurements with the IMA measurements, which is shown, for example, in . Fig. 6B by the pulse rate measurement exchange between IMA 402 and BA 602. Since, in general, feedback spikes in a second BA may also cause spikes in a first BA, as shown in block 918, it should be noted that spikes in a third BA (not shown) can be expected to cause spikes in the second BA, shown in block 918, and that these spikes are similarly folded into the second IMA measurements in block 924 (not shown).

[0062] Method 1000 shows in Fig. 10 Process operations implemented by forward driver gating units and modulators of STDP in accordance with one or more embodiments. Under the present disclosure, modulators of STDP result in a variety of STDP gating functions unique to the BA and the FDG interface, which may be represented, for example, by the inputs 710, 714 of the large loop STDP gating function 704 in Fig. 7B. Further details of the multitude of STDP gating functions unique to the BA and FDG interfaces are also shown at the bottom of Fig. 17, which will be described in greater detail later in this revelation.

[0063] Procedure 1000 begins as in Fig. 10, with block 1002, by spikes in BAs (1 ... n), which excite spikes in the disinhibitory (D1) units and the inhibitory (D2) units of an FDG. The spikes in forward driver gating units exchange their lateral inhibition through the local network in block 1004. An example of this lateral inhibitory exchange is shown in small diagrams at the bottom of the Fig. 18, Fig. 19 and Fig. 20, which are described as forward driver gating units (FDGUs) 1804, 1804A, and 1804B. Although a more complete discussion of the Fig. 18, Fig. 19 and Fig. 20 provided later in this disclosure, certain aspects of the Fig. 18, Fig. 19 and Fig. 20 now describes the lateral inhibition of block 1004 for clarity. In Fig. 18, the FDGU diagrams are represented as three nodes, with the central node representing a unit producing a modulator state, referred to herein as GABA. As shown in Fig. As shown in Figure 18, the GABA modulator is produced by the unit that is active, so the GABA modulator input to the units receiving the GABA modulator state is shown as a dotted circle. The GABA-producing unit produces GABA in Fig. 18, which transmits it along the curved sinusoidal line through its local laterally inhibited network to its neighboring node. Since the GABA-producing node produces GABA and does not receive GABA, which allows it to generate spikes because it is not inhibited, it is in a modulator state, referred to herein as GABA LO In the Fig. 19 and Fig. 20 the GABA node is called GABA HI-state because it receives GABA from the local network. Returning to method 1000, in block 1006, the local network reaches a state through the effect of its exchange of GABA, wherein a subset of units receives inhibited modulation (e.g., GABA) and ceases spiking. In block 1008, the local network reaches a state, also through the effect of its exchange of GABA, wherein a subset of units provides inhibited modulation and continues spiking. In block 1010, the secondary modulators receive copies of the inhibition by spiking-producing groups of forward-driving gating units. Although a more complete discussion of the Fig. 21 and Fig. 22 provided later in this disclosure, certain aspects of the Fig. 21 and Fig. 22 to better describe the secondary modulators of block 1010. The secondary modulators, which are integrators / units, are described in the Fig. 21 and Fig. 22 as DOPA, and are graphically represented as 2104, 2104A in the small diagrams on the left edge of the Fig. 21 and Fig. 22. Line 2108 represents the state of the secondary modulator unit, which, as shown, can be high or low. The low state corresponds to the secondary modulator receiving inhibition from the spikes received by the gating unit network. Therefore, when gating units produce spikes, they inhibit each other, secondary modulators, GPe (external globus pallidus) and GPi (internal globus pallidus). The function of receiving a copy of inhibition from spiking-producing subsets of forward-driving gating units is represented by line 2110 in Fig. 21.

[0064] In block 1012, the secondary modulators integrate the time of inhibition of spiking-producing subgroups as a response current. As in Fig. As shown in Figure 21, line 2108 starts high and goes low when it receives inhibition from gating units. In the low state, there is a current that flows and builds up over time, integrating the input signals and pulling line 2108 in the opposite direction. Beyond a maximum integration period, and with existing inhibition by the gating units, the current begins to disappear again. Ultimately, therefore, the secondary modulators measure a time of inhibition, so that when the inhibition decreases, as in Fig. 22, and when the elapsed time of inhibition is between a certain minimum value and a maximum value, the response event is then expressed as the inhibition fading and the unit, instead of restoring its previous state, enters a state that can produce spikes (e.g., provide modulation). This timer is referred to herein as the response current. If the response current was integrated over a time interval T, any rapid decrease in inhibition in block 1014 causes response spiking. Therefore, the response current is a measure of the state of the system itself, where a gating vector existed for a small minimum time period, inhibited the secondary modulator, and then quickly disappeared. We further teach that the time interval T itself may be under modulatory control of the effector output of the network.In this way, the network can control its own transmission rate by modulating the time interval criterion T using specific effector outputs of the behavioral network.

[0065] Optionally, spiking is permitted in block 1016 if an extrinsic value signal is detected by the secondary modulators. Therefore, block 1016 provides a second, optional way for the secondary modulators to generate spikes by measuring external values ​​that serve as external input. In other words, the secondary modulators can be driven to produce spikes by excitation. The forward driver gating units of the large loop STDP gating function in block 1018 are modified based on inhibitory (e.g., GABA) and secondary (e.g., DOPA) modulators. One possible set of GABA / DOPA criteria for modifying the large loop STDP gating function is illustrated by the four diagrams at the bottom of Fig. 17. Other criteria are possible, provided that over time the system operates in such a way that the sum of the phase shifts of the gating units is zero. For a direct or indirect forward gating unit, the large loop STDP gating function is modified by selecting one of four different STDP diagrams, depending on whether GABA modulation is high or low, whether DOPA is decreasing or increasing at D1 units, or whether DOPA is increasing or decreasing at D2 units.

[0066] Method 1100, shown in Fig. 11 illustrates process operations implemented by forward driver gating units for modifying the STDP gating functions. Therefore, method 1100 further describes the plurality of STDP gating functions implemented in Fig. 17. In addition to describing the STDP gating function that was selected, method 1100 also describes the effects of the selected STDP gating function that are best understood in the Fig. 18, Fig. 19 and Fig. 20. When a specific gating function is selected based on DOPA / GABA correspondence, the gating response to a patterned input by a dSPN (e.g., disinhibitory or D1) or an iSPN (e.g., inhibitory or D2) is either ahead of phase or behind phase. Although either period may be used, in one or more embodiments of the present disclosure, the period of interest is that of the large loop because it escapes the loop-eliminating STDP. Furthermore, this period matches a minimum duration interval of persistent inhibition required for the response event to be expressed in secondary modulators when the inhibition wears off ( Fig. 22).

[0067] Method 1100 begins with block 1102 by receiving primary inhibitory modulator signals from the local gating unit network. Block 1104 receives a second modulator signal from secondary modulators according to the rules in method 1000. If it is a secondary modulator and not a primary modulator in block 1106, the STDP is adjusted to the advancement of the phase of the D1 response to the periodically patterned input. If it is not a primary modulator and not a secondary modulator in block 1108, the STDP is adjusted to the advancement of the phase of the D2 response to the periodically patterned input. If it is a primary modulator and a secondary modulator in block 1110, the STDP is adjusted to the advancement of the phase of the D1 and D2 responses to the periodically patterned input.If it is a primary modulator and not a secondary modulator in block 1108, the STDP is not adjusted to either the advance of the phase of the D1 response or the advance of the D2 response to the periodically patterned input.

[0068] A more detailed description of the exemplary architecture of one or more disclosed embodiments will now be given, with a particular emphasis on the components included in the block diagram layers in the Fig. 4 to 7B. Fig. Figure 12 shows an exemplary implementation of an information-maximizing network 1200 formed from an information-maximizing region 402A and the input measurement values ​​1202. The information-maximizing network 1200 maximizes an output vector ensemble consisting of the vectors "u" relative to an input vector ensemble comprising the vectors "x." Note that the units labeled "x" and "u" correspond to the elements of the vectors "u" and "x" in the description. The effective learning rule is applied to the weights "W" (a matrix in linear algebra terms) such that the information of the output vector ensemble is maximized when transferred to the input vector ensemble. In practice, this is achieved by maximizing the entropy of the output vector ensemble, so that the least-information response of a network is always an output that is always the same.The output, which can be thought of as a vector in a neural network, is very uninformative if the vector always points in the same direction. Therefore, the weighted matrix (W) is dynamically changed to maximize the separation between two output vectors "u" generated when the network provides two input vectors "x." As described above, . Fig. 12 illustrates an example of an information-maximizing network suitable for use with one or more embodiments of the present disclosure. Further details on information-maximizing networks 1200 are disclosed in the publication entitled "A Local Learning Rule That Enables Information Maximization for Arbitrary Input Distributions" by R. Linsker (1997), Neural Computation 9, pages 1661-1665.

[0069] Fig. Figure 13 shows an exemplary configuration of an information-maximizing network 1200 and a behavioral network 1300 of one or more embodiments of the present disclosure. The behavioral network includes a first spike generator 1302, a second spike generator 1304, a feedback path 1306, a phenotypic measurement 1308, input data 1310, a forward driver gate 1312, a feedforward path 1314, and input data 1316, configured and arranged as shown. In general, the behavioral network 1300 establishes a set of loops in a network "governed" by an information-maximizing network 1200. An example of this governed relationship is represented by the directed arrow 1320 extending from the output element of "u" to the spike generator 1304. As previously noted, spike generation involves what is known as a temporal or time-encoded engine.If the element "u" is a scalar (e.g., between zero and one), the spike generator 1304 generates spikes at a rate equal to a function of the scalar element "u." According to the present disclosure, the behavioral network 1300 uses the spikes in a specific way. The controller of the spike generator 1304 is initially the output element "u" of the information-maximizing network 1200, which maximizes the information about input data 1310. Some of the input data 1310 is calculated from effects of the spike generator 1304. Therefore, according to the present disclosure, by connecting spike generators as in . Fig. 13, it is possible to modify the input vector ensemble for the information-maximizing network.

[0070] According to the network configuration shown in Fig. 13, the control of spike generation lies locally with the information-maximizing unit 1200, and spikes can be propagated in various ways. For example, spikes can directly influence phenotypes 1308 through effectors. More specifically, with respect to the present disclosure, spikes can be propagated into a feedback connection from an information-maximizing network 1200 to an ascending network, or spikes can be propagated into a gated feedforward connection, depicted, for example, from spike generator 1302 to forward driver gate 1312.

[0071] Fig. Figure 13 is a simplified representation, and in practice, information-maximizing networks 1200, a behavioral network 1300, and the spike generators 1302, 1304 are connected by a large number of paths, many of which involve loops. A learning rule, such as STDP, can be implemented to eliminate so-called local loops, which are loops that transmit signals within a certain duration. A more detailed example of STDP is shown in the Fig. 14A, Fig. 14B, Fig. 14C and Fig. 14D. As shown in Fig. As shown in Figure 14A, spikes initiate from a preneuron (pre), propagate to a postneuron (post), and can either excite a postneuron to fire or not, depending on the threshold for generating spiking of the postneuron. Fig. Figure 14B shows an initial loop topology that may result from a plurality of paths formed in an information-maximizing network 1200 (shown in the Fig. 12 and Fig. 13) and a behavioral network 1300 (shown in Fig. 13) can exist. Fig. 14C shows the application of an STDP learning rule and Fig. Figure 14D shows the resulting learned topology, with the beginnings and ends of a local loop at node "0" eliminated. A more complete discussion of local loop elimination by STDP is provided by the paper by Kozloski and Cecchi titled "A Theory Of Loop Formation And Elimination By Spike Timing-Dependent Plasticity," published by Front Neural Circuits (March 10, 2010). Additional details of the hardware implementation of STDP loop elimination are provided in U.S. Patent 8,473,439 and U.S. Patent Publication No. US2012 / 0109864.

[0072] Back to Fig. 13, with the implementation of information maximization and loop elimination, the information has been optimized. The configuration shown for the behavioral network 1300 takes the optimized information of the network state and answers the fundamental question of which behavior should occur (or colloquially, "What should happen to it?"). The behavior in the closed-loop implementation of the present invention then relates to a transition of the network from a given state and its present optimized information to a subsequent state, which depends on previously learned useful transitions. It should be noted that the answers to this question are specifically implemented as spiking patterns and are also incorporated into the information-optimizing functionality of the architecture, as shown in Fig. 13, to achieve an even higher level of information optimization given the current behavioral context. The execution of the forward driver gate 1312 is a primary regulator in determining what to do with the optimized local loop-eliminated information. The forward driver gate 1312 also includes STDP, but it is not loop-regulating because connections through the forward driver gate 1312 are negative and therefore existing recurring loop signals are not possible. Instead, STDP serves to shift the phase of negative (inhibitory) gating spike inputs to gate 706 (shown in Fig. 7B) relative to a periodic input pattern attempting to pass through the forward driver gate 1312, of positive spike arrival times from various spike generators into the network. The phase shifts are controlled by two types of negative gating spikes, one type (direct / dSPN, shown in the Fig. 18 - 20) produces a pause in an amplified closed signal from GPi (gate open) and another type (indirect / iSPN, shown in the Fig. 18 - 20) produces an increase in the amplified closed signal from GPi over a pause in an amplified negative signal in GPe, which means that the gate is closed.

[0073] The independent control of open and closed phases of negative gating is controlled by these two pathways as follows: Depending on an additional modulator (e.g., DOPA) and a local competitor (e.g., GABA), phase shifts for "local winner" gates in the presence of the modulator are now described. Note that the competition is determined by lateral inhibition between units and is known from the prior art, such as in a Kohonen network. Phase shifts expand for dSPN units and are constant for iSPN units, effectively temporally separating the open gate windows and the closed gate windows and ensuring that the gating window remains open when activated. When the modulator is withdrawn, the dSPN phase remains constant, while the iSPN phase advances.In this way, the "closed gate" window advances to obscure the "open gate" window, and the loop signal is terminated. For the "losers" of the local competition in the presence of the modulator, the phase shifts for dSPN and iSPN are delayed. In the absence of the modulator, the phase shifts for all "losers" are zero. As noted previously, the concepts of "competition," "winner," and "loser" are described herein as one embodiment of the network. Other "winnerless" spiking networks, as described in the prior art, can also be used, where the relative level of inhibition can still provide the necessary modulation categories for determining the phase shifts and their directions. Furthermore, patterns of lateral connections need not follow those of a Kohonen network and can, for example, be short.

[0074] The modular input for forward gating STDP is positively driven and is rarely determined by external value measurements. In closed-loop systems of the present disclosure, the drivers of large populations of dSPN and iSPN spikes are negative. If this inhibition lasts for a certain time and is then released, the modulator signal is present and strong (rebound spiking). This causes the system to perform gating functions that maximize the entropy of dSPN and iSPN spiking activity over time (e.g., maximizing rapid orthogonal shifts in their gating vectors) as well as the entropy of transitions between large loop dSPN and iSPN controls over their gating functions. In response to inhibition (e.g., the integration times of activation, inactivation, unit compartmentalization, etc.), the system performs gating functions that maximize the entropy of dSPN and iSPN spiking activity over time (e.g., maximizing rapid orthogonal shifts in their gating vectors).), the controls further influence the system to cause gating functions to change their gating vectors at specific intervals (for example, every few hundred milliseconds) according to the temporal conditions imposed by these controls.

[0075] Some loops within a given system (for example, the human brain 200, shown in Fig. 2) are sufficiently long that they are not eliminated by the application of local STDP. Within the brain, for example, such loops are collectively referred to as large loops. Fig. Figure 15 shows a diagram illustrating the various aspects of a large loop covering the sensory, limbic, and motor (SLM) areas of the brain, the concepts of which may be used in connection with one or more embodiments of the present disclosure. The loops long enough not to be eliminated by STDP are loops that are transmitted in large areas of the brain and large regions of information-maximizing networks. In general, the various areas, which are also Fig. 15, to information-maximizing areas and behavioral areas as previously described in this disclosure in connection with the Fig. 4 to 7B, 12 and 13. In accordance with the present disclosure, the forward drive gate 1312 (shown in Fig. 13) as a relay for signals through certain large loops by opening and closing various nodes in the feedforward, forward driver gate connection in the behavioral network 1300 (shown in Fig. 13) to allow certain spikes to pass and to disallow certain spikes to pass.

[0076] Fig. 16 shows a diagram further illustrating an exemplary configuration of a forward driver gate 1600 capable of being used by one or more embodiments of the present disclosure. As previously noted, the present disclosure is directed to a computer system architecture. For ease of reference and explanation, various aspects of the disclosed computer system architecture are described using neurological terminology, such as neurons, synapses, DOPA, and GABA. It will be understood that for any discussion or illustration of a computer system architecture herein, the use of neurological terminology or neurological abbreviations to cover pulsed neural networks equivalent to the described neurological functions or neurological components is for ease of reference. The functionality of forward driver gate 1600 is described in Fig. 17 by Str and Th, which are abbreviations for striatum and thalamus. Forward-driver gate 1600 gated large loops that connect different areas of a network, as in Fig. 17 as S1, S2, and RGA. To get from S2 to RGA in a large loop, S2 must pass through Th, which is under the inhibited control of Str. This means that Str receives inputs from S1 and S2 and, based on its internal weight matrix, determines whether or not Th is inhibited. If the effect of Str inhibits Th, any signal from S1 or S2 attempting to pass through Th to reach RGA will be blocked. If the effect of Str disinhibits Th, any signal from S1 or S2 attempting to pass through Th to reach RGA will be able to do so.

[0077] A more detailed representation of the functionality of Forward Driver Gate 1600 from Fig. 16 is in the diagram at the top of Fig. 17. In Fig. 17, Cx is any unit in a behavioral network that controls the forward driver gate 1600 (shown in Fig. 16), which is represented as the collective functionality of Str (striatum), GP (globus pallidus) and Th (thalamus). The connections from Cx to Cx over the top of Fig. 17 are feedforward information maximization connections between registered IMA units that are also connected to Cx and are not directly relevant to the forward driver gate 1600. For example, if a spike-generating unit, such as Cx-Sensory in a behavioral network, passes a feedforward signal to another Cx unit, such as Cx-Limbic, the signal from Cx-Sensory must pass through Th and then to Cx-Limbic. Nevertheless, Th is under inhibitory control of GP, which is under inhibitory control of Str, which receives the same inputs from Cx-Sensory. Therefore, Str undergoes a form of STDP learning, with STDP receiving inputs from different regions of Cx. Whether the signals successfully pass through Th depends on the effect of Str on Th and whether the Th gate is either open or closed.For example, if GP always inhibits Th, Str can influence GP to trigger short bursts of spikes that pause GP. When GP is paused, its inhibition of Th is also paused, opening a window through which any spike can travel from Cx-Sensory to Cx-Limbic. This function of GP is under the control of a type of Str neuron, referred to herein as DIRECT.

[0078] There is also a competing gate (not shown in Fig. 17), which tries to close GP after it has been opened. Therefore, it can be assumed that GP can exist in two stages, which for simplicity are called GPe and GPi (shown in Fig. 18). Therefore, if Str inhibits GP, there is a way to effectively increase the rate of GP instead of decreasing it through another type of Str neuron, referred to herein as INDIRECT. The interplay of modulatory inputs, large loop STDP, and DIRECT and INDIRECT is represented by the four graphs at the bottom of Fig. 17 and the diagrams in the Fig. 18, Fig. 19 and Fig. 20. A possible set of GABA / DOPA criteria for modifying the large loop STDP gating function is shown by the four diagrams at the bottom of Fig. 17. Other criteria are possible, provided that over time the system operates in such a way that the sum of all phase shifts of the gating units is zero. For a direct or indirect forward gating unit, the large loop STDP gating function is modified by selecting one of the four different STDP diagrams according to whether the GABA modulation is high or low, whether DOPA at D1 units is decreasing or increasing, or whether DOPA at D2 units is decreasing or increasing. For an Str gate, the four diagrams at the bottom of Fig. 17 What happens when the state changes from no DOPA to DOPA (shown by the directed arrow labeled DOPA) for the DIRECT and INDIRECT states. The DIRECT path is different from the INDIRECT path with respect to the shapes of the large loop STDP function. The DIRECT STDP function is essentially symmetric about the x-axis. The INDIRECT STDP function is essentially asymmetric about the x-axis.

[0079] In relation to Fig. Figure 18 shows a group of related diagrams 1802, 1804, and 1806, which together convey the concept of periodically patterned inputs. Diagram 1802 is the central diagram and can be described as a circular timing diagram, showing how a large loop is repeatedly propagated by spiking patterns in behavioral areas. Diagram 1802 includes two ring-shaped openings, designated GPe and GPi. GPi inhibits Th and will either open or close the forward driver gate 1600 (shown in Fig. 16). An open window (uninhibited) for GPi is shown by region 1808 and a closed window with high inhibition for GPi is shown by region 1810.

[0080] Therefore, Str can be considered to be composed of two parts: a direct Str gate and an INDIRECT Str gate. The indirect Str gate inhibits GPe, which disinhibits GPi, resulting in a closed gate state (e.g., region 1810). The direct Str gate directly inhibits GPi, resulting in an open gate state (e.g., region 1808). Both types of Str gates (indirect and direct) have a short-term effect on gating in Th. Indirect closes the Th gate, and direct opens the Th gate. Diagrams 1802, 1802A, 1802B, 1804, 1804A, 1804B, 1806, 1806A, 1806B are shown in the Fig. 18, Fig. 19 and Fig. 20 represent different conditions for the INDIRECT and DIRECT str gates. The independent control of negative (inhibitory) open and closed gating phases is controlled by these two pathways as follows. Depending on an additional modulator (e.g., DOPA) and a local competitor (e.g., GABA), phase shifts for "local winner" units in the presence of the modulator are now described. Note that the competition is determined by lateral inhibition between units and is known from the prior art, e.g., as the Kohonen network. Phase shifts are progressive for dSPN units and constant for iSPN units, effectively temporally separating the open and closed gate windows and ensuring that the gating window remains open when activated.When the modulator is withdrawn, the dSPN phase remains constant while the iSPN phase advances. In this way, the "gate closed" window advances to obscure the "gate open" window, and the loop signal is terminated. For "losers" of the local competition in the presence of the modulator, the phase shifts for dSPN and iSPN are delayed. In the absence of the modulator, the phase shifts are zero for all "losers." Note that the concepts of "competition," "winner," and "loser" are used herein as one embodiment of the network. Other "winnerless" spiking networks, as described in the prior art, can be used, where the relative level of inhibition can still provide the necessary modulation categories to determine phase shifts and their direction.Furthermore, patterns of lateral connections do not have to follow those of a Kohonen network and can, for example, be short.

[0081] The Fig. 21 and Fig. Figure 22 shows diagrams 2102, 2102A, 2104, 2104A, 2106, 2106A, illustrating the source of modulatory inputs for forward-gated STDP in accordance with one or more embodiments of the present disclosure. The source of modulatory inputs for forward-gated STDP is driven positively and rarely by external measurements. In the closed-loop system of the present disclosure, the drivers are negatively driven by large populations of dSPN and iSPN spikes. When this inhibition wears off, the modulator signal is present and strong (rebound spiking).

[0082] This induces the system to perform gating functions that maximize the entropy of dSPN and iSPN spiking activity over time (e.g., maximizing rapid orthogonal shifts in their gating vectors), as well as the entropy of transitions between large-loop dSPN and iSPN controls via their gating functions. In response to inhibition (e.g., integration times of activation, inactivation, unit compartmentalization, etc.), the controls further induce the system to perform gating functions that change their gating vectors at specific intervals (e.g., every few hundred milliseconds) according to the temporal conditions imposed by these controls.

[0083] Therefore, it can be seen from the foregoing detailed description and the accompanying drawings that the present disclosure is directed to modeling biological neural systems using computer systems implementing a type of pulsed artificial neural network, which in the present disclosure is an information-based exchange (IBEX) network with temporal encoders and informative loop signals. The technical advantages of the present disclosure include an architecture that does not require imposed value functions for closed-loop functions. In addition, relatively fast learning results are obtained from loop propagation and the discovery of gating functions that maximize the entropy and diversity of all propagations over information-maximizing regions.Behavioral and network dynamics are deployed as a coding schema and as a means of determining code usefulness. Categories emerge from unsupervised learning of high-entropic loop transitions that terminate other loops (elimination of threats). Open-loop control problems are linked to an internal set of closed-loop selection mechanisms. State estimation components become more robust and undergo faster reorganization if the conditions for loop selection change. Multiple latencies for behavior initiation and termination are available due to extremely large numbers of loop states and the availability of specific states that arise not from external behavior but only from restricting state transitions to certain behavioral outputs and not others.

[0084] Referring to Fig.23, a computer program product 2300 in accordance with one embodiment including a computer-readable storage medium 2302 and program instructions 2304 is generally shown.

[0085] The present disclosure may be implemented as a system, a method, or a computer program product. Accordingly, the present invention may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions for carrying out aspects of the disclosure.

[0086] The computer-readable storage medium may be a tangible device capable of securing and storing instructions used by an instruction-executing device. The computer-readable storage medium may include, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.A partial list of more specific examples of computer-readable storage media includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CDROM), a digital versatile disc (DVD), a memory stick, a floppy disc, a mechanically encoded device such as punch cards or raised structures in a groove in which instructions are stored, or any suitable combination of the foregoing.A computer-readable storage medium, as used herein, cannot be construed as a transmitted signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves transmitted through a waveguide or other transmission medium (for example, light pulses transmitted through an optical fiber cable), or electrical signals transmitted through a cable.

[0087] Computer-readable program instructions as described herein can be downloaded to corresponding processing / computing devices from a computer-readable storage medium or an external computer or external storage device over a network, for example, the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each processing / computing device receives computer-readable program instructions from the network and forwards them to memory in a computer-readable storage medium within the corresponding processing / computing device.

[0088] Computer-readable program instructions for performing operations of the present disclosure may include build instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code in a combination of one or more programming languages, for example, an object-oriented programming language such as Smalltalk, C++, or similar, or conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user computer, partially on the user computer as a stand-alone software package, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server.In the latter case, the remote computer may be connected to the user computer through any network, where the network may be a local area network (LAN) or a wide area network (WAN), or the connection may be established in such an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, the electronic circuits may include, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLA), which execute the computer-readable program instructions by using the state information of the computer-readable program instructions to personalize the electronic circuit to carry out aspects of the present disclosure.

[0089] Aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It should be understood that each block of a flowchart and / or block diagram, and each combination of blocks in the flowcharts and / or block diagrams, may be implemented as computer-readable program instructions.

[0090] These computer-readable program instructions may be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing operations as depicted in the flowchart and / or block diagram block or blocks.These computer-readable instructions may further be stored on a computer-readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable storage medium containing instructions comprises an article of manufacture that includes instructions that perform aspects of the function specified in the flowchart and / or the block diagram block or blocks.

[0091] The computer-readable program instructions may further be loaded into a computer, other programmable data processing apparatus, or other device capable of performing a series of operational steps on a computer, programmable apparatus, or device for producing a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device implement the functions specified in the flowchart and / or block diagram block or blocks.

[0092] The flowchart and block diagrams in the figures illustrate the architecture, functionality, and execution of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this sense, each block in the flowchart or block diagrams may represent a module, a segment, or a portion of instructions comprising one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the described functions may occur in a different order than described in the figures. For example, two blocks depicted consecutively may be executed concurrently or sometimes in reverse order, depending on the corresponding functionality.It should further be noted that each block of the block diagram and / or flowchart illustrations may be implemented by special purpose hardware-based systems that perform the specified functions or combinations of special purpose hardware and computer instructions.

[0093] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms, unless the context indicates otherwise. It is further understood that the term "comprising" refers to the presence of stated features, integers, steps, operations, elements, and / or components when used in the description, but does not preclude the presence of multiple stated features, integers, steps, operations, elements, and / or components.

Claims

[1] Computer network with paths, where the network includes: a plurality of units configured to communicate via the paths, wherein the network is configured to identify informative loop signals in loops formed from a plurality of network paths connecting a first of the plurality of units to a second of the plurality of units, wherein the network is further configured to apply spike timing dependent plasticity (STDP)-dependent inhibitory gating to the plurality of network paths, wherein the network is further configured to open and close gates in the loop in a phase-shifted manner by applying STDP functions to open gate outputs and closed gate outputs, and wherein the network is further configured to make at least a rate or a direction of the phase shift dependent on a regulation signal, wherein the regulatory signal is based at least in part on a change in STDP-dependent inhibitory gating. [2] The network of claim 1, wherein the units comprise temporal encoders. [3] The network of claim 2, wherein the temporal encoders selectively select behavior by coupling the temporal encoders to a selection of external effectors. [4] The network of claim 2, wherein the network is further configured to apply information maximization to the temporal encoders to regulate spike rates. [5] The network of claim 4, wherein an input to the information maximization comprises at least one of the following: • data external to the network; • Feed-forward data resulting from information maximization; • Data calculated from a spike that has undergone STDP-dependent inhibitory gating; and • Data calculated from a spike that has passed through a feedback connection. [6] The network of claim 2, wherein the network is further configured to select spike timings of the temporal encoder as a function of a distribution of feed-forward and feedback spike times under control of the STDP. [7] The network of claim 1, wherein the regulation signal is based at least in part on an external measurement value. [8] A network according to claim 1, wherein a sum of the phase shifts over time is zero. [9] The network of claim 1, wherein the change in the gating signals comprises an orthogonal change in a vector representation of the gating signal. [10] A network according to claim 1, wherein the change is made after a predetermined time interval controlled by the effector output of the network. [11] A computer-implemented method for implementing a network with paths, the method comprising: • Communicating between a plurality of units through the paths through a processor, • Identifying informative loop signals in loops formed from a plurality of network paths connecting a first of the plurality of units to a second of the plurality of units by the processor, • Applying spike-timing-dependent plasticity (STDP)-dependent inhibitory gating to the multitude of network pathways, • phase-shifting opening of gates and closing of gates in the loop by applying STDP functions to open gate outputs and closed gate outputs, and • making at least one rate or one direction of the phase shift dependent on a regulatory signal, wherein the regulatory signal is based at least in part on a change in STDP-dependent inhibitory gating. [12] The method of claim 11, wherein the units comprise temporal encoders. [13] The method of claim 12, wherein the temporal encoders make a behavioral selection by coupling the temporal encoders to a plurality of external effectors. [14] The method of claim 12, further comprising applying information maximization to regulate spike rates to the temporal encoders. [15] The method of claim 12, further comprising selecting spike timings of the temporal encoders as a function of a distribution of feed-forward and feedback spike times under control of STDP. [16] The method of claim 11, wherein the regulation signal is based at least in part on an external measurement value. [17] The method of claim 11, wherein the change is performed after a predetermined time interval under the control of network effectors. [18] A computer program product for implementing a network with paths, the computer program product comprising: a computer-readable storage medium having program instructions stored thereon, the computer-readable storage medium not being a transmitting signal per se, the program instructions being readable by a processor circuit to cause the processor circuit to execute a method, the method comprising: • Communication between a multitude of units through the paths, • Identification of informative loop signals in loops formed by a plurality of network paths connecting a first of a plurality of units to a second of the plurality of units, • Applying spike-timing-dependent plasticity (STDP)-dependent inhibitory gating to the multitude of network pathways, • phase-shifting opening and closing of gates in the loop by applying STDP functions to open gate outputs and closed gate outputs, and • making at least one of a rate and a direction of the phase shift dependent on the regulatory signal, wherein the regulatory signal is based at least in part on a change in STDP inhibiting gating. [19] The computer program product of claim 18, wherein the units comprise temporal encoders. [20] The computer program product of claim 19, wherein the temporal encoders make a behavioral selection by coupling the temporal encoders to a plurality of external effectors. [21] The computer program product of claim 19, further comprising applying information maximization to regulate spike rates to the temporal encoders. [22] The computer program product of claim 19, further comprising selecting spike timings of the temporal encoders as a function of a distribution of feed-forward and feedback spike times under control of STDP. [23] A computer program product according to claim 18, wherein the change is performed after a predetermined time interval under the control of network effectors.

Images