Methods, devices, and systems for generating Bayesian inferences using spiking neural networks

Abstract

Techniques and mechanisms for performing Bayesian inference using a spiking neural network. In one embodiment, a parent node of a spiking neural network receives a periodic first bias signal. The parent node transmits a likelihood signal to a child node, wherein the parent node and the child node correspond to a first condition and a second condition, respectively. The likelihood signal indicates a probability of the first condition based on a phase change applied to the first bias signal. The child node also receives a signal indicating an instance of the second condition; based on the indication and the second bias signal, the child node signals the parent node to adjust the phase change applied to the first bias signal. After the adjustment, the likelihood signal indicates an updated probability of the first condition.

Description

Background Art

[0001]

[0014] Embodiments described herein relate generally to spiking neural networks, and more particularly, but not exclusively, to techniques for performing Bayesian inference operations using spiking neural networks.

[0002] Spiking neural networks (or "SNNs") are increasingly being adapted to provide next-generation solutions for a variety of applications. SNNs rely in various ways on signaling techniques that use time-based relationships between signal spikes to convey information. Compared to typical deep learning architectures, such as those provided with convolutional neural networks (CNNs) or recurrent neural networks (RNNs), SNNs offer communication economies that, in turn, allow for orders of magnitude improvements in power efficiency.

[0003] While neural networks are generally effective for applications such as image recognition, problems have arisen when attempting to adapt such neural networks for higher-order analytical reasoning, such as Bayesian inference processing. Bayesian networks are straightforward in principle, but are difficult to implement in practice due to the exponential relationship between the number of terms in a given problem and the circuitry and / or computational load required to solve that problem. To address these difficulties, existing techniques variously require auxiliary nodes, non-local computations, or design support circuits for additional purposes. As the size, diversity, and proliferation of artificial intelligence networks grow, incremental improvements in solutions that provide higher-order analysis are expected to have increasing added value. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Embodiments of the invention are illustrated by way of example and not limitation in the figures of the accompanying drawings in which:

[0005] Figure 1 Shown are diagrams each illustrating features of a simplified neural network according to an embodiment.

[0006] Figure 2A is a functional block diagram illustrating elements of a spiking neural network according to an embodiment.

[0007] Figure 2B is a functional block diagram illustrating elements of a neural network node according to an embodiment.

[0008] Figure 3 is a flow chart illustrating features of a method for operating a spiking neural network, according to an embodiment.

[0009] Figure 4 is a functional block diagram illustrating features of a spiking neural network for performing Bayesian inference operations, according to an embodiment.

[0010] Figure 5Timing diagrams illustrating in various ways signals generated using a spiking neural network according to an embodiment are shown.

[0011] Figure 6 is a functional block diagram illustrating a computing device according to one embodiment.

[0012] Figure 7 is a functional block diagram illustrating an exemplary computer system according to one embodiment. DETAILED DESCRIPTION

[0013] The embodiments discussed herein provide, in various ways, techniques and mechanisms for determining conditional likelihood information using spiking neural networks, wherein the determined conditional likelihoods represent corresponding probabilities of the Bayesian network used to generate Bayesian inferences. Various embodiments of neural graphical models associated with spiking neural networks are based on interactions between neural oscillations. Each neuron represents a corresponding variable of interest, and their synaptic connections represent the conditional dependency structure between these variables. Spiking activity oscillates, and the amplitude of the oscillation represents the corresponding probability. When (a subset of) random variables are observed at some of the leaf nodes of the graph, evidence propagates from the leaf nodes to the central node as an equivalent phase shift; the combination of the propagating wave and the phase shift modulates the oscillations of the neurons throughout the graph, and thereby modulates the corresponding probabilities of the Bayesian network to generate Bayesian inferences. Multiple nodes of the spiking neural network can each represent or otherwise correspond to corresponding conditions, including, for example, conditions local to the spiking neural network or conditions of the system to be evaluated by the spiking neural network. For a given node, the corresponding condition can result in a signal being received by (or transmitted from) that node. For example, a given node may be configured to receive a signal and identify a predefined pattern and / or timing of the signal as indicative of an instance of a condition to which that node corresponds. In some embodiments, a spiking neural network is used to perform one or more Bayesian inference operations—e.g., where some or all of the plurality of nodes each correspond to a respective condition (or “variable”) of a Bayesian network. As used herein, “Bayesian inference” and “Bayesian inference operation” refer in various ways to the process of changing values, signal characteristics, and / or other system states to indicate updated probabilities of conditions of a Bayesian network. Such updated probabilities may be used when the Bayesian network subsequently selects a condition over one or more other such conditions—e.g., where the Bayesian inference operation further includes such selection.

[0014] A plurality of nodes of a spiking neural network may each receive a respective bias signal, at least some component of which is sinusoidal or otherwise varies periodically with time. Each of such plurality of nodes may receive the same bias signal, or alternatively, some or all of the plurality of nodes may each receive a respective different bias signal. In an embodiment, the periodic components of some or all of such bias signals each have the same frequency. Based on such biases, the plurality of nodes may exhibit resonance relative to each other in various ways—e.g., in terms of their respective membrane potentials and / or synaptic signaling. Such resonance between nodes may be based on the coupling of the nodes to each other and further based on common frequency components of one or more bias signals provided to the nodes in various ways.

[0015] The respective node types of two given nodes - referred to herein as "parent node" types and "child node" types - may be based at least in part on the functional relationship between the nodes. For a given first node and second node, the first node may act as a parent node with respect to the second node, with the second node (in turn) acting as a child node with respect to the first node. The parent node represents an event, entity, and / or state that is causally prior (with a given probability) to the entity or event represented by the respective child node. In some embodiments, a given node is a child node with respect to one node and a parent node with respect to another node. Alternatively or additionally, a node may be a parent node for each of a plurality of respective child nodes, and / or a child node may be a child node for each of a plurality of respective parent nodes. During operation of a spiking neural network, a parent node may send a signal, referred to herein as a "likelihood signal," to a respective child node. Alternatively or additionally, a child node may send another signal, referred to herein as a "modulation signal," to a parent node.

[0016] As used herein, a "likelihood signal" refers to a signal passed from a parent node to a child node, wherein a characteristic of the signal indicates the likelihood (e.g., current probability) of a condition corresponding to the parent node. Such signal characteristics may include, for example, the rate (e.g., frequency), amplitude, predefined pattern, and / or timing (e.g., relative to a reference time) of at least some signal spikes or other components of the likelihood signal. For example, a node may provide a likelihood signal that includes a periodic component during at least some time periods, the periodic component being based at least in part on the periodic characteristics of the bias signal of that node. The periodic component may be further based on one or more other likelihood signals received by that node from the corresponding parent node (each of the one or more other likelihood signals including a corresponding periodic component). In one embodiment, such a "rate-amplitude" characteristic of the likelihood signal indicates the likelihood of the characteristic corresponding to that node.

[0017] As used herein, "rate-amplitude" refers to a characteristic of a signal having a certain rate of spikes (or spike rate) that varies during at least some time period. The spike rate can be a moving average rate based on the number of signal spikes that occur within the time period of a sampling window. The rate of spikes can include a periodic component that oscillates or otherwise varies between the rate of a first signal spike and a second signal spike. For a given time period, the rate-amplitude of such a signal is the magnitude of the variation caused by the spike rate of the signal.

[0018] As used in the text, "adjustment signal" refers to a signal passed from a child node to a parent node for indicating that the parent node will adjust the phase of the bias signal provided to that parent node. The parent node can be configured to receive the adjustment signal and realize that a predefined spike pattern, timing and / or other characteristics of the adjustment signal are used to specify or otherwise indicate the adjustment of the value of the phase change applied to the bias signal. Such adjustment signal can be based at least in part on the current probability of the condition corresponding to the child node. This probability can be indicated by the characteristics of the membrane potential at the child node (such as rate-amplitude) - for example, where this characteristic is based on the phase change (if any) of the bias signal applied to the child node. In some embodiments, given a second condition, the adjustment signal is further based at least in part on the conditional probability of the first condition (wherein the first condition and the second condition correspond to the child node and the parent node, respectively). The conditional probability can be indicated by the weight assigned to the synapse through which the child node receives the likelihood signal. The adjustment signal can be passed from the child node to the parent node based on the child node receiving the corresponding "instance signal".

[0019] As used herein, an "instance signal" refers to a signal provided to a node to indicate an instance of a condition to which the node corresponds. For example, an instance signal may be received by a node from a source external to the spiking neural network. Alternatively, an instance signal may be generated based on the operation of one or more other nodes of the spiking neural network. In response to the instance signal, the receiving node may each transmit one or more conditioning signals to a corresponding parent node.

[0020] The techniques described herein can be implemented in one or more electronic devices. Non-limiting examples of electronic devices that can utilize the techniques described herein include any type of mobile and / or fixed device, such as a camera, a cellular telephone, a computer terminal, a desktop computer, an e-reader, a fax machine, a kiosk, a laptop computer, a netbook computer, a notebook computer, an Internet appliance, a payment terminal, a personal digital assistant, a media player and / or recorder, a server (e.g., a blade server, a rack-mounted server, a combination thereof, etc.), a set-top box, a smartphone, a tablet personal computer, an ultra-mobile personal computer, a landline telephone, a combination thereof, etc. More generally, the techniques described herein can be employed in any of a variety of electronic devices that include a spiking neural network.

[0021] In some embodiments, the nodes of the spiking neural network can be of the leaky integrate and fire (LIF) type, where, for example, based on one or more spike signals received at a given node j, the membrane potential v of that node j is m The value of can reach a peak and then decay over time. m The peak and decay behavior can be described for example according to the following formula:

[0022]

[0023] where v 静息 is the membrane potential v m The resting potential to be achieved, τ m is for the membrane potential v m The time constant of the exponential decay, w ij is the synaptic weight of the synapse from another node i to node j, I ij is the spike signal (or “spike train”) delivered to node j via the synapse, and J b is a value based on, for example, a bias current or other signal supplied to node j from some external node / source. A spiking neural network can be based on a predefined threshold voltage V 阈值 operation, where node j is configured to respond to its membrane potential v m Greater than V 阈值 The output signal spikes.

[0024] Figure 1An example diagram illustrating a simplified neural network 100 is provided, which provides an illustration of connections 120 between a first set of nodes 110 (e.g., neurons) and a second set of nodes 130 (e.g., neurons). Some or all of a neural network, such as the simplified neural network 100, can be organized into multiple layers—for example, including an input layer and an output layer. It will be appreciated that the simplified neural network 100 depicts only two layers and a small number of nodes, but other forms of neural networks can include a large number of nodes, layers, connections, and paths configured in various ways. For example, a neural network can be monolithic, with a single layer, having an arbitrary connectivity graph representing the strength of causal relationships between corresponding conditions represented in various ways by the nodes of the layer.

[0025] Data provided to the neural network 100 may first be processed by the synapses of the input neurons. The interaction between the input, the synapses of the neurons, and the neurons themselves determines whether the output is provided to the synapses of another neuron via the axon. Modeling of synapses, neurons, axons, etc. can be accomplished in various ways. In an example, neuromorphic hardware includes multiple separate processing elements (e.g., neural nuclei) in synthetic neurons and a messaging structure for passing outputs to other neurons. Determining whether a particular neuron "fires" to provide data to further connected neurons depends on the activation function applied by the neuron and the synaptic connections (e.g., w) from neuron i (e.g., in a layer of the first set of nodes 110) to neuron j (e.g., in a layer of the second set of nodes 130). ij ). The input received by neuron i is depicted as the value x i , and the output produced from neuron j is depicted as the value y j Thus, the processing performed in a neural network is based on weighted connections, thresholds, and evaluations performed between neurons, synapses, and other elements of the neural network.

[0026] In an example, the neural network 100 is built from a network of spiking neural network cores that communicate via short packetized spike messages sent from core to core. For example, each neural network core may implement a certain number of primitive nonlinear time-domain computational elements as neurons, such that when the activation of a neuron exceeds a certain threshold level, the neuron generates a spike message that is propagated to a fixed set of fan-out neurons contained in the destination core. The network may distribute the spike message to all destination neurons, and in response, those neurons update their activations in a transient, time-dependent manner, similar to the operation of real biological neurons.

[0027] The neural network 100 further illustrates receiving a value x at a neuron i in the first set of neurons (eg, a neuron in the first set of nodes 110). iThe output of the neural network 100 is also shown as represented by the value y j , which reaches a neuron j in the second set of neurons (e.g., a neuron in the first set of nodes 110) via the path established by connection 120. In a spiking neural network, all communication occurs over event-driven action potentials or spikes. In this example, spikes convey no information other than the spike time and the source and destination neuron pairs. Computation can occur in various ways in each corresponding neuron as a result of dynamic nonlinear integration of weighted spike inputs using real-valued state variables. The time domain sequence of spikes generated by or for a particular neuron can be referred to as the "spike train" of that particular neuron.

[0028] In the example of a spiking neural network, the activation function occurs via a sequence of spikes, which means that time is a factor that must be taken into account. In addition, in a spiking neural network, each neuron can provide functionality similar to that of a biological neuron, because the artificial neuron receives its inputs via synaptic connections to one or more "dendrites" (part of the physical structure of a biological neuron), and these inputs affect the internal membrane potential of the artificial neuron's "soma" (cell body). In a spiking neural network, when the membrane potential of an artificial neuron crosses an excitation threshold, the artificial neuron "fires" (e.g., produces an output spike). Therefore, the effect of the input on a spiking neural network neuron acts to increase or decrease the internal membrane potential of the neuron, making the neuron more or less likely to fire. In addition, in a spiking neural network, the input connections can be excitatory or inhibitory. The membrane potential of a neuron can also be affected by changes in the internal state ("leakage") of the neuron itself.

[0029] Figure 1 Also illustrated is an example inference path 140 in a spiking neural network, such as may be implemented in the form of neural network 100 or other forms of neural networks. The inference path 140 of a neuron includes a presynaptic neuron 142 configured to generate a presynaptic spike sequence x representing a spiking input. i A spike train is a temporal sequence of discrete spike events that provides a set of times at which a given neuron fired.

[0030] As shown in the figure, the spike sequence x i The spike train x is generated by the neuron preceding the synapse (e.g., neuron 142), and is evaluated based on the properties of synapse 144. i For example, a synapse may apply one or more weights, such as weight w ij , these weights are used to evaluate the spike train x i The data from the spike train x i An input spike enters a synapse such as , with weight wij The weight scales the effect of the presynaptic spike on the postsynaptic neuron (e.g., neuron 146). If the integrated contribution of all input connections to the postsynaptic neuron exceeds a threshold, the postsynaptic neuron 146 will fire and generate a spike. As shown in the figure, y j is the postsynaptic spike sequence generated by the neuron following the synapse (e.g., neuron 146) in response to a certain number of input connections. As shown in the figure, the postsynaptic spike sequence y j Distributed from neuron 146 to other postsynaptic neurons.

[0031] Figure 2A Features of a spiking neural network 200 for implementing Bayesian inference operations are shown, according to an embodiment. Figure 2A Also included is a legend 201 showing corresponding symbols for network nodes, bias signals, likelihood signals, instance signals, and adjustment signals. Spiking neural network 200 is an example of an embodiment in which a parent node passes a likelihood signal to a corresponding child node, which then signals back to the parent node that a phase adjustment is to be made to the bias current. Spiking neural network 200 may include, for example, some or all of the features of neural network 100.

[0032] like Figure 2A As shown in FIG, a spiking neural network 200 includes a plurality of nodes coupled to each other via synapses in various manners, each of the plurality of nodes being configured to receive a respective bias signal, which may be sinusoidal or otherwise periodic. By way of illustration and not limitation, nodes 210, 220, 230 of network 200 can receive respective bias signals 212, 222, 232 in various manners. Based at least in part on such bias signals, the plurality of nodes can exhibit resonant coupling in various manners, for example, with respect to at least synaptic signaling and / or respective membrane voltages of the plurality of nodes. For example, the respective signals at nodes 210, 220, 230 can each exhibit at least some resonance at a first frequency. Such resonant coupling can facilitate representing probabilities by applying phase changes to the bias signals.

[0033] In the illustrated example embodiment, nodes 210 and 230 are configured to act as a parent node and a child node, respectively, with respect to each other. Node 210 is further coupled to receive one or more likelihood signals 224, each from a corresponding other node (such as node 220) of a spiking neural network—e.g., where node 210 is also configured to act as a child node for each such other node. Node 210 may apply a change to the phase of bias signal 212, wherein the phase change (and / or a signal based on the phase change) indicates the probability of the condition corresponding to node 210. In such an embodiment, node 210 generates likelihood signal 218 based on bias signal 212 (and any phase change applied thereto) and further based on one or more likelihood signals 224. The one or more operations for generating the signal spikes of likelihood signal 218 may be adapted, for example, from conventional leaky integrate and fire (LIF) neuron technology. The periodic component of likelihood signal 218 may be based on both the phase-adjusted bias signal 212 and the resonant coupling of multiple nodes to each other. Characteristics of such periodic components (eg, including rate-amplitude) may represent or otherwise indicate the probability of the condition corresponding to node 210. Likelihood signal 218 may be passed to one or more child nodes, including node 230 in this example.

[0034] Node 230 may be configured to indicate, using an adjustment signal 234, the adjustment to be made to the phase change applied to bias signal 212 by node 210, if any. The membrane voltage of node 230 may be generated based on bias signal 232—for example, where the periodic component of the membrane voltage is based on the phase change, if any, applied to bias signal 232 by node 230. In embodiments where node 230 also serves as a parent node with respect to some other node or nodes, such phase change applied to bias signal 232 may be based on one or more other adjustment signals (not shown) each received by node 230 from the one or more other nodes.

[0035] Node 230 may further be coupled to receive an example signal 236 indicating an example of a condition to which node 230 corresponds. Example signal 236 may be provided, for example, from a sensor, test unit, or other source external to spiking neural network 200 (or alternatively, from another node of spiking neural network 200). Some embodiments are not limited with respect to the particular source from which example signal 236 is received. Based on the indication provided by example signal 236, node 230 may indicate (via adjustment signal 234) an amount of phase change to adjust bias signal 212. Based on adjustment signal 234, node 210 may adjust the phase change of bias signal 212. Such adjustments may include determining whether to apply a change to the phase of bias signal 212, determining the amount of phase change to change, and / or determining whether to adjust the amount of phase change. Such adjustments to the phase change at node 210 may cause likelihood signal 218 to indicate a different value for the probability of the condition to which node 210 corresponds. As a result, spiking neural network 200 may implement a Bayesian inference operation that changes the corresponding value of one or more probabilities.

[0036] Although some embodiments are not limited in this respect, node 210 may be further coupled to receive one or more other adjustment signals (not shown), each from a corresponding other child node—for example, where likelihood signal 218 is also passed to each other such child node. In such embodiments, the phase change applied to bias signal 212 may be further adjusted based on one or more such other adjustment signals.

[0037] Figure 2B 1 shows features of spiking neural network nodes 240, 250 each participating in a Bayesian inference operation according to an embodiment. Figure 2B Nodes 240 and 250 may include, for example, corresponding features of nodes 210 and 230.

[0038] Nodes 240 and 250 are configured to act as parent and child nodes, respectively, with respect to each other's operations. Nodes 240 and 250 may correspond to condition A and condition B, respectively, with condition B having at least some dependency on condition A. Conditions A and B may be corresponding parameters of a Bayesian network implemented using a spiking neural network.

[0039] Such a spiking neural network can be initialized according to a Bayesian network design that defines, for each given node in a plurality of nodes, an initial probability of a condition to which that given node will correspond. Such a probability can be indicated at least in part based on a phase change to be applied to a corresponding bias current at a given node. For each child node in a plurality of nodes, the Bayesian network design can define a corresponding conditional probability of the condition to which that child node corresponds given another condition to which the corresponding parent node of that child node corresponds. Such conditional probabilities can be indicated, for example, by weights assigned to synapses through which the child node receives likelihood signals. The specific probability value for a given initialization state can be determined based on conventional Bayesian network design techniques and is not limited to some embodiments.

[0040] Reference node 240 may receive a periodic bias signal J at block 242. a , the block 242 changes the phase φ a The periodic bias signal J is applied to a At a given moment, the phase change φ a The amount of adjustment may be based on one or more adjustment signals (such as the illustrative adjustment signal C shown) each passed from a corresponding child node to node 240. ba ). Bias signal J a The phase-changed version of L may be provided to another block 246, which is based on, for example, one or more likelihood signals each from a corresponding other node (not shown). By way of illustration and not limitation, node 240 may receive a likelihood signal L from another node that acts as a parent node with respect to node 240. x1a (Assigned to node 240 through which it receives L x1a The synaptic weight w x1a 244 can be applied by node 240 to L x1a .

[0041] Block 246 can convert the membrane voltage V m-a Generates the bias signal J a The phase change φ applied to it a , and the likelihood signal L x1a (As through w x1a Weighted) function f a Function f a Some features of may be adapted, for example, from any of a variety of leaky integrate and fire (LIF) techniques for generating spike signals—for example, where the function f a Including the features of equation (1) described herein. Another block 248 of node 240 can be based on the membrane voltage V m-a Generate likelihood signal L a .

[0042] For example, block 248 may be responsive to V m-a Crossing or otherwise reaching V th-a The likelihood signal L is generated a One or more peaks. Likelihood signal L a The rate-magnitude and / or other characteristics of L may indicate the current probability of condition A to which node 240 corresponds. a may be sent to node 250 (and, in some embodiments, to one or more other child nodes of node 240). Node 250 may then adjust the signal C ba Delivered to node 240, the regulated signal C ba Indicates the phase change φ to be adjusted a The phase change φ a Such adjustments can result in a likelihood signal L a The updated value of the probability of condition A is subsequently indicated.

[0043] For example, node 254 may convert the synaptic weight w ba 254 is applied to the likelihood signal L a Additionally, node 250 may receive a bias signal (not shown) such as bias signal 232 and change the phase by φ b 252 is applied to the bias signal. In such an embodiment, the adjustment signal C ba The phase change φ may be based at least in part on b 252 and synaptic weight w ba 254 is generated—for example, where block 256 is based on the synaptic weight w ba 254 and phase change φ b The corresponding probability indicated by 252 determines the scaling factor β ba Note that in some embodiments, the adjustment signal C ba Independent of the likelihood signal L a .

[0044] Synaptic weight w ba The value of 254 may represent or otherwise indicate the conditional probability P(B|A) of condition B given condition A. Alternatively or additionally, the phase change φ b 252 may indicate the probability P(B) of condition B. Initial values ​​for such probabilities P(B) and P(B|A) may be predefined as part of the initialization state of the spiking neural network. In the event that node 250 is also a parent node to one or more other nodes, P(B) may be updated subsequently during operation of the spiking neural network—for example, where the conditional probability P(B|A) is a fixed value (e.g., the corresponding synaptic weight w ba254). Any such update of P(B) may include using the same parameters as used to update the phase change φ a The phase change φ is adjusted by similar techniques to those of b The amount is 252.

[0045] In one embodiment, block 256 applies the scaling factor β ba The value of is determined as a function of the ratio {P(B|A) / P(A)}—for example, according to the following function:

[0046] β ba =[{P(B|A) / P(A)}–1] (2)

[0047] In such embodiments, the scaling factor β ba Indicates whether the correlation (dependency) between condition A and condition B is positive or negative. Based on such correlation, adjust signal C ba Phase change φ can be signaled a To increase or decrease.

[0048] Block 256 may apply the scaling factor β ba The value of is passed to another block 258 of node 250 - for example, where block 258 is based on β ba Generates the regulation signal C ba For example, the adjustment signal C ba may indicate or otherwise be based on a product equal to [β ba ]·[Δφ ba ], where Δφ ba is related to the phase change φ a The phase adjustment value corresponding to both the given instance of condition B corresponding to node 250. ba The value of may correspond to the incremental amount by which P(A) is to change in response to a given instance of P(B). For example, Δφ ba The value of may be based on parameters initially defined as part of the Bayesian network design. In one embodiment, Δφ ba For example, based on the instance signal N indicating an instance of condition B b is variable. For example, at any given moment Δφ ba The value of may represent or otherwise indicate the example signal N b In other embodiments, Δφ ba The value of φ indicates or is otherwise based on the phase change a The maximum allowable value of the phase change φ a The range of permissible values ​​of , etc. Node 250 may be preconfigured to baA predefined pattern, timing, and / or other characteristic of the phase change φ is identified as indicative of the phase change to be adjusted. ba amount.

[0049] Accordingly, the node 240 may change the phase φ based on b 252 and weight w ba 254 to generate the adjustment signal C ba , where the weight w ba 254 indicates P(B|A), and wherein, based on the phase change φ b 252, the membrane voltage of node 240 (and / or the likelihood signal L from node 240) b (not shown)) indicates P(B). For example, the adjustment signal C ba Based on the scaling factor β ba and phase adjustment value Δφ ba Although some embodiments are not limited in this respect, node 240 may be further coupled to also receive likelihood signal L a Some other child node (not shown) of - for example, where the other child node corresponds to condition C.

[0050] In such embodiments, the other child node may include additional circuitry and / or execute software to adjust the bias signal based on the phase change φ applied to the bias signal by that child node. c And that child node applies the likelihood signal L a The weight w ca To generate another adjustment signal C in a similar manner ca . Weight w ca The probability P(C|A) may be indicated—for example, where the phase change φ c , the membrane voltage of that other child node (and / or the likelihood signal L from that other child node) c ) indicates P(C). Adjustment signal C ca Based on the scaling factor β ca and phase adjustment value Δφ ca , the phase adjustment value Δφ ca With a scaling factor of e.g. ba and phase adjustment value Δφ ba Block 242 may be coupled to further receive the adjustment signal C ca , wherein block 242 is used to adjust the signal C based on ca The indicated adjustment value is used to further adjust the phase change φ a Although some embodiments are not limited in this respect, the phase change φ may also be adjusted during operation of the spiking neural network. b 252 amount - for example, where such adjustment is based on the phase change φ aThose signals that are regulated exchange similar signals.

[0051] Phase change φ a can be limited to a certain predefined range of permissible values. Such phase changes φ a The limited range of φ may be implemented, for example, by node 210. In an embodiment, the phase change φ a An amount that is allowed to vary equal to or less than pi (π) radians—for example, where the phase changes φ a is always in the range of 0 to π (or, for example, in the range of -π / 2 to π / 2). In some embodiments, such phase changes are allowed to vary by an amount equal to or less than 0.8π radians—for example, where the amount is equal to or less than 0.7π. a A more limited range of (e.g., a range of 0.1π to 0.9π) can facilitate the control of phase changes φ a Multiple simultaneous adjustments (each such adjustment is based on a corresponding adjustment signal) are combined with the likelihood signal L a The corresponding changes in the indicated probabilities may be relatively more linear. As a result, simultaneous changes in probabilities (each in response to a respective different child node) may add more linearly to each other (or be more linearly offset from each other).

[0052] Figure 3 Features of a method 300 for operating a spiking neural network according to an embodiment are shown. The method 300 may be performed using one of the spiking neural networks 100, 200—for example, where the method 300 includes passing one or more signals between the node 240 and the node 250. To illustrate certain features of various embodiments, reference is made herein to Figure 4 4. However, such description can be extended to apply to any of a variety of different spiking neural networks configured to adjust a phase change to a bias current, where, based on the phase change, a likelihood signal or membrane voltage of a node indicates the probability of a condition to which that node corresponds.

[0053] Method 300 may include receiving (at 310) a first bias signal and a second bias signal at a first node and a second node of a spiking neural network, respectively. The first node may correspond to a first condition, wherein the second node corresponds to a second condition, the second condition having, for example, a dependency relationship with the first condition. Figure 4Spiking neural network 400 may be configured to perform Bayesian inference operations according to embodiments. Spiking neural network 400 may include, for example, some or all of the features of neural network 200. Spiking neural network 400 is an example of an embodiment in which a plurality of nodes are each configured to receive a respective bias signal, wherein synapses between respective ones of the plurality of nodes coupled in various manners each correspond to a respective dependency relationship of a Bayesian network. Resonant coupling of the plurality of nodes may facilitate applying phase changes to the bias signal as a mechanism for determining (e.g., including updating) the probability of a given variable.

[0054] exist Figure 4 In FIG. 4 , spiking neural network 400 includes nodes 410, 412, 414, 416, and 418, each corresponding to a corresponding condition (parameter) of a Bayesian network. As illustrated by the example embodiment, a Bayesian network may include conditions by which determinations about a house or other residential property may be made. For example, node 410 may correspond to condition G, "trash may be turned over"—e.g., where node 412 corresponds to condition D, "dog barking," and node 414 corresponds to condition W, "window broken." Node 416 may correspond to condition R, "raccoon on property"—e.g., where node 418 corresponds to condition B, "burglar on property." In such embodiments, nodes 410 and 412 may each be a corresponding child node of node 416—e.g., to reflect a dependency relationship, where condition R may cause (and be indicated by) one or both of condition G or condition D. Alternatively or additionally, nodes 412, 414 can each be a respective child node of node 418—e.g., to reflect a dependency relationship where condition B can cause (and be indicated by causing) one or both of condition D or condition W. Different parent-child node dependency relationships between respective ones of nodes 410, 412, 414, 416, 418 can be represented by respective synapses coupled in various ways to communicate likelihood signals 430, 432, 434, 436. In some embodiments, likelihood signals 430, 432 are equal to one another, and / or likelihood signals 434, 436 are equal to one another.

[0055] Although some embodiments are not limited in this respect, method 300 may further include selecting a condition of the Bayesian network in preference to one or more other conditions of the Bayesian network, wherein the selection is based on the adjustment at 360. For example, evaluation logic coupled to a spiking neural network (or alternatively, coupled to a portion of the spiking neural network) may receive likelihood signals, each from a corresponding different node of the spiking neural network. In such embodiments, the selection may include detecting whether one of the received likelihood signals satisfies one or more predefined test criteria. By way of illustration and not limitation, such test criteria (e.g., including a minimum threshold probability value, a minimum threshold difference between two probability values, etc.) may define when a condition corresponding to a given likelihood signal is considered sufficiently likely (either alone or in conjunction with the probability of one or more other conditions) to justify selection of that condition. In response to detecting that one of the received likelihood signals satisfies the test criteria, the evaluation logic may generate a signal indicating selection of the condition corresponding to the node providing the likelihood signal.

[0056] Reference again Figure 4 One or more additional synaptic signals (such as the illustrative signals 450 and 452 shown) can signal other nodes of network 400 (or signal evaluation circuitry external to network 400) whether an instance of a particular one of conditions R and B is indicated, at least at a certain threshold probability. Evaluation of signals 450 and 452 can detect whether the probability indicated by one of nodes 416 and 418 meets one or more test criteria. For example, such a test can detect whether one of P(R) or P(B) is above a certain minimum threshold probability value. Alternatively or additionally, such a test can detect whether one of the values ​​[P(R) − P(B)] or [P(B) − P(R)] is above a certain minimum threshold probability difference value. In response to such a detection, evaluation circuitry can output or otherwise generate a signal indicating the selection of one of nodes 416 and 418 (and, therefore, the selection of the condition corresponding to that node).

[0057] Referring again to method 300, receiving at 310 may include nodes 410, 412, 414, 416, 418 receiving respective bias signals J, each of which is sinusoidal or otherwise periodic. bg 、J bd 、J bw 、J br 、J bb For example, the bias signal J bg 、J bd 、J bw 、J br 、J bbEach bias signal in may have the same frequency ω. Method 300 may further include: (at 320) applying a change to the phase of the first bias signal. For example, the initial configuration of the spiking neural network 400 may include or otherwise be based on respective predefined initial values ​​of the probabilities P(G), P(D), P(W), P(R), and P(B) for the conditions G, D, W, R, and B. For example, P(R) may be indicated by the likelihood signal 430 or the rate-amplitude (or other characteristic) of the membrane voltage at node 416. The indication of P(R) may be based at least in part on the phase of the bias signal J to be applied by node 416. br Similarly, P(B) may be indicated by a characteristic of the membrane voltage at likelihood signal 430 or node 416—for example, where such an indication of P(B) is based at least in part on the bias signal J to be applied to node 418. bb In such embodiments, the indication of the membrane voltage P(G) at node 410 may be based on the bias signal J to be applied to node 410. bg Phase change - for example, where the membrane voltage at node 412 is indicative of P(D) based on the bias signal J to be applied to node 412 bd Similarly, the membrane voltage at node 414 may be indicative of the phase change of P(W) based on the bias signal J to be applied to node 414. bw phase change.

[0058] The initial configuration of spiking neural network 400 may further include or otherwise be based on conditional probabilities P(G|R), P(D|R), P(D|B), and P(W|B). For example, P(G|R) may be represented or otherwise indicated by the synaptic weight that node 410 applies to likelihood signal 430—e.g., where P(D|R) is indicated by the synaptic weight that node 412 applies to likelihood signal 432. Similarly, P(D|B) may be indicated by the synaptic weight that node 412 applies to likelihood signal 434—e.g., where P(W|B) is indicated by the synaptic weight that node 414 applies to likelihood signal 436. Some or all of these probabilities may be provided a priori as input parameters based on a predefined Bayesian network design. The operation of the spiking neural network may signal changes in given probabilities by adjusting phase changes to the bias signal based on one or more example signals.

[0059] During the initialization state of the spiking neural network 400, the same phase change (if any) may be simultaneously applied to J bg 、J bd 、J bw 、J br 、J bbEach of - for example, wherein nodes 410, 412, 414, 416, 418 exhibit some baseline (e.g., in-phase) coupled resonant state. As a result, P(R) and P(B) may be equal to each other, at least initially, and P(G), P(D), and P(W) may be equal to each other, at least initially, and so on.

[0060] In an embodiment, method 300 further includes: (at 330) in response to a change in the phase of the first bias signal, transmitting a third signal (likelihood signal) from the first node to the second node. Based on the change, the third signal indicates the likelihood of the first condition. Characteristics of the third signal (such as rate-amplitude) may indicate the likelihood of the first condition. Referring again to spiking neural network 400, transmitting at 330 may include: node 416 transmitting one or both of likelihood signals 430, 432, and / or node 418 transmitting one or both of likelihood signals 434, 436.

[0061] Method 300 may further include receiving (at 340) a signal at the second node indicating an instance of a second condition (an instance signal). By way of illustration and not limitation, instance signals 420, 422, and 424 may be communicated to nodes 410, 412, and 414 (respectively). Node 410 may be configured to recognize the spike pattern, timing, and / or other characteristics of instance signal 420 as instance signal 420 indicating an instance of condition G. Similarly, characteristics of instance signal 422 may indicate to node 412 an instance of condition D, and / or characteristics of instance signal 424 may indicate to node 414 an instance of condition W.

[0062] In response to the instance of the second condition, method 300 (at 350) may transmit a fourth signal (and an adjustment signal) from the second node to the first node, the fourth signal (and the adjustment signal) being based on the second bias signal. The fourth signal may be further based on a conditional probability of the second condition given the first condition. For example, the third signal may be transmitted via a synapse coupled between the first node and the second node, wherein a weight assigned to the synapse indicates a conditional probability of the second condition given the first condition. In some embodiments, the fourth signal is based on a ratio of the conditional probability of the second condition to the probability.

[0063] Referring again to spiking neural network 400, the communicating at 350 may include node 410 sending conditioning signal 440, node 412 sending one or both of conditioning signals 442, 444, and / or node 414 sending conditioning signal 446. The generation of some or all of conditioning signals 440, 442, 444, 446 may include operations such as those performed at, for example, 350.

[0064] Based on the fourth signal, the method 300 (at 360) can adjust the amount of change in the phase of the first bias signal. For example, the node 416 can adjust the phase of the bias signal J applied to the bias signal J based on one or both of the adjustment signals 440, 442. br Alternatively or additionally, node 418 may adjust the phase change applied to bias signal J based on one or both of adjustment signals 444, 446. bb In response to such an adjustment to the phase change, a characteristic of one of the likelihood signals 430, 432, 434, 436 may be altered, thereby indicating a change in the probability of condition R or a change in the probability of condition B.

[0065] Although some embodiments are not limited in this respect, one or more other nodes (in addition to the second node) may each serve as a child node of the first node. In such embodiments, method 300 may further include: receiving a third bias signal at a third node corresponding to a third condition; and transmitting the third bias signal from the first node to the third node. For example, nodes 416, 410, and 412 of spiking neural network 400 may be (respectively) a first node, a second node, and a third node—e.g., where both likelihood signals 430 and 432 include the third signal. In such embodiments, method 300 may further include: receiving a signal (e.g., instance signal 422) at the third node indicating an instance of the third condition. In response to the instance of the third condition, the third node may transmit a fifth signal (e.g., adjustment signal 442) to the first node, the fifth signal (e.g., adjustment signal 442) being based on the third bias signal (and based on any phase change applied to the third bias signal). Subsequently, the amount of change in the phase of the first bias signal may be further adjusted based on the fifth signal. In such embodiments, adjusting the amount of change in the phase of the first bias signal based on the fifth signal may be performed simultaneously with the adjustment as a result of operation 360 .

[0066] Alternatively or additionally, one or more other nodes (other than the first node) may each serve as a parent node of the second node. In such embodiments, method 300 may further include: receiving a third bias signal at a third node corresponding to a third condition; and applying a change to the phase of the third bias signal. For example, nodes 416, 412, 418 of spiking neural network 400 may be (respectively) the first node, the second node, and the third node. In such embodiments, method 300 may further transmit a fifth signal (e.g., likelihood signal 434) from the third node to the second node, wherein the likelihood signal 434 is applied to the second node based on the third bias signal (e.g., J bb), the fifth signal indicates the likelihood of the third condition (e.g., P(B)). In response to an instance of the second condition, the second node may transmit a sixth signal (e.g., adjustment signal 444) to the third node, the sixth signal (e.g., adjustment signal 444) being based on the second bias signal. Based on the sixth signal, the third node may adjust the amount of change in the phase applied to the third bias signal.

[0067] In some embodiments, the second node also acts as a parent node for a third node corresponding to the third condition. In such embodiments, method 300 may further include: receiving a third bias signal at the third node; and applying a change to the phase of the second bias signal. Similar to the communication at 330, the second node may transmit a fifth signal to the third node, wherein the fifth signal indicates the likelihood of the second condition based on the change in the phase of the second bias signal. Similar to the reception at 340 and the transmission at 350, the third node may receive a signal indicating an instance of the third condition and, in response to the instance of the third condition, may transmit a sixth signal to the second node, the sixth signal being based on the third bias signal. Based on the sixth signal, the second node may adjust the amount of change in the phase of the second bias signal.

[0068] Figure 5 Timing diagrams 500, 510, 520, 530 are shown that illustrate respective signal characteristics during operations for performing Bayesian inference operations according to an embodiment in various manners, each relative to a time axis 505. Signal characteristics may be determined, for example, at one of spiking neural networks 100, 200, 400 based on respective signals communicated in various manners using nodes 240, 250. The various scales (e.g., for timing, voltage, and frequency) shown in timing diagrams 500, 510, 520, 530 are merely illustrative of some embodiments and may vary depending on implementation-specific details.

[0069] Timing diagram 500 shows a spike generated by membrane voltage Vm 502 at a parent node, such as one of nodes 210, 240, 416, 418. The same spike (or otherwise corresponding spike) can be provided by a likelihood signal generated by the parent node based on Vm 502. The spike can have at least some periodic component, where the amplitude, wavelength, and / or other characteristics of the periodic component indicate the probability of the condition corresponding to the parent node. For example, as illustrated by timing diagram 510, the frequency 512 of the spike generated by membrane voltage Vm 502 can exhibit an oscillatory component. The frequency 512 can be a moving average of the spikes that occur within a given time window.

[0070] Timing diagram 520 illustrates an amplitude 522 of the oscillations generated by frequency 512 (where amplitude 522 is the rate-amplitude of Vm 502). As illustrated by timing diagram 520, amplitude 522 may have a first value (e.g., 4 kHz) during a first time period, such as the period shown between time 0 and time 6. The first value of amplitude 522 may be based on a corresponding phase change applied to the bias signal at the parent node.

[0071] The likelihood signal based on Vm 502 may be passed from the parent node to the child node. Subsequently, the child node may detect instances of another condition corresponding to the child node based on the instance signal provided to the child node. In response to such instances, the child node may communicate to the parent node to adjust the amount of phase change applied to the bias signal. For example, as shown in timing diagram 530, the child node may transmit an adjustment signal C b 532. In response to the spike pattern 534, the value of the amplitude 522 may change. In the example scenario shown, the value of the amplitude 522 increases to a second value (e.g., 13 kHz) after the moment of the spike pattern 534. The variable value of the amplitude 522 may indicate the current probability of the condition corresponding to the parent node.

[0072] Figure 6 A computing device 600 is shown according to one embodiment. Computing device 600 houses a board 602. Board 602 may include multiple components, including, but not limited to, a processor 604 and at least one communication chip 606. Processor 604 is physically and electrically coupled to board 602. In some implementations, at least one communication chip 606 is also physically and electrically coupled to board 602. In further implementations, communication chip 606 is part of processor 604.

[0073] Depending on its application, the computing device 600 may include other components that may or may not be physically and electrically coupled to the board 602. These other components include, but are not limited to, volatile memory (e.g., DRAM), non-volatile memory (e.g., ROM), flash memory, a graphics processor, a digital signal processor, a cryptographic processor, a chipset, an antenna, a display, a touch screen display, a touch screen controller, a battery, an audio codec, a video codec, a power amplifier, a global positioning system (GPS) device, a compass, an accelerometer, a gyroscope, a speaker, a camera, and mass storage devices such as hard drives, compact disks (CDs), digital versatile disks (DVDs), and the like.

[0074] The communication chip 606 enables wireless communication for data transmission to and from the computing device 600. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communication channels, etc. that can transmit data through a non-solid medium using modulated electromagnetic radiation. The term does not imply that the associated device does not contain any wires, but in some embodiments, the associated device may not contain any wires. The communication chip 606 can implement any of a variety of wireless standards or protocols, including but not limited to Wi-Fi (IEEE 802.11 series), WiMAX (IEEE 802.16 series), IEEE 802.20, Long Term Evolution (LTE), Ev-DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, DECT, Bluetooth and its derivatives, and any other wireless protocols known as 3G, 4G, 5G, and higher generations. The computing device 600 may include multiple communication chips 606. For example, the first communication chip 606 may be dedicated to shorter-range wireless communications, such as Wi-Fi and Bluetooth, while the second communication chip 606 may be dedicated to longer-range wireless communications, such as GPS, EDGE, GPRS, CDMA, WiMAX, LTE, Ev-DO, etc.

[0075] The processor 604 of the computing device 600 includes an integrated circuit die packaged within the processor 604. The term "processor" may refer to any device or portion of a device that processes electronic data from registers and / or memory to transform the electronic data into other electronic data that can be stored in registers and / or memory. The communication chip 606 also includes an integrated circuit die packaged within the communication chip 606.

[0076] In various implementations, the computing device 600 may be a laptop computer, a netbook, a notebook, an ultrabook, a smartphone, a tablet, a personal digital assistant (PDA), an ultra-mobile OC, a mobile phone, a desktop computer, a server, a printer, a scanner, a monitor, a set-top box, an entertainment control unit, a digital camera, a portable music player, or a digital video recorder. In further implementations, the computing device 600 may be any other electronic device that processes data.

[0077] Some embodiments may be provided as a computer program product or software, which may include a machine-readable medium having instructions stored thereon, which instructions may be used to program a computer system (or other electronic device) to perform a process according to the embodiments. A machine-readable medium includes any mechanism for storing or transmitting information in a machine (e.g., computer) readable form. For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium (e.g., read-only memory ("ROM"), random access memory ("RAM"), magnetic disk storage medium, optical storage medium, flash memory device, etc.), a machine (e.g., computer) readable transmission medium (electrical, optical, acoustic or other form of propagated signals (e.g., infrared signals, digital signals, etc.)), etc.

[0078] Figure 7 A diagrammatic representation of a machine in the exemplary form of a computer system 700 is shown, within which a set of instructions for causing the machine to perform any one or more of the methodologies described herein may be executed. In alternative embodiments, the machine may be connected (e.g., networked) to other machines in a local area network (LAN), an intranet, an extranet, or the Internet. The machine may operate as a server or client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a web appliance, a server, a network router, a switch or a bridge, or any machine capable of executing a set of instructions (sequential or otherwise) specifying the actions performed by the machine. Further, although only a single machine is shown, the term "machine" should also be construed to include any collection of machines (e.g., computers) that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies described herein.

[0079] The exemplary computer system 700 includes a processor 702, a main memory 704 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM)), etc.), a static memory 706 (e.g., flash memory, static random access memory (SRAM), etc.), and a secondary memory 718 (e.g., a data storage device), which communicate with each other via a bus 730.

[0080] The processor 702 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, the processor 702 may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor that implements other instruction sets, or a processor that implements a combination of instruction sets. The processor 702 may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. The processor 702 is configured to execute processing logic 726 for performing the operations described herein.

[0081] The computer system 700 may further include a network interface device 708. The computer system 700 may also include a video display unit 710 (e.g., a liquid crystal display (LCD), a light emitting diode display (LED), or a cathode ray tube (CRT)), an alphanumeric input device 712 (e.g., a keyboard), a cursor control device 714 (e.g., a mouse), and a signal generating device 716 (e.g., a speaker).

[0082] The secondary memory 718 may include a machine-accessible storage medium (or more specifically, a computer-readable storage medium) 732 having stored thereon one or more sets of instructions (e.g., software 722) that embody any one or more of the methodologies or functions described herein. During execution of the software 722 by the computer system 700, the software 722 may also reside, completely or at least partially, within the main memory 704 and / or within the processor 702; the main memory 704 and the processor 702 also constitute machine-readable storage media. The software 722 may further be transmitted or received over the network 720 via the network interface device 708.

[0083] Although the machine-accessible storage medium 732 is shown as a single medium in the exemplary embodiment, the term "machine-readable storage medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) that store one or more sets of instructions. The term "machine-readable storage medium" should also be taken to include any medium that can store or encode instructions that are executable by a machine and cause the machine to perform any of the one or more embodiments. The term "machine-readable storage medium" should accordingly be taken to include, but not be limited to, solid-state memories and optical and magnetic media.

[0084] Example 1 is a computer device for generating Bayesian inference using a spiking neural network, the computer device comprising circuitry configured to: receive a first bias signal at a first node of the spiking neural network, wherein the first node corresponds to a first condition of the Bayesian network; receive a second bias signal at a second node of the spiking neural network, wherein the second node corresponds to a second condition of the Bayesian network; and apply a change to the phase of the first bias signal. The circuitry further configured to: transmit a third signal from the first node to a second node in response to a change in the phase of the first bias signal, wherein the third signal indicates a likelihood of the first condition; receive a signal at the second node indicating an instance of the second condition; transmit a fourth signal from the second node to the first node in response to the instance of the second condition, the fourth signal being based on the second bias signal; and adjust an amount of change in the phase of the first bias signal based on the fourth signal.

[0085] In Example 2, the subject matter of Example 1 optionally includes: wherein a spike rate of the third signal varies over time based on a change in a phase of the first bias signal, wherein a magnitude of the change caused by the spike rate of the third signal indicates a likelihood of the first condition.

[0086] In Example 3, the subject matter of any one or more of Examples 1 and 2 optionally includes wherein the fourth signal is further based on a conditional probability of the second condition given the first condition.

[0087] In Example 4, the subject matter of Example 3 optionally includes: wherein the third signal is transmitted via a synapse coupled between the first node and the second node, wherein a weight assigned to the synapse indicates a conditional probability of the second condition given the first condition.

[0088] In Example 5, the subject matter of Example 3 optionally includes wherein the fourth signal is based on a ratio of the conditional probability of the second condition to the probability.

[0089] In Example 6, the subject matter of any one or more of Examples 1 and 2 optionally includes: the computer device further including circuitry for applying another change to the phase of the second bias signal, wherein the fourth signal is further based on the change in the phase of the second bias signal.

[0090] In Example 7, the subject matter of any one or more of Examples 1 and 2 optionally includes: the computer device further including circuitry for performing the following steps: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; passing the third signal from the first node to the third node; receiving a signal indicating an instance of the third condition at the third node; passing a fifth signal from the third node to the first node in response to the instance of the third condition, the fifth signal being based on the third bias signal; and further adjusting an amount of change in a phase of the first bias signal based on the fifth signal.

[0091] In Example 8, the subject matter of Example 7 optionally includes wherein the circuit is configured to adjust an amount of change in the phase of the first bias signal based on the fifth signal while adjusting the change in the phase of the first bias signal based on the fourth signal.

[0092] In Example 9, the subject matter of any one or more of Examples 1 and 2 optionally includes:

[0093] The computer device further includes circuitry for performing the following steps: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; applying a change to a phase of the third bias signal; transmitting a fifth signal from the third node to the second node, wherein the fifth signal indicates a likelihood of the third condition based on the change in the phase of the third bias signal; transmitting a sixth signal from the second node to the third node in response to an instance of the second condition, the sixth signal being based on the second bias signal; and adjusting an amount of change in the phase of the third bias signal based on the sixth signal.

[0094] In Example 10, the subject matter of any one or more of Examples 1 and 2 optionally includes: the computer device further including circuitry for performing the following steps: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; applying a change to the phase of the second bias signal; passing a fifth signal from the second node to the third node, wherein the fifth signal indicates a likelihood of the second condition based on the change in the phase of the second bias signal; receiving a signal indicating an instance of the third condition at the third node; passing a sixth signal from the third node to the second node in response to the instance of the third condition, the sixth signal being based on the third bias signal; and adjusting an amount of change in the phase of the second bias signal based on the sixth signal.

[0095] In Example 11, the subject matter of any one or more of Examples 1 and 2 optionally includes: the computer device further including circuitry for selecting a condition of the Bayesian network over one or more other conditions of the Bayesian network based on the adjusted amount of change.

[0096] Example 12 is at least one non-transitory machine-readable medium comprising instructions that, when executed by a machine, cause the machine to perform operations for generating Bayesian inference using a spiking neural network, the operations comprising: receiving a first bias signal at a first node of the spiking neural network, wherein the first node corresponds to a first condition of the Bayesian network; receiving a second bias signal at a second node of the spiking neural network, wherein the second node corresponds to a second condition of the Bayesian network; applying a change to a phase of the first bias signal; and transmitting a third signal from the first node to the second node in response to the change in the phase of the first bias signal, wherein the third signal indicates a likelihood of the first condition. The operations further comprise: receiving a signal at the second node indicating an instance of the second condition; transmitting a fourth signal from the second node to the first node in response to the instance of the second condition, the fourth signal being based on the second bias signal; and adjusting an amount of change in the phase of the first bias signal based on the fourth signal.

[0097] In Example 13, the subject matter of Example 12 optionally includes: wherein the spike rate of the third signal varies over time based on a change in the phase of the first bias signal, wherein a magnitude of the change caused by the spike rate of the third signal indicates the likelihood of the first condition.

[0098] In Example 14, the subject matter of any one or more of Examples 12 and 13 optionally includes wherein the fourth signal is further based on a conditional probability of the second condition given the first condition.

[0099] In Example 15, the subject matter of Example 14 optionally includes: wherein the third signal is transmitted via a synapse coupled between the first node and the second node, wherein a weight assigned to the synapse indicates a conditional probability of the second condition given the first condition.

[0100] In Example 16, the subject matter of Example 14 optionally includes wherein the fourth signal is based on a ratio of the conditional probability of the second condition to the probability.

[0101] In Example 17, the subject matter of any one or more of Examples 12 and 13 optionally includes the operations further comprising applying another change to a phase of the second bias signal, wherein the fourth signal is further based on the change in the phase of the second bias signal.

[0102] In Example 18, the subject matter of any one or more of Examples 12 and 13 optionally includes: the operations further including: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; passing the third signal from the first node to the third node; receiving a signal indicating an instance of the third condition at the third node; passing a fifth signal from the third node to the first node in response to the instance of the third condition, the fifth signal based on the third bias signal; and further adjusting an amount of change in a phase of the first bias signal based on the fifth signal.

[0103] In Example 19, the subject matter of Example 18 optionally includes wherein the amount of change in the phase of the first bias signal is adjusted based on the fifth signal while the change in the phase of the first bias signal is adjusted based on the fourth signal.

[0104] In Example 20, the subject matter of any one or more of Examples 12 and 13 optionally includes: the operations further comprising: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; applying a change to a phase of the third bias signal; transmitting a fifth signal from the third node to the second node, wherein the fifth signal indicates a likelihood of the third condition based on the change in the phase of the third bias signal; transmitting a sixth signal from the second node to the third node in response to an instance of the second condition, the sixth signal being based on the second bias signal; and adjusting an amount of change in the phase of the third bias signal based on the sixth signal.

[0105] In Example 21, the subject matter of any one or more of Examples 12 and 13 optionally includes: the operations further comprising: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; applying a change to a phase of the second bias signal; passing a fifth signal from the second node to the third node, wherein the fifth signal indicates a likelihood of the second condition based on the change in the phase of the second bias signal; receiving a signal indicating an instance of the third condition at the third node; passing a sixth signal from the third node to the second node in response to the instance of the third condition, the sixth signal being based on the third bias signal; and adjusting an amount of change in the phase of the second bias signal based on the sixth signal.

[0106] In Example 22, the subject matter of any one or more of Examples 12 and 13 optionally includes: the operations further comprising: selecting a condition of the Bayesian network in preference to one or more other conditions of the Bayesian network, the selecting being based on the adjusted amount of change.

[0107] Example 23 is a method for generating Bayesian inference using a spiking neural network, the method comprising: receiving a first bias signal at a first node of the spiking neural network, wherein the first node corresponds to a first condition of the Bayesian network; receiving a second bias signal at a second node of the spiking neural network, wherein the second node corresponds to a second condition of the Bayesian network; applying a change to a phase of the first bias signal; and transmitting a third signal from the first node to the second node in response to the change in the phase of the first bias signal, wherein the third signal indicates a likelihood of the first condition. The method further comprises: receiving a signal at the second node indicating an instance of the second condition; transmitting a fourth signal from the second node to the first node in response to the instance of the second condition, the fourth signal being based on the second bias signal; and adjusting an amount of change in the phase of the first bias signal based on the fourth signal.

[0108] In Example 24, the subject matter of Example 23 optionally includes: wherein the spike rate of the third signal varies over time based on a change in the phase of the first bias signal, wherein a magnitude of the change caused by the spike rate of the third signal indicates the likelihood of the first condition.

[0109] In Example 25, the subject matter of any one or more of Examples 23 and 24 optionally includes wherein the fourth signal is further based on a conditional probability of the second condition given the first condition.

[0110] In Example 26, the subject matter of Example 25 optionally includes: wherein the third signal is transmitted via a synapse coupled between the first node and the second node, wherein a weight assigned to the synapse indicates a conditional probability of the second condition given the first condition.

[0111] In Example 27, the subject matter of Example 25 optionally includes wherein the fourth signal is based on a ratio of the conditional probability of the second condition to the probability.

[0112] In Example 28, the subject matter of any one or more of Examples 23 and 24 optionally includes the method further comprising applying another change to a phase of the second bias signal, wherein the fourth signal is further based on the change in the phase of the second bias signal.

[0113] In Example 29, the subject matter of any one or more of Examples 23 and 24 optionally includes: the method further including: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; passing the third signal from the first node to the third node; receiving a signal indicating an instance of the third condition at the third node; passing a fifth signal from the third node to the first node in response to the instance of the third condition, the fifth signal based on the third bias signal; and further adjusting an amount of change in a phase of the first bias signal based on the fifth signal.

[0114] In Example 30, the subject matter of Example 29 optionally includes wherein the amount of change in the phase of the first bias signal is adjusted based on the fifth signal while the change in the phase of the first bias signal is adjusted based on the fourth signal.

[0115] In Example 31, the subject matter of any one or more of Examples 23 and 24 optionally includes: the method further comprising: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; applying a change to a phase of the third bias signal; passing a fifth signal from the third node to the second node, wherein the fifth signal indicates a likelihood of the third condition based on the change in the phase of the third bias signal; passing a sixth signal from the second node to the third node in response to an instance of the second condition, the sixth signal being based on the second bias signal; and adjusting an amount of change in the phase of the third bias signal based on the sixth signal.

[0116] In Example 32, the subject matter of any one or more of Examples 23 and 24 optionally includes: the method further including: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; applying a change to the phase of the second bias signal; passing a fifth signal from the second node to the third node, wherein the fifth signal indicates a likelihood of the second condition based on the change in the phase of the second bias signal; receiving a signal indicating an instance of the third condition at the third node; passing a sixth signal from the third node to the second node in response to the instance of the third condition, the sixth signal being based on the third bias signal; and adjusting an amount of change in the phase of the second bias signal based on the sixth signal.

[0117] In Example 33, the subject matter of any one or more of Examples 23 and 24 optionally includes: the method further comprising: selecting a condition of the Bayesian network in preference to one or more other conditions of the Bayesian network, the selection being based on the adjusted amount of change.

[0118] Techniques and architectures for providing the functionality of spiking neural networks are described herein. In the description above, for ease of explanation, numerous specific details are set forth to provide a thorough understanding of certain embodiments. However, it will be apparent to those skilled in the art that certain embodiments can be practiced without these specific details. In other instances, structures and devices are shown in block diagram form to avoid obscuring the description.

[0119] Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0120] Some parts of the specific embodiments herein are presented in terms of algorithms and symbolic representations of operations on data bits in computer memory. These algorithmic descriptions and representations are the means used by those of ordinary skill in the computer arts to most effectively convey the essence of their work to other persons skilled in the art. An algorithm is generally understood herein to be a self-consistent sequence of steps that leads to a desired result. These steps are those steps that require physical manipulation of physical quantities. Typically, but not necessarily, these quantities take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated. Primarily for common purposes, it has proven convenient to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, etc.

[0121] It should be remembered, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless expressly indicated otherwise, as will be apparent from the discussion herein, it will be understood that discussions throughout the specification utilizing terms such as "process" or "compute" or "calculate" or "determine" or "display" refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities within the computer system's registers and memories and transforms it into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission, or display devices.

[0122] Certain embodiments also relate to apparatus for performing the operations described herein. These apparatuses may be specially constructed for the desired purpose, or they may comprise a general-purpose computer that is selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer-readable storage medium such as, but not limited to, any type of disk, including a floppy disk, an optical disk, a CD-ROM, a magneto-optical disk, a read-only memory (ROM), a RAM such as a dynamic random access memory (RAM) (DRAM), an EPROM, an EEPROM, a magnetic or optical card, or any type of medium suitable for storing electronic instructions and coupled to a computer system bus.

[0123] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general-purpose systems can be used together with the programs taught herein, or it may prove convenient to construct more specialized devices to implement the required method steps. The required structures of various these systems will be presented from the description herein. In addition, certain embodiments are not described with reference to any particular programming language. It will be understood that various programming languages ​​can be used to implement the teachings of such embodiments described herein.

[0124] In addition to what is described herein, various modifications may be made to the disclosed embodiments and implementations thereof without departing from the scope thereof. Therefore, the descriptions and examples herein should be interpreted as illustrative rather than restrictive. The scope of the present invention should be defined solely by reference to the appended claims.

Claims

1. A computer device for generating Bayesian inferences using a spiking neural network, the computer device comprising circuitry for performing the following steps: A first bias signal is received at a first node of the spiking neural network, wherein: The first node corresponds to a first condition of the Bayesian network; receiving a second bias signal at a second node of the spiking neural network, wherein the second node corresponds to a second condition of the Bayesian network; applying a change to a phase of the first bias signal; transmitting a third signal from the first node to the second node in response to the change in phase of the first bias signal, wherein the third signal indicates a likelihood of the first condition; receiving, at the second node, a signal indicating an instance of the second condition; In response to the instance of the second condition, passing a fourth signal from the second node to the first node, the fourth signal being based on the second bias signal; and Based on the fourth signal, the amount of change in the phase of the first bias signal is adjusted.

2. The computer device of claim 1, wherein: Based on the change in the phase of the first bias signal, a spike rate of the third signal changes over time, wherein a magnitude of the change caused by the spike rate of the third signal indicates the likelihood of the first condition.

3. The computer device of claim 1, wherein: The fourth signal is further based on a conditional probability of the second condition given the first condition.

4. The computer device of claim 3, wherein: The third signal is transmitted via a synapse coupled between the first node and the second node, wherein a weight assigned to the synapse indicates a conditional probability of the second condition given the first condition.

5. The computer device of claim 3, wherein: The fourth signal is based on a ratio of the conditional probability of the second condition to the probability.

6. The computer device of claim 1 , further comprising circuitry for applying another change to the phase of the second bias signal, wherein The fourth signal is further based on a change in phase of the second bias signal.

7. The computer device of claim 1 , further comprising circuitry for performing the following steps: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; transmitting the third signal from the first node to the third node; receiving, at the third node, a signal indicative of an instance of the third condition; In response to an instance of the third condition, passing a fifth signal from the third node to the first node, the fifth signal being based on the third bias signal; as well as Based on the fifth signal, the change amount of the phase of the first bias signal is further adjusted.

8. The computer device of claim 7, wherein: The circuit is configured to adjust the phase change of the first bias signal based on the fourth signal while adjusting the phase change of the first bias signal based on the fifth signal.

9. The computer device of claim 1 , further comprising circuitry for performing the following steps: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; applying a change to a phase of the third bias signal; passing a fifth signal from the third node to the second node, wherein the fifth signal indicates a likelihood of the third condition based on a change in a phase of the third bias signal; In response to an instance of the second condition, passing a sixth signal from the second node to the third node, the sixth signal being based on the second bias signal; as well as Based on the sixth signal, the amount of change in the phase of the third bias signal is adjusted.

10. The computer device of claim 1, further comprising circuitry for selecting a condition of the Bayesian network over one or more other conditions of the Bayesian network based on the adjusted amount of variation.

11. At least one non-transitory machine-readable medium comprising instructions that, when executed by a machine, cause the machine to perform operations for generating Bayesian inference using a spiking neural network, the operations comprising: receiving a first bias signal at a first node of the spiking neural network, wherein the first node corresponds to a first condition of the Bayesian network; receiving a second bias signal at a second node of the spiking neural network, wherein the second node corresponds to a second condition of the Bayesian network; applying a change to a phase of the first bias signal; transmitting a third signal from the first node to the second node in response to the change in phase of the first bias signal, wherein the third signal indicates a likelihood of the first condition; receiving, at the second node, a signal indicating an instance of the second condition; In response to the instance of the second condition, passing a fourth signal from the second node to the first node, the fourth signal being based on the second bias signal; and Based on the fourth signal, the amount of change in the phase of the first bias signal is adjusted.

12. The at least one non-volatile machine-readable medium of claim 11, wherein: Based on the change in the phase of the first bias signal, a spike rate of the third signal changes over time, wherein a magnitude of the change caused by the spike rate of the third signal indicates the likelihood of the first condition.

13. The at least one non-transitory machine-readable medium of claim 11, wherein: The fourth signal is further based on a conditional probability of the second condition given the first condition.

14. The at least one non-transitory machine-readable medium of claim 13, wherein: The third signal is transmitted via a synapse coupled between the first node and the second node, wherein a weight assigned to the synapse indicates a conditional probability of the second condition given the first condition.

15. The at least one non-transitory machine-readable medium of claim 13, wherein: The fourth signal is based on a ratio of the conditional probability of the second condition to the probability.

16. The at least one non-transitory machine-readable medium of claim 11, the operations further comprising: Another change is applied to the phase of the second bias signal, wherein the fourth signal is further based on the change in the phase of the second bias signal.

17. The at least one non-transitory machine-readable medium of claim 11, the operations further comprising: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; transmitting the third signal from the first node to the third node; receiving, at the third node, a signal indicative of an instance of the third condition; In response to an instance of the third condition, passing a fifth signal from the third node to the first node, the fifth signal being based on the third bias signal; as well as Based on the fifth signal, the change amount of the phase of the first bias signal is further adjusted.

18. The at least one non-volatile machine-readable medium of claim 17, wherein: While adjusting the change in the phase of the first bias signal based on the fourth signal, the amount of change in the phase of the first bias signal is adjusted based on the fifth signal.

19. The at least one non-transitory machine-readable medium of claim 11, the operations further comprising: receiving a third bias signal at a third node of the spiking neural network, the third node corresponding to a third condition of the Bayesian network; applying a change to a phase of the second bias signal; transmitting a fifth signal from the second node to the third node, wherein the fifth signal indicates a likelihood of the second condition based on a change in the phase of the second bias signal; receiving, at the third node, a signal indicative of an instance of the third condition; In response to an instance of the third condition, passing a sixth signal from the third node to the second node, the sixth signal being based on the third bias signal; and Based on the sixth signal, the amount of change in the phase of the second bias signal is adjusted.

20. The at least one non-transitory machine-readable medium of claim 11, the operations further comprising: A condition of the Bayesian network is selected in preference to one or more other conditions of the Bayesian network, the selection being based on the adjusted amount of variation.

21. A method for generating Bayesian inferences using a spiking neural network, the method comprising: receiving a first bias signal at a first node of the spiking neural network, wherein the first node corresponds to a first condition of the Bayesian network; receiving a second bias signal at a second node of the spiking neural network, wherein the second node corresponds to a second condition of the Bayesian network; applying a change to a phase of the first bias signal; transmitting a third signal from the first node to the second node in response to the change in phase of the first bias signal, wherein the third signal indicates a likelihood of the first condition; receiving, at the second node, a signal indicating an instance of the second condition; In response to the instance of the second condition, passing a fourth signal from the second node to the first node, the fourth signal being based on the second bias signal; and Based on the fourth signal, the amount of change in the phase of the first bias signal is adjusted.

22. The method of claim 21, wherein: Based on the change in the phase of the first bias signal, a spike rate of the third signal changes over time, wherein a magnitude of the change caused by the spike rate of the third signal indicates the likelihood of the first condition.

23. The method of claim 21, wherein: The fourth signal is further based on a conditional probability of the second condition given the first condition.

24. The method of claim 21, further comprising: Another change is applied to the phase of the second bias signal, wherein the fourth signal is further based on the change in the phase of the second bias signal.

25. The method of claim 21, further comprising: A condition of the Bayesian network is selected in preference to one or more other conditions of the Bayesian network, the selection being based on the adjusted amount of variation.

Images