Stochastic Neural Network Selective Node Activation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks (ANNs) face challenges in computing the entire network for every data query and training, which limits parallel computations and dynamic resource allocation based on query difficulty.
Innovation Solution
The implementation of a stochastic neural network (SNN) where nodes are selectively activated based on inputs, allowing for parallel execution of queries and dynamic resource allocation, with each synapse having a processing element to compute and propagate activation probabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the entire neural network is computed for every data query and training, then the network can process all possible computations, but the network cannot run multiple computations in parallel and cannot dynamically allocate computational resources
Solution Approach 1:
The patent divides the neural network into multiple independent computational paths or sub-networks that can be selectively activated. Each synapse is equipped with a processing element that independently evaluates whether to activate based on input characteristics, enabling parallel execution of multiple queries across different network segments without requiring full network computation for each query.
2Adaptability or versatility
If deterministic activation is used where a node fires when activation value exceeds threshold, then the output is consistent across multiple evaluations, but the network cannot dynamically adjust resource allocation based on query difficulty
Solution Approach 1:
The patent introduces stochastic activation where nodes transition from deterministic to probabilistic behavior. Each synapse's processing element generates an activation probability based on the activation value, and the node fires according to this probability rather than a fixed threshold rule. This dynamic mechanism allows the network to adapt computational resources to query difficulty while maintaining reliable output through controlled stochasticity and multiple evaluations.
3Use of energy by moving object
If stochastic activation is used where there is a probability associated with node firing, then the network can dynamically allocate resources, but the output varies across multiple evaluations with fixed input
Solution Approach 1:
The patent implements a feedback mechanism where the stochastic activation outcomes are aggregated across multiple evaluations or parallel paths. The system monitors activation probabilities and adjusts the number of parallel computations or evaluation rounds based on the observed output consistency and query difficulty, thereby controlling information loss while maintaining computational efficiency.
4Productivity
If selective node activation is implemented, then parallel computations can be executed and resources efficiently allocated, but the network requires complex mechanisms to determine which nodes to activate
Solution Approach 1:
The patent empowers each synapse with an embedded processing element that autonomously decides whether to activate the connected node based on local input characteristics and learned parameters. This decentralized self-service mechanism eliminates the need for centralized control or complex global coordination to determine node activation, simplifying the overall system architecture while enabling efficient parallel computation execution.
Data Source
AI summary
Embodiments of the present disclosure implement a stochastic neural network (SNN) where nodes are selectively activated depending on the inputs and which can be trained on multiple objectives. A system and/or network can include one or more nodes and one or more synapses, wherein each synapse connects a respective pair of the plurality of nodes. The system and/or network can further include one or more processing elements, wherein each of the processing elements is embedded in a respective synapse, and wherein each of the processing elements is adapted to receive an input and generate an output based on the input. In various embodiments, a super-imposable stochastic graph is employed with training, regularization and load balancing.