Stochastic Neural Network Path Sampling for Selective Node Activation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks (ANNs) require computing the entire network for every data query and training, limiting parallel computations and variable resource allocation based on query difficulty.
Innovation Solution
Implementing a stochastic neural network (SNN) with selectively activated nodes, allowing parallel execution of queries and variable resource allocation through stochastic activation of nodes and synapses, using activation probabilities to determine node firing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the entire neural network is computed for every data query, then the network can provide accurate deterministic outputs, but the computational load increases and parallel computations cannot be executed
Solution Approach 1:
The patent applies partial action by selectively activating only a subset of nodes in the neural network based on the input data characteristics. Instead of computing all nodes, the system activates nodes probabilistically according to their relevance, reducing computational load while maintaining sufficient accuracy for the query
Solution Approach 2:
The system dynamically adjusts node activation states based on input characteristics and query difficulty. The stochastic activation mechanism allows the network to adapt its computational behavior in real-time, activating more nodes for difficult queries and fewer nodes for simple queries, enabling both parallel computation and accurate output
2Adaptability or versatility
If the entire neural network is computed for every data query, then complete computational coverage is achieved, but the network cannot dedicate variable computational resources based on query difficulty
Solution Approach 1:
The patent changes the activation parameter from deterministic (always active) to stochastic (probabilistically active). Each node has an activation probability that can be adjusted based on input characteristics, allowing the system to allocate computational resources variabley according to query difficulty while maintaining a relatively simple network structure
3Productivity
If stochastic activation is used, then parallel computations can be executed and computational load is reduced, but the output becomes different across multiple evaluations with fixed input
Solution Approach 1:
The system uses feedback mechanisms where the stochastic activation outcomes are aggregated across multiple parallel evaluations. The final output is derived from combining results from multiple stochastic runs, which provides a more stable and reliable output while still benefiting from the computational efficiency of selective activation
Solution Approach 2:
The patent merges results from multiple stochastic evaluations to produce a final deterministic output. By combining outputs from parallel stochastic computations, the system achieves both computational efficiency through selective activation and reliability through result aggregation
Data Source
AI summary
Embodiments of the present disclosure implement a stochastic neural network (SNN) where a subset of the nodes in the network are selectively activated based on sampling a plurality of computational paths traversing the network. In various embodiments, an output of the stochastic neural network is a sequence of the sampled plurality of computational paths with a corresponding sequence of output values that represent approximations of the output of the stochastic neural network. The nodes can include at least one input node, at least one output node and at least two hidden nodes, wherein the hidden nodes are positioned between the input node and the output node, and wherein sampling the plurality of computational paths involves initiating each of the plurality of computational paths from a first of the hidden nodes, wherein the first of the hidden nodes has been activated by a previous computational path.