Stochastic Neural Network Path Sampling for Parallel Query Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks (ANNs) require computing the entire network for every data query, limiting parallel computations and variable resource allocation based on query difficulty.
Innovation Solution
Implementing a stochastic neural network (SNN) with selectively activated nodes, allowing parallel execution of queries and variable resource allocation through stochastic node activation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the entire neural network is computed for every data query, then the network can provide accurate deterministic outputs, but the network cannot run multiple computations in parallel and cannot dedicate variable computational resources
Solution Approach 1:
The neural network is segmented into multiple independent computational paths, where each path can be independently activated or deactivated. This segmentation allows the network to compute only the necessary subset of nodes for each query, enabling parallel execution of multiple queries while maintaining output reliability through the structured organization of computational paths.
Solution Approach 2:
The network implements dynamic node activation where the computational state of each node is determined stochastically based on the input query characteristics. This dynamic approach allows the network to adaptively allocate computational resources - activating more nodes for complex queries and fewer nodes for simple queries - thereby achieving variable resource dedication while maintaining overall system reliability.
2Productivity
If the entire neural network is computed for every data query, then complete computational coverage is achieved, but computational resources cannot be optimized based on query difficulty
Solution Approach 1:
The network implements partial action by computing only the necessary subset of nodes rather than the entire network for each query. The stochastic activation mechanism determines the appropriate level of computation based on query characteristics, allowing the system to perform partial computations when sufficient and full computations when necessary, thereby optimizing computational efficiency without oversimplifying the network structure.
Solution Approach 2:
The network utilizes parameter changes in the form of stochastic activation probabilities assigned to different nodes. These parameters dynamically adjust which nodes are activated during computation, allowing the network to modify its effective structure based on query difficulty. This parameter-based control enables efficient resource allocation while maintaining the underlying network complexity for handling diverse query types.
3Productivity
If nodes are selectively activated stochastically, then parallel computations can be executed and resource allocation optimized, but the output becomes different for multiple evaluations with fixed input
Solution Approach 1:
The network employs periodic action through structured sampling of computational paths. By organizing the stochastic activation into repeatable sampling cycles and using aggregation techniques across multiple samples, the system achieves consistent output statistics despite individual variations. This periodic structure enables parallel computation execution while maintaining output reliability through statistical convergence.
Solution Approach 2:
The network implements feedback mechanisms where the stochastic activation outcomes are aggregated and used to inform subsequent computations. By incorporating feedback from previous sampling results and adjusting activation patterns accordingly, the system maintains output consistency across multiple evaluations while still benefiting from the parallel computation capabilities enabled by stochastic node activation.
Data Source
AI summary
Embodiments of the present disclosure implement a stochastic neural network (SNN) where a subset of the nodes in the network are selectively activated based on sampling a plurality of computational paths traversing the network. In various embodiments, an output of the stochastic neural network is a sequence of the sampled plurality of computational paths with a corresponding sequence of output values that represent approximations of the output of the stochastic neural network. The nodes can include at least one input node, at least one output node and at least two hidden nodes, wherein the hidden nodes are positioned between the input node and the output node, and wherein sampling the plurality of computational paths involves initiating each of the plurality of computational paths from a first of the hidden nodes, wherein the first of the hidden nodes has been activated by a previous computational path.