Partial Activation of Neural Network Pathways for Edge Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning models require specialized hardware for training and prediction, making it impractical to run them on most endpoint devices without dedicated accelerator hardware.
Innovation Solution
Implementing a 'partially-activated' neural network that activates only a subset of neurons along selective pathways across a portion of the neural network multiple times, allowing for efficient execution on devices with basic hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all neurons are activated across the entire neural network in a single pass, then the accuracy of the neural network output is maintained, but the processing time and computational resources increase significantly
Solution Approach 1:
The neural network is segmented into multiple partial pathways, where each pathway processes a subset of neurons. Instead of activating all neurons in a single pass, the system divides the computation into K separate passes, each handling a different pathway. This segmentation reduces the computational load per pass while maintaining overall accuracy through aggregation of results.
Solution Approach 2:
The system applies partial action by activating only a portion of neurons in each pass rather than all neurons. Each pathway processes a subset of the total neurons, and the final output is obtained by aggregating results from multiple partial passes. This approach achieves acceptable accuracy with reduced computational effort.
2Ease of operation
If deep learning models are run on endpoint devices with limited hardware, then local processing capability is improved, but the hardware requirements and complexity increase
Solution Approach 1:
The computational workload is segmented into multiple partial pathways that can be executed sequentially or in parallel on endpoint devices. This segmentation allows the device to process each pathway with limited resources rather than requiring all resources simultaneously, making deployment on hardware-constrained devices feasible.
Solution Approach 2:
The system uses partial activation of neurons in each pass, reducing the instantaneous computational requirements. By processing subsets of neurons across multiple passes rather than activating all neurons simultaneously, endpoint devices with limited hardware can perform deep learning inference locally without requiring high-end accelerators.
3Measurement precision
If continuous communication with the cloud is implemented for deep learning prediction, then processing accuracy is maintained, but the dependency on network connectivity and cloud infrastructure increases
Solution Approach 1:
The neural network computation is segmented into multiple partial pathways that can be executed locally on the endpoint device. This segmentation enables the device to perform inference independently without requiring continuous cloud connectivity, while still achieving accurate results through aggregation of multiple pathway outputs.
Solution Approach 2:
The endpoint device performs deep learning inference locally using the partial pathway activation method, making the system self-sufficient and independent of cloud infrastructure. The device can process data locally even when disconnected from the network, eliminating the need for continuous cloud communication while maintaining prediction capability.
Data Source
AI summary
A device, system, and method for approximating a neural network comprising N synapses or filters. The neural network may be partially-activated by iteratively executing a plurality of M partial pathways of the neural network to generate M partial outputs, wherein the M partial pathways respectively comprise M different continuous sequences of synapses or filters linking an input layer to an output layer. The M partial pathways may cumulatively span only a subset of the N synapses or filters such that a significant number of the remaining the N synapses or filters are not computed. The M partial outputs of the M partial pathways may be aggregated to generate an aggregated output approximating an output generated by fully-activating the neural network by executing a single instance of all N synapses or filters of the neural network. Training or prediction of the neural network may be performed based on the aggregated output.