Spiking Neural Network Frustum Partitioning for Memory Access Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Spiking neural networks (SNNs) face significant challenges in reducing memory access and in-network bandwidth consumption during execution, leading to high power consumption and limited scalability in systems with constrained power and thermal budgets.
Innovation Solution
The implementation of a neural network engine that reorders computations to consume intermediate values soon after generation, reduces data traffic by storing these values in internal memory instead of external RAM, and partitions the neural network into frustums to process independently, allowing for parallel processing and localized data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If spiking neural networks use conventional memory access patterns with external RAM, then data storage capacity is sufficient, but power consumption increases and execution time increases
Solution Approach 1:
The patent segments the neural network computation into independent frustums that can be processed in parallel. Each frustum represents a portion of the network that can be computed independently, allowing multiple frustums to be processed simultaneously without requiring sequential memory access, thereby reducing both power consumption and execution time.
Solution Approach 2:
The patent performs preliminary computation by calculating frustum outputs before final network assembly. Intermediate values are computed and stored in internal memory in advance, reducing the need for repeated memory access during final computation and thereby reducing overall execution time and power consumption.
2Loss of energy
If spiking neural networks store intermediate values in external RAM, then memory capacity is sufficient, but in-network bandwidth consumption increases
Solution Approach 1:
The patent merges the compute units with internal memory resources, creating an integrated architecture where intermediate values are stored locally within the processing unit. This eliminates the need for frequent data transfer between external RAM and compute units, significantly reducing in-network bandwidth consumption while maintaining sufficient memory capacity through the combined compute-memory structure.
3Productivity
If spiking neural networks process the entire network sequentially, then computation accuracy is maintained, but productivity decreases
Solution Approach 1:
The patent divides the neural network into independent frustums that can be processed in parallel while maintaining computational accuracy. Each frustum is processed independently through dedicated compute units, and the results are assembled to produce the final output, thereby increasing processing throughput without sacrificing computation accuracy.
Solution Approach 2:
The patent implements feedback mechanisms where intermediate results from parallel frustum processing are validated and assembled to ensure computational accuracy. The system verifies that parallel processing results match expected outcomes, maintaining reliability while achieving higher productivity through parallel execution.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A spiking neural network having a plurality layers partitioned into a plurality of frustums using a first partitioning may be implemented, where each frustum includes one tile of each partitioned layer of the spiking neural network. A first tile of a first layer of the spiking neural network may be read. Using a processor, a first tile of a second layer of the spiking neural network may be generated using the first tile of the first layer while storing intermediate data within an internal memory of the processor. The first tile of the first layer and the first tile of the second layer belong to a same frustum.