Neural Weight Memory Alignment for Lock-Step Multi-Core Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks with near-zero magnitude parameters face challenges in hardware acceleration due to unstructured sparsity, leading to high pruning rates and irregularities in weight distribution that hinder efficient processing on multiple parallel cores.
Innovation Solution
A system and method for neural network processing that includes processor cores with weight memories, using weight mask bits to indicate pruned or non-pruned weights, and selectively replacing pruned weights with non-pruned weights to align memory sections and ensure consistent processing across multiple cores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If unstructured sparsity is used to achieve high pruning rate, then compressibility is improved, but processing efficiency on parallel cores deteriorates due to irregularities in weight distribution
Solution Approach 1:
The weight memory is divided into multiple sections that are processed concurrently by different processor cores. Each section contains a subset of weight parameters, allowing parallel processing while maintaining consistent data distribution patterns across sections.
Solution Approach 2:
Different sections of the weight memory are designed with locally optimized characteristics to ensure consistent processing behavior. Each section maintains similar sparsity patterns and data distribution, allowing each processor core to operate efficiently on its assigned section without encountering irregularities that would reduce overall productivity.
2Speed
If processor cores process weight memories concurrently, then processing speed is improved, but processing stalls occur due to misalignment of memory sections
Solution Approach 1:
The weight memory is pre-organized into aligned sections during the pruning process. Sections are configured in advance to have consistent boundaries and data layouts, ensuring that when multiple processor cores begin concurrent processing, they all start from synchronized points and process data in a coordinated manner, preventing stalls.
Solution Approach 2:
Memory sections are designed to be equipotential in terms of their processing characteristics. Each section has similar size, data distribution, and access patterns, ensuring that all processor cores experience comparable processing conditions and complete their tasks simultaneously without creating bottlenecks or stalls.
3Reliability
If pruned weights are replaced with non-pruned weights to align memory sections, then processing consistency is improved, but memory requirements increase
Solution Approach 1:
Weight replacement is applied locally only to specific sections that require alignment, rather than uniformly across the entire weight memory. This targeted approach allows memory section alignment to prevent processing stalls while minimizing the overall increase in memory requirements by only expanding specific problematic sections.
Data Source
AI summary
A neural network processing system and a method of generating weights for a neural network processing system is described. The system includes a plurality of processor cores coupled to respective weight memories which store neural network weights. The neural network weights are stored as a plurality of weight mask bits, each weight mask bit indicating whether a corresponding weight is a pruned weight or a non-pruned weight and a plurality of non-pruned weights. At least one of the non-pruned weights has a pruned weight value. Non-pruned weights with a pruned weight value may be selectively added after initial pruning to equalize memory section size, word align memory sections or to ensure processing stalls (hiccups) occur in the same cycle. The resulting pruned weight sets may be used with neural processor accelerators operating in lock step.