Multicore Neural Network Partitioning for Localized Embedded Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing in multicore architectures is inefficient due to hardware limitations, such as pin-limited memory access and high computational complexity, leading to significant energy consumption and performance issues, especially in embedded devices.
Innovation Solution
The proposed solution involves hardware-aware mapping and partitioning of neural networks to exploit locality and sparsity, distributing task scheduling, and decoupling timing requirements between cores, allowing for localized processing and reduced data transfers, thereby minimizing energy consumption and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If neural network processing is implemented in general-purpose programming environments, then hardware availability and software compatibility are improved, but processing efficiency and energy consumption deteriorate
Solution Approach 1:
The patent segments the neural network into distinct computational layers (convolutional layers, fully connected layers, pooling layers) and processes them in parallel across multiple cores. Each layer is independently processed, allowing simultaneous execution of different neural network operations, thereby improving processing efficiency while maintaining hardware compatibility
Solution Approach 2:
The patent transitions from sequential single-core processing to parallel multi-core processing by adding a spatial dimension to the computation. Multiple cores operate simultaneously on different portions of the neural network, converting a one-dimensional sequential process into a multi-dimensional parallel process, which significantly enhances productivity
2Productivity
If data is transferred between memory and processing units, then computation is enabled, but energy consumption increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing input data (resizing, normalization, augmentation) before the main neural network computation. This prepares the data in advance in a format optimized for parallel processing, reducing the need for data manipulation during computation and minimizing energy-intensive data transfers during the critical computation phase
Solution Approach 2:
The patent maintains continuous useful action by implementing overlapping computation and data transfer operations. While one core computes neural network layers, other cores simultaneously prepare data for the next layers or process different batches, ensuring that processing units remain continuously utilized without idle waiting time, thereby improving computational throughput while optimizing energy usage
Data Source
AI summary
Methods and apparatus for localized processing within multicore neural networks. Unlike existing solutions that rely on commodity software and hardware to perform “brute force” large scale neural network processing the various techniques described herein map and partition a neural network into the hardware limitations of a target platform. Specifically, the various implementations described herein synergistically leverage localization, sparsity, and distributed scheduling, to enable neural network processing within embedded hardware applications. As described herein, hardware-aware mapping/partitioning enhances neural network performance by e.g., avoiding pin-limited memory accesses, processing data in compressed formats/skipping unnecessary operations, and decoupling scheduling between cores.