Neural Network Accelerator Scheduling for Variable Layer Parallelism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware configurations for artificial neural networks often have suboptimal performance due to fixed layer configurations that do not accommodate varying parallelisms, leading to over-provisioning, increased costs, reduced reliability, and excess power consumption.
Innovation Solution
A signal processing system comprising a CPU and an accelerator with a plurality of processing engines (PEs) and an instruction scheduler, which dynamically configures operators to match computation demands, allowing for flexible implementation of operators like ReLU and pooling operations, and enables efficient processing of bitmap image files for feature extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If fixed configuration of processing engines is used for each layer, then hardware implementation is simplified, but performance becomes suboptimal due to inability to accommodate varying parallelisms
Solution Approach 1:
The patent implements dynamic configuration of processing engines where the number and arrangement of PEs can be adjusted based on the specific layer requirements. Each layer can be configured with an optimal number of PEs matching its parallelism needs, transforming the static hardware architecture into a dynamically adaptable system that optimizes performance for different neural network operations.
Solution Approach 2:
The patent creates a universal processing engine architecture where a single pool of PEs can serve multiple layers with different computational requirements. The processing engines are designed to be reconfigurable and can be allocated to different layers as needed, allowing the same hardware resources to fulfill multiple functions across various neural network layers.
2Productivity
If over-provisioning of processing engines is done to handle varying parallelisms, then performance requirements are met, but cost increases
Solution Approach 1:
The system dynamically allocates processing engines to layers based on actual computational demands. Rather than having a fixed over-provisioned number of PEs, the architecture allows the active number of processing engines to vary depending on which layers are being executed and their specific parallelism requirements, eliminating the need for excessive hardware provisioning.
Solution Approach 2:
The patent changes the parameter of processing engine allocation from a fixed static value to a dynamic value that adjusts based on layer requirements. This parameter change allows the system to optimize the number of active PEs for each specific operation, avoiding the cost of maintaining a permanently over-provisioned hardware configuration.
3Productivity
If over-provisioning of processing engines is done, then performance requirements are met, but power consumption increases
Solution Approach 1:
The patent implements dynamic power management where processing engines are activated only when needed for specific layer executions. The system adjusts the active processing engine count to match the computational workload, preventing unnecessary power consumption from idle or over-provisioned hardware components while maintaining required performance levels.
Solution Approach 2:
The system dynamically changes the operational parameter of processing engine activation states, transitioning between active and idle states based on real-time computational demands. This parameter adjustment optimizes power consumption by ensuring that processing engines consume energy only when performing useful computational work rather than remaining in a permanently active state.
4Device complexity
If fixed layer configuration is used, then hardware design is simplified, but resource utilization becomes inefficient
Solution Approach 1:
The patent introduces dynamic configurability to the hardware architecture, allowing the processing engine allocation and layer mapping to adapt based on the specific neural network being executed. This dynamic approach maintains relatively simple hardware design while significantly improving resource utilization efficiency through flexible, on-the-fly configuration adjustments.
Data Source
AI summary
A signal processing system includes a central processing unit (CPU) in communication with an accelerator, and an instruction scheduler in communication with the accelerator. A first memory device including a first instruction set is configured to operate the accelerator, a second instruction set is configured to operate the CPU, and a second memory device is configured to receive a datafile. The accelerator includes a plurality of processing engines (PEs) and an instruction scheduler, the instruction set includes a plurality of operators, and the instruction scheduler is configured to implement the operators in the accelerator employing the PEs. The CPU employs the operators implemented in the accelerator to analyze the datafile to extract a feature therefrom.