SIMD VLIW Processor Sequencing for Edge CNN Power Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning accelerators face performance bottlenecks when implemented in edge devices with strict power consumption constraints, particularly in real-time inference tasks for applications like virtual reality, augmented reality, and mobile devices, due to high compute requirements of convolutional neural networks.
Innovation Solution
A processor architecture featuring a VLIW design with a MAC array, multiple vector register arrays, and a multiplexer, supporting SIMD instructions and dynamic configuration for convolution operations, optimized for efficient data processing and power management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing ML accelerators use high compute parallelism to speed up convolutional layers, then processing speed is improved, but power consumption increases making them unsuitable for edge devices
Solution Approach 1:
The processor is divided into multiple function units (FUs) that can be selectively activated. Each FU handles specific computational tasks, allowing the system to segment the compute workload and activate only the necessary units, thereby reducing overall power consumption while maintaining processing speed for convolutional operations.
Solution Approach 2:
The processor implements dynamic configuration of the MAC array shape and selective activation of function units based on the specific computational requirements of the neural network layer being processed. This dynamic adaptation allows the system to optimize between compute throughput and power consumption by activating only the necessary computational resources for each operation.
2Productivity
If a VLIW processor with multiple function units is used to improve compute utilization, then processing efficiency is improved, but device complexity increases
Solution Approach 1:
Multiple function units are designed with universal capabilities to handle various neural network operations (convolutions, activations, etc.). Each FU can be configured to perform different functions, reducing the need for specialized hardware for each operation type and simplifying the overall architecture while maintaining high compute utilization.
Solution Approach 2:
The processor allows dynamic reconfiguration of function unit parameters and MAC array dimensions to match the specific requirements of different neural network layers. This parameter-based configuration enables a single processor architecture to efficiently handle diverse workloads without requiring complex specialized hardware for each operation type.
3Productivity
If SIMD instructions process fixed-size vector data to improve throughput, then processing throughput is improved, but adaptability to different data sizes is reduced
Solution Approach 1:
The processor implements dynamic vector length configuration where the size of vectors processed by SIMD instructions can be adjusted based on the input data characteristics. This allows the system to maintain high throughput by using vectorized operations while adapting to different data sizes by changing the vector length parameter, thus resolving the contradiction between throughput and adaptability.
Data Source
AI summary
In one embodiment, a system comprising a processor and a non-transitory memory coupled to the processor comprising instructions executable by the processor. The processor, comprising an internal memory; a Multiply-Accumulate (MAC) array; a first vector register array; a second vector register array; and a third vector register array, is operable when executing instructions to transfer weights for M filters and an input activation tensor from an external memory to the internal memory, insert paddings to the input activation tensor in the internal memory based on first configuration parameters, configure the MAC array to a required shape based on second configuration parameters for convolution operations between the input activation tensor and the M filters, and calculate a row of the output activation tensor by performing the convolution operations on corresponding R rows of the input activation tensor with the M filters, wherein R is a filter height.