Configurable Accelerator Framework for Mobile DCNN Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep convolutional neural networks (DCNNs) require significant computational resources and power, making them unsuitable for mobile devices due to their large size and high bandwidth, power, and computing resource demands.
Innovation Solution
A hardware accelerator framework is developed, including a system on chip (SoC) with a configurable accelerator framework that incorporates multiple digital signal processor (DSP) clusters, convolution accelerators, and a reconfigurable dataflow accelerator fabric, which enables efficient convolutional operations and reduces memory access, allowing for real-time computer vision applications in mobile devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional fully programmable solutions are used for DCNN processing, then computational flexibility is maintained, but power consumption and processing speed exceed mobile device capabilities
Solution Approach 1:
The system segments DCNN processing into distinct functional units: convolution accelerators for filter operations, DSP clusters for general-purpose computing, and specialized memory structures. This segmentation allows each component to be optimized for its specific function, enabling mobile devices to achieve high-speed DCNN processing while controlling power consumption through selective activation of processing units based on application requirements
Solution Approach 2:
The configurable accelerator framework provides dynamic reconfigurability, allowing the system to adapt its computational resources based on real-time requirements. The framework can dynamically allocate resources between convolution accelerators and DSP clusters, adjusting the processing pipeline to match the specific DCNN layer being executed, thereby optimizing the balance between processing speed and power consumption for different workloads
2Measurement precision
If DCNN models are made larger to improve accuracy, then recognition performance increases, but device size and resource requirements become unsuitable for mobile applications
Solution Approach 1:
The system implements a nested memory hierarchy with embedded on-chip memory structures that are integrated within the accelerator fabric. This nesting allows large DCNN models to be stored and processed efficiently within the mobile device by placing frequently accessed data (kernels, feature maps) in on-chip memory, reducing the need for external memory access and thereby supporting larger models without proportionally increasing device complexity
Solution Approach 2:
The configurable accelerator framework introduces a new dimension of reconfigurability that allows the system to adapt its computational capacity to match model size requirements. By configuring the number and type of processing units dynamically, the system can support varying model complexities without requiring a fixed high-capacity hardware design, thus enabling accurate recognition while controlling device complexity
3Productivity
If bandwidth is increased to support larger DCNN models, then data throughput improves, but power consumption and device complexity increase beyond mobile device capabilities
Solution Approach 1:
The system merges computation and memory functions by integrating embedded memory structures directly within the accelerator fabric and coupling them closely with processing units. This merging reduces the distance data must travel between storage and processing, enabling high data throughput for large DCNN models while avoiding the need for high-bandwidth external memory interfaces that would increase device complexity and power consumption
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Embodiments are directed towards a configurable accelerator framework device (400) that includes a stream switch (500) and a plurality of convolution accelerators (600). The stream switch (500) has a plurality of input ports and a plurality of output ports. Each of the input ports is configurable at run time to unidirectionally pass data to any one or more of the output ports via a stream link. Each one of the plurality of convolution accelerators (600) is configurable at run time to unidirectionally receive input data via at least two of the plurality of stream switch output ports, and each one of the plurality of convolution accelerators (600) is further configurable at run time to unidirectionally communicate output data via an input port of the stream switch.