Depthwise Separable Convolution Accelerator With Multi-Buffer Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing accelerators for neural networks, such as TPU, DaDianNao, and EIE, are not suitable for low-end edge devices due to high memory requirements, and even when computational speed is enhanced, performance is not improved due to significant memory access latency.
Innovation Solution
An accelerator system with an off-chip memory, buffers, and a bus architecture that reduces memory access latency by using multiple buffers to temporarily store data and instructions, and performs depthwise separable convolution to enhance processing unit utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If accelerators use large capacity on-chip memory or frequent off-chip memory access to enhance computational speed, then computational throughput is improved, but memory access latency increases and performance is not relatively improved
Solution Approach 1:
The patent segments the memory system into multiple buffers (first buffer, second buffer, third buffer, fourth buffer) distributed across different locations (off-chip and on-chip). This segmentation allows parallel data loading and processing, where different buffers can simultaneously hold input data, weights, intermediate results, and output data, thereby reducing memory access latency while maintaining high computational throughput.
Solution Approach 2:
The patent implements preliminary action by pre-loading data into buffers before computation is needed. The first buffer pre-stores input data, the second buffer pre-loads instructions, and the fourth buffer prepares to store output data. This preliminary data preparation in buffers reduces waiting time during actual computation, effectively reducing memory access latency while maintaining high throughput.
2Productivity
If accelerators are designed for high computational speed, then processing capability is enhanced, but device complexity increases making them unsuitable for low-end edge devices
Solution Approach 1:
The patent transitions from a single large on-chip memory architecture to a multi-dimensional buffer distribution architecture. Instead of using one large memory block, the system distributes data across multiple smaller buffers located in different dimensions (off-chip first buffer, on-chip second/third/fourth buffers). This dimensional redistribution reduces the complexity of any single memory component while maintaining overall processing capability.
Solution Approach 2:
The patent introduces buffers as intermediary components between the computation units and the main memory systems. These buffers act as mediators that temporarily hold data close to the computation units, reducing the need for frequent access to large capacity memory. This intermediary buffer architecture reduces device complexity while preserving processing capability.
3Loss of time
If multiple buffers are introduced to reduce memory access latency, then performance is improved, but device complexity increases
Solution Approach 1:
The patent designs the buffer architecture with multi-functionality where the same buffer structure can serve multiple purposes: the first buffer handles input data, the second buffer handles instructions, the third buffer handles intermediate data transfer, and the fourth buffer handles output data. This universal buffer design pattern reduces overall system complexity by using a standardized multi-functional approach rather than specialized components for each function.
Data Source
AI summary
An accelerator system includes an off-chip memory, a bus, and an accelerator. The off-chip memory includes a first buffer. The accelerator includes a second buffer, a third buffer, a fourth buffer, an instruction fetching module, a load module, a storage module, a convolution control module, and a computation module. The instruction fetching module fetches instructions from the off-chip memory via the bus. The load module loads input data from the off-chip memory via the bus. The storage module stores the output data to the off-chip memory via the bus. The computation module performs convolution computations based on the input map and multiple kernels allocated by the convolution module. In response to loading the input data or instruction from the off-chip memory, the requested input data or instruction is transferred from the off-chip memory and temporarily stored in the first buffer.