DNN Accelerator Frontend Convolution Data Orchestration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face performance bottlenecks in frontend convolutional layers due to high computation costs and inefficient data loading, particularly when the number of input channels is small, leading to suboptimal acceleration and network-level performance degradation.
Innovation Solution
The implementation of a DNN accelerator with a compute block that includes a local memory, write module, read module, datastore, padding module, and PE array, utilizing an X-major layout for data storage and sparsity bitmaps to optimize data load and computation in frontend layers, allowing for efficient data reuse and reduced power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional data loading and computation methods are used in frontend convolutional layers, then implementation simplicity is maintained, but computation efficiency deteriorates and power consumption increases
Solution Approach 1:
The patent segments the data loading and computation process into distinct phases: data fetching from memory, tiling into intermediate buffers, orchestration into compute blocks, and execution on PE arrays. This segmentation allows each stage to be optimized independently, resolving the contradiction between efficiency and complexity by making the complex processes manageable and optimizable in discrete steps.
Solution Approach 2:
The patent performs preliminary actions by pre-fetching data into intermediate buffers and pre-tiling data before computation. The data is organized into tiles and stored in intermediate buffers ahead of time, allowing the compute blocks to execute without waiting for data retrieval during computation. This preliminary data preparation significantly improves computation efficiency while the complexity is managed through systematic data flow design.
2Loss of energy
If data is loaded and computed in traditional sequences, then implementation simplicity is maintained, but data reuse efficiency deteriorates leading to increased power consumption
Solution Approach 1:
The patent ensures continuous useful action by orchestrating data flow so that processing elements continuously operate on available data tiles without idle cycles. Data is loaded, processed, and reused in a continuous pipeline fashion across multiple compute blocks, maximizing utilization of computational resources and reducing energy waste from idle processing units.
Solution Approach 2:
The patent implements data reuse mechanisms where intermediate buffer data is recovered and reused across multiple compute blocks. Instead of reloading the same data multiple times from main memory, the system recovers and reuses data already in intermediate buffers, significantly reducing energy consumption while managing complexity through systematic data tracking and reuse scheduling.
3Productivity
If standard convolution operations are performed, then algorithm simplicity is maintained, but computation cost increases due to redundant operations
Solution Approach 1:
The patent segments the convolution computation into discrete compute blocks that operate on tiled data segments. Each compute block processes a specific portion of the convolution operation independently, allowing for optimized computation schedules and reducing redundant operations across the entire network while maintaining algorithmic correctness.
Solution Approach 2:
The patent introduces dynamic computation scheduling where the computation pipeline adapts to data availability and processing status. Compute blocks are dynamically activated and deactivated based on data flow, allowing the system to optimize computation speed by executing only necessary operations at each time step rather than following a rigid static schedule.
Data Source
AI summary
An DNN (deep neural network) accelerator may accelerate deep learning, such as convolutions in frontend layers through a scheduler for loading data to be processed. The DNN accelerator may store, in a memory, an input tensor of a convolutional layer in a DNN. The convolutional layer may be the first layer or a layer that is arranged before the one or more other convolutional layers in the DNN such that data processed by the first layer can be efficiently reused across data load rounds. The input tensor includes one or more channels. A channel includes activations arranged in rows and columns. The DNN accelerator may read at least a portion of the input tensor from the memory into a datastore. The datastore includes some databanks. The DNN accelerator may provide a vector of one or more activations to a processing element for operations such as multiplications on the vector.