Circular Buffer for CNN Convolution Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional convolution computations for convolutional neural networks (CNNs) are slow and power-inefficient due to non-contiguous memory access and the requirement for large scratch memory, which hinders real-time processing in embedded systems.
Innovation Solution
Implementing a circular FIFO buffer that transforms the input cube into a contiguous linear array, allowing processors to perform matrix multiplication for convolution computations without memory movement, using a pointer to traverse and update the buffer efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional convolution computations traverse input data for each CNN filter non-contiguously, then complete input processing is achieved, but memory access speed decreases and computation becomes slow
Solution Approach 1:
The patent reorganizes the three-dimensional input cube into a two-dimensional buffer layout where the slowest-varying dimension is collapsed. This dimensional transformation allows the processor to access data contiguously in the remaining dimensions while maintaining the full three-dimensional data structure, thereby improving memory access speed without sacrificing computational completeness.
Solution Approach 2:
The patent introduces a circular buffer as an intermediary data structure between the input cube and the convolution operation. This buffer serves as a mediator that stores input data in a contiguously accessible format, allowing the processor to perform fast sequential reads while the underlying data maintains its original three-dimensional semantics through virtual indexing.
2Productivity
If conventional convolution computations store complete input as a single cube, then all input data is available for processing, but large scratch memory is required
Solution Approach 1:
The patent segments the complete input cube into manageable chunks that fit within the circular buffer. By processing the input data in segments rather than requiring the entire cube to be loaded simultaneously, the system achieves real-time processing capability while minimizing the memory footprint to only what is currently needed for computation.
Solution Approach 2:
The patent implements a dynamic buffer management system where the circular buffer size is optimized to hold only the necessary portion of input data for current processing. As new data arrives and old data is processed, the buffer dynamically updates its contents, ensuring that memory usage remains minimal while maintaining the ability to process complete inputs over time.
3Reliability
If conventional convolution computations require complete input storage, then accurate convolution results are produced, but power efficiency decreases due to large memory and non-contiguous access
Solution Approach 1:
The patent transforms the three-dimensional input cube into a two-dimensional buffer representation, collapsing one dimension to reduce memory access complexity. This dimensional reduction maintains computational accuracy by preserving all necessary data relationships while enabling contiguously sequential access patterns that significantly improve power efficiency by reducing activation energy during memory operations.
Data Source
AI summary
Using a buffer sized according to the size of the filters of a convolutional neural network (CNN), a processor may use a read pointer to generate a two-dimensional virtual matrix of inputs. The number of inputs in each row in the two-dimensional virtual matrix of inputs may match the one-dimensional filter size of the cubic filters. The processor may collapse each of the cubic filters to one-dimensional linear arrays and generate a two-dimensional filter matrix from the one-dimensional linear arrays. The convolution computations for a corresponding layer of the CNN therefore reduce to a single matrix multiplication without any memory movement operations. When the buffer is refreshed using a new input frame, the processor may increment the initial read address of each read pointer by one and increment the final read address by one, circling back to the corresponding initial read address.