Preprocessor Circuit for Strided Convolution Data Formatting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolution techniques for image processing in deep learning are limited by their inability to efficiently implement strided or dilated convolutions due to conflicts between shift register-based data processing and the need for skipping samples in the input image, leading to inefficiencies in memory usage and computational complexity.
Innovation Solution
A preprocessor circuit architecture that utilizes multiplexers and shift registers to format image data into parallel streams, allowing for efficient strided or dilated access patterns, and supports various convolutional filter sizes with minimal area penalty, enabling efficient convolution operations in systolic arrays for CNNs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If shift register-based data processing is used for convolution operations, then data processing capability is improved, but the ability to implement strided or dilated convolutions deteriorates due to conflicts with skipping samples requirement
Solution Approach 1:
The patent divides the input image data into multiple separate shift registers, with each shift register handling a specific column or set of columns. This segmentation allows different shift registers to independently process data with different stride patterns, resolving the conflict between sequential processing and the need to skip samples. Each shift register can be configured to output data at different rates, enabling strided and dilated convolutions while maintaining efficient parallel processing.
Solution Approach 2:
The patent introduces dynamic control mechanisms that allow the shift registers to operate with variable timing and data output rates. By dynamically adjusting the clocking and data flow through each shift register, the system can adapt to different convolution requirements (standard, strided, or dilated) without sacrificing processing efficiency. This dynamic operation enables the system to switch between different processing modes as needed.
2Power
If large number of digital signal processors are used for convolution operations, then computational capability is improved, but device complexity and memory requirements worsen
Solution Approach 1:
The patent combines multiple functions into a unified shift register-based architecture that performs data loading, buffering, and sequential processing in a single integrated structure. Instead of using separate DSPs for each processing function, the system merges these operations into the shift register fabric, reducing the total number of processing elements while maintaining computational capability. This integration significantly reduces device complexity and memory requirements.
Solution Approach 2:
The patent creates a universal shift register architecture that can perform multiple convolution operations (different filter sizes, strides, and dilations) using the same hardware structure. The shift registers are designed to be reconfigurable and can handle various data formats and processing requirements, eliminating the need for dedicated DSPs for each specific operation. This multi-functionality reduces both device complexity and memory requirements while maintaining high computational capability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An example preprocessor circuit for formatting image data into a plurality of streams of image samples includes: a first buffer (705) configured to store a plurality of rows (814) of the image data (802) and output a row of the plurality of rows; a second buffer (708), coupled to the first buffer (705), including a plurality of storage locations (709) to store a respective plurality of image samples of the row (814) output by the first buffer (705); a plurality of shift registers (712); an interconnect network (710) including a plurality of connections (718), each connection (718) coupling a respective one of the plurality of shift registers (712) to more than one of the plurality of storage locations (709), one or more of the plurality of storage locations (709) being coupled to more than one of the plurality of connections (718); and a control circuit (714) configured to load (1608) the plurality of shift registers (712) with the plurality of image samples based on the plurality of connections (718) and shift (1616) the plurality of shift registers (712) to output the plurality of streams of image samples.