Toeplitz Kernel Image Convolution Without Input Buffers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems face challenges in performing computationally intensive tasks like feature extraction with high power consumption and processing time, particularly in resource-constrained environments such as autonomous drones and virtual reality applications.
Innovation Solution
An image convolution accelerator system using a Toeplitz-like kernel, which is generated based on a desired convolution kernel and padded with zeros, activates processing elements to multiply with image data when allocated to non-zero rows, enabling efficient convolution without input buffers and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional convolution methods are used for image processing, then feature extraction can be performed, but power consumption and processing time become excessively high
Solution Approach 1:
The patent divides the convolution operation into two independent phases: a matrix multiplication phase that processes multiple pixels simultaneously using parallel processing elements, and a summation phase that accumulates results. This segmentation allows the computationally intensive multiplication to be performed efficiently in parallel while separating it from the accumulation operation, thereby reducing overall processing time and power consumption.
Solution Approach 2:
The patent pre-computes and stores the convolution kernel as a Toeplitz matrix before processing image data. By preparing the kernel structure in advance with proper zero-padding and arrangement, the system eliminates the need for repeated kernel computations during image processing, significantly reducing real-time processing time and energy consumption.
2Measurement precision
If high-resolution images are processed to improve feature extraction accuracy, then measurement precision improves, but processing time and power consumption increase
Solution Approach 1:
The patent transforms the 2D convolution operation into a structured matrix multiplication problem by rearranging the convolution kernel into a Toeplitz matrix format. This dimensional reorganization allows the system to process multiple pixel values simultaneously through parallel processing elements, achieving high-resolution image processing without the linear increase in processing time that would otherwise occur with higher resolution.
3Productivity
If conventional convolution algorithms are implemented, then image processing can be performed, but device complexity increases due to input buffers and control logic
Solution Approach 1:
The patent extracts and eliminates the need for traditional input buffers and complex control logic by directly implementing the convolution operation through parallel processing elements that multiply kernel values with image pixels in a systematic manner. The Toeplitz matrix structure provides a regular, predictable pattern that simplifies control requirements while maintaining high convolution efficiency.
Data Source
AI summary
An image convolution accelerator system including processing elements having kernel elements corresponding to a generated Toeplitz-like kernel. The Toeplitz-like generated kernel is based on a desired convolution kernel and padded with zeros, and an image controller is configured to activate the processing elements to multiply the kernel elements with image data when image data from a same row of the image is allocated to all processing elements corresponding to a non-zero row of the Toeplitz-like kernel. At least one storage element is configured to store at least the number of columns of the image minus a dimension of the desired convolution kernel+1.