Configurable Convolution Engine Offloads CPU
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing pipelines consume significant CPU bandwidth and power due to the execution of convolution operations, which reduces CPU availability for other processes and increases resource consumption.
Innovation Solution
A configurable convolution engine is introduced, comprising an input buffer circuit, datapath switch circuit, filter switch circuit, and computation core circuit, which configures operations to perform convolution on input data of various channels by interleaving data values and using kernels with different sizes and sparsity, thereby optimizing resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If convolution operations are performed by the CPU, then image processing can be done, but CPU bandwidth and power consumption increase significantly
Solution Approach 1:
The patent segments the convolution operation into distinct hardware components: input buffer for storing pixel data, kernel buffer for storing filter coefficients, and computation units for performing the actual convolution. This segmentation allows the convolution operation to be performed in dedicated hardware rather than through software execution on the CPU, thereby reducing CPU power consumption while maintaining image processing capability.
Solution Approach 2:
The patent introduces an intermediary image processing unit that acts as a mediator between the CPU and the convolution operation. This unit includes dedicated buffers and computation circuits that handle the convolution task, freeing the CPU from direct involvement in pixel-level computations and reducing overall system power consumption.
2Productivity
If convolution operations are performed by the CPU, then image processing can be done, but CPU availability for other processes decreases
Solution Approach 1:
By segmenting the convolution operation into dedicated hardware components (input buffer, kernel buffer, computation units), the patent offloads this specific task from the CPU. This allows the CPU to remain available for other processes while the convolution operation is handled independently by specialized hardware circuits.
Solution Approach 2:
The image processing unit with its configurable buffers and computation units can handle various convolution operations and image processing tasks, providing universal functionality that replaces multiple CPU-based processing routines and frees the CPU for diverse other tasks.
3Productivity
If traditional convolution operations are performed, then image processing is done, but resource consumption increases
Solution Approach 1:
The patent implements local quality optimization by creating specialized buffer structures and computation units tailored specifically for convolution operations. The input buffer and kernel buffer are designed with specific data layouts and access patterns optimized for convolution, reducing unnecessary data movement and computation overhead, thereby improving resource efficiency.
Solution Approach 2:
The patent performs preliminary actions by pre-loading pixel data into the input buffer and kernel coefficients into the kernel buffer before the convolution computation begins. This preliminary organization of data in optimized formats reduces the computational burden during the actual convolution operation, decreasing overall resource consumption.
Data Source
AI summary
Embodiments of the present disclosure relate to a configurable convolution engine that receives configuration information to perform convolution or its variant operations on streaming input data of various formats. To process streaming input data, input data of multiple channels are received and stored in an input buffer circuit in an interleaved manner. Data values of the interleaved input data are retrieved and forwarded to multiplier circuits where multiplication with a corresponding filter element of a kernel is performed. Varying number of kernels with different sizes and sparsity can also be used for the convolution operations.