Configurable Convolution Engine for Interleaved Channel Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing pipelines consume significant CPU bandwidth and power due to the execution of image processing algorithms, particularly convolution operations, which reduces CPU availability for other processes.
Innovation Solution
A configurable convolution engine with multiple convolution circuits and a channel merge circuit that performs convolution and machine learning operations, allowing for various modes of operation such as dual-convolution, cascade, and parallel modes to efficiently process interleaved multi-channel image data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If convolution operations are performed by the CPU, then image processing can be executed with flexibility, but CPU bandwidth and power consumption increase significantly
Solution Approach 1:
The patent divides the convolution operation into two separate convolution circuits (first and second convolution circuits), each handling different sets of channels. This segmentation allows parallel processing of channel data, reducing the computational burden on the CPU while maintaining flexibility through configurable kernel applications.
Solution Approach 2:
The patent introduces a channel merge circuit as an intermediary component that combines the output streams from the two convolution circuits. This mediator handles the complex channel interleaving and merging operations, freeing the CPU from detailed data manipulation tasks while preserving system flexibility through programmable control.
2Adaptability or versatility
If convolution operations are performed by the CPU, then various image processing algorithms can be implemented, but CPU availability for other processes decreases
Solution Approach 1:
By segmenting the convolution workload across two dedicated hardware circuits, the patent enables simultaneous processing of multiple channel streams, thereby increasing overall processing throughput and freeing CPU resources for other tasks.
Solution Approach 2:
The patent replaces CPU-based software execution with dedicated hardware convolution circuits that perform convolution operations in parallel. This substitution transfers computational tasks from the general-purpose CPU to specialized hardware, maintaining algorithmic versatility while significantly improving CPU availability.
3Productivity
If multiple convolution circuits process interleaved channel data separately, then processing throughput increases, but channel data must be merged efficiently
Solution Approach 1:
The channel merge circuit serves as a specialized intermediary that efficiently handles the complex task of merging interleaved channel streams from two convolution circuits. It performs channel interleaving and data reorganization in hardware, managing the complexity of multi-channel data integration while enabling high-throughput parallel processing.
Solution Approach 2:
The patent processes different channel sets in parallel across two convolution circuits, effectively adding a temporal dimension to the processing pipeline. The channel merge circuit then reorganizes these parallel streams into a unified interleaved output, utilizing dimensional transformation to manage data flow complexity.
Data Source
AI summary
Embodiments relate to a configurable convolution engine that receives configuration information to perform convolution and other deep machine learning operations on streaming input data of various formats. The convolution engine may include two convolution circuits that each generate a stream of values by applying convolution kernels to input data. The stream of values may each define multiple channels of image data. A channel merge circuit interleaves the streams of values from the convolution circuits to generate an output stream of output values. The output stream includes the data multiple channels of each input stream arranged in an interleaved manner.