Image Convolution Processing Circuitry with Merged Kernel Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image convolution systems are inefficient due to the inability to execute multiplication and accumulation operations and scalar load operations in the same clock cycle, leading to high memory bandwidth requirements and repeated loading of image rows for multiple kernels, which hampers their performance in executing image processing functions.
Innovation Solution
A system and method that merge kernels column-wise to generate a merged kernel, allowing for parallel execution of multiply-accumulate (MAC) and load instructions in a single clock cycle, with logical shift operations on the merged kernel, enabling efficient convolution of images by loading each row only once and reducing memory bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional convolution systems execute multiplication and accumulation operations separately from scalar load operations, then each operation can be performed with simple hardware, but the system cannot execute multiple operations in the same clock cycle, reducing processing efficiency
Solution Approach 1:
The patent merges the scalar load operation with the multiply-accumulate operation by loading the kernel coefficient into the accumulator register during the same clock cycle as the multiplication and accumulation operations. This is achieved by combining the scalar load instruction with the MAC instruction, allowing both operations to execute in parallel within a single clock cycle, thereby resolving the contradiction between processing efficiency and operation execution complexity
Solution Approach 2:
The accumulator register is designed to serve multiple functions: it accumulates the results of multiplication operations, stores kernel coefficients, and participates in both MAC and scalar load operations simultaneously. This multi-functional design allows the system to perform multiple operations in a single clock cycle without requiring separate dedicated registers for each function, thus improving productivity while managing device complexity
2Adaptability or versatility
If the system loads each row of the image multiple times for convolving with multiple kernels, then various image processing functions can be executed, but the memory bandwidth requirement increases significantly
Solution Approach 1:
The patent loads the kernel coefficients into the accumulator register in advance during the scalar load operation, so that when the multiply-accumulate operation executes, the kernel coefficient is already available in the accumulator register. This preliminary loading of kernel coefficients eliminates the need to reload them for each kernel operation, reducing memory bandwidth requirements while maintaining the ability to execute multiple image processing functions
Solution Approach 2:
Instead of loading the entire image row multiple times for different kernels, the system loads the image row once and stores it in the vector register. Then, different kernel coefficients are loaded into the accumulator register separately for each kernel operation. This copying approach allows the same image row to be reused with different kernels without repeated memory accesses, thereby reducing memory bandwidth while preserving adaptability for multiple image processing functions
3Reliability
If multiple clock cycles are consumed for loading each coefficient of the kernel, then accurate convolution can be performed, but the total execution time increases
Solution Approach 1:
The patent makes the scalar load operation continuous with the multiply-accumulate operation by executing both operations in the same clock cycle. The kernel coefficient is loaded into the accumulator register during the scalar load operation, and immediately used in the multiply-accumulate operation without waiting for the next clock cycle. This continuous execution eliminates idle time between loading and using kernel coefficients, maintaining convolution accuracy while reducing total execution time
Data Source
AI summary
A system for convolving an image includes a processing circuitry that retrieves the image including a set of rows, and a set of kernels, and merges serially all columns of each kernel, to generate a merged kernel. The processing circuitry executes parallelly multiple times, a multiply-accumulate (MAC) instruction on a row loaded in a corresponding vector register and a corresponding coefficient of the merged kernel and a load instruction on a subsequent row in one clock cycle. In the same clock cycle based on the MAC instruction, a logical shift operation is executed on the merged kernel to shift a current coefficient of the merged kernel with a subsequent coefficient such that the MAC instruction is executed on the subsequent row and the subsequent coefficient in the next clock cycle. Thus, each clock cycle is utilized by the system for executing both the MAC and load instructions.