Compiler Kernel Fusion for Image Processing Power Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional image processing technologies face challenges in balancing versatility in software development with power efficiency, as general-purpose processors consume more energy due to finer grained data structures and custom-designed processors are limited in tasks they can perform.
Innovation Solution
A method and apparatus for compiling program code targeted for image processors with programmable stencil processors, involving techniques like kernel fusion, fission, and spatial partitioning to optimize execution on directed acyclic graphs, addressing differences in kernel computational intensity and resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general purpose processors are used for image processing, then versatility in application software development is improved, but power consumption per unit of data increases
Solution Approach 1:
The image processing task is segmented into multiple kernels that can be independently compiled and executed on different stencil processors. Each kernel handles a specific portion of the processing pipeline, allowing the system to achieve high performance for specific tasks while maintaining overall versatility through the combination of multiple specialized kernels.
Solution Approach 2:
The system employs dynamic kernel selection and composition, where the compiler automatically selects and combines appropriate kernels based on the specific image processing task requirements. This dynamic adaptation allows the hardware to efficiently handle diverse image processing applications while maintaining optimized power consumption for each specific task.
2Use of energy by moving object
If custom designed fixed function circuitry is used, then power consumption per unit of data is reduced, but the set of performable tasks becomes limited
Solution Approach 1:
The stencil processor architecture provides a universal platform that can execute multiple different kernels, each optimized for specific image processing tasks. The combination of standardized hardware with programmable kernels enables the system to perform a wide variety of tasks while maintaining the power efficiency of specialized circuitry for each task type.
Solution Approach 2:
The system changes operational parameters by selecting different kernels for different tasks. Each kernel is optimized for specific parameter ranges and task types, allowing the same hardware platform to efficiently handle diverse image processing operations by adjusting which kernel is executed rather than requiring different hardware for each task.
3Adaptability or versatility
If the number of kernels exceeds the number of stencil processors, then task specialization is improved, but device complexity increases
Solution Approach 1:
The compiler merges multiple kernels into combined kernel executions when beneficial, reducing the total number of kernel invocations and simplifying the mapping to available stencil processors. This merging strategy maintains task specialization by preserving the functional logic of individual kernels while reducing orchestration complexity through combined execution units.
4Productivity
If kernel fusion is applied, then execution efficiency is improved, but memory resource requirements increase
Solution Approach 1:
The system applies partial kernel fusion, merging only those kernels that benefit from reduced data transfer and improved execution efficiency, while leaving other kernels separate when their memory requirements would be excessive. This selective fusion approach balances execution efficiency gains against memory resource consumption based on the specific characteristics of each kernel pair.
Data Source
Figure 1
Figure 2a
Figure 2b
AI summary
A method is described. The method includes compiling program code targeted for an image processor having programmable stencil processors composed of respective two-dimensional execution lane and shift register circuit structures. The program code is to implement a directed acyclic graph and is composed of multiple kernels that are to execute on respective ones of the stencil processors, wherein the compiling includes any of: recognizing there are a different number of kernels in the program code than stencil processors in the image processor; recognizing that at least one of the kernels is more computationally intensive than another one of the kernels; and, recognizing that the program code has resource requirements that exceed the image processor's memory capacity. The compiling further includes in response to any of the recognizing above performing any of: horizontal fusion of kernels; vertical fusion of kernels; fission of one of the kernels into multiple kernels; spatial partitioning of a kernel into multiple spatially partitioned kernels; splitting the directed acyclic graph into smaller graphs.