Offload Server Automates GPU Kernel Replacement via Code Pattern Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for automatic offloading of processing to GPUs and FPGAs are inefficient, particularly when trying to increase speed, as they often rely on simple loop statement offloading, which is insufficient compared to manual algorithm modifications, and require significant tuning and technical expertise.
Innovation Solution
An offload server that analyzes application code, detects external library calls, retrieves corresponding libraries and IP cores from a code pattern database, replaces processing descriptions, and generates offload patterns to optimize processing for GPUs or FPGAs, thereby offloading functional blocks in larger units like matrix integration and Fourier transformations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If automatic offloading is performed using simple loop statement extraction, then offloading can be enabled without manual intervention, but processing performance is insufficient compared to manual algorithm modifications
Solution Approach 1:
The system performs preliminary actions by pre-compiling and storing optimized kernel code for common library functions in a code pattern database before runtime. During offloading, the system matches application code against this pre-prepared database to retrieve optimized kernels, avoiding the need for manual algorithm modification while achieving high performance through pre-optimized code patterns.
Solution Approach 2:
The invention introduces an intermediary component - the code pattern database - that bridges automatic offloading and high performance. This database stores pre-optimized kernel code patterns that serve as intermediaries between simple automatic code analysis and complex manual algorithm optimizations, enabling the system to achieve manual-level performance through automated matching.
2Productivity
If manual algorithm modifications are performed to increase speed, then processing performance improves, but significant technical expertise and tuning are required
Solution Approach 1:
The system copies proven, optimized algorithm patterns from the code pattern database into the offloaded application code. Instead of requiring users to manually create optimized algorithms, the system automatically retrieves and inserts pre-optimized kernel code patterns that have been previously tuned by experts, thereby achieving high performance without requiring users to possess specialized knowledge.
Solution Approach 2:
The system automatically changes critical parameters such as block sizes, grid dimensions, and memory allocation strategies by matching application characteristics against stored code patterns. These parameter optimizations are automatically adjusted based on the retrieved patterns, eliminating the need for manual parameter tuning while maintaining high performance.
3Speed
If functional blocks are offloaded in larger units like matrix integration and Fourier transformations, then offload processing speed increases, but code analysis and matching complexity increases
Solution Approach 1:
The system segments complex functional blocks into recognizable code patterns that are stored in the database. By breaking down large functional blocks like matrix integration and Fourier transformations into smaller, standardized pattern units, the system enables efficient automatic matching while maintaining the ability to offload large functional units, thus reducing analysis complexity without sacrificing offload speed.
Data Source
AI summary
An offload server includes: an application code analysis section configured to analyze a source code of an application and detect external library calls included in the source code as replacement sources; a replacement function detection section configured to retrieve libraries and IP cores from a code pattern database by using the detected external library calls as keys, as replacement-destination libraries/IP cores; and a replacement processing section configured to replace processing descriptions of the replacement sources with processing descriptions of the replacement-destination libraries/IP cores retrieved by the replacement function detection section and to generate interfaces of a CPU to the replacement-destination libraries/IP cores.