Adaptive BLAS Acceleration on Embedded GPUs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for accelerating BLAS operations in embedded systems, particularly using OpenCL, are not optimized for the unique memory structures of embedded systems, leading to suboptimal performance and potential overhead, as they are based on desktop PC architectures and do not effectively utilize GPU capabilities.
Innovation Solution
An apparatus and method that automatically sets optimal OpenCL parameters and creates kernels for BLAS operations based on machine-learning data attribute information and OpenCL device information, enabling adaptive acceleration of BLAS operations on GPUs within embedded systems, including memory mapping and local memory optimization, to enhance performance without user intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing OpenCL optimization methods are applied in embedded systems, then some performance improvement may be achieved, but they act as overhead and do not effectively utilize GPU capabilities due to mismatch with unified memory structure
Solution Approach 1:
The patent dynamically changes OpenCL parameters (work group size, local memory size, vector width) based on the specific characteristics of embedded system GPU devices and BLAS operation types. This allows the system to adapt to different hardware configurations and operation requirements, achieving optimal performance without fixed overhead from desktop-optimized parameters
Solution Approach 2:
The system automatically selects and configures optimal OpenCL parameters and kernel implementations without requiring manual intervention or complex user configuration. The automatic selection mechanism analyzes the embedded system's GPU capabilities and BLAS operation characteristics to choose the most appropriate configuration, eliminating the overhead of manual optimization while achieving high performance
2Speed
If local memory is used to minimize memory access expense, then performance may improve, but it may not have performance advantages over global memory in embedded systems with unified memory structure
Solution Approach 1:
The patent dynamically selects between using local memory or global memory for BLAS operations based on the specific operation type and data characteristics. Rather than statically allocating data to local memory, the system adapts its memory access strategy to match the unified memory structure of embedded GPUs, using local memory only when it provides actual performance benefits and falling back to global memory otherwise
Solution Approach 2:
The patent applies different memory access strategies to different parts of the computation based on local requirements. For example, it may use local memory for certain intermediate calculations while using global memory for other operations, optimizing each segment according to its specific needs rather than applying a uniform memory strategy throughout
3Productivity
If work group size is optimized to maximize workload of each kernel, then performance improves, but it requires complex research and configuration that is not sufficient for embedded systems
Solution Approach 1:
The system automatically determines the optimal work group size based on the embedded GPU's capabilities and the BLAS operation characteristics, eliminating the need for manual configuration or complex research. The automatic selection process analyzes device properties and operation parameters to choose the most appropriate work group size, making the system easy to use while achieving high kernel execution efficiency
Solution Approach 2:
The patent dynamically adjusts the work group size parameter based on the specific embedded system configuration and BLAS operation type. This allows the system to optimize kernel execution efficiency for different scenarios without requiring manual intervention, adapting the work group size to match the capabilities of the embedded GPU and the requirements of the operation being performed
Data Source
AI summary
Disclosed herein are an apparatus and method for adaptively accelerating a BLAS operation based on a GPU. The apparatus for adaptively accelerating a BLAS operation based on a GPU includes a BLAS operation acceleration unit for setting optimal OpenCL parameters using machine-learning data attribute information and OpenCL device information and for creating a kernel in a binary format by compiling kernel source code; an OpenCL execution unit for creating an OpenCL buffer for a BLAS operation using information about an OpenCL execution environment and the optimal OpenCL parameters and for accelerating machine learning in an embedded system in such a way that a GPU that is capable of accessing the created OpenCL buffer performs the BLAS operation using the kernel, and an accelerator application unit for returning the result of the BLAS operation to a machine-learning algorithm.