ML Model Reshaping for GPU Acceleration Path Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models, particularly deep learning models like CNNs, are not optimally configured to utilize the acceleration paths of specialized hardware such as GPUs, leading to inefficient computation and execution.
Innovation Solution
The method involves configuring raw input data and machine learning models to fit the prerequisites of specialized computing devices, such as GPUs, by reshaping data and adjusting model functions to align with the hardware's acceleration paths, specifically using techniques like replicating and expanding filters to meet the required dimensions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If machine learning models are trained in a conventional manner, then the models can be developed and deployed, but they cannot effectively utilize the acceleration paths of specialized hardware such as GPU tensor cores
Solution Approach 1:
The patent applies preliminary action by configuring and reshaping input data and model filters before execution on specialized hardware. The system pre-processes the data to match the specific dimensional requirements of GPU tensor cores, ensuring that the acceleration paths can be fully utilized during inference without requiring model retraining or hardware modification.
2Adaptability or versatility
If additional layers are added in CNN models or pitched memory copies are created to accommodate hardware requirements, then the models can execute on specialized hardware, but computation and memory needs increase
Solution Approach 1:
The patent applies dimensionality change by reshaping input data from its original dimensional format into a configuration that matches the requirements of GPU tensor cores. This involves transforming the data structure across different dimensions (e.g., channel, height, width arrangements) to enable efficient parallel processing without adding memory overhead or requiring additional model layers.
3Productivity
If CNN models are retrained with special needs for hardware acceleration, then they can utilize acceleration paths, but training time and computational resources increase
Solution Approach 1:
The patent extracts the hardware-specific configuration requirements from the model training process itself. Instead of retraining models with hardware constraints built-in, the system separates the general model training phase from the hardware optimization phase by applying data and filter configurations that match GPU tensor core requirements, thereby achieving hardware efficiency without sacrificing training flexibility.
4Productivity
If data and models are configured to fit specialized hardware requirements, then computational efficiency is improved, but data preprocessing complexity increases
Solution Approach 1:
The patent applies parameter changes by systematically adjusting data dimensional parameters (channel, height, width) and filter configurations to match the specific requirements of GPU tensor cores. This involves transforming data into formats such as NHWC or NCHW with specific channel counts (e.g., multiples of 8) and adjusting filter dimensions accordingly, thereby enabling efficient execution on specialized hardware through parameter optimization rather than architectural changes.
Data Source
AI summary
Systems and methods of executing a machine learning model on a specialized computing device can comprise obtaining raw input data by a first computing device; obtaining the machine learning model including a function that applies a set of M model parameters to at least one channel of the raw input data; determining a configuration parameter K for the specialized computing device; configuring the raw input data based on the configuration parameter to obtain configured input data; configuring the machine learning model based on the configuration parameter to obtain a configured machine learning model with a configured model dimension corresponding to the data size of the acceleration path; executing the configured machine learning model with the configured model parameter using the configured input data to obtain output data; and providing the output data.