Neural Network Kernel Selection for Matrix Computations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Selecting an optimal kernel for matrix computations is challenging due to various performance-affecting factors, and existing methods require significant time and resources for training deep neural networks to accurately predict kernel relevancies, especially when dealing with diverse input sizes and hardware constraints.
Innovation Solution
A system that utilizes a deep neural network to rank kernels based on input factors and hardware specifications, with a kernel selection process involving a candidate generator, filter, neural network, sorter, and selection engine to efficiently choose the most relevant kernel for computations, and further trains the network using generated data to address difficult prediction areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a deep neural network is trained to accurately predict kernel relevancies, then kernel selection accuracy is improved, but training time and resource consumption increase
Solution Approach 1:
The system performs preliminary actions by generating synthetic training data that represents difficult prediction scenarios before actual kernel selection is needed. This pre-generated training data enables the neural network to be trained more efficiently on targeted cases, improving accuracy without requiring extensive training on all possible input variations.
Solution Approach 2:
The system applies local quality by focusing training efforts on specific difficult prediction areas rather than uniformly training on all possible inputs. By identifying and generating training data for problematic cases (such as specific matrix size ranges or hardware configurations), the neural network achieves higher accuracy in these critical regions while reducing overall training time.
2Measurement precision
If comprehensive training data is used to train the neural network, then prediction accuracy across diverse inputs is improved, but training resource consumption increases
Solution Approach 1:
The system extracts and isolates the most critical training needs by identifying difficult prediction areas and generating targeted synthetic data for those specific cases. This extraction approach allows the neural network to be trained on a smaller, more focused dataset that captures the essential challenging scenarios, reducing overall training resource consumption while maintaining prediction accuracy.
3Productivity
If multiple kernels are evaluated to ensure optimal selection, then computation performance is improved, but selection time increases
Solution Approach 1:
The neural network performs preliminary evaluation and ranking of candidate kernels based on input characteristics before actual kernel execution. This preliminary action allows the system to predict which kernels are likely to perform best without exhaustively evaluating all possible kernels, thus maintaining high computation performance while reducing selection time.
Data Source
AI summary
Apparatuses, systems, and techniques to optimize kernel selection for performing a computation. In at least one embodiment, a neural network is trained and utilized to generate a list of kernels so that an (e.g., optimal) kernel may be identified. The neural network receives characteristics of the input matrices and determines relevancy scores for a list of possible kernels. Based on an ordered listing of kernels by relevant score, a kernel is selected from the list and utilized to perform the computation and provide the result.