Deep Learning Input Segmentation for GPU Thread Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models face performance issues due to unbalanced thread requirements across GPUs, leading to decreased device performance and impaired parallelism, as heavy kernel functions consume all thread resources, leaving others without resources and resulting in significant scheduling overhead.
Innovation Solution
A method that determines a factor associated with a deep learning model input to adjust the number of threads, dividing large inputs into smaller partial inputs, which are processed in parallel, optimizing load balancing and device performance by enabling parallelism in deep learning operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep learning models use a significant amount of calculations to improve performance, then accuracy and capability are improved, but computational resource consumption increases
Solution Approach 1:
The patent segments the input data into multiple partial inputs and processes them through separate threads in parallel. This divides the heavy computational load into manageable chunks that can be executed simultaneously, reducing the time and resource concentration required for any single computation while maintaining overall model accuracy.
2Power
If heavy kernel functions consume all thread resources on GPUs, then computational power is maximized for that function, but other kernel functions cannot execute in parallel, decreasing overall device performance
Solution Approach 1:
The input is divided into multiple partial inputs that are processed by different threads simultaneously. This segmentation ensures that no single kernel function monopolizes all thread resources, enabling multiple kernel functions to execute in parallel and improving overall device productivity while maintaining adequate computational power for each function.
Solution Approach 2:
The patent dynamically adjusts the number of threads and partial inputs based on the computational requirements of different kernel functions. This dynamic approach allows the system to optimize resource allocation in real-time, ensuring that computational power is distributed appropriately across multiple parallel operations rather than being statically allocated to a single function.
3Reliability
If deep learning models require significant scheduling overhead to manage thread resources, then task coordination is improved, but execution efficiency decreases
Solution Approach 1:
By segmenting the input into partial inputs and assigning them to different threads, the patent reduces the scheduling overhead required for any single thread. Each thread manages a smaller, more manageable subset of data, which simplifies resource coordination and reduces the time and complexity of scheduling operations while maintaining task coordination across all threads.
Data Source
AI summary
Embodiments of the present disclosure relate to a method, an electronic device, and a computer program product for processing data. According to exemplary implementations of the present disclosure, a method for processing data includes: determining a factor associated with a first input of a deep learning model, wherein the factor affects the number of threads for executing the deep learning model; generating a plurality of first partial inputs by using the first input based on the factor, wherein each first partial input in the plurality of first partial inputs is a part of the first input; and performing an operation on the plurality of first partial inputs by using the deep learning model, and generating an output of the deep learning model. Thereby, the data processing performance can be improved, and the resource requirement for data processing is lowered.