Neural Network Preprocessing Resource Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The lengthy and compute-intensive process of training neural networks is inefficient due to bottlenecks in preprocessing phases, which can lead to increased total time and cost, especially when preprocessing times exceed computation times, causing high-end GPUs to become idle.
Innovation Solution
Identifying commonality between input data required by multiple neural network training tasks to execute preprocessing phases jointly, allowing for optimized assignment of computation and preprocessing tasks to processors, thereby minimizing total execution time and cost.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If preprocessing is performed separately for each neural network training task, then each task can be processed independently, but the total preprocessing time increases and high-end GPUs become idle waiting for preprocessing to complete
Solution Approach 1:
The patent merges preprocessing operations across multiple neural network training tasks by identifying common input data requirements. When multiple tasks share the same input data, a single preprocessing operation is performed and the results are shared among all tasks that require them, eliminating redundant preprocessing computations and reducing overall processing time.
Solution Approach 2:
The patent creates a universal preprocessing system that can serve multiple training tasks simultaneously. The preprocessing infrastructure is designed to handle multiple tasks with a single preprocessing pipeline, making the system multi-functional and capable of serving different neural network training tasks with shared resources.
2Productivity
If multiple preprocessing phases are executed independently, then each task maintains its own preprocessing pipeline, but hardware resources are underutilized and total execution time increases
Solution Approach 1:
The patent combines multiple independent preprocessing phases into a unified preprocessing architecture that identifies and executes common preprocessing operations once, then shares the results across multiple training tasks. This reduces hardware underutilization while managing complexity through systematic resource sharing.
3Loss of time
If preprocessing time exceeds computation time, then preprocessing becomes the bottleneck, but executing preprocessing sequentially for each task increases total execution time and cost
Solution Approach 1:
The patent merges redundant preprocessing operations across multiple training tasks into single execution units. By identifying that multiple tasks require the same preprocessing operations on the same input data, the system executes these operations once and shares the results, eliminating the sequential bottleneck and improving overall training throughput.
Data Source
AI summary
A method for training a Neural-Network (NN), the method includes receiving a plurality of NN training tasks, each training task including (i) a respective preprocessing phase that preprocesses data to be provided as input data to the NN, and (ii) a respective computation phase that trains the NN using the preprocessed data. The plurality of NN training tasks is executed, including: (a) a commonality is identified between the input data required by computation phases of two or more of the training tasks, and (b) in response to identifying the commonality, one or more preprocessing phases are executed that produce the input data jointly for the two or more training tasks.