Neural Network Operating System for Parallel Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training deep neural networks is inefficient due to the exponential increase in mathematical calculations with the size and complexity of the network, and existing tools struggle to parallelize operations effectively, leading to significant training delays.
Innovation Solution
The neural network operating system arranges calculations in quasi-threads, suspends non-essential threads until dependencies are resolved, and combines operations into batched calculations to optimize training, allowing for efficient parallel processing and automatic mini-batching, even with user-defined code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If neural network size and complexity are increased, then the network's capability and accuracy are improved, but the number of mathematical calculations increases exponentially, leading to longer training time
Solution Approach 1:
The patent segments the neural network training process into multiple independent threads that can be executed concurrently. Each thread handles a portion of the computational workload, allowing parallel processing of mathematical operations. This segmentation enables the system to process larger networks by distributing calculations across multiple threads, thereby reducing overall training time despite the exponential increase in required calculations.
2Reliability
If operations are propagated sequentially through the neural network, then calculation accuracy is maintained, but subsequent operations must wait for previous results, preventing effective parallelization
Solution Approach 1:
The patent implements dynamic thread management where threads are created, suspended, and resumed based on data dependency relationships. When a thread encounters an operation that depends on results from another thread, it suspends itself and allows other independent threads to execute. This dynamic approach maintains calculation accuracy by respecting dependency constraints while maximizing parallelization opportunities, thereby improving productivity without sacrificing reliability.
3Adaptability or versatility
If user code is used to define and train the neural network, then flexibility and customization are achieved, but the user may not arrange operations in an efficient manner for parallel processing
Solution Approach 1:
The patent introduces an intermediary system that sits between the user code and the actual execution engine. This intermediary automatically analyzes the user-defined operations, identifies parallelization opportunities, and manages thread creation and coordination. The intermediary translates user-friendly code into optimized parallel execution plans, allowing users to maintain code flexibility while achieving high training efficiency through automatic operation arrangement and parallel processing.
Data Source
AI summary
Disclosed in some examples are methods, systems, machine-readable media, and devices which operate a neural network defined by user code. A method includes identifying, operations from user code that are integral in operating the neural network, combining a subset of the identified operations into a single processing sequence to be transmitted to an array of hardware processors, performing operations that are not integral in operation of the neural network in a separate thread of execution from the operations that are integral in operating the neural network; and mapping results to the combined operations that were included in the single processing sequence.