Neural Network Layer Grouping for Parallel Compilation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network compilation methods are inefficient due to sequential processing and lack of parallelization, leading to prolonged compile times and suboptimal performance.
Innovation Solution
A method and apparatus that group neural network layers based on dependencies, generate parallelizable passes and threads, and perform compilation in parallel using multi-threading and layer parallelism to optimize operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If sequential compilation is used, then compilation process is simple, but compile time is long
Solution Approach 1:
The neural network layers are segmented into groups based on data dependencies, and compilation passes are divided into independent sets that can be executed in parallel. This segmentation allows multiple compilation operations to proceed simultaneously without violating dependency constraints, thereby reducing total compile time while maintaining process manageability.
Solution Approach 2:
The patent introduces a temporal dimension to the compilation process by creating multiple passes that operate in parallel timeframes. Instead of sequential compilation, the system executes independent passes simultaneously across different time dimensions, effectively transforming a 1D sequential process into a multi-dimensional parallel process that reduces overall execution time.
2Ease of operation
If layer-by-layer compilation is used, then compilation is systematic, but parallelization is limited
Solution Approach 1:
The systematic layer-by-layer compilation approach is enhanced by segmenting layers into groups based on data dependency relationships. This segmentation identifies which layers can be processed in parallel while maintaining systematic progression through dependency-constrained groups, thereby achieving both organization and speedup.
Solution Approach 2:
The compilation system dynamically adapts the compilation order based on dependency analysis. Instead of a fixed sequential approach, the system creates dynamic pass sets that can execute in parallel when dependencies allow, and sequentially when constraints require it, optimizing productivity while maintaining systematic control.
3Loss of time
If caching is implemented, then compile time is reduced, but compilation accuracy may be compromised
Solution Approach 1:
The system performs preliminary analysis of data dependencies between layers before compilation begins. This preliminary action identifies which layers can be safely cached and which require sequential processing, ensuring that caching decisions are made in advance based on accuracy requirements rather than reacting during compilation.
Solution Approach 2:
The dependency graph provides continuous feedback about which layers have been processed and which await processing. This feedback mechanism ensures that caching is applied only where safe, and that compilation accuracy is maintained by respecting dependency constraints, thereby reducing compile time without compromising reliability.
Data Source
AI summary
A compile method for a neural network, the compile method includes receiving data related to the neural network, generating a grouped layer by grouping layers comprised in the neural network based on the data, generating a set of passes executable in parallel based on a dependency between a plurality of passes to process the neural network, generating a set of threads performing a plurality of optimization functions based on whether optimization operations performed by the optimization functions is performed independently for the layers, respectively, or sequentially based on a dependency between the layers, and performing compilation in parallel based on the grouped layer, the set of passes, and the set of threads.