Neural Network Operation Load Balancing via Dynamic Data Traversal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network operation methods using sparsity face load imbalance issues between operation devices due to non-uniform data distribution, leading to decreased performance and limited applicability to both weights and inputs.
Innovation Solution
A method and apparatus that determine and prioritize data traversal paths between operation devices, skipping operations for zero or range-value data, and updating paths based on competition and priority to optimize neural network operations using sparsity, thereby reducing computation cost and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If sparsity is utilized to accelerate neural network operation, then computation time is reduced, but load imbalance occurs between operation devices
Solution Approach 1:
The patent implements dynamic data distribution that adapts to the sparsity patterns of neural network weights and inputs. The system dynamically determines which operation devices process which data elements based on real-time sparsity characteristics, allowing the workload to be continuously optimized as data patterns change, thereby maintaining load balance while exploiting sparsity for acceleration
Solution Approach 2:
The system changes the parameter of data distribution strategy based on sparsity thresholds and operation device capabilities. By adjusting distribution parameters such as block sizes, threshold values, and device assignment rules, the system optimizes both the acceleration benefit from sparsity and the load balance across devices
2Productivity
If conventional sparsity methods are applied, then weight processing is accelerated, but applicability to both weights and inputs is limited
Solution Approach 1:
The patent creates a universal sparsity exploitation mechanism that functions identically for both weight data and input activation data. The same data distribution logic, sparsity detection methods, and operation device assignment strategies are applied regardless of data type, enabling the system to accelerate processing for both weights and inputs without requiring separate specialized pathways
3Reliability
If fine pruning is performed for predetermined processor, then accuracy is maintained, but unique pruning process increases device complexity
Solution Approach 1:
The patent extracts the complexity of fine pruning from the processor hardware by implementing pruning logic in software or firmware. The system extracts sparsity patterns from the data and uses this extracted information to guide operation device selection, rather than requiring dedicated hardware pruning units, thereby maintaining accuracy while reducing device complexity
Data Source
AI summary
Disclosed is a method and apparatus of operating a neural network. The neural network operation method includes receiving data for the neural network operation, verifying whether competition occurs between a first data traversal path corresponding to a first operation device and a second data traversal path corresponding to a second operation device, determining first operand data and second operand data from among the data using a result of the verifying and a priority between the first data traversal path and the second data traversal path, and performing the neural network operation based on the first operand data and the second operand data.