CNN Layer Scheduling With Data Replication for Padding Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI chip architectures face challenges in efficiently scheduling multi-layer convolutional neural networks due to issues with static compilation and repeated data computations, particularly in handling dynamic computational graphs and large-volume input data, leading to suboptimal utilization of computing power.
Innovation Solution
A scheduling method that involves performing first convolution computations on data groups by compute units, determining a data replication-transmission mode based on the configuration of subsequent convolutional layers, and replicating and transmitting intermediate data rows to reduce redundant computations and improve computing power utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If static compilation is used for scheduling convolutional neural networks on AI chips, then the scheduling process is simplified and deterministic, but the system cannot adapt to dynamic computational graphs and large-volume input data, leading to suboptimal computing power utilization
Solution Approach 1:
The patent implements dynamic scheduling by allowing the computational graph to be constructed and modified at runtime based on input data characteristics. The system dynamically determines computation paths and data distribution strategies rather than using fixed static compilation, enabling adaptation to varying input sizes and computational requirements while maintaining manageable complexity through structured runtime decision-making
2Productivity
If data is distributed across multiple compute units without replication, then memory usage is optimized, but compute units require padding operations and experience waiting time, reducing computing power utilization
Solution Approach 1:
The patent implements selective data replication where intermediate data rows are copied from one compute unit to another when needed. Specifically, when a compute unit requires padding to maintain computational continuity, the system replicates the necessary data rows from neighboring compute units rather than performing padding operations, thereby eliminating waiting time and improving computing power utilization while controlling data transmission volume through selective replication
3Productivity
If compute units perform padding operations to handle data distribution, then computational continuity is maintained, but computing power is wasted on non-effective padding computations
Solution Approach 1:
The patent eliminates wasted padding computations by copying effective data rows from neighboring compute units. Instead of performing padding operations that generate non-effective computational results, the system directly replicates the required data rows from adjacent compute units, ensuring that all computational power is used effectively while maintaining computational continuity across the distributed architecture
Data Source
AI summary
A scheduling method, a scheduling apparatus, an electronic device and a storage medium are provided. The scheduling method includes: respectively performing, by a plurality of compute units, a first convolution computation on a plurality of data groups that correspond to the plurality of compute units respectively, to obtain a plurality of first computation result groups; determining a data replication-transmission mode corresponding to the plurality of first computation result groups in the plurality of compute units, according to a configuration rule of a second convolutional layer in the plurality of compute units; and copying a first intermediate data row required by the first compute unit for padding in the second convolution computation process from a first computation result group in the second compute unit and transmitting the first intermediate data row that is copied to the first compute unit based on the data replication-transmission mode.