Die Reduction Engine for Batch Normalization Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multiple-computing core, multiple-die, and multiple-computing device systems, batch normalization for deep neural networks leads to high synchronization overhead and long delays due to remote data transmission and synchronization operations across dies and devices, affecting overall performance.
Innovation Solution
Each die in the system designates a reduction engine to function as a die reduction engine, where computing cores send data and synchronization indicators to these local engines for reduction operations, generating results and acknowledgments, thereby reducing the need for remote synchronization and data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a global synchronization unit and global reduction engine are used for batch normalization across multiple dies and computing devices, then normalization accuracy is improved, but synchronization overhead and data transmission delay increase significantly
Solution Approach 1:
The patent divides the global synchronization system into multiple local synchronization units, each responsible for a specific die. Each die performs batch normalization independently using its own reduction engine and synchronization unit, eliminating the need for data to traverse across multiple dies to a remote global unit. This segmentation reduces transmission distance and synchronization delay while maintaining normalization accuracy within each die.
Solution Approach 2:
The patent implements local reduction engines and synchronization units within each die, allowing batch normalization to be performed with local data processing rather than remote global synchronization. This local processing approach reduces the transmission overhead and delay associated with sending data across multiple dies to a centralized global unit, while still achieving accurate normalization results.
2Stability of the object's composition
If data refreshing, synchronization, and reading are performed at remote ends across dies and devices, then global consistency is achieved, but three times the long delay across dies occurs
Solution Approach 1:
The patent segments the synchronization process into three local operations (data refreshing, synchronization, and reading) that occur within each die rather than remotely across dies. Each die independently performs these operations with its own reduction engine and synchronization unit, eliminating the need for data to traverse the inter-die communication interface three times, thus reducing delay while maintaining consistency within each die.
Solution Approach 2:
The patent performs reduction operations and synchronization locally within each die before data needs to be used in subsequent processing steps. By completing the reduction and synchronization operations in advance within the local die, the system eliminates the need for delayed remote data retrieval, thereby reducing overall processing delay while ensuring data consistency is established before use.
Data Source
AI summary
The embodiments of the disclosure relate to a data processing method and a computing system. For each die: a first reduction engine of multiple reduction engines corresponding to multiple computing cores included in a current die is determined; each computing core sends data to be reduced and a synchronization indicator to the first reduction engines in multiple dies; in response to receiving the data to be reduced and the synchronization indicators from the computing cores in multiple dies, the first reduction engine in the current die performs a reduction operation on the data to be reduced to generate a reduction computing result, and sends synchronization acknowledgments to the computing cores in the current die; and in response to receiving the synchronization acknowledgment, each computing core in the current die reads the reduction computing result from the first reduction engine in the current die.