Fused Data Communication Across Processors With Lower Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face inefficiencies in computational performance and energy consumption due to increased communication between processors, particularly in large-scale deep neural network training, where communication optimization is critical for overall device operation.
Innovation Solution
Implementing fused data generation and communication techniques using augmented components such as a targeted communication module, data generation and communication tracking module, and updates convergence unit to manage concurrent data processing and communication across multiple processors, leveraging near/in-memory processing for synchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple processors are used to improve computational efficiency, then computational performance is improved, but communication overhead between processors increases
Solution Approach 1:
The patent merges data generation and communication operations into a single fused operation. The processor generates data and immediately communicates it to other processors without sequential execution, combining what were previously separate steps into one unified operation that executes concurrently.
Solution Approach 2:
The system performs communication operations in advance or concurrently with data generation. By preparing communication buffers and establishing communication paths before data is fully generated, and by overlapping communication with subsequent computation steps, the system reduces idle waiting time between processors.
2Productivity
If communication between processors is increased for large-scale neural network training, then computational capability is improved, but energy consumption increases
Solution Approach 1:
By fusing data generation and communication into a single operation, the system eliminates redundant memory traffic and avoids sequential execution of communication steps. This merging reduces the total number of memory accesses and communication transactions, thereby lowering energy consumption while maintaining computational capability.
Solution Approach 2:
The fused operation ensures continuous useful action by overlapping data generation with communication and computation. Instead of idle waiting between sequential steps, processors continuously perform productive operations concurrently, improving energy efficiency by reducing idle time and optimizing the compute-to-communication ratio.
3Ease of operation
If separate kernel launches are used for computation and communication, then programming simplicity is maintained, but operational efficiency decreases
Solution Approach 1:
The patent combines multiple kernel launches into a single fused kernel operation that handles both data generation and communication tasks. This unified approach eliminates the overhead of multiple kernel launches and sequential execution, improving operational efficiency while maintaining programmer-friendly syntax through high-level fused operation interfaces.
Data Source
AI summary
Fused data generation and associated communication techniques are described. In an implementation, a system includes processing system having a plurality of processors. A data generation and communication tracking module is configured to track programmatically defined data generation and associated communication as performed by the plurality of processors. A targeted communication module is configured to trigger targeted communication of data between the plurality of processors based on the tracked programmatically defined data generation and associated communication.