Distributed Memory Data Segmentation for Collective Communication Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing processing modes for collective communication in distributed-memory parallel computers, particularly when dealing with data areas of significantly different bit widths, result in inefficient data conversion and increased software overhead, leading to reduced overall processing efficiency.
Innovation Solution
The proposed solution involves determining a 'division-recommended condition' where data with a smaller element size is divided into segment data, converted to match the size of data with a larger element size, and then combined for collective communication, thereby reducing the number of data conversions and communications required.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data areas with different bit widths are combined using conventional piggyback techniques, then communication efficiency is improved, but data conversion overhead increases due to repeated addition and elimination of 0 bits
Solution Approach 1:
The patent segments data areas into multiple regions based on element size characteristics. By dividing the data area into first, second, and third regions with different element sizes, the system avoids uniform conversion of entire data areas, thereby reducing the overhead of adding and eliminating 0 bits while maintaining communication efficiency through targeted segmentation.
2Adaptability or versatility
If collective communication function is used frequently, then data exchange between calculation nodes is achieved, but software overhead increases due to argument checking and dynamic area securing
Solution Approach 1:
The patent combines multiple data areas into a single continuous data area through the piggyback technique, allowing multiple communication operations to be merged into one collective communication function call. This reduces the number of times the collective communication function needs to be invoked, thereby reducing software overhead while maintaining data exchange capability.
Solution Approach 2:
The patent performs preliminary combining of data areas before executing the collective communication function. By pre-processing the data areas to create a continuous data structure, the system reduces the complexity of argument checking and dynamic area securing during the actual communication operation, thereby reducing software overhead.
3Adaptability or versatility
If element sizes of data areas are made equal through conversion, then piggyback combining is enabled, but processing time increases due to addition and elimination of 0 bits
Solution Approach 1:
The patent applies different element sizes to different regions of the data area based on local requirements. Instead of converting the entire data area to a uniform element size, the system maintains first, second, and third regions with element sizes of first, second, and third values respectively. This local differentiation enables piggyback combining without requiring conversion of all data, thereby improving processing speed while maintaining compatibility.
Data Source
AI summary
A computing system includes one or more processors configured to perform generation of a plurality of pieces of segment data by dividing first data, generate a plurality of pieces of extended data by performing an extension process on each of the plurality of pieces of segment data, each extended data having a size equal to a size of second data, generate first combination data by combining first extended data included in the plurality of pieces of extended data and the second data, perform transmission of the first combination data, in response to receiving data of a computation result calculated based on the first combination data, generate a first computation result related to the first extended data by dividing the data of the computation result, and generate a third computation result related to the first data by performing a reconstruction process based on the first computation result.