Parallel Processing Apparatus Dynamic Failure Rerouting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In high-performance computing, retransmitting communication commands through low-level communication libraries can be time-consuming due to repeated calls to transmission and reception confirmation functions, leading to prolonged execution times of parallel programs when communication paths fail.
Innovation Solution
A parallel processing apparatus that includes a management node to collect failure information and transmit this information to compute nodes, allowing them to determine alternative retransmission routes and re-execute inter-node communication using these routes, thereby reducing the need for repeated retransmissions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If communication commands are retransmitted by repeatedly calling transmission and reception confirmation functions of the low-level communication library, then communication reliability is improved, but execution time of parallel programs increases
Solution Approach 1:
The management node proactively collects failure information from all compute nodes and proactively transmits this information to the compute nodes before retransmission is needed. This preliminary provision of failure information allows compute nodes to quickly determine alternative routes without repeatedly calling transmission and reception confirmation functions, thus maintaining communication reliability while reducing execution time
Solution Approach 2:
The management node acts as an intermediary between compute nodes and failure information. Instead of compute nodes directly querying each other for failure status (which would require repeated function calls), the management node centralizes failure information collection and distribution, enabling efficient retransmission routing decisions
2Reliability
If communication paths are set to avoid failed locations at job assignment time, then communication reliability is improved, but adaptability to dynamic failures during execution deteriorates
Solution Approach 1:
The system dynamically updates failure information during job execution. The management node continuously collects failure information from compute nodes and transmits updates to all nodes when changes are detected. This dynamic mechanism allows the system to adapt to failures that occur during execution, maintaining both reliability and adaptability
Solution Approach 2:
The management node implements a feedback mechanism by continuously monitoring failure information from compute nodes and redistributing updated failure information to all nodes. This feedback loop ensures that the system remains adaptive to dynamic failures while maintaining communication reliability through informed routing decisions
Data Source
AI summary
A parallel processing apparatus including a plurality of compute nodes and a management node including a first processor configured to execute a process including collecting failure information regarding a plurality of ports of the plurality of compute nodes, and transmitting, to the plurality of compute nodes, failed port information including information on a failed port of the plurality of ports when an update in the failure information is detected in the collecting, wherein each of the plurality of compute nodes includes a second processor configured to execute a process including determining a retransmission route based on the failed port information when an inter-compute node communication in a low-level communication library has failed, and re-executing the inter-node communication by using the determined retransmission route.