Software Failure Detection for Exascale Computing Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current large-scale processing systems assume that data reads and writes do not fail at the hardware level, leading to inefficiencies and frequent system restarts, especially in future exascale machines, where hardware failures can occur frequently, reducing uptime and performance-per-watt efficiency.
Innovation Solution
Implementing a method that monitors expected and observed data operations within a threshold time period using software and hardware timers and counters, allowing for failure detection and correction without the need for extensive hardware buffers, thereby reducing power consumption and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hardware buffers and extensive hardware monitoring are implemented to detect data failures, then reliability is improved, but device complexity and power consumption increase
Solution Approach 1:
The patent replaces extensive hardware monitoring infrastructure with a software-based failure detection mechanism. The system uses software to track expected data operations versus actual performed operations, eliminating the need for complex hardware buffers and monitoring circuits while achieving reliable failure detection.
Solution Approach 2:
The patent creates a virtual copy of the data operation tracking function through software. Instead of using physical hardware buffers to store and monitor data operations, the system uses software data structures to replicate and track operation counts, achieving the same monitoring function with reduced hardware complexity.
2Reliability
If hardware buffers and extensive hardware monitoring are implemented to detect data failures, then reliability is improved, but power consumption increases
Solution Approach 1:
The patent replaces power-intensive hardware monitoring circuits with software-based tracking. The software mechanism uses minimal computational resources to compare expected versus actual data operations, consuming significantly less power than dedicated hardware monitoring buffers and circuits.
Solution Approach 2:
The patent extracts the failure detection function from the hardware layer and relocates it to the software layer. By removing the need for extensive hardware buffers and monitoring circuits, the system eliminates the associated power consumption while preserving the reliability benefits of failure detection.
3Device complexity
If the system assumes data reads and writes do not fail, then device complexity is reduced, but reliability deteriorates and frequent restarts occur
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously tracks expected data operations versus actual performed operations. This feedback loop enables the system to detect failures, determine when restarts are necessary, and manage system state accordingly, improving reliability without requiring complex hardware infrastructure.
Solution Approach 2:
The system performs self-monitoring of data operation failures using software-based tracking. By independently tracking expected versus actual operations and determining when restarts are needed, the system achieves reliable operation through self-service monitoring without external hardware intervention.
Data Source
AI summary
Methods and apparatus to provide failure detection are disclosed herein. An example method to synchronize data operations between multiple workload units in a computing device to facilitate failure detection includes identifying a number of first data operations to write data from a computing node in a first workload unit to locations that are not in a local cache of the computing node and are not in a memory of the first workload unit, the first data operations corresponding to a set of computing instructions that are assigned to the first workload unit and, when a flag in the first workload unit has been set to a first value, synchronizing the first data operations with second data operations by a second workload unit.