SIMD Replay Unit for Silent Data Corruption Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Silent data corruption (SDC) is a widespread issue affecting memory, storage, networking, and central processing units (CPUs), leading to errors in data processing that can propagate and cause application-level problems such as crashes or model failures in machine learning systems.
Innovation Solution
The proposed solution involves a replay unit within a Single Instruction, Multiple Data (SIMD) compute unit that utilizes unused ALU slots to replay computations and compare results with the original outputs, detecting mismatches as silent data errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If parity mechanism and ECC are used to protect SRAM, then data storage reliability is improved, but device complexity increases and compute logic remains vulnerable to SDC
Solution Approach 1:
The patent segments the detection function by separating storage protection (parity/ECC for SRAM) from compute protection (replay unit for ALU). The replay unit is a dedicated segment that specifically handles compute logic SDC detection, allowing storage and compute to have independent protection mechanisms tailored to their specific vulnerability profiles.
Solution Approach 2:
The replay unit acts as an intermediary between the compute logic and the error detection function. Instead of modifying the compute logic itself or the storage protection, the replay unit intercepts operations and performs verification in parallel, serving as a mediator that protects compute logic without interfering with its normal operation.
2Reliability
If replay computations are performed on all compute units, then SDC detection coverage is improved, but productivity decreases due to resource contention
Solution Approach 1:
The patent applies partial action by having the replay unit operate on a subset of operations rather than all operations continuously. The sampling mechanism selectively replays certain operations based on configured rates, providing sufficient SDC detection coverage while avoiding the productivity penalty of replaying every single operation. This partial monitoring approach balances detection effectiveness with system performance.
Solution Approach 2:
The replay unit performs operations periodically rather than continuously, with configurable sampling rates that determine how often replay operations are executed. This periodic action allows the system to maintain SDC detection capability while managing resource consumption and minimizing impact on overall productivity through time-based scheduling of replay activities.
3Measurement precision
If sampling rate is increased to detect more SDCs, then measurement precision is improved, but loss of time increases due to more replay operations
Solution Approach 1:
The patent implements dynamic sampling rates that can be adjusted based on system conditions, workload characteristics, and detected error patterns. The sampling mechanism is not fixed but can adapt its intensity, allowing the system to increase detection precision when needed while reducing time loss during normal operation. This dynamic adjustment optimizes the trade-off between detection accuracy and performance overhead.
Solution Approach 2:
The system changes the sampling rate parameter based on different operational contexts and requirements. By making the sampling rate a configurable and adjustable parameter rather than a fixed value, the system can optimize detection precision for critical operations while maintaining lower overhead for less critical operations, thereby managing the time loss associated with replay operations.
Data Source
AI summary
An aspect of the disclosed technology is a replay unit that enables replaying computations on unused or empty ALU slots. The replay unit may be added on a per lane basis or within a lane in a SIMD unit or device.