Seeded Stochastic Rounding for Reproducible In-Network Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional stochastic rounding methods lack reproducibility and fail to address issues such as numeric bias accumulation, seed propagation and coordination, race conditions, and interference between computational streams in in-network computing environments.
Innovation Solution
Implementing a system where seed values are derived from a base seed value and propagated throughout network devices, ensuring reproducible stochastic rounding by using seed-based pseudo-random algorithms, and maintaining computational stream isolation while hiding the network topology from users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional stochastic rounding methods are used, then computational operations can be performed, but reproducibility is lost and numeric bias accumulates
Solution Approach 1:
The system implements feedback mechanisms where rounding results are fed back into the computational stream, and seed values are adjusted based on observed biases. The rounding circuit monitors numeric outcomes and modifies subsequent rounding decisions to compensate for accumulated bias, ensuring reproducible results across multiple computational streams.
Solution Approach 2:
The patent changes the parameter of seed value generation from fixed or independent to derived and coordinated. By generating seed values from a base seed through deterministic transformations, the system maintains reproducibility while distributing seed values across different computational streams to prevent bias accumulation.
2Reliability
If seed values are propagated throughout network devices, then reproducible rounding is achieved, but system complexity increases
Solution Approach 1:
The base seed value generation mechanism serves multiple functions: it provides seed values for reproducibility, coordinates seed propagation across network devices, and manages computational stream isolation. This multi-functional approach reduces overall system complexity despite achieving reliable reproducible rounding.
Solution Approach 2:
The patent introduces intermediary structures that facilitate seed propagation between network devices without requiring direct coordination between all devices. These intermediaries manage the distribution of derived seed values, reducing the complexity of maintaining reproducibility across the entire network.
3Reliability
If computational streams are isolated, then interference is reduced, but resource allocation flexibility is limited
Solution Approach 1:
The system segments computational streams into isolated units with dedicated seed values, preventing interference between streams.同时，it maintains flexibility by allowing different resource allocations to be mapped to these segmented streams through the coordinated seed value system, enabling isolation without sacrificing allocation adaptability.
Data Source
AI summary
A system includes at least one processing node to perform one or more compute processes as part of a distributed workload to generate an output. The at least one processing node is configured with a derived seed value that is generated from a base seed value. The system further includes a rounding circuit to perform rounding operations for the at least one processing node according to the derived seed value.