PCIe Ordered Write Striping Across Multi-Socket S2S Links
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Maintaining ordered write observation (OWO) across separate sockets in a system is problematic due to long inter-socket latencies, leading to degraded performance and reduced bandwidth utilization.
Innovation Solution
Striping ordered PCIe writes across multiple socket-to-socket (S2S) links by breaking the writes into CleanUnique and WriteBack transactions, allowing the local S2S bridge to send write transactions to multiple bridges, ensuring coherency is maintained through CleanUnique transactions, and then sending WriteBack transactions once coherency is confirmed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If PCIe writes are performed in order through multiple sockets, then data consistency is maintained, but write throughput decreases due to sequential processing
Solution Approach 1:
The patent segments the ordered write stream into multiple parallel write streams, each assigned to a different socket. The write processor divides incoming writes into groups and distributes them across multiple sockets simultaneously, allowing parallel processing while maintaining order within each segment. This resolves the contradiction by achieving both data consistency (through ordered processing within segments) and improved throughput (through parallel segment processing).
Solution Approach 2:
The patent introduces a new dimension of parallelism by processing writes across multiple sockets simultaneously rather than sequentially. By distributing writes across multiple dimensional paths (sockets) and using multiple write processors, the system achieves parallel processing while maintaining overall order through coordination mechanisms. This dimensional expansion allows throughput to scale without sacrificing data consistency.
2Productivity
If multiple write processors are used to increase throughput, then write performance improves, but system complexity increases
Solution Approach 1:
The patent divides the write processing workload into multiple independent segments handled by separate write processors. Each processor manages a portion of the write stream, reducing the complexity burden on any single processor while collectively achieving high throughput. The segmentation allows parallel processing without requiring complex coordination between processors.
Solution Approach 2:
The patent introduces a write distributor as an intermediary component that manages the complexity of coordinating multiple write processors. The distributor receives incoming writes, makes intelligent routing decisions, and distributes them across processors according to current system state. This intermediary absorbs the coordination complexity, allowing individual processors to remain simple while the system as a whole achieves high throughput.
3Reliability
If writes are buffered to maintain ordering, then data consistency is ensured, but latency increases
Solution Approach 1:
The patent segments the buffering requirement into smaller, localized buffers at each write processor rather than a single large centralized buffer. Each processor maintains ordering within its assigned segment using a small buffer, reducing the average wait time for writes. The segmentation allows most writes to proceed quickly with minimal buffering while still ensuring overall order through the distributed architecture.
Solution Approach 2:
The patent changes the buffering dimension from a single sequential buffer to multiple parallel buffers across different sockets and processors. This allows writes to be buffered and processed in parallel dimensions, reducing the time any single write needs to wait. The multi-dimensional buffering approach maintains ordering constraints while significantly reducing average latency compared to a single sequential buffer.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Innovative techniques to efficient stripe ordered writes across multiple socket-to-socket links. The ordered writes may be PCIe ordered writes. Each socket-to-socket write (or remote write) for an address may be converted into two transactions. The first transaction may ensure that coherency for the address is ensured. The second transaction may be the actual request transaction to write the data of the address. In so doing, when multiple remote writes are involved, the remote writes may be distributed over multiple socket-to-socket links to maximize bandwidth.