Remote Memory Access with Segmented Queues for Low Latency Interconnect

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data communication technologies face challenges in achieving low-latency, reliable, and cost-effective message transfer between computing devices, particularly in clustered computing systems, due to overhead from complex networking protocols and the need for sophisticated routing features that increase latency and processing overhead.

Innovation Solution

A method and system utilizing a circular message queue with advancing pointers and guard flags, mapped to remote memory across a peer-to-peer data link, which reduces latency by minimizing processing steps and eliminating the need for intermediary protocols, ensuring serialized and guaranteed message transfer.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If TCP/IP protocol is used for reliable data exchange, then message delivery reliability is improved, but processing overhead and latency increase

Engineering Contradiction:

Improvemessage delivery reliabilityVSAvoidprocessing overhead and latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent extracts the essential reliability function from the complex TCP/IP protocol stack, implementing a simplified custom protocol that provides only the necessary ordered delivery and acknowledgment mechanisms without the overhead of routing, congestion control, and other Internet-optimized features. This creates a dedicated low-latency data link protocol tailored specifically for cluster communication requirements.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent changes the protocol parameters by using fixed-size messages with sequence numbers instead of variable-length TCP segments, implementing a simpler state machine for connection management, and using direct memory access patterns that eliminate multiple copies between buffer pools. These parameter changes reduce processing overhead while maintaining reliability.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If sophisticated networking features are implemented, then adaptability and versatility are improved, but device complexity and latency increase

Engineering Contradiction:

Improvenetworking capabilitiesVSAvoidprocessing complexity and latency

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent removes sophisticated networking features such as dynamic routing, congestion control, and complex error recovery mechanisms that are unnecessary for dedicated cluster connections. The implementation focuses only on the essential features needed for reliable ordered message delivery between known endpoints, significantly reducing device complexity.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of implementing a full-featured network protocol stack and then stripping away features, the patent inverts the approach by designing a minimal protocol from the ground up that includes only the essential reliability features, thereby avoiding the complexity overhead of legacy networking implementations.

Inventive Principle:

Principle #13The other way round (Inversion)

3Reliability

If message queues are used for reliable delivery, then message ordering is improved, but processing overhead increases

Engineering Contradiction:

Improvemessage ordering guaranteeVSAvoidprocessing overhead

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent implements self-service mechanisms where each endpoint autonomously manages its own message queue state and acknowledgment tracking without requiring complex coordination with the other endpoint. The receiving端 independently tracks which messages it has received and sends acknowledgments only when ready, eliminating the need for complex flow control and reducing processing overhead.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS9811500B2Low latency device interconnect using remote memory access with segmented queues

Publication Date: 2017.11.07 TSX

AI summary

A writing application on a computing device can reference a tail pointer to write messages to message buffers that a peer-to-peer data link replicates in memory of another computing device. The message buffers are divided into at least two queue segments, where each segment has several buffers. Messages are read from the buffers by a reading application on one of the computing devices using an advancing head pointer by reading a message from a next message buffer when determining that the next message buffer has been newly written. The tail pointer is advanced from one message buffer to another within a same queue segment after writing messages. The tail pointer is advanced from a message buffer of a current queue segment to a message buffer of a next queue segment when determining that the head pointer does not indicate any of the buffers of the next queue segment.