Network Interconnect Routing Resilience Against Deadlocks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Massively parallel processing systems face challenges in dynamically updating routing information without causing deadlocks and ensuring packet ordering, especially when network links fail or configurations change, as existing methods often require reinitializing the entire network.

Innovation Solution

A network resiliency system that detects errors, generates new routing information to bypass failed links, and enters a quiescent state to prevent deadlocks by suppressing request packets while allowing response packets to complete, then loads new routing information into routers, ensuring consistent routing without reinitializing the entire network.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If routing information is dynamically updated to bypass failed links, then system reliability is improved, but deadlocks may occur due to inconsistent routing states across routers

Engineering Contradiction:

Improvesystem reliabilityVSAvoiddeadlocks

Core Design Contradiction:

ReliabilityVSObject-generated harmful factors

Solution Approach 1:

The system performs preliminary actions by suppressing request packet injections and entering a quiescent state before routing information is updated. This prevents new packets from entering the network during the transition period, eliminating the source of potential deadlocks while allowing in-flight packets to complete their journeys.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The update process is segmented into distinct phases: quiescing the network, updating routing information in routing tables, and then resuming normal operation. This segmentation allows the system to maintain consistency within each phase while transitioning between states, preventing deadlocks that would occur with simultaneous updates across all routers.

Inventive Principle:

Principle #1Segmentation

2Adaptability or versatility

If routing information is updated dynamically, then adaptability to network errors is improved, but packet ordering may be violated due to mixed old and new routing information

Engineering Contradiction:

Improveadaptability to network errorsVSAvoidpacket ordering

Core Design Contradiction:

Adaptability or versatilityVSStability of the object's composition

Solution Approach 1:

The system enters a quiescent state before updating routing information, which preliminary prevents any new packets from being injected into the network. This ensures that when routing tables are updated, all packets in transit are already committed to their paths and will maintain their relative ordering, while new packets will use the updated routing information consistently.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The quiescent state acts as an intermediary phase between old and new routing information states. During this intermediate state, no packets are injected, allowing the system to transition routing information across all routers uniformly. This intermediary state prevents the mixing of packets routed by different routing tables, preserving packet ordering.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Stability of the object's composition

If the entire network is reinitialized to ensure consistent routing, then routing consistency is improved, but system productivity is reduced due to shutdown

Engineering Contradiction:

Improverouting consistencyVSAvoidsystem productivity

Core Design Contradiction:

Stability of the object's compositionVSProductivity

Solution Approach 1:

Instead of reinitializing the entire network, the system segments the update process into localized routing table updates while maintaining continuous network operation. The quiescent state is temporary and localized to the control plane, while the data plane continues to forward packets, preserving productivity while achieving routing consistency.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically updates routing information without static reinitialization. The routing tables are updated in place while the network continues to operate, with the quiescent state providing a brief window for consistent update propagation. This dynamic approach maintains productivity while ensuring routing consistency through controlled, incremental changes rather than complete system restart.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS8854951B2Dynamically updating routing information while avoiding deadlocks and preserving packet order after a link error

Publication Date: 2014.10.07 CRAY INC

AI summary

A system for allowing dynamic changing of routing information of a network interconnect while avoiding deadlocks and preserving packet ordering. A network resiliency system detects when an error in the network interconnect occurs and dynamically generates new routing information for the routers that factors in the detected error. The network resiliency system then generates new routing information that factors in the failure. The network resiliency system then directs the network interconnect to enter a quiescent state in which no packets are transiting through the network interconnect. After the network interconnect enters the quiescent state, the network resiliency system directs the loading of the new routing information into the routing tables of the network interconnect and then directs the network interconnect to start injecting request packets into the network interconnect.