Hardware Forwarding Element Data Plane Incremental Update
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network forwarding elements face challenges in reconfiguring their data plane components without significant impact on incoming packet traffic, particularly when upgrading encapsulation methods or recovering from failures, as existing methods are slow and disruptive.
Innovation Solution
Implementing a data plane configuration mechanism using direct memory access (DMA) that allows the control plane to reconfigure ingress and egress pipelines, match-action units, and traffic managers, enabling rapid repurposing of IC chips with minimal disruption to packet processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the data plane component is reconfigured using traditional methods, then the encapsulation method can be upgraded, but the reconfiguration process causes significant impact to incoming packet traffic and takes several seconds
Solution Approach 1:
The data plane is divided into multiple independent pipelines (ingress pipelines and egress pipelines) that can be reconfigured independently. During reconfiguration, only affected pipelines are paused while others continue processing packets, maintaining partial throughput. This segmentation allows encapsulation method upgrades without complete system shutdown.
Solution Approach 2:
Configuration data is pre-loaded into DMA buffers before the actual reconfiguration occurs. The control plane prepares the new configuration in advance, and once ready, the DMA engine rapidly transfers it to the data plane components. This preliminary preparation minimizes the actual reconfiguration time and its impact on packet traffic.
2Reliability
If a cold reboot of the data plane component is performed, then the wedge issue is resolved, but the reboot takes several seconds and causes large impact to incoming packet traffic
Solution Approach 1:
The problematic data plane component is extracted and isolated for individual reconfiguration. Instead of rebooting the entire forwarding element, only the specific data plane pipeline or component that wedged is paused and reconfigured. This extraction allows targeted recovery while maintaining operation of other healthy pipelines.
Solution Approach 2:
The system transitions from a static all-or-nothing reboot approach to a dynamic selective reconfiguration approach. The control plane can dynamically identify which specific pipeline or component needs reconfiguration and apply changes incrementally. This dynamic approach reduces downtime from several seconds to minimal duration, maintaining reliability while minimizing service disruption.
3Adaptability or versatility
If new IC is released to support new encapsulation method, then the encapsulation capability is updated, but users have to wait several years for the release
Solution Approach 1:
The data plane components are designed with reconfigurable parameters including encapsulation methods. Instead of requiring new hardware ICs, the system changes software-based configuration parameters that define encapsulation behavior. The match-action tables and pipeline configurations can be dynamically modified to support new encapsulation methods immediately, eliminating multi-year wait times for hardware updates.
Solution Approach 2:
The functionality of new encapsulation methods is implemented by copying and adapting existing pipeline configurations and match-action rules. Rather than waiting for new hardware, the system creates software copies of processing logic with modified encapsulation parameters. This allows rapid deployment of new encapsulation capabilities by replicating and adapting proven processing pipelines.
4Speed
If the data plane is reconfigured through DMA, then the reconfiguration speed is improved to under 100 milliseconds, but the system complexity increases
Solution Approach 1:
A DMA (Direct Memory Access) engine is introduced as an intermediary between the control plane and data plane components. The DMA engine handles the complex data transfer and synchronization operations, buffering configuration data and coordinating with packet processing pipelines. This intermediary simplifies the control plane's burden while achieving rapid sub-100-millisecond reconfiguration speeds through hardware-accelerated memory operations.
Data Source
AI summary
A method of incremental updating of a network forwarding element that includes (i) a set of data plane circuits with a set of ingress buffers and a group of configurable packet processing stages and (ii) a set of control plane circuits comprising a set of direct memory access (DMA) buffers. Configuration data for reconfiguring the data plane packet processing stages is loaded into the DMA buffers while the packet processing stages are processing the packets. The ingress buffers are configured to (i) pause sending the packets to the processing stages and (ii) continue storing the incoming packets while sending the data plane packets to the processing stages is paused. The configuration data is loaded from the DMA buffers into the packet processing stages. The ingress buffers are configured to resume sending the data packet plane packets to the packet processing stages.