Streaming Packet Rewrite Engine for Network Appliances
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for rewriting packet content in multi-core systems require holding entire responses, leading to increased memory consumption and latency, and often involve unnecessary overhead such as copying data to buffers, which reduces throughput and resource efficiency.
Innovation Solution
The system performs a streaming rewrite by holding a limited number of packets, applying rewrites before accumulating complete data, and using an efficient multiple pattern search algorithm to parse and rewrite content in a single pass, avoiding buffer copying and reducing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current systems hold entire responses before rewriting, then rewrite accuracy is improved, but memory consumption and latency increase
Solution Approach 1:
The system segments the response into individual packets and processes them one at a time through the rewrite engine, rather than holding the entire response in memory. Each packet is processed independently as it arrives, reducing memory requirements while maintaining rewrite accuracy through sequential processing of complete packets.
Solution Approach 2:
The system performs preliminary parsing and pattern identification on packets as they arrive, before the complete response is accumulated. By preparing rewrite rules and identifying patterns in advance on individual packets, the system can rewrite content immediately without waiting for the complete response, thereby reducing latency and memory usage.
2Reliability
If current systems hold entire responses before rewriting, then rewrite completeness is improved, but response time latency increases
Solution Approach 1:
The rewrite operation continues uninterrupted as packets arrive sequentially. Instead of waiting for the complete response to accumulate before initiating rewriting, the system continuously processes packets as they arrive, maintaining steady-state rewrite operation. This continuous processing eliminates the idle waiting period and reduces overall response time latency while ensuring complete rewriting through sequential packet processing.
3Adaptability or versatility
If current systems copy response to buffer, then rewrite flexibility is improved, but throughput decreases
Solution Approach 1:
The system extracts only the necessary packet data required for rewriting operations, rather than copying the entire response to a buffer. By taking out only the essential information (packet content, headers, and relevant metadata) and processing it directly in memory, the system eliminates the unnecessary buffer copying step, thereby improving throughput while maintaining rewrite flexibility through direct manipulation of extracted packet data.
4Measurement precision
If multiple body parsing is performed, then pattern matching accuracy is improved, but resource consumption increases
Solution Approach 1:
The system performs preliminary pattern identification and rule preparation on packets as they arrive, rather than performing multiple sequential parsing operations after receiving the complete response. By preparing parsing rules and identifying patterns in advance on individual packets, the system eliminates the need for repeated parsing operations, thereby reducing resource consumption while maintaining high pattern matching accuracy through efficient single-pass processing.
Data Source
AI summary
A streaming rewrite method and system that can execute an efficient multiple pattern search method that parses a response in a data structure of an appliance. The method and system can avoid copying to a buffer by parsing data across a data structure to identify search patterns and phrases that may be identified by one or more actions and/or rules of an appliance or system. A parser can input one or more search patterns, and parse a body of a response or one or more packets for the search patterns. The parser can obtain pattern information about the packets and/or the response, and store this information in a database. The appliance can then perform lookups in the database for pattern information and perform rewrites in accordance with the stored pattern information. The rewritten response and/or packets can then be transmitted to a destination.