Packet Splitting via On-Chip Memory for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory access latencies, including bus snoop logic and cache flush/invalidation, become bottlenecks in packet processing, even with multiple powerful CPUs, as they limit the performance of communication systems like routers and network devices.
Innovation Solution
A system and method that split data structures into header and payload sections, where the header is processed from on-chip memory, minimizing the need for system memory accesses and eliminating cache coherency issues, allowing direct processing and transfer without cache snooping penalties.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If packet processing uses system memory with cache coherency management, then data can be stored centrally, but memory access latencies increase due to bus snoop logic and cache flush/invalidation operations
Solution Approach 1:
The patent segments the packet processing system into two distinct memory spaces: on-chip memory for storing packet headers and system memory for storing packet payloads. This segmentation allows the CPU to access headers quickly from on-chip memory without triggering cache coherency protocols, while payloads remain in system memory. The classification engine determines which packets should be split and routes them appropriately, eliminating unnecessary bus snoop operations and cache flush/invalidation overhead.
2Productivity
If multiple powerful CPUs are used for packet processing, then processing capacity increases, but memory access bottlenecks persist due to cache coherency management overhead
Solution Approach 1:
The patent extracts the packet header data from system memory and places it in on-chip memory, separating the critical path data (headers) from the bulk data (payloads). This extraction eliminates the need for multiple CPUs to participate in cache coherency management for header access, as on-chip memory does not require cache synchronization. Each CPU can independently and efficiently access headers from on-chip memory while maintaining full access to payloads in system memory.
3Ease of manufacture
If packet headers are accessed from system memory, then centralized storage is maintained, but performance decreases due to bus snoop logic and cache flush operations
Solution Approach 1:
The patent introduces on-chip memory as an intermediary between system memory and the CPU for packet header access. Instead of the CPU directly accessing headers from system memory (which triggers cache coherency protocols), the headers are first copied to on-chip memory by a DMA engine. The CPU then accesses this intermediary on-chip memory location, achieving fast access without involving the system memory cache coherency mechanism. This intermediary approach maintains the simplicity of centralized storage while dramatically improving access performance.
Data Source
AI summary
A data structure splitting method is provided for processing data using a minimum number of memory accesses. An SoC is provided with a with a central processing unit (CPU), a system memory, an on-chip memory (OCM), and a network interface including an embedded direct memory access (DMA). The network interface accepts a data structure with a header and a payload. The DMA writes the payload in the system memory, and the header in the OCM. The network interface DMA notifies the CPU of the header address in the OCM. The CPU reads the header in the OCM, performs processing instructions, and writes the processed header in the OCM. The CPU sends the address of the processed header in OCM to the network interface DMA. The network interface DMA reads the processed header from the OCM and sends a data structure with the processed header and the payload.