Direct CPU Instruction Stream Network Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In scenarios where network IO access cannot overlap with the compute phase, the asynchronous nature of network access introduces latency, hindering CPU processing efficiency.
Innovation Solution
The approach involves treating the network as another memory that can be directly accessed by the CPU using loads and stores, allowing for synchronous network access by deriving network access information from virtual addresses, checking access permissions, and managing failures through notifications and management software invocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If asynchronous network access is used, then CPU can continue working during network operations, but latency is introduced when network IO cannot overlap with compute phase
Solution Approach 1:
The patent introduces a network interface card (NIC) as an intermediary device that handles network operations. The NIC translates CPU memory access requests into network communication operations, enabling the CPU to interact with remote memory over the network using standard load/store instructions without managing network protocols directly. This intermediary approach allows synchronous network access to proceed without blocking CPU execution flow.
Solution Approach 2:
The network interface card is designed to perform multiple functions: it acts as both a network communication device and a remote memory access interface. By implementing both network protocol handling and memory-mapped I/O capabilities in a single device, the system achieves universal functionality that resolves the contradiction between maintaining simple CPU operation and enabling efficient network access.
2Loss of time
If direct network access from instruction stream is implemented, then latency is reduced, but system complexity increases due to address translation and access permission checking
Solution Approach 1:
The patent combines address translation, permission checking, and network address resolution functions within the network interface card. Instead of having separate hardware components for each function, the NIC integrates these operations into a unified address translation mechanism that the CPU accesses through standard memory operations. This merging reduces the number of separate components while maintaining the necessary complexity for secure and efficient network access.
Solution Approach 2:
The network interface card performs self-service by automatically handling address translation from virtual addresses to network addresses, checking access permissions against configured security policies, and managing the network communication protocol. This self-service capability eliminates the need for CPU intervention in these complex operations, reducing latency while containing complexity within the NIC itself.
3Productivity
If network is treated as memory for direct access, then CPU operations continue uninterrupted, but reliability challenges arise from network failures
Solution Approach 1:
The system implements beforehand cushioning by configuring access permission tables and error handling protocols in advance. The network interface card is pre-configured with security policies that define which CPU addresses can access which network resources. Error handling routines are established beforehand to manage network failures, timeouts, and protocol errors without requiring CPU intervention. This preparatory approach allows the system to maintain uninterrupted CPU operations while having reliable fallback mechanisms in place.
Data Source
AI summary
A method for network access of remote memory directly from a local instruction stream using conventional loads and stores. In cases where network IO access (a network phase) cannot overlap a compute phase, a direct network access from the instruction stream greatly decreases latency in CPU processing. The network is treated as yet another memory that can be directly read from, or written to, by the CPU. Network access can be done directly from the instruction stream using regular loads and stores. Example scenarios where synchronous network access can be beneficial are SHMEM (symmetric hierarchical memory access) usages (where the program directly reads/writes remote memory), and scenarios where part of system memory (for example DDR) can reside over a network and made accessible by demand to different CPUs.