A method and system for transparent access to memory within a computing node based on a scale-up network of software and hardware collaboration

Abstract

The application discloses a kind of soft and hardware collaborative cross computing node memory transparent access method and system based on Scale-Up network, comprising the following steps: reserving continuous physical address window in the system physical address space of local computing node;Remote computing node allocates virtual address segment;Remote computing node generates the unique address handle of virtual address segment and sends to local computing node;Local computing node binds the address handle received with the physical address window reserved;Remote computing node converts target virtual address into target physical address in discontinuous physical memory by IOMMU and page table, completes data read-write.The application realizes that local CPU accesses remote node discontinuous physical memory with local load / store semantics transparently, reduces cross-node access delay, while realizing that local width is decoupled with cluster scale, remote memory target node autonomous control and cross-node access security isolation.

Description

Technical Field

[0001] This invention relates to the fields of computer architecture and distributed computing technology, and in particular to a method and system for transparent cross-computing node memory access based on scale-up networks through hardware and software collaboration. Background Technology

[0002] In computationally intensive scenarios such as high-performance computing (HPC), artificial intelligence training and inference, and big data analytics, the local physical memory capacity of a single computing node (such as a server or GPU-accelerated node) has a physical limit, making it difficult to meet the ever-increasing demand for large memory workloads. For example, the parameter scale of large-scale AI models has exceeded trillions, and the memory capacity of a single node is far from sufficient to hold the complete model weights and intermediate activation data. To address this, the industry generally adopts a scale-up architecture, which connects multiple computing nodes into a "supernode" through high-speed interconnect technology, enabling the aggregation and sharing of cross-node memory resources.

[0003] Currently, the mainstream technical solutions for achieving cross-compute node memory access include Remote Direct Memory Access (RDMA) and Compute Express Link (CXL) based on cache coherence interconnect. However, these existing technologies all have varying degrees of shortcomings.

[0004] I. Limitations of the RDMA Scheme

[0005] RDMA allows one computer to directly access the memory of another computer without the intervention of the CPUs and operating system kernels at both ends. However, RDMA has the following inherent drawbacks:

[0006] 1. Insufficient software layer transparency. RDMA requires applications to modify code or rely on specific programming interfaces (such as libverbs, OpenFabrics, etc.), and cannot directly use the local CPU's load / store instruction semantics to access remote memory. Applications must explicitly call RDMA read / write primitives and manage complex mechanisms such as memory registration and queue pairs (QPs), which significantly increases the difficulty of software development and migration costs.

[0007] 2. Complex address management. RDMA operations require pre-registering memory regions and obtaining remote memory keys (R_key). If the remote physical memory is not distributed contiguously (such as memory fragmentation), complex software logic is required for address management and mapping maintenance, making it difficult to guarantee access efficiency.

[0008] 3. High hardware resource consumption. Although RDMA implements kernel bypass, it still requires the CPU to participate in control plane operations such as memory registration and queue polling, and lacks an end-to-end hardware-level address translation acceleration mechanism, making it difficult to further reduce cross-node access latency.

[0009] II. Limitations of the CXL Scheme

[0010] CXL, as an emerging cache-coherent interconnect standard, allows host processors to directly access remote memory using load / store instructions via the CXL.mem protocol, offering advantages in single-machine memory expansion scenarios (such as CXL memory expansion cards). However, CXL still has significant shortcomings in scale-up expansion scenarios across compute nodes:

[0011] 1. Lack of cross-node address mapping mechanism. The CXL specification is mainly aimed at memory expansion and pooling within a single host. Although CXL switches can connect multiple hosts, the cross-node address space mapping and routing mechanism lacks a standardized definition, and different vendors implement it differently. In multi-node scenarios, CXL does not provide a complete hardware-level solution for how to efficiently map the non-contiguous physical memory of remote nodes into an address space accessible to local nodes.

[0012] 2. Performance overhead of cross-node cache consistency. CXL requires maintaining cross-node cache consistency (such as a variant of the MESI protocol). When scaling up to multiple nodes on a large scale, the overhead of consistency maintenance increases dramatically, leading to a significant performance degradation.

[0013] 3. Lack of dedicated scale-up network optimization. CXL is based on the PCIe physical layer, and its design goal is to balance general I / O and memory expansion, rather than being specifically optimized for large-scale scale-up interconnects between accelerators.

[0014] III. Common Defects of Existing Cross-Node Memory Access Schemes

[0015] Existing technical solutions have the following common shortcomings in transparent cross-node memory access:

[0016] 1. Lack of a hardware-software coordinating address mapping and transaction translation mechanism. In traditional solutions, the CPU MMU (virtual address → local physical address) and IOMMU (device virtual address → physical address) operate independently, lacking a collaborative design for cross-node access. When a load / store transaction issued by the local CPU needs to access the memory of a remote node, existing solutions cannot automatically translate the local physical address access into a network transaction containing the remote node identifier, address handle, and offset at the hardware level. Cross-node memory access must rely on software intervention for address resolution and routing decisions, increasing latency and consuming computational resources.

[0017] 2. Reliance on unified physical addressing or global address space. Existing solutions require that remote memory be included in a unified global addressing space, causing the local bit width to expand with the total capacity of the cluster, which restricts the scalability of the system.

[0018] 3. Remote memory management relies on a centralized entity. Existing solutions require a centralized management entity to uniformly allocate and maintain cross-node addresses and mappings. They lack a remote memory management mechanism that allows target nodes to be autonomous, which poses a risk of single point of failure and performance bottlenecks in large-scale clusters.

[0019] 4. Inefficient handling of address fragmentation. When remote physical memory is not contiguous, complex software logic is required to manage fragmented memory, making it difficult to hide the fragmented layout of remote physical memory while ensuring access efficiency. Summary of the Invention

[0020] The purpose of this invention is to provide a technical solution for a hardware and software collaborative transparent cross-compute node memory access method and system based on scale-up networks to address the shortcomings of existing technologies. This solution enables the local CPU to transparently access the non-contiguous physical memory of remote nodes with local load / store semantics, reducing cross-node access latency. It also achieves decoupling of local bit width and cluster size, autonomous management and control of remote memory target nodes, and secure isolation of cross-node access.

[0021] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0022] A hardware-software collaborative method for transparent memory access across computing nodes based on scale-up networks, characterized by the following steps:

[0023] S1. Reserve a continuous physical address window in the system physical address space of the local computing node. The physical address window is marked as a remote memory access window, and the local operating system is prohibited from allocating physical address windows.

[0024] S2. The remote computing node allocates a virtual address segment for the remote memory used by the local computing node, and maps the virtual address segment to the non-contiguous physical memory of the remote computing node through a page table.

[0025] S3. The remote computing node generates a unique address handle for the virtual address segment and sends the address handle to the local computing node via the network;

[0026] S4. The local computing node binds the received address handle to the reserved physical address window;

[0027] S5. When a local compute node accesses remote memory via a local physical address, the initiating address translation unit in the scale-up controller of the local compute node translates the local physical address into a network transaction. The target address of the network transaction contains the node identifier, address handle, and offset address of the remote compute node. The network transaction is routed to the remote compute node through the scale-up network. The receiving address translation unit in the scale-up controller of the remote compute node translates the address handle and offset address into the target virtual address in the virtual address segment. The remote compute node translates the target virtual address into the target physical address in the non-contiguous physical memory through the IOMMU and page table to complete the data read and write.

[0028] This access method utilizes a reserved physical address window and address handle binding mechanism, combined with hardware-accelerated transaction translation by the initiating side address translation unit, to automatically convert load / store memory access transactions initiated by the local CPU into network transactions containing the remote node identifier, address handle, and offset address. These transactions are then routed to the remote node via the scale-up network to complete the access. Throughout this process, the local application requires no modification and can transparently access remote memory using local load / store semantics. Simultaneously, the fragmented layout of the remote physical memory is completely hidden by the page table and IOMMU, achieving a synergistic optimization of software transparency and hardware efficiency.

[0029] A hardware-software collaborative transparent memory access system across computing nodes based on Scale-Up networks, used to implement the aforementioned hardware-software collaborative transparent memory access method across computing nodes based on Scale-Up networks, characterized in that it includes:

[0030] Local computing nodes reserve contiguous physical address windows in their system physical address space. These physical address windows are marked as remote memory access windows, and the local operating system is prohibited from allocating physical address windows.

[0031] Remote compute nodes are interconnected with local compute nodes via a scale-up network. They are used to allocate virtual address segments for remote memory allocated to local compute nodes and to map the virtual address segments to the non-contiguous physical memory of remote compute nodes through page tables.

[0032] The handle generation module, located on the remote computing node, is used to generate a unique address handle for the virtual address segment and send the address handle to the local computing node via the network;

[0033] The mapping configuration module, set on the local computing node, is used to bind the received address handle to the reserved physical address window;

[0034] The initiating side address translation module is set in the scale-up controller of the local compute node. It is used to translate the local physical address into a network transaction when the local compute node accesses remote memory through the local physical address. The destination address of the network transaction includes the node identifier, address handle and offset address of the remote compute node.

[0035] The network routing module is used to route network transactions to remote computing nodes via the Scale-Up network;

[0036] The receiving-side address translation module, located in the Scale-Up controller of the remote computing node, is used to translate the address handle and offset address into the target virtual address in the virtual address segment;

[0037] And a third address translation module, located on a remote computing node, is used to translate the target virtual address into the target physical address in non-contiguous physical memory through the IOMMU and page table to complete data reading and writing.

[0038] Through the above design, the modules work together to build a complete end-to-end hardware-accelerated cross-node memory access pipeline. From the physical address window reservation and mapping configuration of the local computing node, to the address translation and network routing on the initiating side, and then to the handle interpretation and IOMMU translation on the receiving side, all address translation and transaction encapsulation are completed at the hardware level without the need for software hop-by-hop intervention, which significantly reduces cross-node memory access latency, while achieving complete transparency of the remote physical memory fragmentation layout to the source node.

[0039] The present invention, by adopting the above-described technical solution, has the following beneficial effects:

[0040] 1. The application of this invention can directly access remote memory through local load / store semantics without modification, is compatible with the existing software ecosystem, greatly reduces the difficulty of software development and migration costs, and achieves transparent access.

[0041] 2. The remote node of the present invention hides physical memory fragments through page tables and IOMMU, presents a continuous virtual address segment to the outside world, simplifies cross-node address management, avoids the complex fragmented memory management logic on the source node side, and achieves efficient memory management.

[0042] 3. This invention completes address translation and network transaction routing through a hardware-accelerated pipeline of the initiating side address translation unit, the receiving side address translation unit, and the IOMMU, reducing software intervention and significantly reducing cross-node memory access latency.

[0043] 4. This invention supports multi-node scale-up expansion, with memory capacity increasing linearly with the number of nodes, meeting the needs of large memory workloads.

[0044] 5. This invention uses address handles to hide the physical layout of remote memory, exposing only logical identifiers. The source node cannot know the physical address information of the remote memory, thus achieving secure isolation for cross-node access.

[0045] 6. Each node in this invention maintains an independent local physical address space with adjustable bit width, eliminating the need to incorporate remote memory into a unified global addressing system. The local bit width does not expand with the number of cluster nodes or the total capacity, reducing hardware design complexity and decoupling bit width from cluster size.

[0046] 7. In this invention, the export, cancellation, migration, and page table update of remote memory are led by the memory owner on the target node side, eliminating the need for a centralized management entity, thus eliminating single points of failure and global lock contention, improving the scalability and availability of the system, and realizing autonomous management of the target node.

[0047] 8. The hot migration of this invention is transparent. When the target virtual machine is migrated or physical pages are reorganized, only the IOMMU page table is updated. The handle and access address held by the initiating side remain unchanged, realizing complete transparency of remote memory hot migration to the source node.

[0048] 9. This invention does not rely on a dedicated Scale-Up interconnect protocol, but only requires the underlying network to have load / store transaction carrying capabilities. It is compatible with various Scale-Up interconnect standards such as ETH-X, SUE, CXL over Fabric, and UALink, as well as a self-developed memory semantic interconnect protocol, and has good versatility and portability. Attached Figure Description

[0049] The present invention will be further described below with reference to the accompanying drawings:

[0050] Figure 1 This is a flowchart of a hardware-software collaborative method for transparent cross-compute node memory access based on Scale-Up networks, and a method within the system of the present invention.

[0051] Figure 2 This is a hardware architecture diagram of the present invention;

[0052] Figure 3 This is a schematic diagram of the address translation relationship and data flow in this invention;

[0053] Figure 4 This is a flowchart of the sending-side address translation process in this invention;

[0054] Figure 5 This is a flowchart of the receiving-side address conversion in this invention. Detailed Implementation

[0055] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0056] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0057] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0058] The Chinese translations of the English terms in this invention are shown in the table below:

[0059] like Figure 1 As shown, this invention provides a hardware-software collaborative method for transparent cross-compute node memory access based on Scale-Up networks, comprising the following steps:

[0060] S1. Reserve a continuous physical address window in the system physical address space of the local computing node. The physical address window is marked as a remote memory access window, and the local operating system is prohibited from allocating physical address windows.

[0061] The physical address window is reserved by the firmware during the system memory mapping initialization phase and declared as device memory, preventing the operating system's physical page allocator from including the physical address window in regular main memory management. The reservation and declaration of the physical address window are completed by the firmware early in the system startup process, without modifying the operating system kernel source code. This ensures compatibility with existing operating system memory management mechanisms and avoids the risk of data corruption and system crashes caused by the operating system misusing remote memory access windows as regular main memory.

[0062] Specifically, during the system memory mapping initialization phase, the firmware (such as OpenSBI on the RISC-V platform) reserves a contiguous range (such as 15-16TB) in the local physical address space as a "remote memory access window" and declares it as device memory (similar to the MMIO address range of PCIe). This prevents the operating system's physical page allocator from including this range in regular main memory management and only allows the Scale-Up interconnect driver to access it through explicit mapping.

[0063] The size of the physical address window is a fixed value in the text, but its base address and size can be changed by adjusting the system address mapping of the on-chip NoC. The specific size of the physical address window is determined by the system design (e.g., 1TB, 512GB, 256GB, etc.), and the amount of remote memory allocated by the remote compute node to the local compute node is less than or equal to the size of the physical address window. This simplifies the address space management logic of the firmware and operating system, reduces the complexity of system implementation, and provides flexible configuration capabilities by adjusting the base address and size of the window through the system address mapping of the on-chip network (NoC).

[0064] S2. The remote compute node allocates a virtual address segment for the remote memory used by the local compute node, and maps the virtual address segment to the non-contiguous physical memory of the remote compute node through a page table.

[0065] Specifically, the local compute node initiates a memory request to the remote compute node through the front-end network (out-of-band control network). The operating system of the remote compute node allocates a contiguous virtual address segment for the remote memory used by the local compute node, and maps this virtual address segment to the non-contiguous physical memory (such as multiple discrete DDR physical address spaces) of the remote compute node through a page table. This mapping relationship is maintained by the operating system of the remote compute node.

[0066] S3. The remote computing node generates a unique address handle for the virtual address segment and sends the address handle to the local computing node via the network.

[0067] Specifically, the scale-up interconnect driver of the remote computing node generates address handles (e.g., 0x1) for the allocated virtual address segments, establishes a mapping relationship between the handles and the virtual address segments, and writes the mapping table to the address translation unit on the receiving side. The handles are generated by the scale-up interconnect driver of the remote computing node and are unique within the remote computing node. In multi-node scenarios, global uniqueness is achieved by combining the address handle with the node identifier of the remote computing node. The address handle is guaranteed to be unique only within a single remote computing node, forming a globally unique identifier when combined with the node identifier. This eliminates the need for a centralized entity to allocate global handles, achieving decentralized handle generation and management and target node autonomy, thus eliminating single points of failure and global lock contention bottlenecks.

[0068] An address handle is a logical identifier used to hide the physical layout of remote memory on a remote computing node. By exposing the address handle as a logical identifier, the physical address layout of remote memory (including the distribution, capacity, and topology information of non-contiguous physical memory) is completely hidden. The source node can only access remote memory through the logical handle, effectively preventing security risks caused by physical address leakage and improving the security of cross-node memory access.

[0069] After the address handle is generated, the remote compute node sends the handle to the local compute node through the front-end network (out-of-band control network). The front-end network is an out-of-band control network outside the scale-up data network, and can be implemented based on TCP / IP or PCIe.

[0070] The message format is customized by the Scale-Up interconnect driver. The message content includes at least a memory allocation success flag and the corresponding address handle. Handle information is transmitted via out-of-band control network, separated from the Scale-Up data network. This decoupling of the control plane and data plane avoids interference from control messages on data plane transmission performance. The customized message format also ensures implementation flexibility and scalability.

[0071] S4. The local computing node binds the received address handle to the reserved physical address window.

[0072] Specifically, the operating system of the local computing node requests local virtual memory and maps the local virtual memory to the corresponding address segment of the physical address window through the local page table.

[0073] The initiating side address translation unit in the scale-up controller of the local compute node translates the local physical address into a network transaction. The target address of the network transaction contains the node identifier, address handle, and offset address of the remote compute node.

[0074] When multiple remote compute nodes exist, each remote compute node generates an independent address handle for its allocated remote memory. The local compute node then binds each address handle to a different address segment of the physical address window. A single physical address window can be divided into multiple address segments, each bound to address handles from different remote compute nodes. This enables aggregated access to the memory resources of remote nodes from a single local window, effectively improving the utilization efficiency of the local physical address window and the scalability of the system.

[0075] S5. When a local compute node accesses remote memory via a local physical address, the initiating address translation unit in the scale-up controller of the local compute node translates the local physical address into a network transaction. The target address of the network transaction contains the node identifier, address handle, and offset address of the remote compute node. The network transaction is routed to the remote compute node through the scale-up network. The receiving address translation unit in the scale-up controller of the remote compute node translates the address handle and offset address into the target virtual address in the virtual address segment. The remote compute node translates the target virtual address into the target physical address in the non-contiguous physical memory through the IOMMU and page table to complete the data read and write.

[0076] The initiating-side address translation unit is configured with address translation rules, which are maintained by the scale-up interconnect driver of the local compute node. These rules are used to convert memory access transactions that access the physical address window into network transactions. Dynamic updates and flexible configuration are supported, allowing adaptation to different remote memory mapping requirements without hardware modifications. Furthermore, the address translation rules employ a fixed number of entries, handling only large memory blocks. Remote physical memory fragmentation is handled by the remote IOMMU, reducing the design complexity of the initiating-side address translation unit.

[0077] The receiving-side address translation unit is configured with a mapping relationship between address handles and virtual address segments. This mapping relationship is maintained by the scale-up interconnect driver of the remote computing node. The scale-up interconnect driver of the remote node is responsible for establishing, updating, and revoking the mapping relationship. The target node of the remote memory has complete autonomy over the export, revocation, migration, and page table updates of memory without the intervention of a centralized entity, thus achieving target node autonomy in remote memory management.

[0078] Specifically, when the CPU of a local compute node accesses local virtual memory, it translates the local virtual memory into a physical address within the physical address window through the local page table. This allows applications to access remote memory without being aware of the process, following a standard virtual-to-physical address translation procedure. This software path is completely consistent with local memory access, achieving transparent access at the application level.

[0079] The virtual address is translated by the IOMMU and finally accessed through the page table to access the data corresponding to the target physical address in the non-contiguous physical memory.

[0080] This access method utilizes a reserved physical address window and address handle binding mechanism, combined with hardware-accelerated transaction translation by the initiating side address translation unit, to automatically convert load / store memory access transactions initiated by the local CPU into network transactions containing the remote node identifier, address handle, and offset address. These transactions are then routed to the remote node via the scale-up network to complete the access. Throughout this process, the local application requires no modification and can transparently access remote memory using local load / store semantics. Simultaneously, the fragmented layout of the remote physical memory is completely hidden by the page table and IOMMU, achieving a synergistic optimization of software transparency and hardware efficiency.

[0081] A hardware-software collaborative transparent memory access system across computing nodes based on Scale-Up networks is used to implement the aforementioned hardware-software collaborative transparent memory access method across computing nodes based on Scale-Up networks. The system includes: a local computing node, a remote computing node, a handle generation module, a mapping configuration module, an initiating side address translation module, a network routing module, and a third address translation module.

[0082] The local compute node reserves a continuous physical address window in its system physical address space. The physical address window is marked as a remote memory access window, and the local operating system is prohibited from allocating physical address windows.

[0083] The local compute node also includes a local page table. The operating system of the local compute node allocates local virtual memory and maps the local virtual memory to the physical address window through the local page table. This ensures that application code can transparently access remote memory without any modification, maintaining compatibility with the existing software ecosystem.

[0084] The CPU of the local compute node accesses remote memory through local load / store semantics. This is achieved via local page tables, an initiating-side address translation module, a network routing module, a receiving-side address translation module, and a third-party address translation module, enabling transparent read / write access to the remote memory of the remote compute node. This covers the entire link from the issuance of load / store instructions by the local CPU to the return of data from the remote physical memory. All address and transaction translations are completed in the hardware pipeline, making the latency of cross-node memory access close to that of local memory access. Furthermore, read / write operations, atomic operations, error handling, and consistency constraints utilize AXI and transport layer semantics, ensuring compatibility with standard bus protocols without additional protocol adaptation overhead.

[0085] Remote compute nodes are interconnected with local compute nodes via a scale-up network. They are used to allocate virtual address segments for remote memory allocated to local compute nodes and to map these virtual address segments to the non-contiguous physical memory of remote compute nodes via page tables.

[0086] The handle generation module is set on the remote computing node and is used to generate a unique address handle for the virtual address segment and send the address handle to the local computing node through the network.

[0087] The mapping configuration module is set on the local compute node and is used to bind the received address handle to the reserved physical address window.

[0088] The initiating side address translation module is located in the scale-up controller of the local compute node. It is used to translate the local physical address into a network transaction when the local compute node accesses remote memory through the local physical address. The destination address of the network transaction includes the node identifier, address handle and offset address of the remote compute node.

[0089] The network routing module is used to route network transactions to remote computing nodes via the Scale-Up network.

[0090] The receiving-side address translation module is located in the scale-up controller of the remote computing node and is used to translate the address handle and offset address into the target virtual address in the virtual address segment.

[0091] The third address translation module is located on the remote computing node and is used to translate the target virtual address into the target physical address in non-contiguous physical memory through the IOMMU and page table to complete data reading and writing.

[0092] The third address translation module includes an IOMMU and page tables. The IOMMU is configured with address translation rules that map virtual address segments to non-contiguous physical memory. These address translation rules are maintained by the operating system of the remote compute node. When a remote virtual machine is migrated or physical pages are consolidated, only the IOMMU page tables need to be updated. The virtual addresses and handles held by the initiating node do not need to be changed, achieving transparency of remote memory hot migration to the source node. This greatly simplifies system operation and maintenance and dynamic resource scheduling for large-scale clusters.

[0093] The system also includes multiple remote computing nodes. Each remote computing node generates an independent address handle for its allocated remote memory, and the local computing node binds each address handle to a different address segment of the physical address window. The system supports parallel access and independent handle management of multiple remote computing nodes. The memory resources of each remote computing node can be accessed uniformly by the local node through different addresses within the same local physical address window. The cluster memory capacity grows linearly with the number of nodes, achieving good horizontal scalability.

[0094] Through the above design, the modules work together to build a complete end-to-end hardware-accelerated cross-node memory access pipeline. From the physical address window reservation and mapping configuration of the local computing node, to the address translation and network routing on the initiating side, and then to the handle interpretation and IOMMU translation on the receiving side, all address translation and transaction encapsulation are completed at the hardware level without the need for software hop-by-hop intervention, which significantly reduces cross-node memory access latency, while achieving complete transparency of the remote physical memory fragmentation layout to the source node.

[0095] Example 1: Dual-Node Memory Expansion Scenario

[0096] System Configuration:

[0097] Compute node 1 (CN1) and compute node 2 (CN2) are interconnected via a Scale-Up interface. The two compute nodes have independent system physical address spaces, with a total address space size of 16TB.

[0098] CN1 has a 15-16TB physical address window (Window_Remote_Addr) reserved locally for allocation to the Scale-Up controller.

[0099] CN2 is allocated 512GB of memory for CN1 to use.

[0100] Step 1: Hardware Address Space Planning

[0101] During the system memory mapping initialization phase, CN1's firmware marks the 15-16TB Scale-Up controller physical address space as a "remote memory access window," declaring it as device memory, so that CN1's local operating system physical page allocator does not include this range in regular main memory management.

[0102] Step 2: Remote memory allocation and mapping

[0103] CN1 requests 512GB of memory from CN2 via the front-end network. CN2's operating system creates process P1 for the memory allocated to CN1, requests 512GB of virtual memory (Virtual_Addr_CN2), and maps it to four non-contiguous 128GB DDR spaces (Physical_Addr_DDR0~3) via page table Page2. CN2's IOMMU is configured with the following address translation rules: Virtual_Addr_CN2 is mapped to Physical_Addr_DDR0~3 via Page2.

[0104] Step 3: Address handle generation and passing

[0105] The CN2 Scale-Up interconnect driver allocates handle 0x1 to Virtual_Addr_CN2, establishes a mapping relationship, and writes this mapping table into the receiving-side address translation unit: CN2_ID + handle 0x1 + offset address → Virtual_Addr_CN2. CN2 sends handle 0x1 to CN1 through the front-end network, with the message format being {memory allocation success flag, handle 0x1}.

[0106] Step 4: Local mapping configuration

[0107] The CN1 operating system requests 512GB of virtual memory (Virtual_Addr_CN1) and maps it to the first 512GB (15-15.5TB) of Window_Remote_Addr through the local page table Page1. The CN1 Scale-Up interconnect driver configures the address translation rules of the initiating side address translation unit: memory access transactions to Window_Remote_Addr are converted into network transactions, and the target address = CN2_ID + handle 0x1 + offset address + source node CN1_ID.

[0108] Address translation rule format: The address translation table of the initiating side address translation unit adopts a fixed number of entries. Each entry contains the local physical address window base address, window size, target node identifier, and target handle. Since address translation only needs to handle the mapping of large blocks of memory, remote physical memory fragmentation is maintained by the remote IOMMU, reducing the design complexity of the address translation unit. Each entry in the address translation table of the receiving side address translation unit contains the source node identifier, handle, mapped virtual address base address, and size.

[0109] Step 5: Transparent Access Verification

[0110] The CPU of CN1 accesses the virtual address Virtual_Addr_CN1+0x1000, which is translated into a memory access transaction for the physical address Window_Remote_Addr+0x1000 via page table Page1. This memory access transaction is routed to the Scale-Up controller via the on-chip network. The initiating address translation unit in the controller translates the physical address Window_Remote_Addr+0x1000 into a network transaction: target = CN2_ID + handle 0x1 + offset address 0x1000 + source node CN1_ID. This network transaction is transmitted to CN2 via the Scale-Up network. The receiving address translation unit in the Scale-Up controller of CN2 obtains the virtual address Virtual_Addr_CN2+0x1000 based on the source node CN1_ID, handle 0x1, and offset address 0x1000. This virtual address is translated by locating Page2 through the IOMMU, ultimately accessing the data corresponding to Physical_Addr_DDR0+0x1000. The data is returned to CN1 via the Scale-Up network, completing the transparent read / write operation.

[0111] In the above process, read / write operations, atomic operations, error handling, and consistency constraints follow the semantics of AXI and the transport layer. The data return path is symmetrical to the request path; read operations require data return, while write operations only require confirmation (and return a completion status if necessary). Atomic operations are supported by atomic transactions provided by the Scale-Up network transport layer.

[0112] like Figure 2As shown, compute node 1 and compute node 2 are interconnected via a scale-up network. The CPU of compute node 1 is connected to the scale-up controller via an on-chip network. The scale-up controller contains an initiating-side address translation unit (ACC MMU), which is responsible for translating local physical addresses into network transactions. The scale-up controller of compute node 2 contains a receiving-side address translation unit (LINK MMU), which is responsible for translating handles and offset addresses in network transactions into local virtual addresses, and then locating the actual physical memory via the IOMMU and page tables. The red curve represents the complete data path for the local CPU to access remote memory in step 5.

[0113] like Figure 3 As shown, the address translation path on the local compute node side is: local virtual address (VA_CN1) → local page table (Page1) → local physical address (PA_Window) → initiating side address translation unit → network transaction (Node_ID+Handle+Offset). After the network transaction is transmitted to the remote compute node via the Scale-Up network, the address translation path on the remote compute node side is: network transaction (Node_ID+Handle+Offset) → receiving side address translation unit → remote virtual address (VA_CN2) → IOMMU+page table (Page2) → remote physical address (PA_DDR).

[0114] like Figure 4 As shown, the initiating side address translation unit receives physical address memory access transactions from the local CPU and determines whether the physical address falls within the reserved physical address window range. If so, it searches the address translation table, matches the corresponding window entry, obtains the target node identifier and target handle, and constructs a network transaction (target node identifier + handle + offset address + source node identifier); otherwise, it processes the access as a regular local memory access. The address translation table is maintained by the local Scale-Up interconnect driver and supports dynamic addition, deletion, and updating of entries.

[0115] like Figure 5 As shown, the receiving-side address translation unit receives network transactions from the Scale-Up network and parses out the source node identifier, target handle, and offset address. It searches the handle mapping table for an entry matching the target handle, obtains the corresponding remote virtual address base address, calculates the target virtual address by combining it with the offset address, and then sends the target virtual address to the IOMMU for subsequent address translation. The handle mapping table is maintained by the remote Scale-Up interconnect driver and supports dynamic establishment and cancellation of mapping relationships.

[0116] Example 2: Multi-node memory expansion scenario

[0117] When the system needs to add computing nodes CN3 and CN4, repeat steps 2 to 4 of Example 1.

[0118] Specifically: CN3 allocates a virtual address segment for the memory used by CN1, generates an independent handle (such as 0x2), and sends handle 0x2 to CN1 through the front-end network. CN1 binds handle 0x2 to the next address segment (15.5-16TB) of the physical address window and configures the corresponding translation rules of the initiating side address translation unit.

[0119] CN4 operates similarly to CN3, generating independent handles (such as 0x3) and mapping them to the corresponding address segments of the physical address window. CN1 accesses the remote memory of CN2, CN3, and CN4 through different address segments of the physical address window, and the memory resources of each remote node are aggregated into a unified locally accessible address space through the same physical address window.

[0120] In multi-node scenarios, the address handles of each remote computing node only need to be unique within their respective nodes. Global uniqueness is achieved through the combination of {handle + node ID}, eliminating the need for a centralized entity to allocate global handles. Each node independently manages its own handle space, eliminating the risk of global lock contention and single point of failure.

[0121] Example 3: Remote Memory Hot Migration Scenario

[0122] When a virtual machine (VM) in the remote compute node CN2 needs to be migrated to another physical node, or when the operating system performs physical page compaction, the CN2 operating system updates the IOMMU page table, remapping the original virtual address segments to new physical memory addresses. Since CN1 only holds the node identifier, address handle, and virtual address segment information (offset address) of CN2, and is unaware of the actual distribution of the underlying physical memory, the update of the IOMMU page table on CN2 is completely transparent to CN1. CN1 continues to access the remote memory using the original handle and offset address, without any configuration changes or software notifications, achieving transparency of remote memory hot migration to the source node.

[0123] Example 4: Cross-node access security isolation scenario

[0124] Address handles are exposed externally as logical identifiers, completely hiding the physical address layout of remote memory (including the distribution, capacity, and topology information of non-contiguous physical memory). In scale-up network transactions, the target virtual address (obtained through handle + offset address decoding) is carried. The receiving-side address translation unit performs hardware verification of the source node identifier, target address range, operation type, and permissions when decoding the handle. Specifically, the handle mapping table stores access permission information (e.g., read-only, read-write, executable, etc.) for each handle. The receiving-side address translation unit performs permission matching checks based on the source node identifier and operation type in the network transaction, and only submits the transaction to the IOMMU for further processing after the permission verification passes. Unauthorized access requests are intercepted by hardware and errors are reported, effectively preventing unauthorized access across nodes and malicious attacks.

[0125] The above are merely specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent substitutions, or modifications made based on the present invention to achieve substantially the same technical effect are all covered within the protection scope of the present invention.

Claims

1. A hardware-software collaborative method for transparent cross-compute node memory access based on Scale-Up networks, characterized in that... Includes the following steps: S1. Reserve a continuous physical address window in the system physical address space of the local computing node. The physical address window is marked as a remote memory access window, and the local operating system is prohibited from allocating the physical address window. S2. The remote computing node allocates a virtual address segment for the remote memory used by the local computing node, and maps the virtual address segment to the non-contiguous physical memory of the remote computing node through a page table. S3. The remote computing node generates a unique address handle for the virtual address segment and sends the address handle to the local computing node via the network; S4. The local computing node binds the received address handle to the reserved physical address window; S5. When the local computing node accesses the remote memory through a local physical address, the initiating address translation unit in the scale-up controller of the local computing node converts the local physical address into a network transaction. The target address of the network transaction includes the node identifier of the remote computing node, the address handle, and the offset address. The network transaction is routed to the remote computing node through the scale-up network. The receiving address translation unit in the scale-up controller of the remote computing node translates the address handle and the offset address into a target virtual address in the virtual address segment. The remote computing node converts the target virtual address into a target physical address in the non-contiguous physical memory through the IOMMU and page table to complete the data read / write operation.

2. The method for transparent cross-compute node memory access based on scale-up networks using hardware and software collaboration as described in claim 1, characterized in that: The physical address window mentioned in step S1 is reserved by the firmware during the system memory mapping initialization phase and declared as device memory, so that the operating system's physical page allocator does not include the physical address window in the regular main memory management.

3. The method for transparent cross-compute node memory access based on scale-up networks using hardware and software collaboration as described in claim 1, characterized in that: The size of the physical address window is a fixed value, and the size of the remote memory allocated by the remote computing node to the local computing node is less than or equal to the size of the physical address window.

4. The method for transparent cross-compute node memory access based on scale-up networks using hardware and software collaboration as described in claim 1, characterized in that: The address handle in step S3 is generated by the Scale-Up interconnect driver of the remote computing node and is unique within the remote computing node; In a multi-node scenario, global uniqueness is achieved by combining the address handle with the node identifier of the remote computing node.

5. The method for transparent cross-compute node memory access based on scale-up networks using hardware and software collaboration as described in claim 4, characterized in that: The address handle is a logical identifier used to hide the remote memory physical layout of the remote computing node.

6. The method for transparent cross-compute node memory access based on scale-up networks using hardware and software collaboration as described in claim 1, characterized in that: Step S3, which involves sending the address handle to the local computing node via the network, specifically includes implementation through an out-of-band control network. The out-of-band control network can be a TCP / IP network or a PCIe network. The message format is customized by the Scale-Up interconnect driver, and the message content includes at least a memory allocation success flag and the corresponding address handle.

7. The method for transparent cross-compute node memory access based on scale-up networks using hardware and software collaboration as described in claim 1, characterized in that: In step S4, the operating system of the local computing node requests local virtual memory and maps the local virtual memory to the physical address window through the local page table; when the CPU of the local computing node accesses the local virtual memory, it converts the local virtual memory into a memory access transaction of a physical address in the physical address window through the local page table.

8. The method for transparent cross-compute node memory access based on scale-up networks using hardware and software collaboration as described in claim 1, characterized in that: In step S4, when there are multiple remote computing nodes, each remote computing node generates an independent address handle for its allocated remote memory, and the local computing node binds each address handle to a different address segment of the physical address window.

9. A method for transparent cross-compute node memory access based on scale-up networks using hardware and software collaboration, as described in claim 1, characterized in that: The initiating side address translation unit in step S5 is configured with address translation rules, which are maintained by the Scale-Up Interconnect driver of the local computing node and are used to convert memory access transactions that access the physical address window into network transactions.

10. A method for transparent cross-compute node memory access based on scale-up networks using hardware and software collaboration, as described in claim 1, characterized in that: The receiving-side address translation unit in step S5 is configured with a mapping relationship between the address handle and the virtual address segment, and the mapping relationship is maintained by the scale-up interconnect driver of the remote computing node.

11. A hardware-software collaborative transparent memory access system across computing nodes based on Scale-Up networks, used to implement the hardware-software collaborative transparent memory access method across computing nodes based on Scale-Up networks as described in any one of claims 1 to 10, characterized in that, include: The local computing node has a continuous physical address window reserved in its system physical address space. The physical address window is marked as a remote memory access window, and the local operating system is prohibited from allocating the physical address window. The remote computing node is interconnected with the local computing node through a scale-up network. It is used to allocate virtual address segments for remote memory allocated to the local computing node and to map the virtual address segments to the non-contiguous physical memory of the remote computing node through a page table. A handle generation module, located on the remote computing node, is used to generate a unique address handle for the virtual address segment and send the address handle to the local computing node via the network; A mapping configuration module, located on the local computing node, is used to bind the received address handle to the reserved physical address window; An initiating side address translation module is set in the scale-up controller of the local computing node. It is used to translate the local physical address into a network transaction when the local computing node accesses the remote memory through the local physical address. The destination address of the network transaction includes the node identifier of the remote computing node, the address handle, and the offset address. The network routing module is used to route the network transactions to the remote computing node through the Scale-Up network; A receiving-side address translation module, located in the Scale-Up controller of the remote computing node, is used to translate the address handle and the offset address into a target virtual address in the virtual address segment. And a third address translation module, located on the remote computing node, is used to translate the target virtual address into the target physical address in the non-contiguous physical memory through the IOMMU and page table to complete data read and write.

12. A hardware-software collaborative cross-compute node transparent memory access system based on scale-up networks as described in claim 11, characterized in that: The third address translation module includes an IOMMU and a page table. The IOMMU is configured with address translation rules that map the virtual address segment to the non-contiguous physical memory. The address translation rules are maintained by the operating system of the remote computing node.

13. A hardware-software collaborative cross-compute node transparent memory access system based on scale-up networks as described in claim 11, characterized in that: The local computing node also includes a local page table. The operating system of the local computing node requests local virtual memory and maps the local virtual memory to the physical address window through the local page table.

14. A hardware-software collaborative cross-compute node transparent memory access system based on scale-up networks as described in claim 11, characterized in that: The system also includes multiple remote computing nodes, each of which generates an independent address handle for its allocated remote memory, and the local computing node binds each address handle to a different address segment of the physical address window.

15. A hardware-software collaborative cross-compute node transparent memory access system based on scale-up networks as described in claim 11, characterized in that: The CPU of the local computing node accesses the remote memory through local load / store semantics, and achieves transparent read and write access to the remote memory of the remote computing node through the local page table, the initiating side address translation module, the network routing module, the receiving side address translation module and the third address translation module.

Images