A heterogeneous storage control system, method, and medium based on multidimensional semantic awareness

Abstract

This invention discloses a heterogeneous storage control system, method, and medium based on multidimensional address structure awareness, belonging to the field of high-performance computing architecture. The system integrates an L1 / L2 hierarchical multidimensional translation backup buffer (MD-TLB), a hardware tensor page table walker with an exception state machine, a pipelined spatial address operation unit (SAMU) supporting indirect step size and lookup table caching, and a consistency engine supporting range invalidation. The controller establishes physical address sovereignty through a memory pinning protocol and an IOMMU exemption mechanism; efficiently processes sparse tensors using indirect step size logic; ensures atomicity and resource reclamation during task switching through pipelined draining and a safety heartbeat watchdog; and eliminates bus storms using a range consistency protocol. This scheme significantly reduces the number of address translation stages, solves the problems of sparse data adaptation and hardware deadlock in heterogeneous computing, and achieves highly reliable, low-latency semantic virtual memory management.

Description

Technical Field

[0001] This invention relates to the fields of computer architecture and integrated circuit design, specifically to a semantic virtual memory system, method, and medium that supports sparse mapping, hierarchical caching, system-level coordination, and security fault tolerance. Background Technology

[0002] In the fields of artificial intelligence and high-performance computing, data is usually in the form of multidimensional tensors. However, the traditional von Neumann architecture uses a linear physical address space, which leads to a "semantic gap". Existing heterogeneous storage solutions have three major bottlenecks: (1) Address translation is missing and the processing depends on the operating system, resulting in long-tail latency, and traditional page table lookups require 4-5 levels of jumps; (2) Traditional linear affine transformations cannot handle sparse physical memory or compressed formats; (3) There is a lack of deep collaboration with the host operating system's memory management (such as IOMMU, page migration), which leads to physical address invalidation or performance zeroing. In addition, in a multi-tenant cloud environment, there is a lack of effective hardware-level isolation and "zombie" resource reclamation mechanisms. Summary of the Invention

[0003] This invention provides a highly available semantic virtual memory system. The core innovations include: (1) L1 / L2 hierarchical architecture and hardware stepper: solving the metadata capacity limitation in large model scenarios, compressing the address translation levels from the traditional 4-5 levels of software page table lookup to 1-2 levels of direct hardware matching. (2) Speculative indirect step size calculation: solving the addressing bottleneck of sparse tensors and non-contiguous physical space through mathematically rigorous table lookup formulas and hardware prefetching. (3) Full lifecycle security contract: introducing IOMMU exemption, memory pinning and security heartbeat mechanisms to ensure system-level robustness from address translation to fault recovery. (4) Range consistency maintenance: using the Range Invalidation mechanism to eliminate bus storms during high-dimensional data reorganization. Attached Figure Description

[0004] (1). Figure 1 This is a system architecture diagram of the heterogeneous storage controller provided in the embodiments of the present invention, showing the topology relationship of L1 / L2 MD-TLB, SAMU, Page Walker and heartbeat watchdog. (2). Figure 2 This is a schematic diagram of the bit field structure of the Multidimensional Virtual Address (MSVA) in an embodiment of the present invention. (3). Figure 3 This is a schematic diagram of the internal range comparison logic circuit of the MD-TLB in this embodiment of the invention. (4). Figure 4 This is a structural diagram of the SAMU's internal pipelined affine transformation engine and Stride LUTCache in this embodiment of the invention. (5). Figure 5This is a timing diagram of protocol interaction and range consistency maintenance based on CXL VDM in this embodiment of the invention. (6) Figure 6 This is a comparison diagram of the address translation paths and delays between existing linear addressing technology and the semantic addressing architecture of this invention. Detailed Implementation

[0005] (I) Example 1: System-level memory coordination (Pinning & IOMMU Bypass). During the tensor registration phase, the host driver first requests the operating system kernel to "pin" the target physical page frame, preventing the OS kernel's memory compression or transparent big page migration thread from moving these physical pages. Subsequently, the driver configures the host IOMMU to establish pass-through rules for the controller's physical aperture. This ensures that the physical address (PA) seen by the SVMC is stable and directly accessible, eliminating the performance overhead of nested translation and establishing the controller's "first sovereignty" over physical memory. (II) Example 2: L1 / L2 hierarchical MD-TLB and hardware walker. For recommendation system scenarios, the MD-TLB adopts a hierarchical design. The L1 MD-TLB is a fully associative structure, storing only a small number of active tensor slices involved in the current batch, resulting in extremely fast matching speed. The L2 MD-TLB stores the base address information of the entire model. The two can be configured with an inclusive, non-inclusive, or exclusive strategy to maximize effective capacity utilization. When L1 misses, the hardware automatically backfills from L2; if L2 also misses, the hardware tensor page table walker is triggered. It is worth noting that the tensor page table adopts a dimension-separated index structure, which may include, but is not limited to, hash tables split by dimension, segmented lookup tables, or radix tree transformations supporting multi-dimensional indexes. Its core feature is the decoupling of addressing logic from the tensor's order (rank). (III). Example 3: Sparse mapping and transpose based on indirect step size. The SAMU enters indirect step size mode, supporting formulas... For example, for transpose access, the computation logic is configured to automatically switch. and The corresponding step index parameter enables logical matrix transposition without moving the original data in physical memory. Furthermore, the Stride LUT Cache uses speculative prefetching logic to preload the step parameters for subsequent coordinates, masking lookup latency. (IV). Example 4: Range Consistency and Alignment Correction. When the controller performs large-scale data reorganization, the consistency engine constructs a range invalid message for the CXL protocol. Before this, the internal alignment correction circuit automatically checks the starting address and length of the request. If the starting address is not aligned to the cache line (64B), the circuit automatically rounds it down (Floor) and increases the length accordingly, while simultaneously aligning the payload granularity of the protocol packet to ensure that the consistency operation sent to the bus covers all potentially dirty data lines. (V). Example 5: Security Heartbeat and Automatic Unpinning. The host driver needs to write a preset authentication sequence to the controller's security heartbeat watchdog register every specific period. If the host OS crashes and causes the heartbeat to stop for more than its time-to-live (TTL), the watchdog triggers a forced reset procedure: (a) terminating all ongoing DMA operations; (b) clearing the MD-TLB; and (c) sending an Unlock message to the CXL bus to release pinned physical memory. This prevents "zombie memory" from permanently occupying memory due to host failure.

Claims

1. A heterogeneous memory controller based on multidimensional address structure awareness, characterized in that, The controller is configured as either standalone hardware or integrated logic to physically connect the host processor and heterogeneous storage media, including: (1). Multidimensional address interface logic, configured to receive memory access requests; the memory access request includes an address space identifier (ASID) and a multidimensional coordinate index for describing the tensor geometry; (2). Multidimensional transformation backup buffer (MD-TLB) adopts an L1 / L2 hierarchical structure to store the mapping relationship between the logical tensor block with access control and the discrete physical page frame set; (3) Hardware Tensor Page Walker, coupled to the MD-TLB; configured to automatically suspend the current request when the multidimensional coordinate index is not hit in the MD-TLB, and traverse the multi-level tensor page tables in physical memory according to the tensor dimension definition; the walker includes an exception reporting state machine, which writes the error status to the register and triggers a host interrupt when an invalid page table entry or permission violation is found during traversal, and releases the suspended request queue to prevent bus deadlock; (4). Spatial Address Operation Unit (SAMU), which includes a pipelined affine computation array and a stride LUT cache; configured to support indirect stride mode, obtain stride parameters through dynamic table lookup to adapt to sparse tensors or non-contiguous physical space, and calculate discrete physical addresses in real time in a pipelined manner. (5) A consistency and protocol management engine that supports the consistency semantics of interconnect protocols; it includes range-based invalidation logic, configured to send a consistency signal containing an address range descriptor instead of sending it line by line when a large-scale data reorganization operation is detected that causes changes to memory data, thereby reducing interconnect bus overhead.

2. The heterogeneous storage controller according to claim 1, characterized in that, The L1 / L2 hierarchical structure of the MD-TLB is configured as follows: (1). L1 MD-TLB: Configured as a fully associative or highly associative structure, it stores the tensor slice mapping of the currently active computing task and provides single-clock-cycle hit latency; (2). L2 MD-TLB: Configured as a large-capacity group-associative structure to store a wide range of tensor base addresses and global context information; (3). The L1 and L2 MD-TLBs are configured to use an inclusive, non-inclusive or exclusive cache consistency strategy; (4) The numerical comparison logic circuit generates a hit signal only when the ASID matches, the permission bit is enabled, and the input dimension index falls within the closed interval defined by the upper and lower bound registers.

3. The heterogeneous storage controller according to claim 1, characterized in that, The hardware tensor page table walker is configured as a sparse page table with a parsed dimension-separated index structure; the exception reporting state machine is configured to generate an MSI-X interrupt message to notify the host operating system when a page fault or segmentation fault is detected, and return a specific error code to the requesting source to ensure that the hardware pipeline does not enter an infinite waiting state.

4. The heterogeneous storage controller according to claim 1, characterized in that, The Spatial Address Operation Unit (SAMU) is configured to perform address offset calculations based on a non-linear mapping table; for the indirect step mode, the affine computation array is configured to perform the following logical address translations: P A = B a s e + ∑ d = 0 n - 1 C o o r d d × S t r i d e _ T a b l e d C o o r d d in, For physical address, Physical base address For the first Dimensional coordinates The step-size lookup table is used; the affine computation array supports equivalent mathematical transformations of the above address translation logic, including but not limited to incremental accumulation, multi-stage pipeline reduction, or base address offset calculation based on tiling; the Stride LUT Cache is configured to include speculative prefetch logic, which, when the hardware detects coordinates... When a linear trend is observed, the indirect step size parameters for subsequent dimensions are automatically loaded from memory into the cache in advance.

5. The heterogeneous storage controller according to claim 1, characterized in that, The controller is configured to support address pass-through mode and work in conjunction with the host IOMMU; the controller is mapped as a physical aperture in the host physical address space; the controller is configured to register exemption rules with the host IOMMU through the driver during the system initialization phase, so that transactions issued by the host processor for the physical aperture can directly bypass host-level address translation and be directly passed to the controller for unique semantic-level address mapping.

6. The heterogeneous storage controller according to claim 1, characterized in that, The MD-TLB also includes an overlap detection and arbitration circuit; it is configured to detect in real time whether the range of a newly written tensor overlaps with the address of an existing valid entry during the tensor registration phase; if an overlap is detected, the hardware logic parses the entry or triggers a hardware error based on the preset longest prefix matching principle.

7. The heterogeneous storage controller according to claim 1, characterized in that, The consistency and protocol management engine also includes an alignment correction circuit; configured to automatically align any tensor boundary address of the software request down to the nearest cache line boundary or memory page boundary before sending an invalid range signal, and to perform alignment correction on the granularity of the protocol packet payload, adjusting the length field accordingly to ensure that consistency operations cover all relevant dirty data.

8. The heterogeneous storage controller according to claim 1, characterized in that, The controller also includes a double-buffered shadow register set, a pipeline drain circuit, and a security heartbeat watchdog; the shadow register set is used to preload the configuration for the next task; the drain circuit is configured to pause receiving new requests and wait for the pipeline to be cleared when a context switch instruction is received; the security heartbeat watchdog is configured to receive periodic heartbeat signals from the host driver, the signals containing a preset authentication sequence or a specifically encoded token; If no valid heartbeat is received within the preset **TTL**, the host system is determined to be abnormal, and the hardware automatically performs a Force Unpinning operation to release the locked physical memory resources.

9. A heterogeneous memory access method based on multidimensional address structure awareness, using the controller as described in any one of claims 1-8, characterized in that... include: (1). Memory lock registration steps: Receive configuration instructions, lock (Pin) the corresponding physical page frame through the host driver to prevent operating system migration, and establish a tensor mapping relationship containing the life cycle; (2). Hierarchical isolation addressing steps: hierarchically search and verify ASID and coordinate range in L1 / L2 MD-TLB; (3). Sparse acceleration calculation steps: Utilize the stride LUT Cache to speculatively prefetch the step size parameter and calculate the nonlinear physical address through a pipeline array; (4). Atomic switching and protection steps: Lock-free switching is achieved by using a pipelined drain mechanism and shadow registers, and the host status is monitored by a heartbeat watchdog to prevent resource deadlock.

10. A computing system comprising a host processor, an artificial intelligence accelerator, high-bandwidth memory (HBM), and a heterogeneous storage controller as claimed in claim 1; the system being configured to utilize the controller to manage a mapping from a multidimensional semantic space to a heterogeneous physical space.

11. A computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method of claim 9; or the storage medium containing hardware description language (HDL) code for configuring a programmable logic device (FPGA) or describing an application-specific integrated circuit (ASIC), the code being synthesized to generate the hardware logic of a heterogeneous memory controller as described in any one of claims 1-8.

Images