A storage control system and a storage control method

Abstract

The application discloses a storage control system and a storage control method, relates to the technical field of computers, and is characterized in that cold and hot data identification work in a computer fast interconnection memory heterogeneous system is executed in a storage controller of a storage device, the identification precision of data access hotness of a data unit in the storage controller is significantly improved, candidate data units are reported by the storage controller according to data access hotness, a source device and a target device are determined from the storage device according to information of the candidate data units reported by the storage controller, a data unit to be migrated in the source device is determined, the data unit to be migrated is migrated from the source device to the target device, and an address mapping table in an exchange controller between the storage device and a computing node is updated, so that a cold and hot data identification and data migration scheme without host awareness is realized, and the cold and hot data migration performance of the computer fast interconnection memory heterogeneous system is significantly improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a storage control system and storage control method. Background Technology

[0002] With the development of computer interconnect technology, memory expansion technology based on Compute Express Link (CXL) allows high-performance computing nodes to be flexibly connected to Compute Express Link (CXL) memory via CXL switches, thus enabling flexible memory connection methods. In heterogeneous CXL memory systems, storage devices with varying read / write performance exist. Therefore, it is necessary to rationally allocate matching storage media based on the frequency of data access to the storage devices to improve system performance and resource utilization. However, the accuracy of hot / cold data identification, which relies on the host operating system, is relatively low, and the performance improvement of heterogeneous CXL memory systems after controlling data migration is limited.

[0003] Improving the performance of cold and hot data migration in heterogeneous computer memory interconnect systems is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] This invention provides a storage control system and storage control method to at least solve the problem in related technologies where the accuracy of hot and cold data identification is low due to the reliance on the host operating system, resulting in poor hot and cold data migration performance.

[0005] This invention provides a storage control system, comprising: a management controller and a storage controller;

[0006] The storage controller is used to monitor the data access frequency of the storage device it is located in, and to determine candidate data units based on the data access frequency.

[0007] The management controller is used to determine the source device and target device from the storage device based on the information of the candidate data unit reported by the storage controller, and to determine the data unit to be migrated in the source device, control the migration of the data unit to be migrated from the source device to the target device, and update the address mapping table in the exchange controller between the storage device and the computing node.

[0008] The switching controller is a computer fast interconnect switching controller, and the storage device is a computer fast interconnect storage device; the address mapping table is a mapping table between the system physical address of the computing node and the device physical address of the storage device.

[0009] The present invention also provides a storage control method applied to a management controller, comprising:

[0010] Receive candidate data units from the storage controller that monitor the data access activity of the storage device it is located in and report them.

[0011] Based on the information of the candidate data units reported by the storage controller, the source device and the target device are determined from the storage devices, and the data units to be migrated in the source device are determined.

[0012] The system controls the migration of the data unit to be migrated from the source device to the target device.

[0013] Update the address mapping table in the switching controller between the storage device and the compute node;

[0014] Wherein, the switching controller is a computer fast interconnect switching controller, the storage device is a computer fast interconnect storage device; the address mapping table is a mapping table between the system physical address of the computing node and the device physical address of the storage device.

[0015] This invention, by offloading the identification of hot and cold data in a heterogeneous computer fast interconnect memory system to the storage controller of the storage device, significantly improves the accuracy of identifying the data access frequency of data units in the storage controller compared to traditional hot and cold data monitoring schemes based on the host operating system. The storage controller reports candidate data units based on the data access frequency, and the management controller determines the source and target devices from the storage device based on the information of the candidate data units reported by the storage controller, identifies the data units to be migrated in the source device, controls the migration of the data units to be migrated from the source device to the target device, and updates the address mapping table in the exchange controller between the storage device and the compute node. This achieves a host-insensitive hot and cold data identification and data migration scheme, and by improving the accuracy of hot and cold data identification, it significantly improves the performance of hot and cold data migration in a heterogeneous computer fast interconnect memory system. Attached Figure Description

[0016] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 An architecture diagram of a storage control system provided in an embodiment of the present invention;

[0018] Figure 2 A flowchart of a storage control method provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0020] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0021] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Here, we will first explain some key terms used in the embodiments of the present invention.

[0023] In traditional operating systems, cross-media data migration is typically scheduled by kernel threads, achieved through locking page tables, pausing access, copying data, and updating mappings. This mechanism has significant drawbacks: firstly, the host needs to participate in every page migration operation, increasing system complexity and scheduling overhead; secondly, to ensure data consistency, it often blocks upper-layer applications from accessing the target page, impacting response latency, especially in high-concurrency, high-real-time scenarios. This problem is particularly severe in high-concurrency, low-latency scenarios of rapidly interconnected computer systems. Therefore, there is an urgent need for a cross-media data migration mechanism that is transparent to host access, requires no modification to page tables, and does not trigger interruptions. Furthermore, consideration should be given to how to push control logic down to the device side, enabling asynchronous, automatic, and transparent data transfer through hardware.

[0024] To address the aforementioned issues, this invention provides a storage control system, method, storage medium, and computer program product. It offloads the hot / cold data identification process in heterogeneous computer fast interconnect memory systems to the storage controller of the storage device. Compared to traditional hot / cold data monitoring schemes based on the host operating system, this significantly improves the accuracy of identifying the data access frequency of data units within the storage controller. The storage controller reports candidate data units based on data access frequency. The management controller, based on the information of the candidate data units reported by the storage controller, determines the source and target devices from the storage device, identifies the data units to be migrated in the source device, controls the migration of the data units to be migrated from the source device to the target device, and updates the address mapping table in the exchange controller between the storage device and the compute node. This achieves a host-insensitive hot / cold data identification and migration scheme, and by improving the accuracy of hot / cold data identification, it significantly improves the hot / cold data migration performance of heterogeneous computer fast interconnect memory systems.

[0025] Figure 1 This is an architecture diagram of a storage control system provided in an embodiment of the present invention.

[0026] like Figure 1 As shown, the storage control system provided in this embodiment of the invention may include: a management controller and a storage controller; wherein, the storage controller is used to monitor the data access heat of the storage device it resides in, and determine candidate data units based on the data access heat; the management controller is used to determine the source device and the target device from the storage device based on the information of the candidate data units reported by the storage controller, and determine the data units to be migrated in the source device, control the migration of the data units to be migrated from the source device to the target device, and update the address mapping table in the switching controller between the storage device and the computing node; the switching controller is a computer fast interconnect switching controller, and the storage device is a computer fast interconnect storage device; the address mapping table is a mapping table between the system physical address of the computing node and the device physical address of the storage device.

[0027] Compute Express Link (CXL) is an open standard that supports cache coherency, high bandwidth, and low latency interconnection. It can be used to build system architectures that pool computing power, pool memory, and decouple resources. Through the Compute Express Link protocol, the system can connect remote Compute Express Link memory devices (CXL Type 3 memory devices) to the host system, enabling cross-node sharing and dynamic on-demand expansion of memory resources.

[0028] like Figure 1As shown, in a computer rapid interconnect system, the first port of the switch controller connects to the compute node, and the second port connects to the storage device. At the hardware level, the compute node and the switch controller, as well as the switch controller and the storage device, can be connected via Peripheral Component Interconnect Express (PCIe). At the protocol level, the compute node and the switch controller, as well as the switch controller and the storage device, are connected via the computer rapid interconnect protocol.

[0029] The number of compute nodes can be one or more. The types of compute nodes can include general-purpose compute nodes and accelerated compute nodes. General-purpose compute nodes are the host machines, with a Central Processing Unit (CPU) as their computing core. Accelerated compute nodes use accelerator cards as their computing core, which can include, but are not limited to, Graphics Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAs).

[0030] The number of storage devices can be one or more, configured as high-speed interconnect memory for the computer. The type of storage device can be one or more, including but not limited to Dynamic Random Access Memory (DRAM) and Solid State Drive (SSD). Because different types of storage media have significant differences in latency, bandwidth, cost, and energy efficiency—for example, DRAM has lower latency and higher bandwidth, suitable for hot data; while SSDs have larger capacities but higher latency, suitable for cold data—it is necessary to accurately identify cold and hot data in the memory media and allocate them accordingly to the matching memory media to ensure system performance and resource utilization.

[0031] The switching controller is used to forward data between compute nodes and storage devices, that is, to forward the memory access tasks of compute nodes to storage devices. The types of memory access tasks include read data tasks and write data tasks, and forward the data read from the storage device or the write completion information back to the compute node.

[0032] There can be one or more switching controllers. If there are multiple switching controllers, they can be fully interconnected, meaning that any two switching controllers are directly connected to each other. This ensures that the access path length is the same when any computing node in a heterogeneous memory system accesses any storage device.

[0033] The management controller provided in this embodiment of the invention can be implemented based on the management controller in a CXL switch. The management controller in the switch is a higher-level controller used to manage the switching controller and can be implemented using a microprocessor (mCPU). The management controller provided in this embodiment of the invention can also be implemented using another device besides those described above.

[0034] like Figure 1 As shown, the management controller can deploy a scheduling control module, a hybrid media-aware mapping module, a migration control module, and an address mapping controller (RemappingUnit).

[0035] The compute node issues a memory access task to the storage device based on the system's physical address. The switching controller queries its local address mapping table to determine the corresponding storage device and the physical address of the target device, thereby forwarding the memory access task to the appropriate storage device. During this process, the management controller obtains information about the memory access task and the address mapping table by accessing the switching controller, enabling it to identify hot and cold data units within the storage device and control data migration.

[0036] The access frequency of data units in the storage device is monitored by the data popularity identification module deployed on the storage controller of the storage device to identify candidate cold data and candidate hot data as candidate data units.

[0037] In this embodiment of the invention, the data unit used for monitoring can be a page, which is the basic unit for memory management by the operating system. The operating system divides the virtual memory space into fixed-size blocks, each called a page. Common page sizes include 4KB and 8KB. The system physical address of a page is the physical memory address used by the host operating system to maintain the page, i.e., the physical memory address seen by the host. Each page is allocated a fixed-size contiguous memory region in physical memory. Assuming the page size in the system is 4KB, physical memory allocation starts from address 0. Page 0: physical address range is 0 to 4095 (0x0000 to 0x0FFF), page 1: physical address range is 4096 to 8191 (0x1000 to 0x1FFF)... and so on. The device physical address is the actual address of the page in the storage device. In this embodiment of the invention, the data heat identification module can use the initial address and page size of the page as address information for monitoring the page.

[0038] The management controller deploys a scheduling control module to coordinate and manage memory data migration in heterogeneous memory systems with rapid computer interconnection. If the management controller is a management controller within a switch, the scheduling control module can be the Fabric Management (FM) module deployed within that management controller. If the management controller is a device other than a switch, after the Fabric Management module initializes the storage device, the scheduling control module interacts with the Fabric Management module to implement subsequent functions.

[0039] Based on the structured management module, the management controller performs unified scheduling and coordination management of compute nodes and storage devices of multiple storage media types connected to the computer fast interconnect memory heterogeneous system. Specifically, it can execute tasks such as storage device initialization, resource scheduling, data migration task instruction issuance, and migration status tracking, serving as the core of the control logic for memory data migration tasks. During the system initialization phase of the computer fast interconnect memory heterogeneous system, the structured management module performs discovery and registration operations on the storage devices connected to the system. This includes recording performance parameters such as capacity, bandwidth, and latency of each storage device, and establishing a system topology diagram that clarifies the connection relationships, routing paths, and load distribution between devices. The structured management module also supports hot-swapping capabilities and updates the memory pool topology and configuration table in real time, providing dynamic support for subsequent resource allocation and migration.

[0040] During system operation, the data heat identification module monitors the access heat of data units in the storage devices of the computer's rapidly interconnected memory heterogeneous system based on the system topology diagram.

[0041] The data heat identification module in the storage controller may include a hot data identification module and / or a cold data identification module. It is used to select candidate data units from the data units according to the pre-configured filtering conditions for selecting data units that need to be migrated, and report the candidate data units to the scheduling control module of the management controller so that the scheduling control module can determine the data units to be migrated from the candidate data units.

[0042] Then, the management controller determines the data unit to be migrated and its target device based on the information of the candidate data units reported by the storage controller.

[0043] Specifically, for candidate hot data reported by the data heat identification module, the scheduling control module can check the availability of storage devices with higher performance than the source device in the computer's fast interconnect memory heterogeneous system. If available, it selects a target device, allocates a target physical address, and coordinates the source and target devices to initiate migration. For candidate cold data reported by the data heat identification module, the scheduling control module can check the availability of storage devices with lower performance than the source device in the computer's fast interconnect memory heterogeneous system. If available, it selects a target device, allocates a target physical address, and coordinates the source and target devices to initiate migration. In other words, the scheduling control module performs pairing and access arbitration between the source address in the source device and the target address in the target device before data migration, monitors the migration process, ensures the migration is complete and error-free, and promptly cleans up old data in the source device.

[0044] In addition, the structured management module can also be used to maintain interface with system-level topology configuration interfaces (such as Advanced Configuration and Power Interface (ACPI), System Resource Affinity Table (SRAT), etc.). To support hot-swapping of devices, sharing among multiple hosts, or to optimize host scheduling strategies when the migration scale is large, the structured management module can selectively update the system description table (in addition to ACPI and SRAT, it can also include the Heterogeneous Memory Attribute Table (HMAT), Cache Hierarchy Discovery Table (CDAT), etc.) to help the host operating system perceive macro-level changes in topology or resource status, but this does not affect the access transparency of the individual page migration process.

[0045] The address mapping controller maintains the mapping relationship between the system physical address of the host system and the device physical address of the storage device, and works in conjunction with each CXL Type 3 memory device. In this embodiment of the invention, the address mapping controller needs to ensure address transparency during cross-media page migration, ensuring that the host's logical address (i.e., the system physical address) remains unchanged, while the actual physical location of the data unit (device physical address) is dynamically adjusted according to the system policy, thereby achieving data migration capability that is imperceptible to the application.

[0046] During system operation, an address mapping controller maintains an address mapping table between system physical addresses and device physical addresses. Each entry records the device physical address and media information corresponding to a system physical address. When a compute node initiates a memory access request, the request first reaches the switch. The address mapping controller then queries the mapping relationship of the target data unit in the memory access request, resolves the target storage device to be accessed and its physical address within the target storage device, and forwards the memory access request to the target storage device for execution. This process is completely transparent to the host (compute node), requiring no modification to page tables or virtual address mappings, ensuring no awareness at the operating system level.

[0047] During the data migration process, after the data unit to be migrated is successfully migrated to the target device, the address of the data unit to be migrated is redirected by the address mapping controller. The system physical address of the data unit to be migrated points to the device physical address of the source device, which is then updated to point to the device physical address of the target device.

[0048] To enhance resource awareness and cross-media scheduling efficiency during the hot and cold data migration process, this invention can use a hybrid media-aware decision module to coordinate with the scheduling control module and the address mapping controller to comprehensively evaluate the status parameters of each storage device in a heterogeneous computer memory interconnection system. By combining the access parameters of candidate data units, it can intelligently determine the most suitable target device for storing candidate data units, thereby achieving the dual goals of optimal performance and maximized resource utilization, and realizing an efficient memory migration strategy.

[0049] During the data migration process, the migration control module pre-deployed on the storage controller invokes the Direct Memory Access (DMA) channel between the source and target devices, enabling data migration from the source device to the target device without the host's awareness.

[0050] The storage control system provided in this invention decentralizes the identification of hot and cold data in a heterogeneous computer fast interconnect memory system to the storage controller of the storage device. Compared with traditional hot and cold data monitoring schemes based on the host operating system, this significantly improves the accuracy of identifying the data access heat of data units in the storage controller. The storage controller reports candidate data units based on the data access heat, and the management controller determines the source and target devices from the storage device based on the information of the candidate data units reported by the storage controller. It also determines the data units to be migrated in the source device, controls the migration of the data units to be migrated from the source device to the target device, and updates the address mapping table in the exchange controller between the storage device and the computing node. This achieves a host-insensitive hot and cold data identification and data migration scheme, and significantly improves the performance of hot and cold data migration in a heterogeneous computer fast interconnect memory system by improving the accuracy of hot and cold data identification.

[0051] Based on the above embodiments, the present invention further describes the storage controller in the storage control system.

[0052] In this embodiment of the invention, the storage device can identify hot and cold data stored locally by deploying a data heat identification module.

[0053] In some optional embodiments of the present invention, the storage device may include two types, corresponding to different read and write rates; wherein, the storage controller located in the storage device with a higher read and write rate is used to identify candidate cold data in the storage device, and the storage controller located in the storage device with a lower read and write rate is used to identify candidate hot data in the storage device; candidate cold data and candidate hot data are candidate data units.

[0054] Among them, storage devices with higher read and write speeds can be CXL-DRAM, while storage devices with lower read and write speeds can be CXL-SSD.

[0055] In other words, when monitoring cold data in storage devices with high-performance storage media, it is necessary to select data units with the lowest access frequency, thus requiring monitoring the access frequency of all data units in the storage device. When monitoring cold data in storage devices with low-performance storage media, since it is only necessary to monitor data units with higher access frequency, and it is not necessary to know the access frequency of all data units, it is sufficient to record the access frequency of the most frequently accessed data units.

[0056] In some optional embodiments of the present invention, the storage device may include three or more types, corresponding to different read and write rates; wherein, the storage controller located in the storage device with the highest read and write rate is used to identify candidate cold data in the storage device, the storage controller located in the storage device with the lowest read and write rate is used to identify candidate hot data in the storage device, and other types of storage devices are used to identify candidate cold data and candidate hot data in the storage device; candidate cold data and candidate hot data are candidate data units.

[0057] In this embodiment of the invention, the storage controller identifies candidate hot data in the storage device, which may include: sampling read and write operations from the computer fast interconnect link, storing the corresponding data unit address and the corresponding access frequency into a first record table; and scanning the first record table every first preset time interval, recording data units with access frequencies greater than the first access frequency as candidate hot data.

[0058] Specifically, a two-dimensional array table can be created in the storage controller to sample all remote read / write operations from the computer's fast interconnect link to the local machine. The address of the destination data unit is obtained and input into the two-dimensional array table as a key, and the corresponding access count in the table is incremented by 1. Every first preset time interval (which can be 10 seconds), the two-dimensional array table is scanned, and a threshold judgment is made on all count estimates. If the access frequency of a certain data unit exceeds the first access frequency, it is marked as candidate hot data, and the information of the candidate hot data (which can be the page number) is stored in a dedicated hot page candidate cache for subsequent use by the migration control module to call the direct memory access module.

[0059] In this embodiment of the invention, the storage controller identifies candidate cold data in the storage device, which may include: pre-deploying two bitmap registers, the bitmap registers recording the access status corresponding to the data address in the storage device; using one of the bitmap registers as the current cycle register, when a read / write operation is received from the computer fast interconnect link, setting the access status of the corresponding data unit address in the current cycle register to the first access status; after a second preset time interval, switching to the other bitmap register as the current cycle register, and comparing the access status in the two bitmap registers, if there is a data unit address with no access record for two consecutive cycles, it is determined to be candidate cold data.

[0060] Specifically, two bitmap registers (Bitmap_A and Bitmap_B) can be created in the storage controller, each corresponding to the total number N of data units in the storage device. One bitmap register serves as the current cycle register. Each time a memory access task is received, the bit corresponding to the target data unit address is set to 1, and other unaccessed bits are set to 0. The bitmap registers are swapped every second preset time interval, with the other bitmap register becoming the current cycle register. By comparing the access records of corresponding bits in the two bitmaps, it is determined whether cold data (e.g., data units without access records for two consecutive cycles) exists. Information about candidate cold data (which can be page numbers) is stored in a dedicated cold page candidate cache for later use by the migration control module when calling the direct memory access module.

[0061] The embodiments of the present invention employ different hot and cold data monitoring strategies for storage devices with different read and write speeds, which is beneficial for targeted optimization of hot and cold data identification in different storage devices with different read and write speeds.

[0062] In this embodiment of the invention, the management controller can also send information about candidate hot data to the switching controller, so that the switching controller can cache the mapping information between the system physical address and the device physical address corresponding to the candidate hot data. This can improve the table lookup latency of the switching controller and enhance its performance in performing memory access tasks.

[0063] The embodiments of the present invention further describe the method for determining the data unit to be migrated.

[0064] In some optional embodiments of the present invention, the management controller determines the source device and the target device from the storage devices based on the candidate data unit information reported by the storage controller, and determines the data unit to be migrated in the source device. This may include: if a candidate data unit exists, detecting the status information of the storage device; for candidate hot data in the candidate data unit, if a first storage device that meets the first load condition is detected, determining the storage device where the candidate hot data is located as the source device and the first storage device as the target device; for candidate cold data in the candidate data unit, if a second storage device that meets the second load condition is detected, determining the storage device where the candidate cold data is located as the source device and the second storage device as the target device; wherein, the read / write rate of the first storage device is higher than that of the second storage device.

[0065] In some optional embodiments of the present invention, the management controller is used to determine the source device and the target device from the storage device based on the information of the candidate data units reported by the storage controller, and to determine the data units to be migrated in the source device. This may include: calculating a performance score after storing the candidate data units in the storage device based on the access parameters of the candidate data units and the status parameters of the storage device; and determining the data units to be migrated and the target device based on the performance score.

[0066] Specifically, calculating the performance score after storing the candidate data unit in the storage device based on the access parameters of the candidate data unit and the status parameters of the storage device may include: determining a first matching degree parameter between the candidate data unit and the storage device based on the access parameters of the candidate data unit and the performance attribute parameters of the storage device; and calculating the performance score after storing the candidate data unit in the storage device based on the first matching degree parameter and the status parameters of the storage device.

[0067] In specific implementation, the first matching degree parameter between the candidate data unit and the storage device is determined based on the access parameters of the candidate data unit and the performance attribute parameters of the storage device. This may include: pre-establishing a first correspondence table between the access parameters of the data unit and the performance attribute parameters of the storage device; and querying the first correspondence table based on the access parameters of the candidate data unit and the performance attribute parameters of the storage device to determine the first matching degree parameter.

[0068] The access parameters for candidate data units can include at least one of the following: access frequency, read / write ratio, sequential parameters, and random parameters. Specifically, one or more types of access parameters can be set into multiple levels according to a range. For a type of storage device, each level can correspond to a matching degree parameter. For example, high-frequency random access is more suitable for DRAM, while sequential read / write is more suitable for SSD. The higher the matching degree parameter, the more suitable the data unit is for storage in that type of storage medium.

[0069] Based on the first matching degree parameter and the state parameters of the storage device, the performance score after storing the candidate data units in the storage device can be calculated, which may include: obtaining the first weight corresponding to the first matching degree parameter and the second weight corresponding to the state parameters of the storage device; and using the first weight and the second weight to perform a weighted summation calculation on the first matching degree parameter and the state parameters of the storage device to obtain the performance score.

[0070] The status parameters of the storage device include at least one of the following: bandwidth utilization of the storage device, average access latency of the storage device, and remaining capacity of the storage device.

[0071] In other words, by collecting the access parameters of candidate data units and the status parameters of each storage device during the current scheduling cycle, and substituting them into a preset performance scoring model, the performance score of the candidate data units after storage on the storage device can be obtained. This performance scoring model can be expressed by the following formula:

[0072] ;

[0073] in, For the first The candidate data unit is stored in the first... Performance ratings after each storage device; For the first The candidate data unit and the first Matching parameters for each storage device The weights for the matching degree parameter; For the first The bandwidth evaluation parameter of the first storage device can be used to represent the bandwidth evaluation parameter of the second storage device. The bandwidth usage of the storage device reflects the bandwidth usage of the first storage device. The current bandwidth availability of each storage device The weights for bandwidth evaluation parameters; For the first Average access latency per storage device Weights for average access latency; For the first Remaining available capacity of each storage device The weight of the remaining available capacity.

[0074] In the above performance scoring model, That is, the first weight. , , This is the second weight.

[0075] The aforementioned performance scoring model can be implemented based on the software module of the management controller. The memory data migration method provided in this embodiment of the invention may further include: obtaining system operating status parameters of the host computer rapid interconnect system; and updating the first weight and the second weight based on the system operating status parameters. That is, the first weight and the second weight can be dynamically adjusted and adaptively updated based on system operating feedback.

[0076] The performance scoring model described above can also be embedded as hardware logic, meaning it can be implemented using the logic circuitry of a programmable controller. Programmable controllers can be, but are not limited to, field-programmable gate arrays (FPGAs) and complex programmable logic devices (CPLDs).

[0077] In this embodiment of the invention, determining the data unit to be migrated and the target device based on the performance score may include: determining the first migration quantity of the data unit to be migrated corresponding to the current scheduling cycle; sorting the data units in descending order of performance score, determining the combination of the candidate data units of the first migration quantity and the source devices of the corresponding storage devices that are not candidate data units, and using the candidate data units as the data unit to be migrated in the current scheduling cycle, and using the corresponding storage devices as the target devices.

[0078] Traditional data migration schemes, which directly migrate cold data to low-performance storage media and hot data to high-performance storage media, are prone to resource mismatch. However, this invention provides a method for determining the data unit to be migrated and its corresponding target device. By considering the compatibility between the data unit's access characteristics and media attributes, it constructs an access-aware, storage media modeling, and learning-based scoring mechanism, ensuring that the data unit falls into its most suitable storage medium. This improves performance while further reducing overall memory costs. Furthermore, the hybrid media-aware decision module provided in this invention has good scalability and evolution capabilities. With the diversification of memory media (such as the introduction of new types of memories like NVDIMM, ReRAM, and PCM), the performance scoring model only needs to expand the media attribute dimensions and matching rules to support scheduling decisions for new types of devices. The entire scheduling strategy can be embedded in hardware logic or implemented through hardware-software collaboration, making it suitable for various deployment modes and computing platforms.

[0079] The embodiments of the present invention further describe the process of controlling memory data migration.

[0080] In this embodiment of the invention, the management controller controls the migration of the data unit to be migrated from the source device to the target device and updates the address mapping table in the switching controller between the storage device and the compute node. This may include: the management controller controlling the switching controller to freeze the mapping table entries of the system physical address corresponding to the data unit to be migrated, and then controlling the migration of the data unit to be migrated from the source device to the target device; after the migration is completed, the switching controller controls the switching controller to update the device physical address corresponding to the system physical address of the data unit to be migrated in the address mapping table to the device physical address in the target device.

[0081] In this embodiment of the invention, updating the address mapping table in the switching controller between the storage device and the computing node by the management controller may include: sending a migration completion message to the switching controller for each migration of a data unit to be migrated, so that the switching controller performs an atomic operation to update the address mapping table, thereby updating the device physical address corresponding to the system physical address of the data unit to be migrated in the address mapping table to the device physical address in the target device.

[0082] Specifically, after the scheduling control module issues a migration instruction, the address mapping controller first freezes the mapping table entry for the corresponding system physical address to prevent new access requests from entering the page being migrated; then, after the direct memory access module completes the data transfer, it immediately performs an atomic-level system physical address mapping update operation, pointing the system physical address originally mapped to the device physical address of the source device to the new device physical address in the target device.

[0083] In its implementation, the migration control module first collaborates with the local direct memory access engine to initialize the source address (DPA_src) and target address (DPA_dst), and then initiates the data transfer process. The direct memory access channel, using page granularity (typically 4KB) or larger block granularity (such as 2MB big pages), writes the contents of the data units to be migrated from the source device to the target device via the computer's Fast Interconnect Memory (CXL.mem) path, achieving end-to-end data transfer.

[0084] Because the entire migration process is asynchronous within the device, and host access requests may continue to occur before direct memory access is completed, it is necessary to address the issues of "read / write conflicts during migration" and "data consistency." To address this, this embodiment of the invention employs a lightweight dual-path buffering mechanism. Its basic principle is to maintain a pair of mirrored buffer structures between the source and destination addresses. During the data migration process involving direct memory access, the source device still acts as the server for host access requests, and all read / write requests are responded to normally.

[0085] In some optional embodiments of the present invention, the management controller controls the migration of the data unit to be migrated from the source device to the target device, which may include: if the management controller detects that the switching controller is executing a write request for the data unit to be migrated from the computing node during the process of the source device reading the data unit to be migrated and writing it to the target address of the target device, the management controller caches the data of the write request in the target buffer of the target device at the same time as the switching controller executes the write request, so that after the migration of the data unit to be migrated is completed, the target device updates the data at the target address according to the data in the target buffer.

[0086] In some optional embodiments of the present invention, the management controller controls the migration of the data unit to be migrated from the source device to the target device, which may include: during the process of the source device reading the data to be migrated and writing it to the target address of the target device, if the management controller detects that the switching controller executes a write request for the data unit to be migrated by the computing node, the management controller records the write request information in the write replay log of the target device, so that the target device updates the data at the target address according to the write replay log after the migration of the data unit to be migrated is completed.

[0087] In other words, the page data seen by the host can always remain consistent by either synchronously writing the data units to be migrated from the source device to the target buffer of the target device or rewriting them into the target page after migration. Once the entire data transfer process is complete, the migration control module triggers a consistency check to confirm that the target page has been completely overwritten and is consistent with the source page state. At this point, the module sends a mapping switch request to the address mapping controller in the management controller, which then performs an atomic update of the SPA system physical address to the device physical address. The host is completely unaware of this process; the update of the access path is automatically redirected to the new address of the target device by the address mapping controller or the address mapping table, without causing any application interruption.

[0088] In this embodiment of the invention, the management controller controls the migration of data units to be migrated from the source device to the target device, which may include: calling the direct memory access module of the source device and the direct memory access module of the target memory to write the data units to be migrated from the source device to the target device.

[0089] The embodiments of the present invention provide a storage control method. The method is described in detail below in conjunction with the execution flow of the storage control method.

[0090] Figure 2 A flowchart of a storage control method provided in an embodiment of the present invention.

[0091] like Figure 2 As shown, the storage control method provided in this embodiment of the invention, applied to the management controller, may include: S201: receiving candidate data units reported by the storage controller based on the data access popularity of the storage device it is monitoring.

[0092] S202: Based on the candidate data unit information reported by the storage controller, determine the source device and target device from the storage devices, and determine the data unit to be migrated in the source device.

[0093] S203: Controls the migration of data units to be migrated from the source device to the target device.

[0094] S204: Update the address mapping table in the switching controller between the storage device and the compute node.

[0095] Among them, the switching controller is a computer fast interconnect switching controller, the storage device is a computer fast interconnect storage device, and the address mapping table is a mapping table between the system physical address of the computing node and the device physical address of the storage device.

[0096] This invention also provides a memory data migration method, on which the computer rapid interconnect system can be deployed in a CXL switching architecture rack with multi-host capability, including the following key hardware units: at least one computing node capable of running a general x86 architecture server operating system; a management controller deploying the scheduling control module, hybrid media-aware decision module, and address mapping controller provided by this invention, which can be based on a management controller implementation in the switch; at least two CXL Type 3 memory expansion devices, such as a high-performance CXL-DRAM and a large-capacity CXL-SSD respectively; and a storage controller for each storage device deploying a migration control module and a data heat identification module, which can be a hot data identification module and / or a cold data identification module.

[0097] The implementation steps may include: Step 1: System initialization and topology awareness.

[0098] During the system power-on startup phase, the structured management module in the management controller starts first, identifying and initializing all connected compute nodes and storage devices that serve as CXL Type 3 memory. This includes: enumerating the CXL Type 3 devices present in the system; identifying their device type (DRAM or SSD), capacity, bandwidth, latency, and other key performance indicators; constructing a system topology map and registering resource tables, identifying inter-device link paths, NUMA attributes, etc.; initializing the mapping table structure of the address mapping controller, establishing the initial system physical address → device physical address mapping relationship; and interfacing with the host operating system's ACPI / SRAT and other topology management tables to complete system awareness.

[0099] Step 2: Collect page access behavior and identify hot and cold pages.

[0100] During system operation, compute nodes continuously send read and write requests to the CXL memory pool, and the storage controller of each storage device deploys a data heat identification module to monitor access behavior.

[0101] Specifically, a hot data identification module can be deployed on the CXL-SSD. This module continuously samples read and write requests from the host, counts the access frequency of page addresses, and performs scans at fixed intervals (e.g., every 10ms) to filter out data units whose access frequency exceeds a first access frequency and mark them as candidate hot data.

[0102] A cold data identification module can be deployed on CXL-DRAM devices. The module periodically (e.g., every 20ms) rotates the active bitmap and compares data units that have not been accessed during two cycles to identify candidate cold data.

[0103] The data heat identification module reports the address information of candidate data units to the scheduling control module for scheduling decisions.

[0104] Step 3: Scheduling decision and target medium evaluation.

[0105] After receiving information about candidate data units, the scheduling and control module invokes the hybrid media awareness module to select target devices. The hybrid media awareness module collects access parameters (such as frequency, read / write ratio, and randomness) of the candidate data units and evaluates the status parameters of all storage devices in the system (such as remaining capacity, bandwidth utilization, and average access latency). It then uses a performance scoring model to calculate the performance score of the candidate data units on each storage device, selects the optimal combination of candidate data units and their corresponding storage devices as the data units to be migrated and their corresponding target devices, and returns the address information of the data units to be migrated and the information of the target devices (which can be device numbers).

[0106] Step 4: Migration preparation and address mapping freeze.

[0107] The scheduling and control module generates migration instructions based on the address information of the data unit to be migrated and the information of the target device. These instructions may include the physical address of the source device, the physical address of the target device, and the corresponding system physical address. The migration instructions are then sent to both the source and target devices to initiate the direct memory access data transfer process. Furthermore, the host's access to the data unit to be migrated still points to the physical address of the source device, and the source device will respond accordingly.

[0108] Step 5: Page data transfer and consistency assurance.

[0109] The migration control modules within both the source and target devices initiate the Direct Memory Access engine, copying the content of the data units to be migrated from the source device to the target device at a granularity of data units (which can be pages) along the computer's high-speed interconnect link. To ensure data consistency during the migration process, the following mechanisms can be employed: During migration, the source device continues to respond to host requests; for write operations, a double-shot caching or write log mechanism is used to synchronously update the data to the target buffer; after the migration is complete, a consistency check is performed to ensure that the content of the target page is consistent with the source page.

[0110] Step 6: Address mapping switch and resource update.

[0111] Once data migration is complete and target address consistency verification passes, the migration control module sends a "mapping switch request" to the address mapping controller. The address mapping controller then performs the following operations: atomically updates the system physical address → device physical address mapping entry, setting the target address as the new mapping address; updates the corresponding entry for the device physical address in the address mapping controller for subsequent quick table lookups; and notifies the scheduling control module to release the resources of the source address and reclaim the old device physical address. This switch is completely transparent to the host operating system, the access path is automatically updated, and no page table modification or service interruption is required.

[0112] Step 7: Migration complete and status cleanup.

[0113] After the migration is completed, the scheduling and control module updates the system topology resource table and equipment load status. The system can then continue to perform a new round of hot and cold data identification and migration scheduling, forming a continuously dynamically optimized resource management cycle.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0115] Embodiments of the present invention also provide a storage control device, which may include: a scheduling control module, configured to receive candidate data units reported by the storage controller based on the data access frequency of the storage device it is monitoring; a hybrid media-aware decision module, configured to determine a source device and a target device from the storage device based on the candidate data unit information reported by the storage controller, and to determine the data units to be migrated in the source device; a migration control module, configured to control the migration of the data units to be migrated from the source device to the target device; and an address mapping controller, configured to update the address mapping table in the switch controller between the storage device and the compute node. The switch controller is a Computer Fast Interconnect Switch Controller, the storage device is a Computer Fast Interconnect Storage Device, and the address mapping table is a mapping table between the system physical address of the compute node and the device physical address of the storage device.

[0116] For a description of the features in the embodiment corresponding to the storage control device, please refer to the relevant description in the embodiment corresponding to the storage control method, which will not be repeated here.

[0117] Embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described storage control method embodiments.

[0118] Embodiments of the present invention also provide a non-volatile storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described storage control method embodiments when running.

[0119] In one exemplary embodiment, the aforementioned non-volatile storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0120] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described storage control method embodiments.

[0121] Embodiments of the present invention also provide another computer program product, including a non-volatile storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described storage control method embodiments.

[0122] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0123] The present invention has provided a detailed description of a storage control system and storage control method. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of these embodiments are only intended to aid in understanding the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A storage control system, characterized in that, include: Management controller and storage controller; The storage controller is used to monitor the data access frequency of the storage device it is located in, and to determine candidate data units based on the data access frequency. The management controller is used to determine the source device and target device from the storage device based on the information of the candidate data units reported by the storage controller, and to determine the data units to be migrated in the source device. After controlling the exchange controller between the storage device and the compute node to freeze the mapping table entry of the system physical address corresponding to the data unit to be migrated, the management controller controls the migration control module pre-deployed in the storage controller to call the direct memory access channel between the source device and the target device to migrate the data unit to be migrated from the source device to the target device without the host's awareness. After each migration of the data unit to be migrated is performed, the corresponding migration completion message is sent to the exchange controller so that the exchange controller performs an atomic operation to update the address mapping table in the exchange controller, so as to update the device physical address corresponding to the system physical address of the data unit to be migrated in the address mapping table to the device physical address in the target device. The switching controller is a computer fast interconnect switching controller, and the storage device is a computer fast interconnect storage device; the address mapping table is a mapping table between the system physical address of the computing node and the device physical address of the storage device.

2. The storage control system according to claim 1, characterized in that, The storage devices include two types, corresponding to different read and write speeds; The storage controller located in the storage device with a high read / write speed is used to identify candidate cold data in the storage device, and the storage controller located in the storage device with a low read / write speed is used to identify candidate hot data in the storage device. The candidate cold data and the candidate hot data are the candidate data units.

3. The storage control system according to claim 1, characterized in that, The storage devices include three or more types, corresponding to different read and write speeds; Among them, the storage controller located in the storage device with the highest read / write speed is used to identify candidate cold data in the storage device, the storage controller located in the storage device with the lowest read / write speed is used to identify candidate hot data in the storage device, and other types of storage devices are used to identify the candidate cold data and the candidate hot data in their respective storage devices; The candidate cold data and the candidate hot data are the candidate data units.

4. The storage control system according to claim 2 or 3, characterized in that, The storage controller identifies the candidate hot data in the storage device, including: Sample read and write operations from the computer's fast interconnect link, and store the corresponding data unit address and the corresponding access frequency into the first record table; Every first preset time interval, the first record table is scanned, and data units with an access frequency greater than the first access frequency are recorded as candidate hot data.

5. The storage control system according to claim 2 or 3, characterized in that, The storage controller identifies the candidate cold data in the storage device, including: Two bitmap registers are pre-deployed, and the bitmap registers record the access status corresponding to the data addresses in the storage device; Using one of the bitmap registers as the current cycle register, when a read / write operation is received from the computer fast interconnect link, the access state of the corresponding data unit address in the current cycle register is set to the first access state; After a second preset time interval, the data is switched to another bitmap register as the current cycle register, and the access status of the two bitmap registers is compared. If there is a data unit address that has no access record for two consecutive cycles, it is determined as the candidate cold data.

6. The storage control system according to claim 1, characterized in that, The management controller, based on the information of the candidate data units reported by the storage controller, determines the source device and the target device from the storage devices, and determines the data units to be migrated in the source device, including: If the candidate data unit exists, then the status information of the storage device is detected; For candidate hot data in the candidate data unit, if a first storage device that meets the first load condition is detected, then the storage device where the candidate hot data is located is determined to be the source device, and the first storage device is the target device; For candidate cold data in the candidate data unit, if a second storage device that meets the second load condition is detected, then the storage device where the candidate cold data is located is determined to be the source device, and the second storage device is the target device; The read / write speed of the first storage device is higher than that of the second storage device.

7. The storage control system according to claim 1, characterized in that, The management controller controls the migration of the data units to be migrated from the source device to the target device, including: During the process of reading the data unit to be migrated from the source device and writing it to the target address of the target device, if the management controller detects that the switching controller is executing a write request for the data unit to be migrated from the computing node, then while the switching controller is executing the write request, the data of the write request is cached in the target buffer of the target device, so that after the migration of the data unit to be migrated is completed, the target device updates the data of the target address according to the data in the target buffer.

8. The storage control system according to claim 1, characterized in that, The management controller controls the migration of the data units to be migrated from the source device to the target device, including: During the process of reading the data to be migrated from the source device and writing it to the target address of the target device, if the management controller detects that the switching controller executes a write request for the data unit to be migrated by the computing node, it records the write request information in the write replay log of the target device, so that the target device updates the data at the target address according to the write replay log after the migration of the data unit to be migrated is completed.

9. A storage control method, characterized in that, Applied to management controllers, including: Receive candidate data units from the storage controller that monitor the data access activity of the storage device it is located in and report them. Based on the information of the candidate data units reported by the storage controller, the source device and the target device are determined from the storage devices, and the data units to be migrated in the source device are determined. After the switching controller between the storage device and the computing node freezes the mapping table entry of the system physical address corresponding to the data unit to be migrated, the migration control module pre-deployed in the storage controller calls the direct memory access channel between the source device and the target device to control the migration of the data unit to be migrated from the source device to the target device without the host's awareness. For each migration of the data unit to be migrated, a migration completion message is sent to the switching controller, so that the switching controller performs an atomic operation to update the address mapping table in the switching controller, so as to update the device physical address corresponding to the system physical address of the data unit to be migrated in the address mapping table to the device physical address in the target device. Wherein, the switching controller is a computer fast interconnect switching controller, the storage device is a computer fast interconnect storage device; the address mapping table is a mapping table between the system physical address of the computing node and the device physical address of the storage device.

Images