Data storage system and electronic device

Abstract

The application discloses a data storage system and electronic equipment, and relates to the technical field of data processing, and at least one acceleration hard disk containing multiple ports is arranged, a hard disk controller of the acceleration hard disk is used to establish a direct link with a processor of at least one node based on a computing express link (CXL) protocol and the multiple ports, shared access of the acceleration hard disk by multiple nodes is realized, and communication delay between the node and the acceleration hard disk is shortened based on the characteristics of low delay and high bandwidth of the CXL protocol; meanwhile, the processor of the node executes read and write operations on any acceleration hard disk through the direct link, data synchronization between multiple nodes is ensured, and on the other hand, data is actively migrated between different storage levels based on data access frequency, performance difference of storage media and capacity state of each storage level, so that the storage efficiency of the data storage system is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more particularly to a data storage system and electronic device. Background Technology

[0002] With the rapid development of technologies such as big data, cloud computing, and artificial intelligence, the scale of data is growing exponentially, placing higher demands on the performance, response speed, and data sharing capabilities of data storage systems. Such data storage systems typically consist of multiple nodes and solid-state drives (SSDs). Nodes interact and access data through processors, internal memory, and SSDs. During data processing, the processor prioritizes data reading and writing through the node's memory to ensure low latency. When the data volume exceeds the memory capacity or long-term storage is required, the data is migrated to the SSD.

[0003] Communication between nodes and access to solid-state drives (SSDs) are all done through traditional buses or network protocols. Since SSDs are external storage devices, they have high latency. Furthermore, data synchronization between multiple nodes via traditional buses or network protocols results in significant transmission latency and overhead, leading to low storage efficiency in the data storage system. Summary of the Invention

[0004] This application provides a data storage system and an electronic device to at least solve the problem of low storage efficiency in data storage systems in the related art.

[0005] This application provides a data storage system, which includes at least one node, at least one accelerated hard disk, and at least one solid-state drive, wherein...

[0006] The node includes node memory, processor and switch. The switch is connected to at least one accelerated hard disk and at least one solid-state hard disk. The accelerated hard disk includes a hard disk controller, multiple memory modules and multiple ports.

[0007] The hard disk controller is used to establish direct links with the processor of at least one node via the Fast Link (CXL) protocol and multiple ports, so that at least one node can have shared access to the accelerated hard disk.

[0008] The processor is used to perform read and write operations on any accelerated hard drive via a direct link to achieve data synchronization between at least one node;

[0009] The processor is also used to migrate data in at least one accelerated hard disk, at least one solid-state drive, and in the node memory of at least one node.

[0010] This application also provides an electronic device including the aforementioned data storage system.

[0011] This application establishes at least one accelerated hard drive with multiple ports. The accelerated hard drive's controller, based on the Compute Fast Link (CXL) protocol and its multiple ports, establishes direct links with the processors of at least one node. This enables shared access to the accelerated hard drive by multiple nodes and, leveraging the low latency and high bandwidth of the CXL protocol, reduces communication latency between nodes and the accelerated hard drive. Simultaneously, the node's processor performs read and write operations on any accelerated hard drive through these direct links, ensuring data synchronization among multiple nodes. Furthermore, based on data access frequency, storage medium performance differences, and the capacity status of each storage tier, it actively migrates data between different storage tiers. This adapts frequently accessed hot data to the high-speed storage capabilities of the node's memory or the accelerated hard drive, while storing infrequently accessed cold data in solid-state drives. Ultimately, this constructs a low-latency, highly shared, and self-adaptive data storage architecture, improving the storage efficiency of the data storage system. Attached Figure Description

[0012] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This application provides a schematic diagram of the system architecture of a related data storage system as an embodiment of the present application.

[0014] Figure 2 This application provides a schematic diagram of the system architecture of a data storage system according to an embodiment of the present application.

[0015] Figure 3 This is a schematic diagram of the structure of an accelerated hard disk provided in an embodiment of this application;

[0016] Figure 4 This application provides a schematic diagram of the connection relationship of a data accelerator.

[0017] Figure 5 A flowchart illustrating a data migration method provided in an embodiment of this application;

[0018] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0020] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0021] With the rapid development of technologies such as big data, cloud computing, and artificial intelligence, the scale of data is growing exponentially, placing higher demands on the performance, response speed, and data sharing capabilities of data storage systems. Such data storage systems typically consist of multiple nodes and solid-state drives (SSDs). Nodes interact and access data through processors, internal memory, and SSDs. During data processing, the processor prioritizes data reading and writing through the node's memory to ensure low latency. When the data volume exceeds the memory capacity or long-term storage is required, the data is migrated to the SSD.

[0022] Below, in conjunction with Figure 1 The system architecture of the relevant data storage system is illustrated with examples.

[0023] Figure 1 This is a schematic diagram of the system architecture of a related data storage system provided in an embodiment of this application. Please refer to... Figure 1 , Figure 1 It can include a data storage system.

[0024] A data storage system can include multiple nodes and multiple storage devices. Nodes can include processors, local memory, cache synchronization modules, and network interfaces. Storage devices can include multiple solid-state drives (SSDs) and multiple hard disk drives (HDDs). Nodes can handle computing tasks and cache synchronization, while storage devices can provide tiered storage capabilities, forming a collaborative and efficient distributed storage system.

[0025] Nodes can perform computational tasks through processors, prioritize the use of local memory to accelerate data access, and only send requests to external storage through the network interface when the data is not in local memory.

[0026] The cache synchronization module in a node can be used to determine whether the cached data of this node is consistent with that of other nodes, thus avoiding errors caused by data inconsistency (such as duplicate writing or reading old data).

[0027] Cache synchronization between nodes can be achieved through direct synchronization via a non-transparent bridge or through synchronization based on remote direct memory access.

[0028] For example, if node A modifies some data, the cache synchronization module will immediately notify other nodes that "the data has expired", forcing other nodes to read the latest data from the memory again.

[0029] A network interface can be used to enable data transmission between nodes and memory; it is the physical interface through which nodes access a distributed network.

[0030] Communication between nodes and access to solid-state drives (SSDs) are all done through traditional buses or network protocols. Since SSDs are external storage devices, they have high latency. Furthermore, data synchronization between multiple nodes via traditional buses or network protocols results in significant transmission latency and overhead, leading to low storage efficiency in the data storage system.

[0031] To address the aforementioned technical problems, this application provides a data storage system. By configuring at least one accelerated hard drive with multiple ports, and utilizing the accelerated hard drive's controller based on the Compute Fast Link (CXL) protocol and its multiple ports, direct links are established with the processors of at least one node. This enables shared access to the accelerated hard drive by multiple nodes and, leveraging the low latency and high bandwidth of the CXL protocol, reduces communication latency between nodes and the accelerated hard drive. Simultaneously, the node's processor performs read and write operations on any accelerated hard drive through these direct links, ensuring data synchronization among multiple nodes. Furthermore, based on data access frequency, storage medium performance differences, and the capacity status of each storage level, it actively performs dynamic migration of data between different storage levels. This adapts frequently accessed hot data to the high-speed storage capabilities of the node's memory or the accelerated hard drive, while storing infrequently accessed cold data in solid-state drives. Ultimately, this constructs a low-latency, highly shared, and self-adaptive data storage architecture, improving the storage efficiency of the data storage system.

[0032] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] Below, in conjunction with Figure 2 The data storage system described in this application will be explained.

[0034] Figure 2 This is a schematic diagram of the system architecture of a data storage system provided in an embodiment of this application. Please refer to [link / reference]. Figure 2 , Figure 2It can include a data storage system.

[0035] The data storage system includes at least one node, at least one accelerated hard disk, and at least one solid-state drive, wherein,

[0036] The node includes node memory, processor and switch. The switch is connected to at least one accelerated hard disk and at least one solid-state hard disk. The accelerated hard disk includes a hard disk controller, multiple memory modules and multiple ports.

[0037] The hard disk controller is used to establish direct links with the processor of at least one node via the Fast Link (CXL) protocol and multiple ports, so that at least one node can have shared access to the accelerated hard disk.

[0038] The processor is used to perform read and write operations on any accelerated hard drive via a direct link to achieve data synchronization between at least one node;

[0039] The processor is also used to migrate data in at least one accelerated hard disk, at least one solid-state drive, and in the node memory of at least one node.

[0040] Nodes can be considered the smallest unit for data processing and scheduling.

[0041] Node memory can use high-speed storage media such as Double Data Rate 5 Synchronous Dynamic Random Access Memory (DDR5).

[0042] Node memory can be used to temporarily store frequently accessed data during processor computation, i.e., hot data.

[0043] The processor can be used to execute data read and write instructions, schedule data migration strategies, and coordinate data synchronization between nodes.

[0044] Switches can be used to establish communication links between nodes and acceleration hard drives, and between nodes and solid-state drives, supporting the transmission of data between different storage devices.

[0045] Accelerated hard drives can be used to store data that is accessed at the next highest frequency.

[0046] The memory module can use high-speed storage media with performance close to that of node memory, with access latency between node memory and solid-state drives, and provides large-capacity high-speed caching capabilities.

[0047] The hard disk controller can support the Compute Express Link (CXL) protocol, which can establish direct links with the processor of at least one node through multiple ports, enabling multiple nodes to access the accelerated hard disk in parallel.

[0048] The port can use CXL 2.0 or higher interface, with a single port bandwidth of up to 32GB / s, meeting the requirements for high-throughput data transmission.

[0049] Below, in conjunction with Figure 3 The structure of an accelerated hard drive will be illustrated with an example.

[0050] Figure 3 This is a schematic diagram of an accelerated hard disk provided in an embodiment of this application. Please refer to... Figure 3 , Figure 3 This can include accelerating the hard drive.

[0051] In a two-node data storage system, the accelerated hard drive may include a hard drive controller, multiple memory modules, and two ports. The two ports establish direct links with the two nodes respectively to enable parallel shared access to the accelerated hard drive by multiple nodes.

[0052] This results in faster hard drives with better memory consistency and lower latency, which is crucial for certain artificial intelligence or high-performance computing scenarios.

[0053] Solid-state drives (SSDs) can be used as persistent storage devices to store infrequently accessed data, i.e., cold data. They employ the Non-Volatile Memory Host Controller Interface Specification (NVMe) protocol and the fourth-generation PCI-Express bus technology (Peripheral Component Interconnect Express, PCIe 4.0) interface, providing storage capacities ranging from TB to PB, with access latency of 20-100 microseconds.

[0054] The accelerated hard drive's controller establishes a point-to-point direct link with the processors of each node via the CXL protocol and multiple ports. The processor can directly address the accelerated hard drive's memory modules without going through a switch, achieving low-latency read and write operations to the accelerated hard drive, reducing latency to the microsecond level. At the same time, multiple nodes access the accelerated hard drive in parallel through their respective direct links, avoiding the queuing problem in traditional shared storage.

[0055] When multiple nodes need to share data, the processor of any node writes the data to the memory module of the accelerated hard disk through the CXL direct link. The processors of other nodes can directly read the data through their own direct links without additional protocol conversion or intermediate forwarding, thus achieving efficient data synchronization between nodes.

[0056] The node's processor dynamically performs data migration based on data access frequency, storage media performance, and capacity status at each level:

[0057] When data access frequency increases, the processor migrates it from the solid-state drive to the accelerated drive, or from the accelerated drive to the node memory, to reduce access latency;

[0058] When data access frequency decreases, the processor migrates it from node memory to accelerated hard disks, or from accelerated hard disks to solid-state drives, to free up high-speed storage resources and optimize storage costs.

[0059] In this way, the direct link based on the CXL protocol reduces the communication latency of multi-node shared storage, enhances the cluster's collaborative processing capabilities, and the intelligent data migration strategy matches the storage medium according to the access requirements, thereby improving the utilization of storage resources, balancing performance and cost, and improving the storage efficiency of the data storage system.

[0060] In one possible implementation, the processor is specifically used for:

[0061] Receive a data read instruction, determine a first target accelerated hard disk among at least one accelerated hard disk according to the data read instruction, and read data from the first target accelerated hard disk through a direct link according to the data read instruction;

[0062] Receive a data write instruction, determine a second target accelerated hard disk among at least one accelerated hard disk according to the data write instruction, and write data to the second target accelerated hard disk through a direct link according to the data write instruction.

[0063] The data reading command can be initiated by the upper-layer application, such as a database query or a real-time computing task in the upper-layer application. The command contains the identification information of the data to be read.

[0064] Data write commands can be initiated by applications, such as data generation tasks or intermediate result storage within applications. The commands contain the data content to be written and the data identifier.

[0065] The processor can parse and process the data read command to obtain the data identification information in the data read command. Based on the data identification information, it can perform a query in the data location mapping table to determine the first target accelerated hard disk.

[0066] Optionally, if the mapping table shows that the data to be read is stored on a certain accelerated hard disk, the processor determines that accelerated hard disk as the first target accelerated hard disk.

[0067] Optionally, if data is cached on multiple accelerated hard drives simultaneously, the processor can determine the first target accelerated hard drive based on load balancing strategies or link status to avoid overloading a single device.

[0068] Based on the CXL protocol, the processor sends a read request to the hard disk controller of the first target accelerated hard disk through a pre-established direct link. The request contains the physical address of the data to be read in the memory module of the accelerated hard disk.

[0069] A direct link can be a CXL channel between a processor port and an accelerator hard drive port.

[0070] After receiving the request, the hard disk controller directly reads the corresponding data from the local memory module and sends the data back to the processor via a direct link;

[0071] After receiving the data, the processor can temporarily store it in the node's memory and return it to the upper-layer application that initiated the read command.

[0072] In one possible implementation, the processor is specifically used to generate a data read request according to a data read instruction, and send the data read request to the first hard disk controller corresponding to the first target accelerated hard disk via a direct link. The data read request includes a target address.

[0073] The first hard disk controller is used to receive data read requests, perform corresponding read operations according to the data read requests, obtain first response data, and send the first response data to the processor through a direct link.

[0074] For example, assuming that transaction record data is stored in memory module 3 of accelerated hard disk A, after the processor parses the instruction, it sends a request to accelerated hard disk A through a direct link. The hard disk controller reads the data from memory module 3 and sends it back. The latency of the whole process can be controlled within 100 nanoseconds, which is far lower than the latency of accessing the SSD through a switch.

[0075] After parsing and processing the data write command, the processor selects a suitable acceleration hard drive as the second target acceleration hard drive based on data characteristics, such as access frequency prediction, data size, and system status, such as the remaining memory capacity of each acceleration hard drive and the current write load.

[0076] In one possible implementation, the processor is specifically used to generate a data write request according to the data write instruction, and send the data write request to the second hard disk controller corresponding to the second target accelerated hard disk via a direct link. The data write request includes a target address and data to be written.

[0077] The second hard disk controller is used to receive data write requests, perform corresponding write operations based on the data write requests, obtain second response data, and send the second response data to the processor through a direct link.

[0078] For example, intermediate result data generated by a real-time computing node needs to be temporarily stored. The processor predicts that it will be accessed by three other nodes within the next 10 minutes. Therefore, it selects the accelerated hard disk B, which has established CXL links with these three nodes, as the second target accelerated hard disk and completes the writing through the direct link. The whole process takes less than 500 microseconds.

[0079] Thus, the memory semantic access characteristics of the CXL protocol are as follows: the processor can directly address the memory module of the accelerated hard drive, reducing protocol conversion and link forwarding overhead, and reducing read and write latency to the microsecond or even nanosecond level; multiple nodes can read and write the same accelerated hard drive in parallel through their respective CXL direct connection links, improving the overall data interaction efficiency of the cluster.

[0080] In one possible implementation, the processor is also used to:

[0081] Predict thermal data in at least one solid-state drive for a future period using a pre-defined model;

[0082] Store hot data to at least one accelerated hard drive.

[0083] The preset modules can be machine learning models trained based on historical access patterns, such as Long Short-Term Memory (LSTM) time series models or gradient boosting tree models.

[0084] The processor can predict hot data in the solid-state drive for future periods, such as the next 1 hour or 24 hours, using a preset model. After determining the future hot data, it triggers a data migration mechanism to migrate this hot data from the solid-state drive to at least one accelerated hard drive through a switch and a direct link.

[0085] Optionally, hot data in at least one solid-state drive (SSD) for a future period can be predicted using a preset model in the following manner: input multiple metadata corresponding to at least one SSD into the preset model to obtain the output results, which include multiple data identifiers; and determine the data corresponding to each of the multiple data identifiers as hot data.

[0086] Metadata can include the historical access time series of the data, the access frequency trend over the past 7 days, the business type to which the data belongs, and the periodic characteristics of related businesses.

[0087] In this way, by predicting hot data in the solid-state drive in the future and migrating it to the accelerated drive in advance by using a preset model, the access latency of future hot data can be significantly reduced, the resource contention during sudden hot access can be avoided, and the resource utilization of the accelerated drive and solid-state drive can be optimized, thereby improving the system response speed and operational stability.

[0088] In one possible implementation, the processor is specifically used for:

[0089] Obtain the access frequency of data in at least one accelerated hard drive, the access frequency of data in at least one solid-state drive, and the access frequency of data in node memory;

[0090] Based on the access frequency of data in at least one accelerated hard disk, at least one solid-state drive, and at least one node's memory, the data is migrated.

[0091] The processor monitors the access frequency of data in each storage tier: node memory, accelerator disk, and solid-state disk, and dynamically adjusts the data storage location to match the performance characteristics of the data and the storage medium.

[0092] The processor can be deployed with an access log module that automatically records key information when data is read or written.

[0093] Key information may include data identifier, access time, access type, etc.

[0094] Data identifiers can be used to distinguish different data; they can be data block identifiers, logical addresses, or file unique identifiers.

[0095] Access time can be used to calculate the number of accesses per unit of time, and the access time can be accurate to the microsecond level of the access occurrence.

[0096] Access type can distinguish between read and write operations.

[0097] For data in accelerator disks and solid-state drives, the processor communicates with the controller of the corresponding device via a direct link or switch to obtain the access logs recorded locally.

[0098] For example, the hard drive controller of an accelerator hard drive and the storage controller of a solid-state drive both have built-in access monitoring units that can count the number of accesses in real time.

[0099] The processor analyzes the collected access logs, determines the access frequency based on the number of accesses per unit time, and performs migration processing on the data in at least one accelerated hard disk, at least one solid-state hard disk, and at least one node's memory based on the access frequency of data in at least one accelerated hard disk, at least one solid-state hard disk, and at least one node's memory.

[0100] For example, for short-term, high-frequency access data, a 1-minute statistical period is used to update the access frequency value.

[0101] For example, for data that is accessed infrequently over a long period of time, a 24-hour statistical cycle is used to update the access frequency value, thereby reducing computational overhead.

[0102] In this way, by acquiring the access frequency of data in node memory, accelerating hard disks, and solid-state drives, and migrating data accordingly, the data storage location can be precisely matched with the access demand. This optimizes the resource utilization of each storage level, reduces the access latency of high-frequency data, and improves the overall storage efficiency of the system.

[0103] In one possible implementation, the processor includes a data accelerator.

[0104] Below, in conjunction with Figure 4 This section explains and describes the data accelerator.

[0105] Figure 4 This is a schematic diagram illustrating the connection relationship of a data accelerator provided in an embodiment of this application. Please refer to... Figure 4 , Figure 4 It can include a processor, node memory, accelerated hard disk and solid-state drive. The processor includes a data accelerator. The data in the node memory, accelerated hard disk and solid-state drive can be migrated according to their respective access frequencies.

[0106] The processor is specifically used to determine a first frequency range corresponding to the acceleration hard disk, a second frequency range corresponding to the solid-state hard disk, and a third frequency range corresponding to the node memory, and to determine at least one piece of data to be migrated and the target storage space corresponding to the data to be migrated in at least one acceleration hard disk, at least one solid-state hard disk, and node memory based on the access frequency of data in at least one acceleration hard disk, at least one solid-state hard disk, and the access frequency of data in at least one solid-state hard disk, the first frequency range, the second frequency range, and the third frequency range.

[0107] The processor is also used to send migration instructions to the data accelerator. The migration instructions include an identifier of at least one piece of data to be migrated, an identifier of the current storage space where the data to be migrated is located, and an identifier of the target storage space corresponding to the data to be migrated. The current storage space is any one of a solid-state drive, an accelerated hard drive, or node memory.

[0108] The data accelerator is used to migrate at least one piece of data to the corresponding target storage space according to migration instructions.

[0109] The processor can adjust its frequency range in real time according to system load and business scenarios.

[0110] For example, when node memory utilization exceeds 90%, the lower limit of the third frequency range can be increased to release memory space occupied by low-frequency data; when the acceleration disk has sufficient free capacity, the first frequency range can be expanded to increase the cache size of the second-highest frequency data and reduce the dependence on node memory.

[0111] The adjusted frequency range is synchronized to all nodes and data accelerators through the system configuration table to ensure consistent global judgment standards.

[0112] The processor can periodically collect the access frequency of all data blocks in the node memory, accelerator disk, and solid-state drive, and match them one by one to the frequency range of the current storage level.

[0113] For example, if a certain data is stored in the node's memory, but its access frequency is less than the lower limit of the third frequency range, it is determined to be data to be migrated;

[0114] If a certain data is stored on an accelerated hard drive, but its access frequency is greater than or equal to the upper limit of the first frequency range, or its access frequency is less than the lower limit of the first frequency range, then it is determined to be data to be migrated.

[0115] If a certain data is stored on a solid-state drive, but its access frequency is greater than or equal to the upper limit of the second frequency range, it is determined to be data to be migrated.

[0116] For identified data to be migrated, the processor can determine the corresponding target storage space based on the frequency range to which its access frequency belongs.

[0117] After the processor determines the data to be migrated and the target storage space, it can generate migration instructions and send them to the data accelerator.

[0118] Optionally, the processor sends migration instructions to the data accelerator via an internal system communication link.

[0119] After receiving the migration instruction, the data accelerator can select the corresponding transmission link to perform the migration based on the types of the current storage space and the target storage space.

[0120] In this way, the data accelerator is dedicated to the migration operation, reducing the processor load and selecting the optimal link based on the storage type to improve the migration rate.

[0121] In one possible implementation, the processor is specifically used for:

[0122] Based on the access frequency and a first frequency range of data in at least one accelerated hard disk, determine M first data items to be migrated in at least one accelerated hard disk, where M is an integer;

[0123] Based on the access frequency and second frequency range of data in at least one solid-state drive (SSD), determine N second data items to be migrated in at least one SSD, where N is an integer;

[0124] Based on the access frequency and third frequency range of the data in the node memory, K third data to be migrated are determined in the node memory, where K is an integer, and at least one data to be migrated includes M first data to be migrated, N second data to be migrated, and K third data to be migrated.

[0125] Based on the first frequency range, the second frequency range, and the third frequency range, determine the target storage space corresponding to at least one piece of data to be migrated.

[0126] Among them, the access frequency of the M first data to be migrated is outside the first frequency range;

[0127] The access frequency of N second data items to be migrated is outside the second frequency range;

[0128] The access frequency of the K third data items to be migrated is outside the third frequency range.

[0129] For example, the first frequency range is 30-100 times / minute, the second frequency range is < 30 times / minute, and the third frequency range is ≥ 100 times / minute.

[0130] For example, in the first batch of data to be migrated, data with a frequency > 100 times / minute is targeted for storage in node memory; data with a frequency < 30 times / minute is targeted for storage in solid-state drives; in the second batch of data to be migrated, data with a frequency ≥ 30 times / minute is targeted for storage in accelerated drives; and in the third batch of data to be migrated, data with a frequency < 100 times / minute is targeted for storage in accelerated drives, thus achieving precise matching between data and storage media.

[0131] In one possible implementation, for any piece of data to be migrated, the target storage space corresponding to the data to be migrated is determined based on a first access frequency, a second access frequency, and a third access frequency, including:

[0132] If the access frequency of the data to be migrated is within the first frequency range, then the target storage space corresponding to the data to be migrated is determined to be at least one accelerated hard disk.

[0133] If the access frequency of the data to be migrated is within the second frequency range, then the target storage space corresponding to the data to be migrated is determined to be at least one solid-state drive.

[0134] If the access frequency of the data to be migrated is within the third frequency range, then the target storage space corresponding to the data to be migrated is determined to be the node memory.

[0135] In one possible implementation, for any given set of data to be migrated, the data accelerator is specifically used for:

[0136] Based on the current storage space and the target storage space, the data to be migrated is migrated to the corresponding target storage space using a data accelerator.

[0137] If the current storage space is a solid-state drive (SSD) and the target storage space is an accelerated hard drive (AHD), the data is read from the SSD via the switch and written to the target AHD via a direct link.

[0138] If the current storage space is an accelerated hard drive and the target storage space is node memory, the data can be directly transferred from the accelerated hard drive to the node memory via a direct connection link;

[0139] If the current storage space is node memory and the target storage space is accelerated hard disk, read the memory data through the node's internal bus and write it to the target accelerated hard disk via a direct link;

[0140] If cross-node memory storage is involved, for example, if the current storage space is the memory of node 1 and the target storage space is the memory of node 2, data is transmitted through links between nodes.

[0141] Optionally, the data accelerator is specifically used for:

[0142] If the current storage space is an accelerated hard disk and the target storage space is node memory, the data to be migrated will be migrated from the accelerated hard disk to the node memory via a direct link.

[0143] If the current storage space is node memory and the target storage space is accelerated hard disk, the data to be migrated will be migrated from node memory to accelerated hard disk via a direct link.

[0144] If the current storage space is a solid-state drive and the target storage space is an accelerated hard drive or node memory, the data to be migrated is decompressed to obtain decompressed data, and the decompressed data is stored in the target storage space.

[0145] If the current storage space is an accelerated hard drive or node memory, and the target storage space is a solid-state drive, then the data to be migrated is compressed to obtain compressed data, and the compressed data is stored in the target storage space.

[0146] In this way, by adopting differentiated processing for migration scenarios between different storage tiers, the efficient migration between hard drives and node memory via direct links is accelerated, and solid-state drives and high-performance storage are decompressed / compressed on demand. This not only leverages the low latency of direct links to improve the efficiency of high-frequency data migration, but also adapts the capacity requirements of solid-state drives and the fast access requirements of high-performance storage through compression / decompression, thereby achieving precise adaptation of storage resources and coordinated optimization of overall system performance and space utilization.

[0147] In one possible implementation, combined with Figure 5 This section explains the process of migrating data from node memory to accelerated hard disk via a direct link.

[0148] Figure 5 This is a flowchart illustrating a data migration method provided in an embodiment of this application. Based on the above embodiments, see also... Figure 5 The method includes:

[0149] S501. Generate an initial migration task based on the data to be migrated, node memory, and acceleration disk.

[0150] The initial migration task can be a migration task based on the data to be migrated, node memory, and acceleration disk configuration.

[0151] It can obtain preset migration tasks, configure and process the preset migration tasks according to the data to be migrated, node memory, and acceleration disk, and generate the initial migration task.

[0152] Optionally, an initial migration task can be generated based on the data to be migrated, node memory, and accelerated disk as follows: the address corresponding to the node memory is determined as the source memory address; the address corresponding to the accelerated disk is determined as the target memory address; the number of bytes to be migrated is determined based on the data to be migrated; and the preset migration task is configured based on the source memory address, target memory address, and number of bytes to be migrated to obtain the initial migration task.

[0153] The processor queries the storage location of the data to be migrated in the node's memory to obtain its physical address. This physical address is the start and end address of the data in the node's memory, and is then determined as the source memory address.

[0154] The processor communicates with the hard disk controller of the accelerated hard disk to request the allocation of a free storage area in the accelerated hard disk memory module and obtains the target physical address, which is used as the target memory address for data migration.

[0155] The processor can calculate the number of bytes to be migrated based on the size of the data to be migrated.

[0156] The system pre-sets a migration task template. The processor fills the source memory address, target memory address, and number of bytes to be migrated into the template, and specifies the transmission link as a direct link between the node and the accelerated hard disk to generate the initial migration task.

[0157] S502. Configure the transmission mode corresponding to the initial migration task to asynchronous migration mode to obtain the target migration task.

[0158] Asynchronous migration mode can refer to the migration process being executed independently in the background without blocking other business operations of the node processor.

[0159] After the processor sends the target migration task to the data accelerator, it can return to execute other tasks without waiting for the migration to complete. Hardware-level link scheduling ensures that data migration and business processing are carried out in parallel, reducing the impact on system real-time performance.

[0160] Below, we can illustrate the determination of the target migration task with code examples.

[0161] struct io_uring_sqe *sqe = io_uring_get_sqe(ring);

[0162] io_uring_prep_memcpy(sqe, dst_addr, src_addr, size, 0 / *flags* / );

[0163] sqe->flags |= IOSQE_ASYNC; / / Enable asynchronous mode

[0164] io_uring_submit(ring);

[0165] Here, sqe represents the migration task; dst_addr represents the target memory address; src_addr represents the source memory address; size represents the number of bytes to migrate; and / *flags* / are additional flags.

[0166] S503. Execute the target migration task on the direct link to migrate the data to be migrated from the node memory to the accelerated hard disk.

[0167] The data accelerator can transmit the data to be migrated to the accelerated hard disk in CXL protocol format via a direct link. After receiving the data to be migrated, the hard disk controller of the accelerated hard disk verifies the checksum to confirm the data integrity, parses the metadata and writes the data to the target memory address, and then generates a response message, which is sent back to the data accelerator via the direct link. After receiving the response message, the data accelerator sends a message to the processor to indicate that the migration is complete. The processor updates the data location mapping table to mark that the data has been migrated from the node memory to the accelerated hard disk, thus completing the entire migration process.

[0168] Optionally, the target migration task can be performed on the direct link to migrate the data to be migrated from the node memory to the accelerated disk in the following manner: according to the target migration task, the data to be migrated is encapsulated to obtain encapsulated data; the encapsulated data is sent to the accelerated disk through the direct link; when the response information sent by the accelerated disk is received through the direct link, it is determined that the data to be migrated is migrated from the node memory to the accelerated disk, and the response information is used to indicate that the accelerated disk has received the encapsulated data.

[0169] Optionally, the data accelerator can encapsulate the data to be migrated according to the target migration task by adding metadata to the data header. The metadata includes the source address identifier, the target address identifier, the data checksum, and the migration task, ensuring that the accelerated hard disk can accurately identify the data ownership and integrity after receiving it.

[0170] In this way, an initial migration task is generated based on the data to be migrated, node memory, and accelerated hard disk; the transmission mode corresponding to the initial migration task is configured as asynchronous migration mode to obtain the target migration task; the target migration task is executed on the direct link to migrate the data to be migrated from node memory to accelerated hard disk. The asynchronous mode ensures non-blocking business, the direct link ensures low-latency transmission, and the encapsulation and response mechanism ensures data reliability, further optimizing the efficiency of data scheduling in the three-level storage architecture.

[0171] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Figure 6 As shown, the electronic device 60 provided in this embodiment includes a data storage system.

[0172] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0173] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0174] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0175] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above embodiments of the hard disk state control method when running.

[0176] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0177] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described embodiments of the hard disk state control method.

[0178] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described embodiments of the hard disk state control method.

[0179] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0180] The above provides a detailed description of a hard disk status control method provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data storage system, characterized in that, The data storage system includes at least one node, at least one accelerated hard disk, and at least one solid-state drive, wherein, The node includes node memory, a processor, and a switch. The switch is connected to the at least one accelerated hard disk and the at least one solid-state hard disk. The accelerated hard disk includes a hard disk controller, multiple memory modules, and multiple ports. The hard disk controller is used to establish direct links with the processor of at least one node by calculating the Fast Link (CXL) protocol and multiple ports, so that the at least one node can have shared access to the accelerated hard disk. The processor is used to perform read and write operations on any one of the accelerated hard disks via the direct link, so as to achieve data synchronization between the at least one node; The processor is also used to migrate data in the node memory of the at least one accelerated hard disk, the at least one solid-state hard disk, and the at least one node.

2. The system according to claim 1, characterized in that, The processor is specifically used for: Receive a data read instruction, determine a first target accelerated hard disk among the at least one accelerated hard disks according to the data read instruction, and read data from the first target accelerated hard disk through the direct link according to the data read instruction; The system receives a data write instruction, determines a second target accelerated hard disk among the at least one accelerated hard disks according to the data write instruction, and writes data to the second target accelerated hard disk through the direct link according to the data write instruction.

3. The system according to claim 2, characterized in that, The processor is specifically configured to generate a data read request according to the data read instruction, and send the data read request to the first hard disk controller corresponding to the first target accelerated hard disk through the direct link. The data read request includes a target address. The first hard disk controller is used to receive the data read request, perform the corresponding read operation according to the data read request, obtain the first response data, and send the first response data to the processor through the direct connection link.

4. The system according to claim 2, characterized in that, The processor is specifically used to generate a data write request according to the data write instruction, and send the data write request to the second hard disk controller corresponding to the second target accelerated hard disk through the direct link. The data write request includes a target address and data to be written. The second hard disk controller is used to receive the data write request, perform the corresponding write operation according to the data write request, obtain the second response data, and send the second response data to the processor through the direct connection link.

5. The system according to claim 1, characterized in that, The processor is specifically used for: The access frequency of data in the at least one accelerated hard disk, the access frequency of data in the at least one solid-state hard disk, and the access frequency of data in the node memory are obtained. Based on the access frequency of data in the at least one accelerated hard disk, the access frequency of data in the at least one solid-state hard disk, and the access frequency of data in the node memory, the data in the at least one accelerated hard disk, the at least one solid-state hard disk, and the node memory of the at least one node are migrated.

6. The system according to claim 5, characterized in that, The processor includes a data accelerator; The processor is specifically used to determine a first frequency range corresponding to the accelerated hard disk, a second frequency range corresponding to the solid-state hard disk, and a third frequency range corresponding to the node memory; and based on the access frequency of data in the at least one accelerated hard disk, the access frequency of data in the at least one solid-state hard disk, the access frequency of data in the node memory, the first frequency range, the second frequency range, and the third frequency range, determine at least one piece of data to be migrated and the target storage space corresponding to the data to be migrated in the at least one accelerated hard disk, the at least one solid-state hard disk, and the node memory. The processor is further configured to send a migration instruction to the data accelerator, the migration instruction including an identifier of the at least one piece of data to be migrated, an identifier of the current storage space where the data to be migrated is located, and an identifier of the target storage space corresponding to the data to be migrated, wherein the current storage space is any one of the solid-state drive, the acceleration drive, or the node memory; The data accelerator is used to migrate at least one piece of data to be migrated to the corresponding target storage space according to the migration instruction.

7. The system according to claim 6, characterized in that, The processor is specifically used for: Based on the access frequency of data in the at least one accelerated hard disk and the first frequency range, M first data items to be migrated are determined in the at least one accelerated hard disk, where M is an integer. Based on the access frequency of data in the at least one solid-state drive and the second frequency range, N second data items to be migrated are determined in the at least one solid-state drive, where N is an integer. Based on the access frequency of the data in the node memory and the third frequency range, K third data to be migrated are determined in the node memory, where K is an integer, and the at least one data to be migrated includes the M first data to be migrated, the N second data to be migrated, and the K third data to be migrated. Based on the first frequency range, the second frequency range, and the third frequency range, the target storage space corresponding to each of the at least one data to be migrated is determined.

8. The system according to claim 7, characterized in that, The access frequency of the M first data to be migrated is outside the first frequency range; The access frequency of the N second data to be migrated is outside the second frequency range; The access frequency of the K third data to be migrated is outside the third frequency range.

9. The system according to claim 7 or 8, characterized in that, For any piece of data to be migrated; determine the target storage space corresponding to the data to be migrated based on the first frequency range, the second frequency range, and the third frequency range, including: If the access frequency of the data to be migrated is within the first frequency range, then the target storage space corresponding to the data to be migrated is determined to be the at least one accelerated hard disk; If the access frequency of the data to be migrated is within the second frequency range, then the target storage space corresponding to the data to be migrated is determined to be the at least one solid-state drive; If the access frequency of the data to be migrated is within the third frequency range, then the target storage space corresponding to the data to be migrated is determined to be the node memory.

10. The system according to claim 6, characterized in that, For any given piece of data to be migrated; the data accelerator is specifically used for: Based on the current storage space and the target storage space, the data to be migrated is migrated to the corresponding target storage space through the data accelerator.

11. The system according to claim 10, characterized in that, The data accelerator is specifically used for: If the current storage space is the accelerated hard disk and the target storage space is the node memory, then the data to be migrated is migrated from the accelerated hard disk to the node memory through the direct connection link.

12. The system according to claim 10, characterized in that, The data accelerator is specifically used for: If the current storage space is the node memory and the target storage space is the accelerated hard disk, then the data to be migrated is migrated from the node memory to the accelerated hard disk through the direct connection link.

13. The system according to claim 10, characterized in that, The data accelerator is specifically used for: If the current storage space is the solid-state drive and the target storage space is the accelerated hard drive or node memory, then the data to be migrated is decompressed to obtain decompressed data, and the decompressed data is stored in the target storage space.

14. The system according to claim 10, characterized in that, The data accelerator is specifically used for: If the current storage space is the accelerated hard disk or node memory, and the target storage space is the solid-state drive, then the data to be migrated is compressed to obtain compressed data, and the compressed data is stored in the target storage space.

15. The system according to claim 12, characterized in that, Migrating the data to be migrated from the node's memory to the accelerated hard disk via the direct link includes: An initial migration task is generated based on the data to be migrated, the node memory, and the acceleration hard disk; Configure the transmission mode corresponding to the initial migration task to asynchronous migration mode to obtain the target migration task; The target migration task is executed on the direct link to migrate the data to be migrated from the node memory to the accelerated hard disk.

16. The system according to claim 15, characterized in that, Based on the data to be migrated, the node memory, and the acceleration disk, an initial migration task is generated, including: The address corresponding to the node's memory is determined as the source memory address; The address corresponding to the accelerated hard disk is determined as the target memory address; Based on the data to be migrated, determine the number of bytes to migrate; Based on the source memory address, the target memory address, and the number of bytes to be migrated, a preset migration task is configured to obtain an initial migration task.

17. The system according to claim 15, characterized in that, Executing the target migration task on the direct link to migrate the data to be migrated from the node memory to the accelerated hard disk includes: According to the target migration task, the data to be migrated is encapsulated to obtain encapsulated data; The encapsulated data is sent to the accelerated hard disk via the direct connection link; When the response information sent by the accelerated hard disk is received through the direct link, it is determined that the data to be migrated will be migrated from the node memory to the accelerated hard disk. The response information is used to indicate that the accelerated hard disk has received the encapsulated data.

18. The system according to any one of claims 1-8 or 10-17, characterized in that, The processor is also used for: Predict thermal data in at least one solid-state drive for a future time period using a preset model; The hot data is stored in the at least one accelerated hard disk.

19. The system according to claim 18, characterized in that, Predicting hot data in the at least one solid-state drive for a predicted future period using a preset model includes: Multiple metadata corresponding to the at least one solid-state drive are input into the preset model to obtain an output result, which includes multiple data identifiers; The data corresponding to the multiple data identifiers are respectively identified as hot data.

20. An electronic device, characterized in that, Includes the data storage system as described in any one of claims 1 to 19.

Images