Data storage method, device, cluster, system, and storage medium

Abstract

The application provides a data storage method, device, cluster, system and storage medium. The method is applied to a first storage cluster. The first storage cluster is managed by a second storage cluster. The storage rate of the first storage cluster is greater than that of the second storage cluster. The method comprises the following steps: creating a data block for storing cold data in a data storage pool of the second storage cluster through a creation interface provided by the second storage cluster, the data storage pool being a virtual pool of the second storage cluster managed by the first storage cluster; recording metadata of the data block and loading the metadata to the memory for caching; and migrating the cold data to a storage space corresponding to the data block when a preset migration condition is reached. According to the scheme, the cold data is migrated when the migration condition is reached, migration is avoided when the business load is high, the migration efficiency is improved, the performance of the storage cluster in processing regular business read-write requests is avoided from being affected, and the normal business processing speed is prevented from being reduced or read-write blocking.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and in particular to a data storage method, apparatus, cluster, system and storage medium. Background Technology

[0002] In the field of distributed storage, servers used for storing data are generally divided into two categories: all-flash and hybrid flash. All-flash servers are equipped with solid-state drives (SSDs) using the Non-Volatile Memory Host Controller Interface (NVM Express, NVMe) protocol, which have higher read and write speeds but also higher costs. Hybrid flash servers mainly use mechanical hard drives (HDDs) and may contain no or only some SSDs, resulting in lower read and write speeds but also lower costs.

[0003] To reduce costs, these two types of servers can be used together. A cluster of all-flash servers stores hot data, while a cluster of hybrid flash servers stores cold data. To prevent cold data from occupying the space of the all-flash servers, the cold data will be migrated to the hybrid flash servers. Traditional migration solutions will perform the migration when the business load is high, which can easily cause the migration task to crowd out normal business (such as the business of clients reading and writing data to the server) read and write requests, resulting in a decrease in the processing speed of normal business. Summary of the Invention

[0004] In view of this, this application provides a data storage method, apparatus, cluster, system, and storage medium to improve the efficiency of data migration between different storage clusters.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] This application provides a data storage method applied to a first storage cluster, wherein the first storage cluster manages a second storage cluster, and the storage rate of the first storage cluster is greater than the storage rate of the second storage cluster. The method includes:

[0007] Through the creation interface provided by the second storage cluster, data blocks for storing cold data are created in the data storage pool of the second storage cluster. The data storage pool is a virtual pool in the second storage cluster that is managed by the first storage cluster.

[0008] The metadata of the data block is recorded and loaded into memory for caching; when the preset migration conditions are met, the cold data is migrated to the storage space corresponding to the data block.

[0009] A second aspect of this application provides a data storage device applied to a first storage cluster, wherein the first storage cluster manages a second storage cluster, and the storage rate of the first storage cluster is greater than the storage rate of the second storage cluster. The device includes:

[0010] A creation unit is used to create data blocks for storing cold data in the data storage pool of the second storage cluster through the creation interface provided by the second storage cluster. The data storage pool is a virtual pool in the second storage cluster that is managed by the first storage cluster.

[0011] The recording unit is used to record the metadata of the data block and load it into memory for caching; when the preset migration conditions are met, the cold data is migrated to the storage space corresponding to the data block.

[0012] A third aspect of this application provides a first storage cluster, including a storage array and at least one control node; any one of the control nodes is used to execute a data storage method as provided in any one of the first aspects of this application.

[0013] The storage array is used to store data.

[0014] A fourth aspect of this application provides a storage system, including a first storage cluster provided in the third aspect of this application and at least one second storage cluster, wherein the first storage cluster is connected to each of the second storage clusters respectively;

[0015] For any second storage cluster, the second storage cluster is used to connect all disks through a unified management module to form a data storage pool;

[0016] The second storage cluster is also used to provide a creation interface through a unified management module, which is used to create data blocks in the data storage pool.

[0017] The fifth aspect of this application provides a computer storage medium for storing a computer program, wherein when the computer program is executed by a processor of an electronic device to which the computer storage medium belongs, it is used to implement the data storage method provided in any one of the first aspects of this application.

[0018] The advantage of this solution is that by creating data blocks and migrating cold data asynchronously, the storage resources of the second storage cluster can be planned in advance and the migration delay can be reduced. The migration is then performed when the cluster is in a suitable state, thus avoiding the migration affecting the read and write efficiency of the first storage cluster for normal business read and write requests. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1This is a flowchart illustrating a data storage method disclosed in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram illustrating the interaction between a first storage cluster and a second storage cluster as disclosed in an embodiment of this application.

[0022] Figure 3 This is a schematic diagram of the network architecture of a data storage method disclosed in an embodiment of this application;

[0023] Figure 4 This is a flowchart illustrating a data storage method disclosed in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram illustrating a metadata storage method disclosed in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the architecture of a second storage cluster disclosed in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram illustrating the migration of cold data to a second storage cluster, as disclosed in an embodiment of this application.

[0027] Figure 8 This is a flowchart illustrating how storage balancing is achieved between second storage clusters, as disclosed in an embodiment of this application.

[0028] Figure 9 This is a flowchart illustrating reading data from a second storage cluster, as disclosed in an embodiment of this application.

[0029] Figure 10 This is a schematic diagram of a data storage device disclosed in an embodiment of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] To facilitate understanding of the technical solutions of the embodiments of this application, some terms in this embodiment will be explained first.

[0032] The Persistence (P) layer refers to the data persistence layer in distributed storage. Plog refers to the linear storage unit provided by the P layer. Slog (Storage log) data blocks, or simply data blocks, are two-layer encapsulation structures based on Plog, serving as storage units for file data in the file system. They record plog information, data compression type, garbage rate, and other information within the data block. In this embodiment, Slog data blocks can be considered as the specific storage format of data in the first and second storage clusters; that is, all data is stored in the first and second storage clusters in the form of Slog data blocks.

[0033] Object storage service (OSD), as the server of the P layer, is a process that manages a physical disk. The OSD is responsible for writing data to the disk in the form of objects, handling data verification, and handling logic such as data verification, copying and recovery.

[0034] This application provides a data storage method applied to a first storage cluster. Please refer to [link to relevant documentation]. Figure 1 The method includes the following steps.

[0035] S101, obtain access information for accessing the second storage cluster, wherein the read / write rate of the first storage cluster is higher than that of the second storage cluster.

[0036] The data storage method of this embodiment can be applied to any distributed storage scenario, such as enterprise-level distributed storage (EDS) scenarios.

[0037] The execution entity in this embodiment can be a first storage cluster. The first storage cluster can be an all-flash storage cluster, that is, a distributed storage cluster composed of multiple all-flash servers as nodes; the second storage cluster can be a hybrid flash storage cluster, that is, a distributed storage cluster composed of multiple hybrid flash servers as nodes.

[0038] The method in this embodiment can be executed through the management module of the first storage cluster. Please refer to [link / reference]. Figure 2 In step S101, the management module executes A1 to obtain the network address, username, and password to obtain access information for the second storage cluster.

[0039] The management module can be considered a software module running on the first storage cluster. The network address obtained by A1 can be the network address of the second storage cluster, and the username and password can be the username and password of the administrator account of the second storage cluster.

[0040] The management module can display a management interface on any terminal device. Users can enter the above access information on the management interface. After entering the information, they can click the control in the management interface to trigger management, such as clicking the "Execute Management" control. The management module responds to this click operation and obtains the network address, username and password entered in the management interface as the access information of the second storage cluster.

[0041] S102, based on the access information, a management request is sent to the second storage cluster. The management request is used to request the use of the second storage cluster to store the data of the first storage cluster.

[0042] See Figure 2 Step S102 is equivalent to the management module responding to obtaining access information by executing A2 and initiating a management request. Specifically, the management module generates a management request with a username and password and sends the management request to the second storage cluster according to the network address of the second storage cluster.

[0043] Upon receiving the management request, the second storage cluster first verifies the username and password. After successful verification, the second storage cluster can check whether it is eligible for management. Specifically, if the following conditions are met, the second storage cluster is determined to be eligible for management; otherwise, it is determined that it is ineligible for management:

[0044] The second storage cluster and the first storage cluster meet the version compatibility requirements;

[0045] The second storage cluster currently has no data storage pool;

[0046] The second storage cluster is not currently managed by any of the first storage clusters.

[0047] If the second storage cluster determines that it cannot be managed, it sends a message to the management module refusing to be managed, and the method in this embodiment ends.

[0048] If the second storage cluster determines that it can be managed, then proceed to step A3 to create a data storage pool, and after creation, proceed to step A4 to provide storage pool information. The storage pool information may include one or more of the following: the maximum capacity (or maximum available capacity) of the data storage pool, the used capacity, the remaining available capacity, and the storage status (specifically, it may be abnormal or normal), and may also include other information related to the data storage pool.

[0049] S103, receive storage pool information fed back by the second storage cluster in response to the management request, and store the data of the first storage cluster in the data storage pool created by the second storage cluster according to the storage pool information.

[0050] In step S103, after receiving the storage pool information, the management module executes A5 based on the storage pool information to expand the data storage pool to the cold pool list. After the expansion is completed, the data of the first storage cluster can be stored in the data storage pool of the second storage cluster.

[0051] The management module can maintain a cold pool list in the first storage cluster. This cold pool list can be stored persistently in the key-value (KV) database of the first storage cluster. The cold pool list (also called cold_pool_table) can record the storage pool information of each data storage pool created by the second storage cluster that has been managed by the first storage cluster. Step A5 is equivalent to adding the storage pool information of the newly added data storage pool to the cold pool list to complete the expansion. In this embodiment, a cold pool can refer to a data storage pool created by the second storage cluster using the above method.

[0052] When the first storage cluster needs to migrate data to the cold pool, it can query the storage pool information of each data storage pool from the cold pool list, thereby determining a target data storage pool, and then migrate the output to the target data storage pool for storage.

[0053] After A5 is executed, the aforementioned second storage cluster is equivalent to a second storage cluster that has been managed by the first storage cluster.

[0054] Each time a data storage pool is expanded into the cold pool list, the management module can start a timer for that data storage pool to periodically execute step A6 and periodically obtain storage pool information. Based on A6, the management module can update the cold pool list with the obtained storage pool information each time, replacing the storage pool information of the same data storage pool recorded in the previous time, thereby realizing the periodic update of storage pool information in the cold pool list.

[0055] The following is combined Figure 3 The structure shown illustrates the beneficial effects of the method in this embodiment.

[0056] Figure 3 In this context, the hot pool of the first storage cluster refers to the data storage pool of the first storage cluster itself, used to store local data of the first storage cluster. The cold pool of the first storage cluster is equivalent to the aforementioned cold pool list. Figure 3 As can be seen, when the storage space of the first storage cluster is insufficient, multiple second storage clusters can be managed into the first storage cluster. Thus, the first storage cluster can automatically migrate excess data to the data storage pool of the second storage cluster through the cold pool list, realizing automatic data migration between all-flash servers and hybrid flash servers, thereby improving migration efficiency and success rate, and avoiding errors in the migration process.

[0057] Optionally, when the storage performance of the first storage cluster itself is insufficient, the storage performance of the first storage cluster can be improved by adding all-flash servers as nodes in the first storage cluster.

[0058] In the first storage cluster Figure 1 The corresponding method, assuming at least one second storage cluster is managed, allows the first storage cluster to further perform actions such as... Figure 4 The data storage method shown migrates data from the first storage cluster to a managed data storage pool. Figure 4 This is a flowchart of a data storage method applied to a first storage cluster, which manages a second storage cluster, and the storage rate of the first storage cluster is greater than the storage rate of the second storage cluster.

[0059] S401 creates data blocks for storing cold data in the data storage pool of the second storage cluster through the creation interface provided by the second storage cluster. The data storage pool is a virtual pool in the second storage cluster that is managed by the first storage cluster.

[0060] S402 records the metadata of the data block and loads it into memory for caching; when the preset migration conditions are met, the cold data is migrated to the storage space corresponding to the data block.

[0061] The creation interface can be included in the storage pool information fed back by the second storage cluster. The creation interface is used to access the data storage pool and to create data blocks in the data storage pool. A data block is equivalent to an abstract concept that maps to a small storage space in the storage cluster. That is, the second storage cluster manages its own storage space in units of data blocks and can read and write data from the corresponding storage space in units of data blocks. The first storage cluster can execute S401 when specific triggering conditions are met.

[0062] In the storage method of this embodiment, the migration of cold data can be performed asynchronously. That is, after the data block is created, the cold data can be temporarily not migrated, but the migration can be performed only when the migration conditions are met. If it is found that the migration conditions are not met during the migration, the migration can be stopped. The migration task will be resumed when the migration conditions are met again, and the remaining cold data can be migrated.

[0063] The beneficial effects of this embodiment are as follows:

[0064] First, in related technologies, the metadata of data blocks is usually stored together with the data blocks in a hybrid flash storage cluster with low read and write speeds. Therefore, each time data migration is performed, the all-flash storage cluster needs to remotely query the metadata of the hybrid flash storage cluster to confirm the status of each data block (idle or used). This query will occupy the read and write bandwidth of the all-flash storage cluster, and the migration efficiency is low due to the long query latency.

[0065] In this embodiment, the metadata of the data block is directly recorded by the first all-flash storage cluster when the data block is created and loaded into the memory cache. In this way, the status of each data block can be directly obtained from the cached metadata each time data is migrated, without having to query from the second hybrid flash storage cluster. This can save the read and write bandwidth of the all-flash storage cluster and eliminate query latency, thereby improving migration efficiency.

[0066] Second, each data block in this scheme can adopt a lock-free design, that is, when migrating data to one data block, there is no need to lock other data blocks. Furthermore, the first storage cluster manages the metadata of each data block through atomic operations, that is, after reading or writing data to a data block, the metadata of this data block is updated immediately and independently without affecting the metadata of other data blocks.

[0067] Based on this, this solution can support writing a large amount of cold data into multiple data blocks in batches, thereby better supporting the simultaneous execution of multiple migration tasks without conflict, and improving data migration speed in scenarios where a large amount of cold data is simultaneously evicted from the first storage cluster.

[0068] Third, this solution will create data blocks and migrate cold data asynchronously, which can plan the storage resources of the second storage cluster in advance and reduce migration delay. The migration will be performed when the cluster is in a suitable state, avoiding the migration affecting the read and write efficiency of the first storage cluster for normal business read and write requests.

[0069] The first storage cluster can execute S401 to create a data block when any creation condition is met. One optional creation condition is that, in response to a cold data migration task, the first storage cluster obtains the cold data migration task when it discovers that the amount of cold data it stores has reached a certain threshold, or when it receives an instruction from a user to actively migrate cold data. Then, it can execute S401 in response to the cold data migration task. Alternatively, the first storage cluster can periodically update the data popularity of its various data blocks. When it detects that the popularity of any of its data blocks is lower than a first threshold, it obtains the cold data migration task.

[0070] Another optional creation condition is that the storage level of the first storage cluster 2 reaches a preset utilization threshold (e.g., 80%, 75% or other values). The storage level of the first storage cluster can be equal to the ratio obtained by dividing the used capacity of the first storage cluster by the maximum capacity of the first storage cluster.

[0071] By using the above creation conditions, data blocks can be created in the second storage cluster in a timely manner when there is a current or potential need to migrate cold data in the first storage cluster, thereby supporting the smooth progress of subsequent cold data migration tasks.

[0072] The preset migration conditions may include any one or more combinations of these migration conditions: Migration condition 1, the performance of the first storage cluster is lower than the first preset performance threshold; Migration condition 2, the performance of the second storage cluster is lower than the second preset performance threshold; Migration condition 3, the sum of the sizes of all data blocks to be migrated in the first storage cluster reaches the preset size.

[0073] The performance of a storage cluster can be characterized by any metric. For example, it can be represented by the amount of data read and written per unit time. For instance, the performance of a storage cluster can be represented by the amount of data read or written to the storage cluster in one second. The first and second preset performance thresholds can be the same or different thresholds for the number of reads or writes per unit time.

[0074] The advantage of using migration conditions 1 and / or 2 is that the performance of the first and second storage clusters is relatively low, indicating that there are fewer requests for normal business read and write operations on these storage clusters at this time. Migrating data at this time can reduce the impact of data migration on normal business read and write operations.

[0075] In migration condition 3, the size of a data block refers to the amount of data stored in that data block. The sum of the sizes of all data blocks to be migrated is equivalent to the total amount of cold data in the first storage cluster. The advantage of using migration condition 3 is that it avoids frequently migrating a small number of small data blocks (data blocks with less data), and instead migrates a large number of data blocks in the first storage cluster in a concentrated manner, which helps to reduce the read and write bandwidth of the storage cluster occupied by the migration task.

[0076] Optionally, the metadata for recording data blocks may include:

[0077] The logical address of the data stored in the data block, as well as the storage pool to which the data block belongs, are recorded as the metadata of the data block.

[0078] Specifically, the metadata of a data block can include at least the address information of the data block, and may also include the mapping information of the data block. The address information may include the logical address (or virtual address) of the data stored in the data block, and may include the storage pool to which the data block belongs (i.e., which data storage pool in which storage cluster the data block belongs to). The mapping information indicates the mapping relationship between the data stored in the storage space corresponding to the data block (i.e., the data contained in the data block) and the file system. Specifically, it can indicate which file in the file system the data contained in the data block belongs to.

[0079] There are no restrictions on how metadata can be cached in memory. For example, a key-value database can be created in the memory of the first storage cluster, and metadata can be cached in the first and second data tables contained in the key-value database.

[0080] like Figure 5As shown, the first data table can also be referred to as the Manifest_table data table, which caches the mapping information of metadata; the second data table can also be referred to as the slog_table data table, which caches the address information of metadata.

[0081] Cold data can include data whose popularity meets a first threshold (e.g., is less than the first threshold). The popularity of data represents the frequency of data access. Generally, after identifying cold data, the first storage cluster can migrate the cold data to the data storage pool of the managed second storage cluster. Therefore, cold data can also be understood as data stored in the data storage pool of the second storage cluster. Conversely, hot data includes data in the first storage cluster whose popularity is greater than or equal to the first threshold.

[0082] like Figure 5 As shown, the first storage cluster also includes a storage management (cache_policy) module and a storage balancing (cache_balance) module. Based on the storage management module, when data needs to be migrated to the cold pool storage, the first storage cluster intelligently selects a cold pool with normal status and low utilization to prioritize the migration of cold data to that cold pool. Utilization can be considered as the ratio of used capacity to maximum capacity. The storage balancing module can be used to intelligently balance the data capacity among the second storage clusters, making the utilization rates of multiple cold pools approximately consistent.

[0083] The beneficial effect of this embodiment is that storing metadata in the first storage cluster can take advantage of the high read and write speed of the first storage cluster, ensuring that users' read, write and query operations on metadata can be responded to quickly, and meeting users' needs for fast metadata query.

[0084] Please see Figure 6 This is a schematic diagram of the architecture of the second storage cluster provided in this embodiment.

[0085] The second storage cluster includes multiple nodes, such as... Figure 6 The diagram shows N nodes, where the specific value of N is not limited; for example, it could be 10, 50, etc. Each node includes multiple disks, for example... Figure 6Node 1 includes Disk 1, Disk 2, and other disks. Upon receiving a management request and confirming its applicability, the second storage cluster creates an independent storage OSD service for each disk on each node. Then, through the persistence layer of the second storage cluster, it connects to and manages all OSD services. Finally, based on information such as the location and capacity of all disks, all these disks are abstracted to form a virtual storage resource pool. This virtual storage resource pool is the data storage pool created by the second storage cluster in response to the management request in the aforementioned embodiment. The method of abstracting disks into a virtual data storage pool is not limited; for example, it can specify fault domain replication or erasure coding (EC) type virtual pools as data storage pools. For specific creation methods, please refer to relevant existing technologies.

[0086] Based on the created data storage pool, the second storage cluster can implement and provide a set of interfaces for operating the capacity space of the data storage pool. The number and type of these interfaces are unlimited. As an example, the provided set of interfaces may include... Figure 6 The interface shown includes a create interface, a write interface, a read interface, and a delete interface. The create interface is used to create several data blocks of a specified size in the data storage pool of the second storage cluster. The write interface is used to write data to the specified data blocks. The read interface is used to read data from the specified data blocks. The delete interface is used to delete the specified data blocks.

[0087] Based on this, when a data block needs to be migrated from the hot pool to the cold pool, the data block management module (slog_manager) of the first storage cluster first calls the creation interface through an asynchronous background call based on the private protocol pre-agreed between the first and second storage clusters. An empty data block is pre-created in the cold pool of the second storage cluster, the metadata of this data block is obtained, and the metadata of this data block is persistently stored in the key-value database of the first storage cluster.

[0088] The storage management module obtains the metadata of the created data block from the data block management module, reads the cold data that needs to be migrated from the hot pool data block, and calls the write interface based on the obtained metadata to write the data read from the hot pool into this newly created data block. The written data will be persistently stored on the disk of the second storage cluster in the storage space corresponding to the data block through the persistence layer of the second storage cluster.

[0089] Finally, the storage management module writes the metadata of this newly created data block into the second data table to update the data storage pool and logical storage address of the data block recorded in the second data table. This completes the process of migrating the cold data stored in the first storage cluster to the data storage pool created by the second storage cluster.

[0090] The beneficial effect of this embodiment is that the physical storage resources (i.e., disks) of the second storage cluster are abstracted into a virtual data storage pool, and several interfaces for accessing the data storage pool are provided to the outside world based on the data storage pool. On the one hand, it can meet the needs of the first storage cluster to read and write data to the second storage cluster. On the other hand, it can decouple the physical resources of the second storage cluster from the first storage cluster, which can improve the security of the second storage cluster and avoid compatibility issues between the physical resources of different storage clusters that prevent normal data reading and writing.

[0091] like Figure 7 As shown, the data blocks that migrate cold data to the data storage pool can be implemented by the storage management module of the first storage cluster.

[0092] The first storage cluster can handle user write and read requests through front-end write and read service interfaces.

[0093] When a write request is received, the front-end write business interface executes B1, writes the file to the hot pool data block, and B2, updates the metadata. Specifically, when the front-end write business interface executes B1, it can divide the file to be written carried by the write request into multiple parts, and write the data to each data block of the hot pool of the first storage cluster. Each data block stores one copy of the data. Then, B2 can be executed to record the metadata of the data blocks that were written in the first data table and the second data table. At this point, the write request is completed.

[0094] When a read request is received, the front-end read interface queries the metadata based on the read request to determine if the read request is successful. If the metadata indicates that the data to be read is stored in the hot pool, the read request is considered successful; otherwise, it is considered unsuccessful. The data to be read refers to the data requested in the read request. In response to B3, the read request is successful, and the front-end read interface can retrieve the data to be read and send it back to the client. At this point, the read request processing is complete. Both front-end read and write operations fall under the aforementioned normal business logic.

[0095] The storage management module can manage the popularity of each data block (or the data stored in the data block) in the following way:

[0096] In response to the front-end write business interface adding new data to the data block, configure the initial heat of the newly written data; in response to the front-end write business interface performing a write operation on the existing data of the data block, execute step B4 to update the heat of the data block to which the write operation was performed; in response to the front-end read business interface executing B3, execute step B4 to update the heat of the data block, thereby updating the heat of one or more data blocks read by the front-end read business interface.

[0097] As time progresses, the popularity of each data block in the hot pool of the first storage cluster will change, and some data blocks may become cold data because they have not been accessed for a long time.

[0098] While maintaining the hotness of each data block in the hot pool in real time as described above, the storage management module periodically checks whether the data in each data block has become cold data. If it finds that the data in one or more data blocks in the hot pool has become cold data, it responds to B5, detects cold data, and starts the migration process for these cold data to migrate them from the hot pool to the pre-created data blocks in the cold pool.

[0099] The migration process includes:

[0100] B6. Select a cold pool based on the remaining available capacity;

[0101] B7 migrates cold data to the cold pool;

[0102] B8, update the second data table to point to the cold pool.

[0103] When executing step B6, the storage management module can obtain the remaining available capacity of each cold pool from the key-value database, and select a cold pool based on the remaining available capacity. Specifically, it can select the cold pool with the largest remaining available capacity from multiple cold pools as the cold pool to be used for this migration (denoted as the target cold pool), or it can select the cold pool with the lowest utilization rate from multiple cold pools as the target cold pool. The utilization rate of a cold pool is equal to the remaining available capacity of the cold pool divided by the maximum capacity of the cold pool.

[0104] During execution of B7, the storage management module can read cold data from data blocks in the hot pool, call the write interface provided by the second storage cluster to write the cold data to the storage space corresponding to the pre-created data block in the target cold pool, and repeat this process until all the cold data that needs to be migrated has been written to the corresponding data block. During the execution of B7, after writing a piece of cold data to a data block in the target cold pool, the cold data can be deleted from the data block in the hot pool that originally stored the cold data. For example, cold data stored in data block 1 of the hot pool can be read and written to data block 2 of the target cold pool. After the writing is completed, the previously stored cold data can be deleted from the storage space corresponding to data block 1 of the hot pool.

[0105] When B8 is executed, the storage management module can access the second data table stored in the key-value database, query the data blocks that were written to cold data in B7, and update the metadata of these data blocks in the key-value database, such as updating the logical address of these data blocks, the mapping information between data blocks and files, etc., and the cold data migration process ends.

[0106] Through the above migration method, this embodiment can store frequently accessed hot data in the hot pool, utilize the high read and write speed of the first storage cluster to meet the needs of users to access hot data frequently, and migrate cold data that has not been accessed for a long time to the cold pool with low read and write speed. This can reduce the storage cost of this cold data and expand the amount of data stored.

[0107] The aforementioned write interface can be included in the storage pool information fed back by the second storage cluster. Steps B6 and B8 above are equivalent to one implementation of migrating cold data to the storage space corresponding to the data block in S402. The implementation of S402 can also be summarized as follows: through the write interface provided by the second storage cluster, the cold data stored in the first storage cluster is written to the storage space corresponding to the created data block; the cold data written to the second storage cluster is deleted from the storage space of the data block in the first storage cluster. The advantage of writing cold data to the data block in the second storage cluster and deleting the corresponding data from the data block in the first storage cluster is that timely deletion of redundant cold data in the first storage cluster can effectively save the storage resources of the first storage cluster.

[0108] Optionally, in S402, migrating cold data to the storage space corresponding to the data block may include: obtaining data whose popularity meets a first threshold as cold data, and migrating the cold data to the storage space corresponding to the data block, where the popularity of the data represents the frequency of data access.

[0109] In this embodiment, each data block stores data with a corresponding "heat" value. This heat value can be equal to the elapsed time from the last time the data in that data block was accessed to the current time, referred to as the last access duration. For example, if data block 1 was last accessed 1 minute ago, then the heat value of data block 1 is 1 minute; if data block 2 was last accessed 2 hours ago, then the heat value of data block 2 is 120 minutes. The longer the last access duration, the lower the heat value, and vice versa.

[0110] A data block being written to the hot pool of the first storage cluster, read by a user, or updated by a user can all be considered as the data block being accessed.

[0111] The popularity of a data block can also be measured by other metrics, as long as they can characterize the frequency of access to the data block. For example, the popularity of a data block can also be equal to the cumulative number of times it has been accessed within the most recent preset time period (such as the last 24 hours) or the average number of times it has been accessed per minute. The higher the average or cumulative number of accesses, the higher the popularity, and vice versa.

[0112] When the popularity equals the last access duration, the first threshold can be a preset first duration threshold, such as 30 minutes, 90 minutes or other values, without limitation. Correspondingly, cold data can include data whose last access duration is greater than the first duration threshold, and hot data includes data whose last access duration is less than or equal to the first duration threshold.

[0113] For example, 10 visits per minute can be used as the first threshold, and the average number of visits per minute within a certain period of time (10 minutes, 1 hour, or other time) can be used as the popularity.

[0114] The advantage of differentiating cold data based on whether its access frequency exceeds a first threshold is that frequently accessed data is stored in a first storage cluster with high read / write speeds. This allows frequently accessed data to be read and written at a high rate, enabling the front end to quickly obtain this high-access data and ensuring high business processing efficiency. Meanwhile, infrequently accessed data is stored in a second storage cluster. Since this data is accessed less frequently, even if read and written at a slower rate, it will not significantly affect the front end's business processing efficiency. In summary, this solution can fully utilize the high-cost, low-capacity first storage cluster and the low-cost, high-capacity second storage cluster while maintaining the efficiency of the front end for reading and writing data, thus balancing high read / write performance and large storage space.

[0115] The first threshold can be determined based on the capacity of the first storage cluster. If the capacity of the first storage cluster is large, a lower threshold can be set to store more data in the first storage cluster and make full use of its capacity. If the capacity of the first storage cluster is small, a higher threshold can be set to store more data in the second storage cluster and avoid the first storage cluster from storing too much data, which could cause normal read and write operations to be blocked.

[0116] Optionally, during the storage of data using the first storage cluster and at least one second storage cluster, the amount of data stored in multiple data storage pools may be uneven due to factors such as file deletion and the inclusion of a new second storage cluster. Therefore, in the method of this embodiment, the used capacity of each data storage pool can be intelligently balanced through the storage balancing (cache_balance) module of the first storage cluster. The multiple data storage pools mentioned above may belong to the same second storage cluster or to different second storage clusters; that is, the second storage cluster may include one or more data storage pools.

[0117] Based on the above, the method of this embodiment may further include the following steps:

[0118] In response to multiple data storage pools not meeting the load balancing condition, data migration is performed between multiple data storage pools distributed across multiple secondary storage clusters based on the capacity information of the multiple data storage pools until the load balancing condition is met.

[0119] For detailed implementation methods of the above steps, please refer to [link / reference]. Figure 8 .

[0120] After determining that the balancing conditions are met, the storage balancing module first executes C1 on the key-value database to obtain capacity information. Specifically, it can obtain the capacity information of each cold pool from the key-value database. The capacity information of the cold pool can include the maximum capacity and the used capacity of the cold pool.

[0121] For each cold pool, the storage balancing module obtains the storage level of that cold pool. The storage level of the cold pool can also be called the utilization rate of that cold pool. The storage level (or utilization rate) is equal to the ratio of the used capacity of the cold pool to the maximum capacity of the cold pool. Then, the storage balancing module performs C2 based on the storage levels of all cold pools to calculate the average level. The average level is equal to the average of the storage levels of all cold pools.

[0122] The storage balancing module executes C3, selecting the cold pools with the highest and lowest water levels. In other words, the storage balancing module can identify the cold pool with the highest corresponding water level and the cold pool with the lowest corresponding water level among all cold pools. The cold pool with the highest corresponding water level can be designated as the "migration pool," and the cold pool with the lowest corresponding water level can be designated as the "migration pool." Figure 8 For example, cold pool A with the largest corresponding storage water level is used as the outflow pool, and cold pool B with the smallest corresponding storage water level is used as the inflow pool.

[0123] The storage balancing module calculates the amount of data to be migrated based on the storage level of the outgoing pool, the storage level of the incoming pool, and the average storage level. For the outgoing pool, it calculates the amount of data that needs to be migrated to lower its storage level to the average level, denoted as the first data amount. For the incoming pool, it calculates the amount of data that needs to be migrated to raise its storage level to the average level, denoted as the second data amount. The total amount of data to be migrated is determined based on the first and second data amounts, including selecting the minimum of the first and second data amounts, selecting the maximum of the two, or calculating the average of the two. After determining the amount of data to be migrated, the storage balancing module executes C4 to perform the balancing task.

[0124] In C4, the storage balancing module selects a data block from the outgoing pool, reads the data contained in the selected data block, writes the read data into an idle (currently data-free) data block in the incoming pool, updates the mapping information of the written data block in the first data table of the key-value database, and finally deletes the data originally stored in the selected data block from the outgoing pool. This completes the migration of a data block from the outgoing pool to the incoming pool. This migration process is repeated until the balancing conditions are met based on the amount of migrated data, at which point the migration stops and the balancing task ends.

[0125] During the cyclic migration process, the storage balancing module can select a data block from the migration pool each time according to a preset selection strategy. The selection strategy can be to select the data block that was written to the migration pool earliest, select the data block that has been read the least in the migration pool, select a data block of a specified data type, etc. The specific selection strategy can be set as needed, including but not limited to the selection strategies listed above.

[0126] The beneficial effect of this embodiment is that by performing a balancing task, it can ensure that the storage space utilization rate of each data storage pool is basically the same, avoiding the situation where some data storage pools are overloaded while other data storage pools hardly store any data, thereby improving the storage space utilization efficiency of each data storage pool.

[0127] Optionally, in response to multiple data storage pools not meeting the load balancing criteria, at least one of the following may be included:

[0128] Balance condition one, in response to receiving storage pool information from a new second storage cluster;

[0129] The second equilibrium condition is that the utilization rate difference is greater than a preset second threshold. The utilization rate difference represents the degree of difference in the utilization rates of multiple data storage pools.

[0130] Data migration can be stopped when the equilibrium condition is met, which may include: the difference in usage rate is less than or equal to the second threshold.

[0131] The first balancing condition is equivalent to triggering a balancing task when a new second storage cluster is added to the first storage cluster, migrating the data stored in the original second storage cluster to the newly added second storage cluster, thus preventing the newly added second storage cluster from having no data storage for a long time.

[0132] To detect whether the second balancing condition is met, the storage balancing module can create a timer to periodically detect the storage level of each of the currently managed second storage clusters and calculate the utilization difference based on the storage level of each second storage cluster.

[0133] The utilization rate difference can be equal to the difference between the highest and lowest storage water level in multiple secondary storage clusters; it can also be equal to the standard deviation of the storage water levels in multiple secondary storage clusters; or it can be equal to other statistical indicators that can characterize the degree of difference in utilization rates among multiple data storage pools, without limitation. The second threshold can be preset according to actual conditions, and the specific value is not limited.

[0134] By configuring the above balancing conditions, the storage space utilization rate of each secondary storage cluster can be monitored in real time, and the balancing task can be triggered in a timely manner when the utilization rate difference is too large.

[0135] Optionally, the method of this embodiment can also read data from the second storage cluster and feed it back to the user through the following steps:

[0136] In response to receiving a read request from a client, the system reads the cold data (i.e., the data to be read) corresponding to the read request from the data block of the data storage pool through the read interface provided by the second storage cluster.

[0137] The read cold data is fed back to the client and saved in the data block of the first storage cluster.

[0138] The read interface is provided to the first storage cluster by the second storage cluster, which can be specifically included in the aforementioned storage pool information.

[0139] Please see Figure 9 The front-end read service of the first storage cluster responds to the client's read request and executes step D1, initiating a read request. Then, the front-end read service module executes D2 to read the mapping information of the first data table and D3 to read the metadata of the data block.

[0140] When executing D2 and D3, the front-end read module reads the first data table from the key-value database, and then reads the mapping information recorded in the first data table to determine which data blocks the data to be read, as indicated by the read request, is stored in. Specifically, the read request can carry the identifier of the file to be read, such as the inode corresponding to the file node to be read. The front-end read module can use this identifier to query the mapping information containing this identifier from the first data table, and then query the mapping information to find out which data blocks the data of the file (i.e., the data to be read) is stored in, and then read the metadata of these data blocks (such as the data block identifier slog_id) from the first data table. The data block storing the data to be read can be denoted as the target data block.

[0141] Next, the front-end read module executes D4 to obtain the cold pool and logical storage address of the data block based on its metadata. During D4 execution, the front-end read module queries the mapping relationship recorded in the second data table based on the target data block's metadata obtained in D3. This allows it to determine the cold pool where the target data block resides and the logical storage address of the target data block's data within the cold pool.

[0142] Finally, the front-end read module executes D5 based on the logical storage address, reads data, and executes D6, then feeds the data back to the front end. During the execution of D5 and D6, the front-end read module calls the read interface of the cold pool where the target data block is located based on the logical storage address, thereby reading the data to be read stored in the target data block and feeding back the read data to the client that initiated the read request. At this point, the process of reading the corresponding cold data from the data block in the data storage pool through the read interface provided by the second storage cluster and feeding back the read cold data to the client ends.

[0143] Optionally, when the read data is fed back to the client, it can also be saved in the local data block of the first storage cluster, so that when the client reads the data again, it can be fed back directly from the first storage cluster to the client, thereby improving the efficiency of feeding back data to the client.

[0144] Through the methods described in the above embodiments, the first storage cluster can feed back the cold data stored in the second storage cluster to the client according to the client's needs, thereby meeting the user's needs for persistent storage and querying cold data.

[0145] This embodiment provides a data storage device applied to a first storage cluster, which manages a second storage cluster. The storage rate of the first storage cluster is greater than the storage rate of the second storage cluster. See [link to documentation]. Figure 10 The device includes:

[0146] The creation unit 1001 is used to create data blocks for storing cold data in the data storage pool of the second storage cluster through the creation interface provided by the second storage cluster. The data storage pool is a virtual pool in the second storage cluster that is managed by the first storage cluster.

[0147] Recording unit 1002 is used to record the metadata of the data block and load it into memory for caching; when the preset migration conditions are met, the cold data is migrated to the storage space corresponding to the data block.

[0148] This embodiment also provides a first storage cluster, including a storage array and at least one control node; any control node is used to execute the data storage method provided in any of the foregoing embodiments.

[0149] Storage arrays are used to store data.

[0150] This embodiment also provides a storage system, including a first storage cluster and at least one second storage cluster provided in any embodiment of this application, wherein the first storage cluster is connected to each second storage cluster respectively;

[0151] For any given second storage cluster, the second storage cluster is used to connect all disks through a unified management module to form a data storage pool;

[0152] The second storage cluster is also used to provide a creation interface through a unified management module, which is used to create data blocks in the data storage pool.

[0153] This embodiment provides a computer storage medium for storing computer programs. When the computer program is executed by the processor of the electronic device to which the computer storage medium belongs, it is used to implement the data storage method provided in any embodiment of this application.

[0154] The working principles of the data storage devices, storage clusters, storage systems, and computer storage media described above can all be found in the relevant steps of the data storage method in the foregoing embodiments, and will not be repeated here. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data storage method, characterized in that, Applied to a first storage cluster, which manages a second storage cluster, wherein the storage rate of the first storage cluster is greater than the storage rate of the second storage cluster, the method includes: Through the creation interface provided by the second storage cluster, data blocks for storing cold data are created in the data storage pool of the second storage cluster. The data storage pool is a virtual pool in the second storage cluster that is managed by the first storage cluster. The metadata of the data block is recorded and loaded into memory for caching; when the preset migration conditions are met, the cold data is migrated to the storage space corresponding to the data block.

2. The method according to claim 1, characterized in that, The step of migrating the cold data to the storage space corresponding to the data block includes: Data whose popularity meets the first threshold is identified as cold data, and the cold data is migrated to the storage space corresponding to the data block. The popularity of the data represents the frequency of data access.

3. The method according to claim 1, characterized in that, The metadata of the recorded data block includes: The logical address of the data stored in the data block, and the storage pool to which the data block belongs, are recorded as the metadata of the data block.

4. The method according to claim 2, characterized in that, The step of migrating the cold data to the storage space corresponding to the data block includes: The cold data stored in the first storage cluster is written to the storage space corresponding to the created data block through the write interface provided by the second storage cluster. Delete the cold data written to the second storage cluster from the storage space of the data block in the first storage cluster.

5. The method according to claim 1, characterized in that, The second storage cluster includes multiple data storage pools; The method further includes: In response to multiple data storage pools not meeting the balance condition, data migration is performed among multiple data storage pools distributed across multiple second storage clusters based on the capacity information of the multiple data storage pools until the balance condition is met.

6. The method according to claim 1, characterized in that, Also includes: In response to receiving a read request from a client, the cold data corresponding to the read request is read from the data block of the data storage pool through the read interface provided by the second storage cluster; The read cold data is fed back to the client and saved in the data block of the first storage cluster.

7. A data storage device, characterized in that, The device, applied to a first storage cluster that manages a second storage cluster, wherein the storage rate of the first storage cluster is greater than the storage rate of the second storage cluster, comprises: A creation unit is used to create data blocks for storing cold data in the data storage pool of the second storage cluster through the creation interface provided by the second storage cluster. The data storage pool is a virtual pool in the second storage cluster that is managed by the first storage cluster. The recording unit is used to record the metadata of the data block and load it into memory for caching; when the preset migration conditions are met, the cold data is migrated to the storage space corresponding to the data block.

8. A first storage cluster, characterized in that, It includes a storage array and at least one control node; any one of the control nodes is used to execute the data storage method according to any one of claims 1 to 6; The storage array is used to store data.

9. A storage system, characterized in that, It includes the first storage cluster as described in claim 8 and at least one second storage cluster, wherein the first storage cluster is connected to each of the second storage clusters respectively; For any second storage cluster, the second storage cluster is used to connect all disks through a unified management module to form a data storage pool; The second storage cluster is also used to provide a creation interface through a unified management module, which is used to create data blocks in the data storage pool.

10. A computer storage medium, characterized in that, Used to store a computer program, which, when executed by a processor of an electronic device to which the computer storage medium belongs, is used to implement the data storage method according to any one of claims 1 to 6.

Images