Data storage method and computing device

Abstract

A data storage method and a computing device, the method is applied to a storage device, the storage device comprises a first storage pool and a second storage pool, the first storage pool comprises a first solid state disk and a mechanical hard disk, and the second storage pool comprises a shared storage area. The shared storage area is used for providing an extended storage space for the first storage pool; the method comprises the following steps: receiving a first data writing request; under the condition that the data volume of the first data is smaller than or equal to the available capacity of the first solid state disk of the first storage pool, writing the first data into the first solid state disk of the first storage pool; and when the data volume of the first data is greater than the available capacity of the first solid state disk of the first storage pool and is less than or equal to the available capacity of the shared storage area of the second storage pool, writing the first data into the shared storage area of the second solid state disk of the second storage pool.

Description

Technical Field

[0001] The present invention relates to the field of servers, and in particular to a data storage method and computing equipment. Background Art

[0002] In the server field, distributed storage is a data storage technology that combines storage resources scattered on multiple servers into a virtual storage device, which can then be used by applications over the network. The virtual storage device can also be called a storage pool. Storage resources on servers are generally divided into two types of storage devices: hard disk drives (HDD) and solid state disks / flash drives (SSD). Depending on the hardware that constitutes the storage resources (such as hard disks and flash drives), distributed storage is divided into hybrid storage and all-flash storage.

[0003] Hybrid flash storage is a type of storage pool supported by distributed storage. It combines the advantages of solid-state drives / flash drives (SSDs) and traditional mechanical hard disks (HDDs) to give full play to the advantages of both. Specifically, the hybrid flash storage architecture includes a cache tier and a backing storage tier. The cache tier is a storage pool created using a relatively fast I / O interface, such as a solid-state drive, which is used to accelerate access to hot data (frequently accessed data); the backing storage tier is a storage pool created using a relatively slow I / O interface, such as a mechanical hard disk, which is used to store cold data (infrequently accessed data). In addition, all-flash storage, as a type of storage pool supported by distributed storage, only uses SSDs as main storage and does not require cache. In other words, in all-flash storage, all data is stored in solid-state drives and no longer relies on mechanical hard disks. Although business data is directly written to SSDs, the performance is high, but all-flash storage has small capacity and high cost.

[0004] In hybrid flash storage, when the SSD cache reaches the threshold, it is necessary to flush the disk and write the data to the HDD main memory to ensure that there is enough space in the SSD cache. Therefore, if large amounts of data are read and written, there will be frequent flushing of the disk. Since the business cannot be distinguished during the flushing, the business I / O performance is affected. Summary of the invention

[0005] In order to solve the above-mentioned problems, a data storage method and a computing device are provided in an embodiment of the present application, which adopt two storage methods: a hybrid storage pool of distributed storage and an all-flash storage pool. Part of the capacity of the solid-state hard disk in the all-flash storage pool is planned as dynamic shared capacity as a cache disk of the hybrid storage pool to reduce the number of disk refreshes of the solid-state hard disk in the hybrid storage pool, take into account the available capacity of the distributed storage, and improve data I / O performance.

[0006] To this end, the following technical solutions are adopted in the embodiments of the present application:

[0007] In a first aspect, the present application provides a data storage method, which is applied to a storage device, wherein the storage device includes a first storage pool and a second storage pool, wherein the first storage pool includes a first solid-state hard disk and a mechanical hard disk, the second storage pool includes a second solid-state hard disk, and the second solid-state hard disk includes a shared storage area, and the shared storage area is used to provide extended storage space for the first storage pool; the method includes: receiving a first write data request, the first write data request is used to write first data to the first storage pool; when the data amount of the first data is less than or equal to the available capacity of the first solid-state hard disk of the first storage pool, writing the first data to the first solid-state hard disk of the first storage pool; when the data amount of the first data is greater than the available capacity of the first solid-state hard disk of the first storage pool, and is less than or equal to the available capacity of the shared storage area of ​​the second storage pool, writing the first data to the shared storage area of ​​the second solid-state hard disk of the second storage pool.

[0008] In this implementation, part of the capacity of the solid-state hard disk in the second storage pool (all-flash storage pool) is planned as dynamic shared capacity as the cache disk of the first storage pool (hybrid storage pool), making full use of the high-speed reading and writing capabilities of the solid-state hard disk. At the same time, the cache area can still be used as the available storage capacity of the all-flash storage pool, thereby avoiding the problem of small all-flash storage capacity and high cost caused by shared capacity. By using part of the space of the all-flash storage pool as the cache area in the hybrid storage pool, even when the solid-state hard disk of the first storage pool in the hybrid storage pool is exhausted, the cache area can still be used for data storage, thereby maintaining a high data access speed and responsiveness. At the same time, the frequent migration operations (flush disk) of cached data in the solid-state hard disk of the first storage pool to the mechanical hard disk of the first storage pool are delayed or reduced, thereby reducing the impact on business I / O performance and improving the stability and performance of the storage device.

[0009] As an achievable implementation, the second solid-state hard disk further includes a reserved storage area, and the method further includes: receiving a second write data request, the second write data request being used to write the second data into the second storage pool; when the data amount of the second data is less than or equal to the available capacity of the shared storage area of ​​the second storage pool, writing the second data into the shared storage area of ​​the second solid-state hard disk of the second storage pool; when the data amount of the second data is greater than the available capacity of the shared storage area of ​​the second storage pool and is less than or equal to the available capacity of the reserved storage area of ​​the second solid-state hard disk in the second storage pool, writing the second data into

[0010] In this embodiment, the storage area of ​​the second storage pool excluding the shared storage area is used as the reserved storage area of ​​the second storage pool, ensuring that the second storage pool has a large available capacity and ensuring that the hybrid storage pool will not use this part of the capacity. The purpose of this is to ensure that most of the solid-state hard disks can be included in the second storage pool, and then there is enough space to cache the data stored in the second storage pool, thereby ensuring the reading performance. In this way, the data is directly written into the all-flash storage pool by utilizing the fast writing speed and high performance of the all-flash storage pool, so as to improve the speed and efficiency of data writing.

[0011] As an achievable implementation, the method also includes: when the data volume of the second data is greater than the available capacity of the shared storage area and the reserved storage area in the second solid-state hard disk, migrating the third data in the shared storage area to the mechanical hard disk of the first storage pool so that the available capacity of the shared storage area is greater than or equal to the data volume of the second data; wherein the third data is the first data written into the shared storage area of ​​the second solid-state hard disk in the second storage pool when the storage device receives the first write data request because the available capacity of the first solid-state hard disk in the first storage pool is less than the data volume of the first data; and writing the second data into the shared storage area.

[0012] In this implementation, when the available capacity space in the shared storage area cannot write the second data to be written, it means that the capacity of the shared storage area stores too much third data from the first storage pool, and it is necessary to flush these third data to the mechanical hard disk of the first storage pool of the hybrid storage pool, and release space in time to facilitate writing the second data to the shared storage area of ​​the second storage pool. In this way, the resource utilization of the storage device is optimized, and the data writing operation can be effectively processed, and the second data is written to the shared storage area to ensure the smooth writing of data and avoid data loss, thereby ensuring data security and storage efficiency.

[0013] As an achievable implementation, before writing the first data to the shared storage area of ​​the second solid-state hard disk in the second storage pool, the method also includes: when the data volume of the first data is larger than the shared storage area, migrating the third data in the shared storage area to the mechanical hard disk in the first storage pool; wherein the third data is the first data written to the shared storage area of ​​the second solid-state hard disk in the second storage pool when the storage device receives the first write data request because the available capacity of the first solid-state hard disk in the first storage pool is less than the data volume of the first data.

[0014] As an achievable implementation method, after migrating the third data in the shared storage area to the mechanical hard disk of the first storage pool, the method also includes: determining whether the available capacity of the shared storage area is greater than or equal to the data volume of the first data; if the available capacity of the shared storage area is less than the data volume of the first data, migrating the data in the first solid-state hard disk in the first storage pool to the mechanical hard disk of the first storage pool so that the available capacity of the first solid-state hard disk is greater than or equal to the data volume of the first data; and writing the first data to the first solid-state hard disk in the first storage pool.

[0015] In this embodiment, when the shared storage area still cannot write the first data after the disk is flushed, it means that the second data stored in the shared storage area of ​​the second storage pool is large, and there is no extra free available capacity to share with the first cache pool as a cache space to store the first data. At this time, the first solid-state hard disk of the first storage pool is triggered to flush the disk, and the data in the first solid-state hard disk in the first storage pool is flushed to the mechanical hard disk HDD of the first storage pool, and finally the first data is written to the first solid-state hard disk of the first storage pool. In this way, by recycling the capacity space of the first solid-state hard disk of the first storage pool and the shared storage area of ​​the second storage pool, that is, when the shared storage area and the first solid-state hard disk of the first storage pool cannot write data, the data in the first solid-state hard disk of the first storage pool is flushed to the mechanical hard disk of the first storage pool, and finally the first data is written to the solid-state hard disk of the first storage pool, which further improves the storage efficiency and performance stability of the data. This method not only ensures the security of the data, but also maximizes the advantages of all available capacity of the all-flash storage pool and the hybrid storage pool, and provides better support for the reading and writing of business data. Moreover, according to the performance characteristics of different storage devices, the data writing path is flexibly allocated to fully utilize the high-speed writing performance of the first solid-state hard disk of the first storage pool and the shared storage area of ​​the second storage pool, thereby improving the utilization rate of storage resources.

[0016] As an achievable implementation, the method also includes: when the capacity usage rate of the first solid-state hard disk in the first storage pool reaches or exceeds a first preset water level, migrating the data of the first solid-state hard disk in the first storage pool to the mechanical hard disk in the first storage pool.

[0017] In this embodiment, the proportion of the used capacity of the first solid-state hard disk of the first storage pool can be periodically obtained based on a preset time. In this way, by periodically obtaining the proportion of the used capacity of the solid-state hard disk of the first storage pool, the capacity usage of the first solid-state hard disk of the first storage pool as a cache disk can be monitored in real time, and the situation of excessive capacity occupancy can be discovered in time. When the proportion of the used capacity of the first solid-state hard disk of the first storage pool reaches the set water level, the stored data is automatically flushed to the mechanical hard disk of the first storage pool to release space, thereby avoiding excessive occupancy of the cache disk and causing a decrease in write performance. By regularly flushing the disk, it is ensured that the capacity of the first solid-state hard disk of the first storage pool is always within a controllable range, avoiding data loss or write delays, and improving the stability and reliability of data storage. Moreover, regular flushing operations can optimize the utilization of storage resources, avoid waste of resources and unnecessary occupation, and improve the overall efficiency of storage devices.

[0018] As an achievable implementation, the method further includes: when the capacity usage rate of the shared storage area of ​​the second solid-state hard disk in the second storage pool reaches or exceeds the second preset watermark, migrating the third data stored in the shared storage area of ​​the second solid-state hard disk in the second storage pool to the mechanical hard disk in the first storage pool. The third data is the first data written into the shared storage area of ​​the second solid-state hard disk in the second storage pool by the storage device when receiving the first write data request because the available capacity of the first solid-state hard disk in the first storage pool is less than the data volume of the first data.

[0019] In this implementation, the proportion of the used capacity of the shared storage area can be periodically obtained based on a preset time, and when the proportion of the used capacity of the shared storage area exceeds a second preset watermark, the third data in the shared storage area is flushed to the mechanical hard disk of the first storage pool. Through regular monitoring and automatic flushing operations, effective management of the capacity of the shared storage area is achieved, the stability and performance of data storage are improved, and the utilization of storage resources is optimized, which helps to improve the overall performance and reliability of the distributed storage system.

[0020] As an achievable implementation manner, writing the first data into the shared storage area of ​​the second solid state drive in the second storage pool includes: marking the first data written into the shared storage area in the second storage pool.

[0021] In this implementation, a mark is added to the first data to distinguish the first data and the second data stored in the cache area of ​​the second storage pool, so as to facilitate data volume statistics and disk flushing.

[0022] In a second aspect, an embodiment of the present application also provides a storage device, comprising: a first storage pool and a second storage pool, wherein the first storage pool comprises a first solid-state drive and a mechanical hard disk, the second storage pool comprises a second solid-state drive, the second solid-state drive comprises a shared storage area, and the shared storage area is used to provide expanded storage space for the first storage pool.

[0023] In a third aspect, an embodiment of the present application further provides a computing device, which is used for data storage and includes: a receiving unit and a processing unit. The receiving unit is used to receive a first write data request, and the first write data request is used to write the first data into the first storage pool. The processing unit is used to write the first data into the first solid-state hard disk of the first storage pool when the data volume of the first data is less than or equal to the available capacity of the first solid-state hard disk of the first storage pool; and write the first data into the shared storage area of ​​the second solid-state hard disk of the second storage pool when the data volume of the first data is greater than the available capacity of the first solid-state hard disk of the first storage pool and is less than or equal to the available capacity of the shared storage area of ​​the second storage pool.

[0024] In a fourth aspect, the present application also provides a storage processing device, including: an initialization configuration module, a data IO module and a disk brushing module. Among them, the initialization configuration module is used to configure the first solid-state hard disk of the hybrid storage pool and the mechanical hard disk of the first storage pool, configure the second solid-state hard disk of the all-flash storage pool, and divide the reserved storage area and the shared storage area from the second solid-state hard disk. The data IO module is used to record the cache record table of the hybrid storage pool data, and record the specific location of the data cached in the all-flash storage pool. The data IO module is also used to determine the processing flow according to the available capacity of the hybrid storage pool cache and the available capacity of the all-flash storage pool when writing data. The disk brushing module is used to trigger the disk brushing when writing data, if the available cache space is insufficient, and brush the data cached in the first solid-state hard disk of the first storage pool or the shared storage area to the mechanical hard disk of the first storage pool, and then continue to write the data to the solid-state hard disk of the first storage pool or the shared storage area. The disk brushing module is also used for routine disk brushing. Specifically, the routine inspection task is started to periodically check whether the available cache space is sufficient; if not, the disk is brushed.

[0025] In a fifth aspect, an embodiment of the present application further provides a computing device, comprising: a memory and a processor; the memory and the processor are coupled; the memory is used to store programs; and the processor is used to call the programs stored in the memory to execute the method of the first aspect.

[0026] In a sixth aspect, an embodiment of the present application provides a storage medium, in which program instructions are stored, and when the program instructions are executed, the data storage method involved in the first aspect and any possible implementation thereof is executed. The beneficial effects thereof are as described in the first aspect and will not be repeated here.

[0027] In a seventh aspect, an embodiment of the present application provides a computer program product including program instructions, which, when the program instructions are executed, executes the data storage method involved in the first aspect and any implementable manner thereof. Its beneficial effects are as described in the first aspect and will not be repeated here.

[0028] As mentioned above, the data storage method and computing device provided by the present invention divide the free available capacity of the all-flash storage pool into two parts: reserved capacity (reserved storage area) and dynamic shared capacity (shared storage area): This data storage method realizes intelligent allocation and management of data between the hybrid storage pool and the all-flash storage pool, and improves the performance and stability of the storage device by rationally utilizing the characteristics of the solid-state drive and the mechanical hard disk, and reducing the number of disk refreshes, thereby providing better support for the reading and writing of business data. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In each of the drawings, the same elements are represented by similar reference numerals. For the sake of clarity, the various parts in the drawings are not drawn to scale, and some features may be exaggerated or omitted to more clearly illustrate and explain the present application.

[0030] Figure 1 An architectural diagram of a distributed storage system provided in an embodiment of the present application;

[0031] Figure 2 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0032] Figure 3 A flow chart of a data storage method provided by an embodiment of the present application is shown;

[0033] Figure 4 A flowchart of another data storage method provided by an embodiment of the present application is shown;

[0034] Figure 5 A flowchart of another data storage method provided by an embodiment of the present application is shown;

[0035] Figure 6 A flowchart of another data storage method provided in an embodiment of the present application is shown;

[0036] Figure 7 A flowchart of a data storage method provided in Example 1 of the present application is shown;

[0037] Figure 8A server provided in an embodiment of the present application;

[0038] Fig. 9 A schematic diagram of the structure of a storage and processing device provided in an embodiment of the present application is shown;

[0039] Fig.10 A schematic diagram of the structure of a computing device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0041] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0042] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple terminals refers to two or more terminals.

[0043] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0044] In the description of the embodiments of the present application, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0045] In the description of the embodiments of the present application, the terms "first\second\third, etc." or module A, module B, module C, etc. are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0046] In the description of the embodiments of the present application, the numbers representing the steps, such as S31, S32, etc., do not necessarily mean that the steps will be executed in this manner. If permitted, the order of the previous and next steps can be interchanged, or they can be executed simultaneously.

[0047] Relevant terms involved in the embodiments of this application:

[0048] Hard disk drive (HDD): is a traditional computer storage device. A mechanical hard disk is usually connected to the computer's motherboard or management server card through a SATA (serial advanced technology attachment) interface or a SAS (serial attached SCSI, full name serial attached small computer system interface, serially connected small computer system interface) interface. Its working principle is based on rotating disks and moving magnetic heads. Specifically, a mechanical hard disk consists of one or more metal or glass disks that are driven by an electric motor when rotating at high speeds. The surface of each disk is divided into one or more magnetic areas for storing data. Data is read and written through magnetic heads, which can move on the surface of the disk and position themselves on different tracks as needed to read or write data.

[0049] Solid state disk (SSD): A computer storage device based on solid-state memory technology that stores data in electronic memory (such as flash memory chips) rather than traditional mechanical parts (such as rotating platters and moving heads). SSDs work by storing data in non-volatile memory, which is usually based on flash memory technology, such as NAND flash or 3D NAND flash. These memory chips can read and write data directly without mechanical movement, so they have faster data access speeds and lower access latency, which is better than traditional mechanical hard disks (HDDs). SSDs are usually connected to computer systems through SATA interfaces, PCIe (peripheral component interconnect express) interfaces, or the latest NVMe (non-volatile memory express) interfaces. They have higher read and write speeds, lower energy consumption, and better shock resistance, so they are widely used in application scenarios that require high performance and high reliability, such as operating system boot disks, game loading, big data analysis, cloud computing services, etc.

[0050] A distributed storage system is a system designed to store and manage data on multiple independent computing devices (commonly referred to as nodes). These computing devices are connected through a network and work together to provide a unified pool of storage resources that can be extended across multiple physical locations. Compared with traditional centralized storage, distributed storage distributes data across multiple nodes, each of which can store and access data independently. This distributed storage method can provide higher reliability, scalability, and performance. In a distributed storage system, data is usually divided into multiple blocks or objects and distributed across multiple storage nodes.

[0051] Storage pool: Also known as logical pool, in the storage system, logical pool is an abstract layer that combines multiple storage resources (such as hard disks, SSDs, storage volumes, etc.). Storage pool aggregates the capacity of physical storage resources to achieve more flexible storage management.

[0052] Cache tier: A cache tier is a storage device that can exchange data at high speed and is used to cache data. For example, the cache tier can be a storage device composed of a solid state disk (SSD).

[0053] Backing storage tier: For distributed storage systems with tiered storage strategies, cache disks provide better I / O performance for clients, while backing storage tiers are used to store inactive data. Backing storage tiers are usually made of relatively economical hard disk drives (HDDs), but in some tiered storage strategies, backing storage tiers can also be made of solid state disks (SSDs).

[0054] Disk flushing: When the capacity usage of the cache disk reaches a certain percentage, it will trigger the migration of cache disk data to the main storage disk, which is called disk flushing.

[0055] Water level: The storage space of a mechanical hard drive or solid-state drive.

[0056] Water level, the ratio of cache disk used capacity when disk flushing is triggered.

[0057] Volume: refers to a logical storage space unit, similar to a virtual hard disk or partition. Volumes are usually created by storage administrators or system administrators on physical storage resources and can be allocated to different applications or users to store data.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0059] With the continuous development of information technology, there are more and more types of client applications. Among them, some applications require high-performance all-flash storage, and some applications require large-capacity hybrid storage. In the hybrid flash storage supported by the distributed storage system, application data is not directly written to the HDD main memory, but written to the SSD cache to maximize IO performance. When the SSD cache reaches the threshold, it is necessary to flush the disk and write the data to the HDD main memory to ensure that there is enough space in the SSD cache. If the SSD cache is too small, it will cause frequent flushing, which will affect IO performance. In the all-flash storage supported by distributed storage, the SSD is used as the main storage disk and does not require a cache disk. Data is written directly to the SSD, which has high performance, but high cost and small capacity.

[0060] Based on this, an embodiment of the present application provides a data storage method and a computing device, which adopt two storage methods: a hybrid storage pool and an all-flash storage pool, and use part of the capacity of the solid-state hard disk in the all-flash storage pool as a cache disk of the hybrid storage pool to reduce the number of disk refreshes and improve business data I / O performance.

[0061] In order to more completely understand the present application, the following examples are given. These examples are used to specifically illustrate the embodiments of the present application, and should not be understood in any way as limiting the scope of the present application.

[0062] Figure 1 This is an architecture diagram of a distributed storage system provided in an embodiment of the present application. Figure 1 As shown, the distributed storage system 100 includes multiple nodes. Each node in the distributed storage system 100 may be one or more servers, specifically, a blade server, a high-density server, a rack server, a cabinet server, and the like. It should be understood that different nodes may be divided according to functions in different distributed storage systems 100. For example, in some embodiments, the nodes in the distributed storage system 100 may include two types, namely, a management node 101 and multiple storage nodes 102.

[0063] Among them, the storage node 102 is mainly used to store data, execute data operation instructions (such as data query, data modification, data writing, etc.) issued by the management node 101, and return the execution result to the management node 101. The management node 101 is mainly used to manage and monitor each storage node 102, and can issue data operation instructions to each storage node 102, and can also monitor the resource usage of each storage node 102 (such as disk IO resources, disk remaining storage capacity), and can also maintain the entire distributed storage system 100 to keep it in a healthy state. In some embodiments, the management node 101 can serve as the communication entrance of the entire distributed storage system 100, and all operations on data (such as data writing, deletion, modification, query, etc.) can be completed through the management node 101. It should be noted that the data storage method provided in the embodiment of the present application can be executed by the management node 101.

[0064] Exemplarily, the management node 101 and the storage node 102 may transmit data via the NFS (NetFileSystem) / CIFS (Common Internet File System) protocol in the network; or they may be connected based on the secure shell protocol (SSH); specifically, the embodiments of the present application do not limit the communication method between the management node 101 and the storage node 102.

[0065] In a specific embodiment, the management node 101 may include a distributed storage engine, which is mainly used to receive a write data request sent from the client 103, and process the write data request to confirm that the write data request is used to indicate that data is written to any of the above storage nodes 102.

[0066] Exemplarily, the client 103 may be various electronic devices having a display screen and supporting application operations, including but not limited to terminal devices such as tablet computers, laptops, and desktop computers. It should be noted that the client 103 may be installed with various desktop applications, mobile applications, web applications, or web-based applications, etc. For example, the application may be any one or more of a virtualization platform, a database, a K8S container cloud, a streaming media application, and a backup archive, which are not limited here.

[0067] The following takes the storage node 102 as a single server as an example to introduce the storage device of the storage node.

[0068] Figure 2 FIG. 1 is a schematic diagram showing the structure of a computing device provided in an embodiment of the present application. The computing device may be the management node 101 or the storage node 102. Figure 2 As shown, the computing device as a storage node 102 can be a computing device with storage capabilities, such as a server, wherein one storage node 102 can include a first storage pool 21 and a second storage pool 22. Exemplarily, the first storage pool 21 includes a first solid-state drive 211 and a mechanical hard disk 212; the second storage pool 22 includes a second solid-state drive, and the second solid-state drive includes a shared storage area 221 and a reserved storage area 222; the shared storage area 221 is used for the second storage pool 22 to store its own data, and can also be used to provide extended storage space for the first storage pool 21. Of course, the above-mentioned first storage pool 21 and the second storage pool 22 may not be deployed in one storage node, but may be deployed on multiple different storage nodes, which is not specifically limited here.

[0069] In some embodiments, the available capacity of the second storage pool 22 is divided into a shared storage area 221 and a reserved storage area 222 according to a preset ratio. The shared storage area 221 may include a continuous address interval or the sum of multiple discontinuous address intervals; a mapping relationship between the first solid-state drive 211 and the shared storage area 221 is established, so that the shared storage area 221 can be used as a cache disk of the first storage pool 21. Exemplarily, the first solid-state drive of the first storage pool can be associated with the shared storage area of ​​the second storage pool by creating a mapping rule, for example, by configuring a mapping table between the first solid-state drive and the shared storage area.

[0070] In this way, the shared storage area 221, as a dynamic shared capacity, can be used as the main memory capacity of the second storage pool 22 and as the cache capacity of the first storage pool 21, thereby realizing capacity sharing of the SSD. The reserved storage area 222 may include a continuous address interval or the sum of multiple address intervals. It is worth mentioning that the remaining capacity of the second storage pool 22 after removing the shared storage area 221 is the reserved storage area 222, and the reserved storage area 222 is not available to the first storage pool 21, so as to ensure that the available capacity of the second storage pool 22 itself is sufficient.

[0071] In some embodiments, the capacity of the shared storage area 221 may be a preset ratio of 30% to 80% of the total capacity of the second storage pool 22. In a specific embodiment, the preset ratio may also be 80%, 70%, 60%, 50%, 40%, or 30%, and this application does not impose any specific restrictions thereto. For example, assuming that the available capacity of the second storage pool 22 is 100G, the capacity of the shared storage area 221 is set to 80G according to the preset ratio of 80%, which is used as a shared capacity for storing cache data from the first storage pool 21 in addition to the second storage pool 22 storing its own data. The capacity of the reserved storage area 222 is the remaining capacity after deducting the capacity of the shared storage area 221 from the total capacity of the second storage pool 22, which is 20G.

[0072] In another achievable implementation, the first solid state drive 211 and the mechanical hard disk 212 in the first storage pool 21 can be distributed in multiple storage nodes or one storage node; similarly, the shared storage area 221 and the reserved storage area 222 included in the second solid state drive in the second storage pool 22 can all be deployed in one storage node or deployed separately on multiple storage nodes. Exemplarily, it is assumed that the distributed storage system 100 has 20 storage nodes 102, such as node 1-node 20. Each storage node 102 includes 4 hard disks, of which three hard disks can be mechanical hard disks (hard diskdrive, HDD), and the other hard disk can be a solid state disk (solid state disk, SSD). At this time, two storage pools can be configured, for example, the first storage pool 21 and the second storage pool 22. Specifically, the first storage pool 21 can include the storage space corresponding to the mechanical hard disk of node 1-node 20 as its own main storage disk, and the storage space corresponding to the solid state drive of node 1-node 2 as its own cache disk. The second storage pool 22 may include the storage space corresponding to the solid-state hard disks of nodes 3-node 20 as a shared storage area 221 and a reserved storage area 222; of course, the second storage pool 22 may also be deployed in a storage node, which includes a second solid-state hard disk, and the second solid-state hard disk is divided into a shared storage area 221 and a reserved storage area 222.

[0073] The above is an introduction to the functional architecture of the distributed storage system provided by the embodiment of the present application. Next, based on the above content, the data storage method provided by the embodiment of the present application is introduced. It should be noted that the data storage method provided by the embodiment of the present application can generally be executed by the management node 101. Among them, the relevant introduction of the concepts, modules, etc. involved in the following content can be found above.

[0074] Figure 3 A flow chart of a data storage method provided by an embodiment of the present application is shown. Figure 4 As shown, the data storage method in this embodiment is applied to a storage device, the storage device includes a first storage pool and a second storage pool, wherein the first storage pool includes a first solid-state hard disk and a mechanical hard disk, the second storage pool includes a second solid-state hard disk, the second solid-state hard disk includes a shared storage area, and the shared storage area is used to provide an extended storage space for the first storage pool. The method includes the following steps:

[0075] S301: Receive a first data write request, where the first data write request is used to write first data into a first storage pool.

[0076] In this step, you can Figure 1 The client 103 shown in FIG. 1 sends a first write data request, and receives the first write data request through the distributed storage engine of the management node 101 .

[0077] Exemplarily, the first data may be ordinary business data, such as configuration files, snapshots, log files, monitoring data, etc. of virtual machines generated in a virtualization platform; or, log files, backup files, monitoring and performance data, configuration files, etc. generated by a database; or, container logs, configuration files, monitoring data, event logs, etc. generated by a K8S container cloud; log files, user behavior data, monitoring data, configuration files, etc. generated by streaming media applications; and logs of backup tasks, backup policy configurations, monitoring data, etc. generated by backup archiving.

[0078] In a specific implementation, the first data may also be core business data, such as disk images of virtual machines generated in a virtualization platform, operating system files, application data, user data, etc.; or, actual data records generated by a database, such as customer information, transaction records, product catalogs, order data, etc.; or, container images generated by a K8S container cloud, data in persistent storage volumes, application status data, user data, etc.; or, video and audio content, live streaming data, user-generated content, copyrighted content, etc. generated by streaming media applications; or, actual backup data generated by backup archiving, including database backup, file system backup, virtual machine backup, application data backup, etc. This application does not limit the specific type of the first data, and only needs to confirm that the first data is to be written to the first storage pool.

[0079] Specifically, in this step, the business type of the data to be written can be obtained through the business tag. It should be understood that the first write data request can carry a business tag, which is used to identify the business type of the data (such as ordinary business data or core business data) and the storage pool to be written (such as the first storage pool or the second storage pool). In this embodiment, the business tag can be identified by numbers or letters, or a combination of numbers and letters, for example, "0" represents ordinary business data and "1" represents core business data.

[0080] S302: Determine whether the data volume of the first data is less than or equal to the available capacity of the first solid-state hard disk in the first storage pool. In one scenario, if the data volume of the first data is less than or equal to the available capacity of the first solid-state hard disk in the first storage pool, execute S303. If the data volume of the first data is greater than the available capacity of the solid-state hard disk in the first storage pool, execute S304.

[0081] S303: When the data volume of the first data is less than or equal to the available capacity of the first solid-state hard disk in the first storage pool, write the first data into the first solid-state hard disk in the first storage pool.

[0082] For example, the data size of the first data is 1GB, and the available capacity of the first solid-state hard disk in the first storage pool is 2GB. At this time, the available capacity of the first solid-state hard disk in the first storage pool is sufficient to accommodate the writing of the first data, so the first data is written to the first solid-state hard disk in the first storage pool.

[0083] S304. When the data volume of the first data is greater than the available capacity of the first solid-state hard disk of the first storage pool and is less than or equal to the available capacity of the shared storage area of ​​the second storage pool, write the first data to the shared storage area of ​​the second solid-state hard disk of the second storage pool.

[0084] For example, the size of the first data is 3GB, the available capacity of the first solid-state drive in the first storage pool is 2GB, and the available capacity of the shared storage area of ​​the second solid-state drive in the second storage pool is 80GB. At this time, the available capacity of the first solid-state drive in the first storage pool is insufficient to accommodate the writing of the first data, while the shared storage area of ​​the second solid-state drive in the second storage pool is sufficient to accommodate the writing of the first data, and the first data is written to the shared storage area of ​​the second solid-state drive in the second storage pool.

[0085] In one embodiment, Figure 4 As shown, a data storage method provided in an embodiment of the present application may further include: S401-S406. Exemplarily, S401-S406 may be performed after step S302 and before step S304.

[0086] S401, determine whether the data volume of the first data is less than or equal to the available capacity of the shared storage area of ​​the second storage pool. In one case, if the data volume of the first data is less than or equal to the available capacity of the shared storage area of ​​the second storage pool, execute S402. In another case, if the data volume of the first data is greater than the available capacity of the shared storage area of ​​the second storage pool, execute S403.

[0087] S402: When the data volume of the first data is less than or equal to the available capacity of the shared storage area of ​​the second storage pool, write the first data into the shared storage area of ​​the second solid state drive of the second storage pool.

[0088] For example, the size of the first data is 3GB, the available capacity of the shared storage area is 5GB, and the available capacity of the shared storage area is sufficient to write 3GB of the first data, so the first data is written to the shared storage area of ​​the second solid state drive in the second storage pool.

[0089] S403: When the amount of the first data is greater than the available capacity of the shared storage area of ​​the second storage pool, migrate the third data in the shared storage area to the mechanical hard disk of the first storage pool. The third data is the first data written into the shared storage area of ​​the second solid-state hard disk in the second storage pool when the storage device receives the first data write request because the available capacity of the first solid-state hard disk in the first storage pool is less than the amount of the first data.

[0090] In other words, when the available capacity of the shared storage area of ​​the second solid state drive in the second storage pool is insufficient, the first data previously stored in the shared storage area is migrated to the mechanical hard disk in the first storage pool.

[0091] For example, the size of the first data is 3GB, and the available capacity of the shared storage area is 2GB. The available capacity of the shared storage area is insufficient to write 3GB of the first data, triggering a disk flush of the shared storage area. Suppose that 30GB of the used capacity of the shared storage area is the third data (the first data previously stored in the shared storage area). The disk flushing operation is specifically as follows: copying the 30GB third data from the shared storage area to the mechanical hard disk of the first storage pool, and deleting the corresponding data in the shared storage area to release the used capacity of the shared storage area. It should be understood that the used capacity is the capacity in the second storage pool after being occupied by data.

[0092] S404, again determine whether the data volume of the first data is less than or equal to the available capacity of the shared storage area of ​​the second storage pool. In one case, if the data volume of the first data is less than or equal to the available capacity of the shared storage area of ​​the second storage pool, then execute S402. In another case, if the data volume of the first data is still greater than the available capacity of the shared storage area of ​​the second storage pool, then execute S405.

[0093] S405. When the data volume of the first data is greater than the available capacity of the shared storage area of ​​the second storage pool, migrate the data stored in the first solid-state hard disk in the first storage pool to the mechanical hard disk of the first storage pool, so that the available capacity of the first solid-state hard disk is greater than or equal to the data volume of the first data.

[0094] In this step, after the shared storage area is flushed, its own available capacity is still insufficient to accommodate the writing of the first data, indicating that the shared storage area no longer has sufficient storage space for the writing of the first data. For example, all the space in the shared storage area is occupied, and 1GB of the used capacity of the shared storage area is the first data previously stored. If the 1GB of first data is migrated to the mechanical hard disk of the first storage pool, the released 1GB of capacity space is still unable to write the 2GB of first data to be stored this time. At this time, the data stored in the first solid-state hard disk in the first storage pool is migrated to the mechanical hard disk of the first storage pool.

[0095] S406: Write the first data into the first solid state drive of the first storage pool.

[0096] For example, after the solid-state hard disk in the first storage pool is flushed, its own 20 GB available capacity is released, and then the first data of 2 GB can be written to the first solid-state hard disk in the first storage pool as a cache disk.

[0097] In one embodiment, Figure 5As shown, a data storage method provided by an embodiment of the present application may further include: S501-S506. Exemplarily, S501-S506 may be executed synchronously with S301-S304 or before S301 or after S304.

[0098] S501: Receive a second data write request, where the second data write request is used to write second data into a second storage pool.

[0099] Similarly, in this step, you can Figure 1 The client shown in sends a second write data request, and the second write data request is received by the distributed storage engine in the management node. Exemplarily, the type of the second data is the same as the first data. Through the identifier carried in the second write data request, it can be determined that the second write data request is a request to write the second data into the second storage pool. For details, please refer to step S301, and this embodiment will not be repeated here.

[0100] S502, determine whether the data volume of the second data is less than or equal to the available capacity of the shared storage area of ​​the second storage pool. In one case, if the data volume of the second data is less than or equal to the available capacity of the shared storage area of ​​the second storage pool, then execute step S503. In another case, if the data volume of the second data is greater than the available capacity of the shared storage area of ​​the second storage pool, then execute step 504.

[0101] S503: When the amount of the second data is less than or equal to the available capacity of the shared storage area of ​​the second storage pool, write the second data to the shared storage area of ​​the second solid state drive of the second storage pool.

[0102] In this step, the priority of the shared storage area is higher than the reserved storage area for storing the second data. Thus, when the capacity of the reserved storage area and the shared storage area is sufficient to accommodate the writing of the second data, the shared storage area is preferentially stored to ensure that the second storage pool has enough reserved capacity to store its own data.

[0103] For example, if the size of the capacity space occupied by the second data is 4 GB, the capacity of the shared storage area is 80 GB, and the available capacity of the reserved storage area is 20 GB, then the capacity of the reserved storage area and the shared storage area are both sufficient to accommodate the writing of the second data, and the second data is written to the shared storage area of ​​the solid state drive in the second storage pool.

[0104] S504: When the amount of the second data is greater than the available capacity of the shared storage area of ​​the second storage pool and is less than or equal to the available capacity of the reserved storage area of ​​the solid state drive in the second storage pool, write the second data into the reserved storage area.

[0105] For example, if the size of the second data is 4 GB, the available capacity of the shared storage area is 3 GB, and the available capacity of the reserved storage area is 10 GB, then the available capacity of the shared storage area is insufficient to accommodate the writing of the second data, while the available capacity of the reserved storage area is sufficient to accommodate the writing of the second data, and the second data is written to the reserved storage area of ​​the second solid state drive of the second storage pool.

[0106] S505: When the available capacities of the shared storage area and the reserved storage area are insufficient to write the second data, migrate the third data in the shared storage area to the mechanical hard disk of the first storage pool so that the available capacity of the shared storage area is larger than the data volume of the second data.

[0107] Among them, the third data is the first data written into the shared storage area of ​​the second solid-state hard disk in the second storage pool when the storage device receives the first write data request, because the available capacity of the first solid-state hard disk in the first storage pool is less than the data amount of the first data. For example, the size of the second data is 4GB, the available capacity of the reserved storage area has 2GB remaining, and the available capacity of the shared storage area is 2GB. The available capacity of the reserved storage area and the shared storage area are both insufficient to write 4GB of second data, triggering the flushing of the shared storage area. Exemplarily, 30GB of space in the used capacity of the shared storage area is the third data, and the 30GB of third data is migrated to the mechanical hard disk to release the 30GB capacity of the shared storage area.

[0108] S506: Write the second data into the shared storage area.

[0109] In this step, after the shared storage area is flushed, its own available capacity is sufficient to accommodate the writing of the second data. For example, after the shared storage area is flushed, 30GB of available capacity is released, which is sufficient to accommodate the writing of 4GB of second data, and the second data is written to the shared storage area.

[0110] In one embodiment, Figure 6 As shown, a data storage method provided in an embodiment of the present application may further include: S601-S602. Exemplarily, S601-S602 may be executed before S301 or after S304. Alternatively, S601-S602 may be executed before S501 or after S506. At the same time, S601 and S602 may be executed simultaneously or alternately, without limitation here.

[0111] S601. Based on a preset time, periodically obtain the proportion of the used capacity of the first solid-state hard disk in the first storage pool, and when the proportion of the used capacity of the first solid-state hard disk in the first storage pool reaches or exceeds a first preset water level, flush the data in the first solid-state hard disk in the first storage pool to the mechanical hard disk in the first storage pool.

[0112] In this step, the preset time may be set to 5 minutes. The processor starts a routine check task after 5 minutes to periodically check whether the cache space (the available capacity of the first solid-state hard disk in the first storage pool) is sufficient; if not, the disk is refreshed.

[0113] In an optional embodiment, the water level is used to indicate the proportion of the used capacity of the first solid-state hard disk in the first storage pool as a cache disk when the disk flushing is triggered. For example, assuming that the total capacity of the first solid-state hard disk in the first storage pool is 20GB, the first preset water level can be set to 90% illustratively. Specifically, when the used capacity of the solid-state hard disk in the first storage pool reaches 18GB, the proportion of used capacity is 18GB / 20G×100%=90%. In this way, when the management node obtains that the proportion of the used capacity of the solid-state hard disk in the first storage pool reaches 90%, that is, when the available capacity in the first solid-state hard disk in the first storage pool is equal to 10% of the total capacity of the first solid-state hard disk, the disk is flushed to increase the idle ratio (available capacity ratio) to 30%. Alternatively, according to the size of the first data to be written, the cached data of the corresponding size can be flushed to the mechanical hard disk to ensure that the solid-state hard disk in the first storage pool has enough cache space to write new data.

[0114] It should be noted that if the available capacity of the SSD is too low, the write operation may be slowed down because the SSD needs to frequently perform data writing and migration operations. 30% of the available capacity can provide a balance between performance and capacity utilization. For example, 30% of the available capacity can be used as a write buffer, allowing the SSD to have a certain buffer space when processing a large number of concurrent write requests, avoiding write failures or delays due to insufficient space. In other words, the available capacity ratio of 30% is a preset cache free capacity target value, which is used to ensure that the first solid-state drive (SSD) in the first storage pool has enough space to process new write requests. By regularly restoring the free capacity to 30%, there is enough free capacity to ensure that the system can quickly process new write requests when the system load suddenly increases. The value of 30% is set based on the experience of the system designer and the system performance requirements. Different systems may have different optimal free capacity ratios, which need to be determined according to the specific system characteristics and usage scenarios. This application does not strictly limit the specific value of the free ratio.

[0115] S602: periodically obtain the proportion of used capacity of the shared storage area based on a preset time, and when the proportion of used capacity of the shared storage area reaches or exceeds a second preset watermark, flush the third data in the shared storage area to the mechanical hard disk of the first storage pool. The third data is the first data written into the shared storage area of ​​the second solid-state hard disk in the second storage pool when the storage device receives a first write data request because the available capacity of the first solid-state hard disk in the first storage pool is less than the data amount of the first data.

[0116] In this step, the preset time can be set to 5 minutes. After 5 minutes, the processor starts the routine check task to periodically check whether the available capacity of the shared storage area is sufficient; if not, the disk is refreshed.

[0117] If the total capacity of the shared storage area of ​​the second solid-state hard disk in the second storage pool is 80GB, in an optional implementation, the second preset watermark can be set to 90%. For example, when the used capacity of the shared storage area of ​​the solid-state hard disk in the second storage pool reaches 72GB, the proportion of used capacity is 72GB / 80G×100%=90%. When the management node obtains that the proportion of used capacity of the shared storage area reaches 90%, that is, when the available capacity of the cache in the shared storage area is equal to 10% of the total cache capacity, the disk is flushed to increase the proportion of available capacity, ensuring that the shared storage area of ​​the solid-state hard disk in the second storage pool has enough cache space to write new data.

[0118] Example 1

[0119] Figure 7 A flow chart of a data storage method provided in Example 1 of the present application is shown. Figure 7 As shown, the data storage method in this embodiment is applied in a storage device, the storage device includes a first storage pool and a second storage pool, the first storage pool includes a first solid-state hard disk and a mechanical hard disk; the second storage pool includes a second solid-state hard disk, the second solid-state hard disk includes a shared storage area and a reserved storage area, the shared storage area is used to provide an extended storage space for the first storage pool, and the method includes the following steps:

[0120] S701: Receive a write data request.

[0121] Exemplarily, in this step, the write data request includes a first write data request and a second write data request, the first request and the second request carry tags (such as a and b) marking data types, and the data types include ordinary business data and core business data. The data to be written carried by the first write data request is the first data; the data to be written carried by the second request is the second data. For the sake of clarity, the first storage pool is referred to as a hybrid storage pool, and the second storage pool is referred to as an all-flash storage pool.

[0122] S702: Determine the storage pool type.

[0123] The type of storage pool is determined by determining the type of write data request. If the write data request is the first write data request, the type of storage pool is a hybrid storage pool. If the write data request is the second write data request, the type of storage pool is an all-flash storage pool. The first write data request or the second write data request can be determined based on an identifier carried in the write data request. For example, if the identifier carried in the write data request is "a", the write data request can be determined to be the first write data request. If the identifier carried in the write data request is "b", the write data request can be determined to be the second write data request.

[0124] In the case that the storage pool type to which the to-be-written data carried in the write data request is to be written is a hybrid storage pool, the following steps S703 - S709 may be performed.

[0125] S703, determine whether the data volume of the first data is less than or equal to the available capacity of the first solid-state hard disk in the hybrid storage pool. In one case, if the data volume of the first data is less than or equal to the available capacity of the first solid-state hard disk in the hybrid storage pool, then execute step S704. In another case, if the data volume of the first data is greater than the available capacity of the first solid-state hard disk in the hybrid storage pool, then execute step S705.

[0126] S704: Write the first data to the first solid state drive in the hybrid storage pool.

[0127] S705, determine whether the data volume of the first data is less than or equal to the available capacity of the shared storage area in the all-flash storage pool. In one case, if the data volume of the first data is less than or equal to the available capacity of the shared storage area in the all-flash storage pool, then execute step S706. In another case, if the data volume of the first data is greater than the available capacity of the shared storage area in the all-flash storage pool, then execute step S707.

[0128] S706: Write the first data into the shared storage area.

[0129] S707, flush the third data in the shared storage area to the mechanical hard disk of the hybrid storage pool.

[0130] Among them, the third data is the first data written into the shared storage area of ​​the second solid-state hard disk in the second storage pool when the storage device receives the first write data request because the available capacity of the first solid-state hard disk in the first storage pool is less than the data amount of the first data.

[0131] Exemplarily, the third data stored in the shared storage area is common business data.

[0132] S708, again determine whether the data volume of the first data is less than or equal to the available capacity of the shared storage area in the all-flash storage pool. In one case, if the data volume of the first data is less than or equal to the available capacity of the shared storage area in the all-flash storage pool, then execute step S706. In another case, if the data volume of the first data is greater than the available capacity of the shared storage area in the all-flash storage pool, then execute step S709.

[0133] S709: If the first data still cannot be written to the shared storage area after the disk flushing, flush the data of the first solid state disk in the hybrid storage pool to the mechanical hard disk. Then, execute step S704.

[0134] In the case that the storage pool type to which the second data carried by the second data write request is to be written is an all-flash storage pool, the following steps S710 - S714 may be performed.

[0135] S710: Determine whether the data volume of the second data is less than or equal to the available capacity of the shared storage area of ​​the all-flash storage pool.

[0136] In one case, if the available capacity of the shared storage area of ​​the all-flash storage pool is sufficient to write the second data to be written, step S711 is executed. In another case, if the available capacity of the shared storage area of ​​the all-flash storage pool is insufficient to write the second data to be written, step S712 is executed.

[0137] S711. Write the second data into the shared storage area of ​​the all-flash storage pool.

[0138] S712, determine whether the data volume of the second data is less than or equal to the available capacity of the reserved storage area of ​​the all-flash storage pool. In one case, if the data volume of the second data is less than or equal to the available capacity of the reserved storage area of ​​the all-flash storage pool, execute S713. In another case, if the data volume of the second data is greater than the available capacity of the reserved storage area of ​​the all-flash storage pool, execute S714.

[0139] S713: Store the second data in a reserved storage area of ​​the all-flash storage pool.

[0140] S714: flush the third data in the shared storage area to the mechanical hard disk of the hybrid storage pool, so that the available capacity of the shared storage area is greater than or equal to the data volume of the second data. Then, execute step S711.

[0141] It can be understood that the third data is the first data written into the shared storage area of ​​the second solid-state hard disk in the second storage pool when the storage device receives the first write data request because the available capacity of the first solid-state hard disk in the first storage pool is less than the data amount of the first data.

[0142] As described above, the data storage method provided in the embodiment of the present application adopts a distributed storage system that combines the two storage methods of a hybrid storage pool and an all-flash storage pool, wherein the all-flash storage pool contains most of the SSD capacity (shared storage area and reserved storage area) to ensure a large available capacity. When the free capacity of the all-flash storage pool is sufficient, a portion of the space (shared storage area) can be provided as a cache for the hybrid storage pool; when the free capacity of the all-flash pool is insufficient, the disk can be flushed and the cache can be released to ensure that the data can be written normally. In this way, when the cache capacity of the hybrid storage pool is insufficient, the data can be cached in the free capacity of the all-flash storage pool, and the flushing of the solid-state hard disk in the hybrid storage pool can be avoided as much as possible to provide data IO performance.

[0143] It is understandable that the size of the sequence number of each step in the above-mentioned embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, in some possible implementations, the steps in the above-mentioned embodiments can be selectively executed according to actual conditions, and can be partially executed or fully executed, which is not limited here. In addition, all or part of any features of any of the above-mentioned embodiments can be freely and arbitrarily combined without contradiction; the combined technical solutions are also within the scope of the present application.

[0144] Figure 8 A server is provided in an embodiment of the present application, which can be the above-mentioned Figure 1 Management nodes or storage nodes in the Figure 8 As shown, the server may be a rack server, a cabinet server, a blade server, etc., or a general server, an AI server, a master control server (or a management server), a storage server, etc. Optionally, the server specifically includes:

[0145] The receiving unit 801 is configured to receive a first write data request and a second write data request.

[0146] Processing unit 802 is used to determine that the first write data request is a request to write the first data to the first storage pool; when the available capacity of the first solid-state hard disk in the first storage pool is sufficient, write the first data to the first solid-state hard disk in the first storage pool; when the available capacity of the first solid-state hard disk in the first storage pool is insufficient, write the first data to the shared storage area of ​​the second solid-state hard disk in the second storage pool.

[0147] In one embodiment, the processing unit 802 is used to determine that the second write data request is a request to write the second data to the second storage pool; when the available capacity of the shared storage area of ​​the second solid-state hard disk in the second storage pool is sufficient, the second data is written to the shared storage area of ​​the second solid-state hard disk in the second storage pool; when the available capacity of the shared storage area is insufficient and the available capacity of the reserved storage area is sufficient, the second data is written to the reserved storage area.

[0148] In one embodiment, the processing unit 802 is further configured to migrate the third data in the shared storage area to the mechanical hard disk in the first storage pool when the available capacity of both the shared storage area and the reserved storage area is insufficient, so that the available capacity of the shared storage area is greater than or equal to the data volume of the second data. The third data is the first data written into the shared storage area of ​​the second solid-state hard disk in the second storage pool when the storage device receives the first write data request because the available capacity of the first solid-state hard disk in the first storage pool is less than the data volume of the first data; and the second data is written into the shared storage area.

[0149] In one embodiment, the third data in the shared storage area is migrated to the mechanical hard disk of the first storage, and the processing unit 802 is also used to determine whether the available capacity of the shared storage area satisfies the writing of the second data, wherein, when the available capacity of the shared storage area is insufficient to write the second data, the data stored in the first solid-state hard disk in the first storage pool is migrated to the mechanical hard disk of the first storage pool, and the first data is written to the first solid-state hard disk of the first storage pool.

[0150] In one embodiment, the processing unit 802 migrates data of the first solid-state drive in the first storage pool to the mechanical hard disk in the first storage pool when the capacity usage rate of the first solid-state drive in the first storage pool reaches or exceeds a first preset waterline.

[0151] In one embodiment, when the capacity usage rate of the shared storage area of ​​the second solid-state hard disk in the second storage pool reaches or exceeds a second preset water level, the processing unit 802 migrates the third data stored in the shared storage area of ​​the second solid-state hard disk in the second storage pool to the mechanical hard disk in the first storage pool.

[0152] Based on the above, see Fig. 9 The present application also provides a storage processing device 1000, which can be configured in Figure 1 In the management node shown, specifically, the storage processing device 1000 includes: an initialization configuration module 1001 , a data IO module 1002 and a disk flushing module 1003 .

[0153] Among them, the initialization configuration module 1001 is used to configure the first solid-state drive SSD of the hybrid storage pool and the mechanical hard disk HDD of the first storage pool, and configure the reserved storage area SSD and shared storage area SSD in the second solid-state drive SSD of the all-flash storage pool.

[0154] Exemplarily, the initialization configuration module 1001 configures the capacity ratio of the first solid-state drive SSD and the mechanical hard disk HDD in the hybrid storage pool. It is only necessary to configure the first storage pool solid-state drive SSD according to the minimum threshold, such as 2%. It is understandable that the capacity ratio described here is the ratio of the first storage pool solid-state drive SSD and the first storage pool's mechanical hard disk HDD to the total capacity of the hybrid storage pool. For example, if the total capacity of the hybrid storage pool is 1000GB, then according to the capacity ratio of 2%, the capacity of the first storage pool solid-state drive SSD is configured to 20GB, and the capacity of the first storage pool's mechanical hard disk HDD is 98GB. It is worth mentioning that the first storage pool solid-state drive SSD comes from Figure 2 After configuring the first solid-state drive SSD of the hybrid storage pool, all the capacity of the remaining solid-state drives in the storage device is configured as the available capacity of the all-flash storage pool as the main storage disk of the all-flash storage pool.

[0155] Specifically, the initialization configuration module 1001 configures part of the capacity in the all-flash storage pool, such as 20%, as "all-flash reserved capacity", that is, a reserved storage area; the reserved storage area can only be used as the main memory of the all-flash pool, and cannot be used as a cache disk of the hybrid storage pool. The initialization configuration module 1001 configures the remaining capacity in the all-flash storage pool, such as 80%, as a shared storage area SSD. It should be noted that the shared storage area SSD as "dynamic shared capacity" can be used as an extended storage space of the hybrid storage pool to store the first data that needs to be stored in the first solid-state hard disk. For example, assuming that the capacity of the remaining solid-state hard disk in the storage device is 100GB, the initialization configuration module 1001 can configure the reserved storage area to 20GB and the cache capacity to 80GB. Exemplarily, the initialization configuration module 1001 can also configure the mapping relationship between the hybrid storage pool and the all-flash storage pool so that the hybrid storage pool can use the shared storage area of ​​the associated all-flash storage pool as an extended storage space to store the first data.

[0156] The data IO module 1002 is used to record the cache record table of the hybrid storage pool data, and record the specific location of the data cached in the all-flash storage pool. Specifically, a mark can be added to the data stored in the all-flash storage pool to distinguish whether it is the first data of the all-flash storage pool or the second data of the hybrid storage pool, which is used for data volume statistics and disk flushing. The mark can be a business tag identified by metadata, which will not be repeated in this application.

[0157] The data IO module 1002 is also used to determine the processing flow when writing data according to the available capacity of the hybrid storage pool cache and the available capacity of the all-flash storage pool. For details, please refer to Figure 3-7 , I will not go into details here.

[0158] The disk flushing module 1003 is used to trigger a disk flush when writing data if the cache space is insufficient, and flush the first data stored in the first solid-state hard disk or shared storage area of ​​the hybrid storage pool to the mechanical hard disk HDD of the hybrid storage pool, and then continue to write the data to the first storage pool solid-state hard disk or shared storage area.

[0159] Exemplarily, the first solid-state drive in the mixed flash storage pool is flushed: according to the capacity actually required for the amount of first data to be written, the cache data of the corresponding size is flushed to the mechanical hard disk HDD in the first storage pool to ensure that there is enough cache space to write new data.

[0160] Exemplarily, the shared storage area of ​​the all-flash storage pool is flushed: according to the capacity actually required by the second data volume, the cached data (first data) is flushed to the mechanical hard disk HDD of the first storage pool to ensure that there is enough cache space to write new data.

[0161] In one embodiment, the disk flushing module 1003 is also used for routine disk flushing. Specifically, when no data is written, a routine check task is started to periodically check whether the cache space is sufficient; if not, the disk is flushed.

[0162] Exemplarily, the hybrid storage pool flushes the disk: if the cache free capacity (the first solid-state hard disk available capacity) is less than 10% of the total cache capacity, the disk is flushed to increase the free ratio to 30%.

[0163] Exemplarily, the all-flash storage pool flushes the disk: if the "dynamic shared capacity" (the second solid-state drive shared storage area) is less than 10% of the total capacity of the all-flash storage pool shared storage area, the disk is flushed to increase the proportion to 30%.

[0164] Among them, the initialization configuration module 1001, the data IO module 1002 and the disk brushing module 1003 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the initialization configuration module 1001 is introduced below by taking the initialization configuration module 1001 as an example. Similarly, the implementation of the data IO module 1002 and the disk brushing module 1003 can refer to the implementation of the initialization configuration module 1001.

[0165] As an example of a software functional unit, the initialization configuration module 1001 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the initialization configuration module 1001 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same AZ or in different AZs, and each AZ includes a data center or multiple data centers with similar geographical locations. Among them, usually a region can include multiple AZs.

[0166] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud VPC or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0167] As an example of a hardware functional unit, the initialization configuration module 1001 may include at least one computing device, such as a server, etc. Alternatively, the initialization configuration module 1001 may also be a device implemented using an application specific integrated circuit ASIC, or a programmable logic device PLD, etc. The PLD may be implemented using a CPLD, FPGA, GAL, or any combination thereof.

[0168] The multiple computing devices included in the initialization configuration module 1001 can be distributed in the same region or in different regions. The multiple computing devices included in the initialization configuration module 1001 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the initialization configuration module 1001 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0169] In this way, the storage processing device 1000 provided by the present application configures the mapping relationship between the hybrid storage pool and the all-flash storage pool through the initialization configuration module 1001, and determines the data location according to the available capacity, and uses the all-flash storage pool as the cache of the hybrid storage pool, so that the SSD solid-state hard disk can be dynamically shared. Moreover, by configuring the "all-flash reserved capacity" through the initialization configuration module 1001, the main memory capacity dedicated to the all-flash storage pool is guaranteed. At the same time, the disk flushing module 1003 performs real-time and routine disk flushing to ensure that the cache space is available. Finally, the cache disk of the hybrid storage pool and the main memory disk of the all-flash storage pool are reasonably utilized through the data IO module 1002, which not only simplifies the process, but also takes into account the available capacity and IO performance of the entire distributed storage.

[0170] Based on the above, see Fig.10 The present application also provides a computing device 1100, which can be Figure 1 The management node or storage node in the computing device 1100 includes: a bus 1101, a processor 1102, a memory 1103 and a communication interface 1104. The processor 1102, the memory 1103 and the communication interface 1104 communicate with each other through the bus 1101. The computing device 1100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1100.

[0171] The bus 1101 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 The bus 1101 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 1101 may include a path for transmitting information between various components of the computing device 1100 (for example, the memory 1103, the processor 1102, and the communication interface 1104).

[0172] The processor 1102 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0173] The memory 1103 may include a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0174] The memory 1103 stores executable program instructions, and the processor 102 executes the executable program instructions to respectively implement the data storage methods involved in the above embodiments.

[0175] The communication interface 1103 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0176] The present application also provides a computer-readable storage medium. The computer-readable storage medium is used to store computer program instructions. When the computer program instructions are executed, the data storage method involved in the above embodiment is executed. The computer-readable storage medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0177] The embodiment of the present application further provides a computer program product including instructions. When the computer program product is run, the data storage method involved in the above embodiment is executed.

[0178] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0179] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application. Those skilled in the art should understand that, although the present application is described in detail with reference to the above embodiments, the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions in the embodiments of the present application.

Claims

1. A data storage method, characterized in that: Applied to a storage device, the storage device includes a first storage pool and a second storage pool, wherein the first storage pool includes a first solid-state hard disk and a mechanical hard disk, the second storage pool includes a second solid-state hard disk, the second solid-state hard disk includes a shared storage area, and the shared storage area is used to provide an extended storage space for the first storage pool; The method comprises: receiving a first data write request, where the first data write request is used to write first data into a first storage pool; When the amount of the first data is less than or equal to the available capacity of the first solid-state hard disk in the first storage pool, writing the first data to the first solid-state hard disk in the first storage pool; When the data volume of the first data is greater than the available capacity of the first solid-state hard disk of the first storage pool and is less than or equal to the available capacity of the shared storage area of ​​the second storage pool, the first data is written to the shared storage area of ​​the second solid-state hard disk of the second storage pool.

2. The method according to claim 1, characterized in that The second solid state drive also includes a reserved storage area, and further includes: receiving a second data write request, where the second data write request is used to write second data into a second storage pool; When the amount of the second data is less than or equal to the available capacity of the shared storage area of ​​the second storage pool, writing the second data into the shared storage area of ​​the second solid state drive of the second storage pool; When the amount of the second data is greater than the available capacity of the shared storage area of ​​the second storage pool and is less than or equal to the available capacity of the reserved storage area of ​​the second solid state drive in the second storage pool, the second data is written into the reserved storage area.

3. The method according to claim 2, characterized in that Also includes: In a case where the data volume of the second data is larger than the available capacity of the shared storage area and the reserved storage area in the second solid-state hard disk, the third data in the shared storage area is migrated to the mechanical hard disk in the first storage pool, so that the available capacity of the shared storage area is greater than or equal to the data volume of the second data; wherein the third data is the first data written into the shared storage area of ​​the second solid-state hard disk in the second storage pool when the storage device receives the first write data request because the available capacity of the first solid-state hard disk in the first storage pool is smaller than the data volume of the first data; The second data is written into the shared storage area.

4. The method according to any one of claims 1 to 3, characterized in that: Before writing the first data to the shared storage area of ​​the second solid state hard disk of the second storage pool, the method further includes: When the data volume of the first data is larger than that of the shared storage area, the third data in the shared storage area is migrated to the mechanical hard disk of the first storage pool; wherein the third data is the first data written into the shared storage area of ​​the second solid-state hard disk in the second storage pool when the storage device receives the first write data request because the available capacity of the first solid-state hard disk in the first storage pool is less than the data volume of the first data.

5. The method according to claim 4, characterized in that After migrating the third data in the shared storage area to the mechanical hard disk of the first storage pool, the method further includes: Determining whether the available capacity of the shared storage area is greater than or equal to the data volume of the first data; When the available capacity of the shared storage area is less than the data volume of the first data, migrating the data in the first solid-state hard disk in the first storage pool to the mechanical hard disk in the first storage pool, so that the available capacity of the first solid-state hard disk is greater than or equal to the data volume of the first data; The first data is written to a first solid state drive in the first storage pool.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: When the capacity usage rate of the first solid-state hard disk in the first storage pool reaches or exceeds a first preset watermark, data of the first solid-state hard disk in the first storage pool is migrated to the mechanical hard disk in the first storage pool.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When the capacity usage rate of the shared storage area of ​​the second solid-state hard disk in the second storage pool reaches or exceeds the second preset water level, the third data stored in the shared storage area of ​​the second solid-state hard disk in the second storage pool is migrated to the mechanical hard disk in the first storage pool.

8. The method according to claim 4, characterized in that The step of writing the first data into the shared storage area of ​​the second solid state hard disk of the second storage pool includes: The first data written into the shared storage area in the second storage pool is marked.

9. A computing device, characterized in that include: Memory and processor; The memory is coupled to the processor; The memory is used to store programs; The processor is used to call the program stored in the memory to execute the method according to any one of claims 1-8.

Images