An adaptive hybrid storage system

Abstract

The adaptive hybrid storage system belongs to the field of computers and comprises a request buffer, a high-performance solid state disk, a data compaction module and a high-density solid state disk.The request buffer is used for receiving user requests and temporarily storing write data in the user requests.The high-performance solid state disk dynamically calculates and limits the user write bandwidth through an adaptive smoothing mechanism according to its own write cache space and the data migration speed of the data compaction module, and buffers the write data from the request buffer.The data compaction module is connected between the high-performance solid state disk and the high-density solid state disk, and realizes the migration of the write data to release the write cache space.The high-density solid state disk receives and finally stores the migrated write data, performs a garbage collection operation based on the data migration write bandwidth and its own storage space, and releases new available storage space.The application effectively avoids the significant fluctuation of the available space size of the write buffer in the high-performance solid state disk, and prevents the decline of the user write performance caused by garbage collection under a burst load.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computers, and particularly relates to a self-adaptive hybrid storage system. BACKGROUND

[0002] In recent years, high-density, low-cost quad-level cell (QLC) NAND flash memory has been widely used in the latest solid-state drives (SSDs), significantly improving the storage capacity of SSDs. However, to solve the performance and durability problems of high-density SSDs such as QLC-SSD, the industry has proposed solutions such as hybrid SSDs (Hybrid SSDs) and hybrid storage architectures, which use a hierarchical data management strategy to fully utilize the characteristics of different storage media and achieve a balance between performance and cost.

[0003] A hybrid SSD refers to a unified storage device that integrates high-performance storage media and high-density (QLC NAND) flash memory within a single hardware device, providing a standard storage interface consistent with traditional SSDs to the outside. For example, Intel Optane H10 hybrid storage disk, HyFlex. However, due to the lack of widespread popularity of such devices, cloud storage providers rarely use them for cloud storage backend storage services.

[0004] Hybrid storage architectures represented by CSAL (Cloud Storage Acceleration Layer) utilize multiple independent storage devices and manage data access at the software level with the help of the host CPU to provide a unified block device interface to the system kernel.

[0005] The cloud storage acceleration layer architecture manages the available space in the entire storage system and the data migration between storage devices. Specifically, the cloud storage acceleration layer architecture divides the upper layer of high-performance disks and the lower layer of large-capacity disks into multiple storage chunks (Chunk) and storage zones (Zone), and treats both types of devices as append-only write storage devices.

[0006] When processing user write requests, the cloud storage acceleration layer architecture tends to use most of the storage space on the upper layer of high-performance disks as a write buffer to capture all write requests from user applications, avoiding the write amplification problem caused by small write requests in the lower layer of large-capacity disks, thereby improving the overall write performance of the storage system. For user-initiated read requests, the cloud storage acceleration layer architecture directly reads from the location where the data is stored (high-performance disk or large-capacity disk).

[0007] However, due to the limited capacity of the high-performance disk, in order to reserve sufficient available space for subsequent user write requests, when the available capacity of the upper high-performance disk is gradually exhausted (i.e., the number of free storage blocks is insufficient, and by default, the proportion of free storage blocks is less than 20%), the cloud storage acceleration layer architecture will perform a Compaction operation, which refers to an operation of compacting data in the cache disk to the large-capacity disk in the CSAL architecture. Through the Compaction operation, the effective data in the selected storage block is continuously integrated and migrated to the lower large-capacity disk. In order to ensure the relative stability of the available space of the write buffer in the upper high-performance disk, the storage architecture usually dynamically adjusts the user write bandwidth according to the data migration speed of the Compaction and the remaining space of the write buffer. Specifically, the cloud storage acceleration layer hybrid storage architecture calculates the arithmetic mean speed of the Compaction in a fixed-size historical window, which is used as the upper limit of the user write bandwidth. However, the fixed-size historical data window has the following problems: when the window is too small, the bandwidth performance of the user write request is easily affected by the write performance fluctuations of the lower large-capacity disk, and significant jitter phenomenon occurs; and when the window is too large, the reaction of adjusting the user write bandwidth with the significant monotonic change of the Compaction speed becomes too slow, resulting in fluctuations in the available space size of the write buffer in the upper high-performance disk.

[0008] Similarly, when the number of free storage areas in the large-capacity disk is insufficient (by default, the number of free storage areas is less than or equal to 5), the cloud storage acceleration layer architecture triggers garbage collection to integrate the effective data in the old storage area and discard the invalid data to recover the available storage space. Specifically, when the number of available storage areas in the large-capacity disk is lower than the preset threshold, the storage architecture triggers the garbage collection operation. The system first selects a group of continuous storage areas with the highest invalid data proportion in the large-capacity disk, and transfers the effective data therein to the buffer area in the host memory. Subsequently, the storage system rewrites the effective data after reorganization to the free storage area of the large-capacity disk, completing the recovery of the old storage area.

[0009] However, the read and write operations generated by the cloud storage acceleration layer when performing garbage collection on the large-capacity disk will compete with the Compaction process of migrating data from the upper high-performance disk to the lower large-capacity disk in the storage system, resulting in a decrease in the Compaction speed, and further causing a significant decrease in the user I / O performance. SUMMARY

[0010] In view of the deficiencies of the prior art, the purpose of the adaptive hybrid storage system is to use a high-performance solid state disk as a write cache, use a high-density solid state disk as a data storage device, introduce an adaptive smoothing mechanism to effectively suppress user write performance jitter while responding to monotonic changes in compaction speed in a timely manner, and avoid significant fluctuations in the available space size of the write buffer in the high-performance solid state disk; and further combine a load-aware garbage collection mechanism to prevent user write performance degradation caused by garbage collection under bursty loads.

[0011] The embodiment of the application provides an adaptive hybrid storage system for optimizing write workloads, comprising:

[0012] A request buffer is configured to receive user requests and temporarily store write data in the user requests;

[0013] A high-performance solid state disk is configured to dynamically calculate and limit user write bandwidth through an adaptive smoothing mechanism based on its own write cache space and the data migration speed of a data compaction module, and cache write data from the request buffer;

[0014] A data compaction module is connected between the high-performance solid state disk and the high-density solid state disk, and is configured to migrate write data from the high-performance solid state disk to the high-density solid state disk to release write cache space;

[0015] A high-density solid state disk is configured to receive and finally store the migrated write data, judge whether the hybrid storage system is in an idle state and whether its own storage space is tight based on the data migration write bandwidth of the data compaction module and its own storage space, and perform a garbage collection operation according to the judgment result to release new available storage space.

[0016] In one embodiment, the request buffer is further configured to, for a read request in the user request, access a storage device corresponding to read data based on the location of the read data in the read request, and return the read read data to the user.

[0017] In one embodiment, the adaptive smoothing mechanism uses an exponential weighted smoothing algorithm for smoothing time series, gives a high weight to recent data migration speed, and exponentially decreases the weight of historical data migration speed to suppress short-term fluctuations in write bandwidth, and the calculation formula is as follows:

[0018] ,

[0019] In the formula, is a smoothing factor for controlling the relative proportion of recent data migration speed and historical data migration speed in the weighted average; represents the speed at which the data compaction module cleans up available storage space from the high-performance solid-state drive write buffer at the last time; is a threshold value for limiting user write bandwidth, is the current time step.

[0020] In one embodiment, the smoothing factor is dynamically adjusted according to the deviation between the user write bandwidth upper limit and the data migration speed, and when the deviation increases, the smoothing factor is increased to improve the response agility of the hybrid storage system; and when the deviation decreases, the smoothing factor is decreased to enhance the stability of the user bandwidth; the calculation formula of the smoothing factor is as follows:

[0021] ,

[0022] In the formula, represents the deviation between the user write bandwidth upper limit and the data migration speed, and are adjustable parameters.

[0023] In one embodiment, the deviation is calculated by the following formula:

[0024] ,

[0025] In the formula, is the time window size, used to control the adjustment speed of the smoothing factor ; is the user write bandwidth upper limit at the time step ; is the data migration speed at the time step , i.e., the speed at which the data compaction module cleans up available storage space from the high-performance solid-state drive write buffer.

[0026] In one embodiment, the following conditions need to be met to perform the garbage collection operation:

[0027] When the data migration write bandwidth is lower than a preset first threshold value, it is determined that the hybrid storage system is in an idle state; wherein the first threshold value is determined by offline analysis before deployment of the hybrid storage system, and is used to ensure that the influence of performing the garbage collection operation on the data migration write bandwidth of the data compaction module is negligible at the first threshold value;

[0028] and the number of available storage areas in the high-density solid-state drive is lower than a preset upper limit, it is determined that the storage space of the high-density solid-state drive is tight.

[0029] In one embodiment, the garbage collection operation comprises: selecting one or more continuous storage areas with the highest proportion of invalid data from the storage areas of the high-density solid state disk; reading valid data in the selected storage areas; writing the read valid data to a new available storage area in the high-density solid state disk; and clearing the invalid data in the original storage area to release new available storage space.

[0030] In another aspect, the application also provides an adaptive hybrid storage method using the adaptive hybrid storage system, comprising the following steps:

[0031] receiving a user request, temporarily storing write data in the user request in a request buffer, for a read request in the user request, accessing a storage device corresponding to read data according to a location of the read data, and returning the read read data to the user;

[0032] The high-performance solid state disk dynamically calculates and limits the user write bandwidth through an adaptive smoothing mechanism according to the write cache space and the data migration speed of the data compaction module, and caches write data from the request buffer;

[0033] The data compaction module connected between the high-performance solid state disk and the high-density solid state disk migrates write data from the high-performance solid state disk to the high-density solid state disk to release the write cache space;

[0034] The high-density solid state disk receives and finally stores the migrated write data, judges whether the hybrid storage system is in an idle state and whether the storage space of the high-density solid state disk is tight based on the data migration write bandwidth of the data compaction module and the storage space of the high-density solid state disk, and performs a garbage collection operation according to the judgment result to release new available storage space.

[0035] In another aspect, the application also provides a computer readable storage medium, the storage medium stores a computer program, and the computer program uses a computer to realize the adaptive hybrid storage method.

[0036] Compared with the prior art, the application has at least the following beneficial effects:

[0037] (1) The adaptive hybrid storage system provided by the application uses an exponential weighted smoothing algorithm to replace a simple moving average algorithm, gives higher weight to recent data, and improves the response ability of the user I / O bandwidth upper limit to the monotonic change of the data migration speed.

[0038] (2) On the basis of using the exponential smoothing algorithm to process data migration speed fluctuation, an adaptive adjustment scheme of user bandwidth upper limit is proposed, according to the change trend of data migration speed, the balance factor of using the exponential smoothing algorithm is dynamically adjusted, and the stability and timeliness of user bandwidth are realized.

[0039] (3) For the problem that frequent garbage collection operation caused by burst write request further leads to limited user write bandwidth, and peak bandwidth performance cannot be achieved, the present scheme proposes a load-aware garbage collection method, according to the data compaction module to HD-SSD write data speed, whether the hybrid storage system is in idle or low load state is judged, and then whether to actively trigger garbage collection operation is determined, before the burst traffic comes, using the idle I / O resource, a large amount of available space is cleaned for HD-SSD, and passive triggered garbage collection operation is avoided to affect user bandwidth performance. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description.

[0041] Figure 1 The structure schematic diagram of the adaptive hybrid storage system provided by the present application.

[0042] Figure 2 The running flowchart of the adaptive hybrid storage system provided by the embodiment of the present application.

[0043] Figure 3 The user write bandwidth effect diagram using the adaptive smoothing mechanism.

[0044] Figure 4 The bandwidth performance improvement result of using the garbage collection operation.

[0045] Figure 5 The flowchart of the adaptive hybrid storage method provided by the embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined application purpose, hereinafter the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application. On the contrary, they are only examples of methods and systems consistent with some aspects of the present application as detailed in the appended claims.

[0047] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in this application and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or," as used herein, refers to

[0048] The specific embodiments according to the present application, features and advantages thereof are described in detail in the following with reference to the accompanying drawings and preferred embodiments.

[0049] The present solution is a new adaptive hybrid storage system designed for bursty workloads. The system uses a high-performance solid-state disk as a write cache and another high-density solid-state disk as a large-capacity data storage device to achieve a balance between performance and cost-effectiveness. The hybrid storage system provides a block device to users, and the maximum logical capacity of the block device is equal to the capacity of the underlying disk.

[0050] As shown in the embodiment, Figure 1 The present solution proposes an adaptive hybrid storage system, which includes:

[0051] A request buffer is configured to receive user requests and temporarily store write data in the user requests.

[0052] A high-performance solid-state disk (HP-SSD) is configured to dynamically calculate and limit the user write bandwidth through an adaptive smoothing mechanism according to the write cache space of the HP-SSD and the data migration speed of a data compaction module, and cache write data from the request buffer.

[0053] A data compaction module (Compaction) is connected between the HP-SSD and a high-density solid-state disk (HD-SSD), and is configured to migrate write data from the HP-SSD to the HD-SSD to release write cache space.

[0054] The HD-SSD is configured to receive and finally store the migrated write data, and based on the data migration write bandwidth of the data compaction module and the storage space of the HD-SSD, determine whether the hybrid storage system is in an idle state and whether the storage space of the HD-SSD is tight, and perform a garbage collection operation according to the determination result to release new available storage space.

[0055] The operation of the adaptive hybrid storage system is as shown in the embodiment, Figure 2 and specifically as follows:

[0056] All user requests are first submitted to the request buffer of the hybrid storage system, for read requests in the user request, the hybrid storage system directly accesses the corresponding solid state disk (SSD) according to the location of the read data, and returns the read data to the user application; for write requests, if the write buffer of the upper HP-SSD is about to be full at this time, the hybrid storage system will limit the user write bandwidth according to the recent data migration speed information, maintain the available capacity of the write buffer stable, and if the write buffer space is sufficient, the user write request is not limited.

[0057] In order to effectively suppress the jitter of user I / O performance while responding to the monotonic change of Compaction speed in time, the adaptive smoothing mechanism is proposed, which uses the adaptive exponential weighted moving average mechanism to dynamically adjust the user write bandwidth according to the speed of Compaction cleaning the storage space from the upper HP-SSD.

[0058] Specifically, the adaptive smoothing mechanism provided by the present scheme can effectively overcome the limitations of the fixed window size simple moving average scheme. The exponential weighted smoothing algorithm (EWMA) used by the adaptive smoothing mechanism is an algorithm for smoothing time series, which gives higher weight to the recent data migration speed of the data compaction module on the basis of the simple moving average (SMA) algorithm, and the weight of the historical data migration speed decreases exponentially, thereby suppressing the short-term fluctuations of the write bandwidth while maintaining the ability to respond to the monotonic change of the Compaction speed in time. Specifically, the formula for adjusting the upper limit of the user write bandwidth based on the EWMA algorithm is as follows:

[0059] ,

[0060] In the formula, is a smoothing factor used to control the relative proportion of recent data migration speed and historical data migration speed in weighted average; represents the speed of the last Compaction cleaning the available storage space from the high-performance solid state disk write buffer; is a threshold value for limiting the user write bandwidth, is the current time step.

[0061] By introducing a smoothing factor, the EWMA algorithm achieves a flexible trade-off between stability and agility: when the smoothing factor is relatively small, more historical information is retained in the user bandwidth limit, resulting in smoother changes in user I / O bandwidth; when the smoothing factor is large, the hybrid storage system responds more quickly to sudden and significant changes in compaction speed. Based on this, this solution proposes an adaptive smoothing scheme: dynamically adjusting the smoothing factor according to the short-term difference between compaction speed and user I / O bandwidth limiting, to simultaneously achieve user bandwidth stability and timely responsiveness.

[0062] Therefore, this solution continuously monitors the cumulative difference between the user's I / O rate limiting bandwidth and the compaction speed over a recent period, calculated using the following formula:

[0063] ,

[0064] In the formula, This represents the deviation between the user I / O rate limiting bandwidth and the compaction speed; The time window size is used to control the smoothing factor. Adjustment speed; For time steps User write bandwidth limit; In time step The data migration speed refers to the speed at which the data compaction module clears available storage space from the high-performance solid-state drive's write buffer.

[0065] When the deviation is large, the hybrid storage system will appropriately increase the smoothing factor to enhance the weight of the most recently written data, making bandwidth adjustment more agile. Conversely, when the deviation is small, the hybrid storage system will decrease the smoothing factor to enhance user bandwidth stability. The adaptive adjustment algorithm for the smoothing factor is as follows:

[0066] ,

[0067] In the formula, and These are adjustable parameters, which are related to the actual storage devices used. They need to be set reasonably based on the specific system configuration and through offline analysis or other methods before the hybrid storage system is put into operation.

[0068] like Figure 3 The diagram shows a comparison of user write bandwidth fluctuations and response times using the adaptive smoothing mechanism provided by this invention with those of the existing Cloud Storage Acceleration Layer (CSAL). The results show that the adaptive smoothing mechanism proposed in this solution based on the exponential weighted smoothing algorithm can effectively suppress user write bandwidth fluctuations while maintaining a timely response to monotonic changes in compaction speed.

[0069] As mentioned above, when the available capacity of the write buffer of the upper HP-SSD is insufficient, the adaptive hybrid storage system limits the user's write bandwidth in real time according to the speed at which the data compaction module cleans up the storage space from the write buffer of the upper HP-SSD. At this time, if the hybrid storage system frequently triggers garbage collection and competes for the write speed of Compaction to the lower HD-SSD, it will indirectly cause a substantial decline in the user's write bandwidth.

[0070] Specifically, in the hybrid storage system, the storage system usually needs to frequently perform garbage collection on the lower HD-SSD to ensure sufficient available space. The garbage collection operation selects a group of continuous storage areas with the highest proportion of invalid data from the HD-SSD, reads and removes the invalid data therein, and rewrites the valid data to another storage area of the HD-SSD. However, the existing hybrid storage system such as CSAL usually adopts a passive garbage collection mechanism, that is, garbage collection is triggered only when the available space of the lower HD-SSD is insufficient. This will cause frequent triggering of garbage collection during user write operations. Especially under bursty workloads, the frequently triggered garbage collection will seriously affect the user's write performance. And during the idle stage of the hybrid storage system, the I / O resources are idle for a long time, causing serious resource waste.

[0071] Therefore, to solve the problem of the existing hybrid storage system that the user's I / O performance is seriously reduced by the passive triggered garbage collection under bursty load, the present scheme proposes a load-aware garbage collection mechanism. By actively monitoring the load of the hybrid storage system, the idle I / O resources are efficiently utilized to perform garbage collection when the hybrid storage system is idle or in a low load state, thereby reserving sufficient I / O resources and storage space for the subsequent possible bursty traffic scenarios, and improving the overall performance.

[0072] Since garbage collection indirectly affects the user's write performance by competing for the bandwidth of Compaction to write data to the lower HD-SSD, the load-aware garbage collection mechanism determines whether the hybrid storage system is idle by continuously monitoring the real-time Compaction I / O throughput. Specifically, an offline analysis needs to be performed before the hybrid storage system is officially put into use to determine a Compaction I / O throughput threshold. When the Compaction I / O throughput is below the threshold, the impact of garbage collection on the Compaction speed of the hybrid storage system can be ignored, so as not to significantly affect the user's I / O bandwidth. The threshold varies with different storage devices, so it needs to be determined through offline analysis before the system is actually deployed.

[0073] After the hybrid storage system starts running, if it is detected that the current state is idle and the number of available storage areas in the underlying HD-SSD is lower than the preset upper limit, the hybrid storage system initiates garbage collection actively to clean up invalid data in the underlying HD-SSD to release new available storage space.

[0074] As Figure 4 shown is a comparison of the user write bandwidth performance of the load-aware garbage collection operation provided by the application and the passive garbage collection operation of the existing cloud storage acceleration layer hybrid storage system for bursty load, and the results show that, compared with the cloud storage acceleration layer hybrid storage architecture, the load-aware garbage collection mechanism proposed in the application can achieve a bandwidth performance improvement of up to 110% when user burst write requests arrive.

[0075] In the embodiment, an adaptive hybrid storage method is also provided, which uses the adaptive hybrid storage system, as Figure 5 shown, and includes the following steps:

[0076] The user request is received, and the write data in the user request is temporarily stored in the request buffer. For the read request in the user request, the storage device corresponding to the read data is accessed according to the location of the read data in the read request, and the read read data is returned to the user;

[0077] The high-performance solid state disk dynamically calculates and limits the user write bandwidth through the adaptive smoothing mechanism according to the write cache space and the data migration speed of the data compaction module, and caches the write data from the request buffer;

[0078] The data compaction module connected between the high-performance solid state disk and the high-density solid state disk migrates the write data from the high-performance solid state disk to the high-density solid state disk to release the write cache space;

[0079] The high-density solid state disk receives and finally stores the migrated write data, judges whether the hybrid storage system is in an idle state and whether the storage space of the high-density solid state disk is tight based on the data migration write bandwidth of the data compaction module and the storage space of the high-density solid state disk, and performs a garbage collection operation according to the judgment result to release new available storage space.

[0080] In the embodiment, a computer readable storage medium is also provided, and the storage medium stores a computer program. When the computer program is used by a computer, the adaptive hybrid storage method is implemented.

[0081] In summary, the present application aims at the problem of limiting user I / O using the simple moving average algorithm, considering that the Compaction bandwidth is affected by various factors, it is difficult to determine a fixed window size that can both suppress performance jitter and quickly respond to changes in workload, an adaptive smoothing mechanism is proposed, which can effectively suppress user write performance jitter while responding to changes in Compaction speed in a timely manner, avoiding significant fluctuations in the available space size of the HP-SSD upper layer write buffer. Further considering that the user write bandwidth performance degradation phenomenon is mainly due to the fact that the existing garbage collection strategy only considers the number of remaining free storage areas in the lower HD-SSD, and ignores the workload characteristics such as Compaction write bandwidth, even in the case of burst I / O requests, the hybrid storage architecture will frequently trigger garbage collection, competing with user write requests for bandwidth resources, therefore a load-aware garbage collection mechanism is proposed to prevent user write performance degradation caused by garbage collection under burst I / O traffic.

[0082] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; obviously, the drawings are only some examples or embodiments of the present application, and those skilled in the art can also apply the present application to other similar situations according to these drawings without creative labor. In addition, it can be understood that although the work done in this development process may be complex and long, for those skilled in the art, some design, manufacture or production changes according to the technical content disclosed in the present application are only routine technical means, and should not be regarded as insufficient disclosure of the present application.

[0083] The above is only the preferred embodiment of the present application, and is not intended to limit the present application in any form. Although the present application has been disclosed as above with the preferred embodiment, it is not intended to limit the present application, and any person skilled in the art can make some changes or modifications to the equivalent embodiments within the scope of the technical solutions of the present application without departing from the technical solutions of the present application. Any brief modification, equivalent change and modification of the above embodiments made according to the technical essence of the present application are still within the scope of the technical solutions of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An adaptive hybrid storage system for optimizing write workloads, comprising: The adaptive hybrid storage method comprises the following steps: a request buffer is configured to receive a user request and temporarily store write data in the user request; a high-performance solid state disk is configured to dynamically calculate and limit the user write bandwidth according to the write cache space of the high-performance solid state disk and the data migration speed of the data compaction module, and buffer the write data from the request buffer, wherein the adaptive smoothing mechanism uses an exponential weighted smoothing algorithm to smooth the time series, and the recent data migration speed is given a high weight, and the weight of the historical data migration speed decreases exponentially, so as to suppress the short-term fluctuation of the write bandwidth, and the calculation formula is as follows: ， In the formula, is a smoothing factor used to control the relative proportion of recent data migration speed and historical data migration speed in the weighted average; represents the speed at which the most recent data compaction module cleans up available storage space from the high-performance solid-state disk write buffer; is a threshold value for limiting user write bandwidth, is the current time step; a data compaction module is connected between the high-performance solid state disk and the high-density solid state disk, and is configured to migrate the write data from the high-performance solid state disk to the high-density solid state disk to release the write cache space; a high-density solid state disk is configured to receive and finally store the migrated write data, and judge whether the hybrid storage system is in an idle state and whether the storage space of the high-density solid state disk is tight based on the data migration write bandwidth of the data compaction module and the storage space of the high-density solid state disk, and perform a garbage collection operation according to the judgment result to release new available storage space.

2. The adaptive hybrid storage system of claim 1, wherein, The request buffer is further configured to: for a read request in the user request, accessing a storage device corresponding to the read data according to a position of the read data in the read request, and returning the read read data to the user.

3. The adaptive hybrid storage system of claim 1, wherein, The smoothing factor According to the deviation between the upper limit of the user writing bandwidth and the data migration speed, the smoothing factor is dynamically adjusted. When the deviation increases, the smoothing factor is increased , for improving the response agility of the hybrid storage system; when the deviation decreases, the smoothing factor is decreased , for enhancing the user bandwidth stability; the calculation formula of the smoothing factor is as follows: ， In the formula, represents the deviation amount between the upper limit of the user write bandwidth and the data migration speed, and is an adjustable parameter.

4. The adaptive hybrid storage system of claim 1, wherein, The deviation is calculated by the following formula: ， wherein is the time window size, used to control the smoothing factor is the adjustment speed of the smoothing factor is the time step is the user write bandwidth upper limit is the data migration speed at time step is the data migration speed at time step 5. The adaptive hybrid storage system of claim 1, wherein, The following conditions are required to perform the garbage collection operation: when the data migration write bandwidth is lower than a preset first threshold, it is judged that the hybrid storage system is in an idle state; wherein the first threshold is determined by offline analysis before deployment of the hybrid storage system, and is used to ensure that the influence of the garbage collection operation on the data migration write bandwidth of the data compaction module is negligible at the first threshold; and the number of available storage areas in the high-density solid state disk is lower than a preset upper limit, it is judged that the storage space of the high-density solid state disk is tight.

6. The adaptive hybrid storage system of claim 5, wherein, The garbage collection operation comprises: selecting one or more continuous storage areas with the highest proportion of invalid data from the storage areas of the high-density solid state disk; reading the valid data in the selected storage area; writing the read valid data to a new available storage area in the high-density solid state disk; and clearing the invalid data in the original storage area to release new available storage space.

7. An adaptive hybrid storage method, characterized by, The adaptive hybrid storage method uses the adaptive hybrid storage system of any one of claims 1-6, comprising the following steps: receiving a user request, temporarily storing write data in the user request in the request buffer, and for a read request in the user request, accessing a storage device corresponding to the read data according to a position of the read data in the read request, and returning the read read data to the user; the high-performance solid state disk dynamically calculates and limits the user write bandwidth according to the write cache space of the high-performance solid state disk and the data migration speed of the data compaction module, and buffers the write data from the request buffer; the data compaction module is connected between the high-performance solid state disk and the high-density solid state disk, and is configured to migrate the write data from the high-performance solid state disk to the high-density solid state disk to release the write cache space; The high-density solid state disk receives and finally stores the migrated write data, judges whether the hybrid storage system is in an idle state and whether the storage space of the high-density solid state disk is tight based on the data migration write bandwidth of the data compaction module and the storage space of the high-density solid state disk, executes a garbage collection operation according to a judgment result, and releases new available storage space.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program uses a computer to implement the adaptive hybrid storage method in claim 7.

Images