A data storage and write optimization method and device for adaptive cache

Abstract

The present invention discloses a data storage write optimization method and device of adaptive caching, including creating three independent transaction groups in a storage pool, namely an open transaction group, an intermediate transaction group and a synchronous transaction group, and initializing the capacity of the open transaction group to cache dirty data; for dirty data newly written into the storage pool, firstly storing the dirty data into the open transaction group, and when the capacity of the open transaction group is full or the dirty data is received for a first preset time, transferring the dirty data in the open transaction group to the intermediate transaction group, and creating a new open transaction group to receive the newly written dirty data. The data storage write optimization method and device of adaptive caching dynamically adjusts the cache capacity of the open transaction group in the storage pool, avoiding the time taken for the transaction group to write dirty data into the back-end disk due to a decrease in the speed of writing data into the storage pool to the back-end disk, thereby improving reliability and reducing the impact on the upper-layer business I / O.

Description

Technical Field

[0001] The present invention belongs to the technical field of data storage, and in particular relates to a method and device for optimizing data storage writing of an adaptive cache. Background Art

[0002] With the rapid development of information technology, the amount of data has exploded. How to efficiently and securely store and quickly access this data has become a major challenge facing all industries. Especially driven by cutting-edge technologies such as the Internet of Things, cloud computing, and artificial intelligence, the complexity and diversity of data storage have increased significantly, placing higher demands on the I / O performance and I / O stability of data storage. I / O performance and I / O stability are two key indicators for measuring data storage. How to improve the read and write performance and stability of data storage is a core issue that every storage manufacturer is closely concerned about.

[0003] At present, storage systems, especially domestic storage arrays, often experience performance degradation, increased I / O latency, and even I / O zero drop when facing long-rate, high-pressure continuous data writing. These problems are mainly due to system defects in domestic storage systems when processing large amounts of high-speed data read and write operations. For example, some solid-state drives (SSDs) experience significant performance degradation and increased I / O latency when writing long-rate, high-pressure data under continuous high-intensity write operations due to the limited number of erases and writes and the low efficiency of the garbage collection mechanism, which exacerbates I / O jitter and has an adverse impact on the stability of upper-layer services.

[0004] Most storage vendors have optimized the cache of storage systems, such as increasing cache capacity or optimizing cache scheduling strategies to improve the read and write performance of storage systems. However, in actual deployment applications, such as the existing patent CN111338579A, which provides a read and write cache optimization method, system, terminal, and storage medium based on a storage pool, when experiencing long-term continuous high-pressure data reading and writing, the underlying disk may experience performance degradation or fluctuations, causing the performance of the storage system to also decline and become unstable. In particular, some domestic storage solid-state disks may experience large performance jitters when the data write discreteness is high and the disk data is full and triggers garbage collection. In extreme cases, I / O may even drop to zero. At this time, the storage pool may have an I / O write timeout error, causing business interruption, which seriously affects the upper-layer business of the storage system. Summary of the invention

[0005] The purpose of the present invention is to solve the problems raised in the background technology and to provide a data storage write optimization method and device for adaptive caching.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] The present invention proposes an adaptive cache data storage write optimization method, which includes creating three independent transaction groups in a storage pool, namely an open transaction group, an intermediate transaction group and a synchronous transaction group, and initializing the capacity of the open transaction group to cache dirty data;

[0008] For dirty data newly written into the storage pool, the dirty data is first stored in the open transaction group. When the capacity of the open transaction group is full or the dirty data reception reaches the first preset time, the dirty data in the open transaction group is transferred to the intermediate transaction group, and a new open transaction group is created to receive the newly written dirty data.

[0009] Transfer the dirty data received by the intermediate transaction group to the synchronous transaction group, and then write the dirty data received in the synchronous transaction group to the backend disk;

[0010] In the process of writing the dirty data received in the synchronous transaction group to the backend disk, recording a first average rate at which the dirty data in the current synchronous transaction group is written to the backend disk within a second preset time and a second average rate at which all the dirty data in the current synchronous transaction group is written to the backend disk;

[0011] The first average rate is compared with the second average rate, and the capacity of the dirty data cached by the next open transaction or the next interval open transaction group is dynamically adjusted.

[0012] Preferably, during the dirty data transfer process, after the current dirty data in the synchronous transaction group is completely written to the backend disk, the dirty data in the intermediate transaction group begins to be transferred to the synchronous transaction group, otherwise the dirty data in the intermediate transaction group is in a waiting state;

[0013] When the current dirty data in the intermediate transaction group is completely transferred to the synchronization transaction group, the dirty data in the open transaction group will be written into the intermediate transaction group. Otherwise, the dirty data in the open transaction group will be in a waiting state.

[0014] Preferably, comparing the first average rate with the second average rate and dynamically adjusting the capacity of the next open transaction or the next interval open transaction group to cache dirty data includes:

[0015] When the second average rate is equal to the first average rate, the capacity of the dirty data cache of the open transaction group is not adjusted;

[0016] When the second average rate is greater than the first average rate, the ratio of the second average rate to the first average rate is calculated, and then the capacity of the dirty data cache of the next open transaction group is adjusted to be reduced by the same ratio;

[0017] When the second average rate is less than the first average rate, the ratio of the second average rate to the first average rate is calculated, and then the capacity of the dirty data cached in the next interval open transaction group is increased by the same ratio, and the capacity of the dirty data cached in the next open transaction group is not adjusted.

[0018] Preferably, the adaptive cache data storage write optimization method also includes opening a shared cache area in the storage pool, and when the second average rate is greater than the first average rate, the capacity of the dirty data cached by the next open transaction group is adjusted to be reduced by the same proportion, and the dirty data in the next open transaction group that exceeds the adjusted cache capacity is transferred to the shared cache area, and for the next interval open transaction group, the dirty data in the shared cache area is preferentially written, and then the newly written dirty data continues to be received until the capacity of the open transaction group is full or the dirty data is received for a first preset time.

[0019] Preferably, the adaptive cache data storage write optimization method further includes: when the writing of dirty data into an open transaction group fails, setting an adjustment strategy, and the adjustment strategy includes:

[0020] First, determine whether the amount of dirty data in the current open transaction group has reached the most recently adjusted capacity. If so, wait for the third preset time before retrying to write the dirty data to the open transaction group. If the write fails again, the waiting time is extended by twice the last waiting time. Repeat the operation until the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time.

[0021] When the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time, wait until all dirty data in the current synchronization transaction group is written to the backend disk before retrying to write the dirty data to the open transaction group.

[0022] A data storage write optimization device with adaptive caching, comprising:

[0023] The transaction group module is used to create three independent transaction groups in the storage pool, namely, the open transaction group, the intermediate transaction group, and the synchronous transaction group, and initialize the capacity of the open transaction group to cache dirty data;

[0024] The transaction group module is used to store the newly written dirty data in the storage pool into the open transaction group. When the capacity of the open transaction group is full or the dirty data is received for a first preset time, the dirty data in the open transaction group is transferred to the intermediate transaction group, and a new open transaction group is created to receive the newly written dirty data.

[0025] The transaction group module is used to transfer the dirty data received by the intermediate transaction group to the synchronization transaction group, and then write the dirty data received in the synchronization transaction group to the backend disk;

[0026] A detection module, used for recording a first average rate at which dirty data in the current synchronous transaction group is written to the backend disk and a second average rate at which all dirty data in the current synchronous transaction group is written to the backend disk during a second preset time period when dirty data received in the synchronous transaction group is written to the backend disk;

[0027] The cache adjustment module is used to compare the first average rate with the second average rate and dynamically adjust the capacity of the next open transaction or the next interval open transaction group to cache dirty data.

[0028] Preferably, during the dirty data transfer process, the transaction group module is configured as follows:

[0029] When the current dirty data in the synchronous transaction group is completely written to the backend disk, the dirty data in the intermediate transaction group will be transferred to the synchronous transaction group. Otherwise, the dirty data in the intermediate transaction group will be in a waiting state.

[0030] When the current dirty data in the intermediate transaction group is completely transferred to the synchronization transaction group, the dirty data in the open transaction group will be written into the intermediate transaction group. Otherwise, the dirty data in the open transaction group will be in a waiting state.

[0031] Preferably, during the dirty data transfer process, the transaction group module is configured as follows:

[0032] When the current dirty data in the synchronous transaction group is completely written to the backend disk, the dirty data in the intermediate transaction group will be transferred to the synchronous transaction group. Otherwise, the dirty data in the intermediate transaction group will be in a waiting state.

[0033] When the current dirty data in the intermediate transaction group is completely transferred to the synchronization transaction group, the dirty data in the open transaction group will be written into the intermediate transaction group. Otherwise, the dirty data in the open transaction group will be in a waiting state.

[0034] Preferably, the adaptive cache data storage write optimization device further comprises opening a shared cache area in the storage pool, and the cache adjustment module is further configured as follows:

[0035] When the second average rate is greater than the first average rate, and the capacity of the next open transaction group to cache dirty data is adjusted to be reduced by the same proportion, the dirty data in the next open transaction group that exceeds the adjusted cache capacity is transferred to the shared cache area, and for the next interval open transaction group, the dirty data in the shared cache area is written first, and then the newly written dirty data continues to be received until the capacity of the open transaction group is full or the dirty data is received for a first preset time.

[0036] Preferably, the data storage write optimization device of the adaptive cache is further configured as follows:

[0037] When dirty data fails to be written to an open transaction group, a regulation policy is set and operates as follows:

[0038] First, determine whether the amount of dirty data in the current open transaction group has reached the most recently adjusted capacity. If so, wait for the third preset time before retrying to write the dirty data to the open transaction group. If the write fails again, the waiting time is extended by twice the last waiting time. Repeat the operation until the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time.

[0039] When the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time, wait until all dirty data in the current synchronization transaction group is written to the backend disk before retrying to write the dirty data to the open transaction group.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The adaptive cache data storage write optimization method and device dynamically adjusts the cache capacity of the open transaction group in the storage pool to avoid the slowdown of writing data in the storage pool to the back-end disk, which causes the transaction group dirty data to be written to the back-end disk for a long time, resulting in business I / O timeout interruption, thereby improving reliability and reducing the impact on upper-layer business I / O.

[0042] The data storage write optimization method and device of the adaptive cache sets an adjustment strategy when the writing of dirty data to the open transaction group fails. When the number of times the dirty data cannot be written to the development transaction group reaches the threshold of the number of rewrites or the waiting time reaches max_timeout, the writing to the open transaction group will no longer be retried, thereby reducing the overhead of system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flow chart of a method and device for optimizing data storage and writing in an adaptive cache according to the present invention;

[0044] Figure 2 A schematic diagram of adjusting the cache capacity of an open transaction group according to the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] In one embodiment, Figure 1-Figure 2 As shown, a data storage write optimization method for adaptive caching is provided, comprising:

[0047] S1. Create three independent transaction groups in the storage pool, namely, the open transaction group, the intermediate transaction group, and the synchronous transaction group, and initialize the capacity of the open transaction group to cache dirty data (for the first time receiving dirty data, set the initial value of the capacity of the open transaction group to cache dirty data);

[0048] The capacity of the intermediate transaction group and the synchronous transaction group changes with the capacity of the open transaction group. The data written into the storage pool is managed through three independent transaction groups and written to the backend disk.

[0049] S2. For dirty data newly written into the storage pool, the dirty data is first stored in an open transaction group. When the capacity of the open transaction group is full or the dirty data reception reaches a first preset time, the dirty data in the open transaction group is transferred to the intermediate transaction group, and a new open transaction group is created to receive the newly written dirty data.

[0050] Among them, for the dirty data (business I / O data) newly written into the storage pool, it is first inserted into the open transaction group cache list (that is, the dirty data is first stored in the open transaction group). When the capacity of the open transaction group is full or the dirty data is received for a first preset time (when the dirty data written to the open transaction group this time reaches the current cache capacity of the open transaction group or the current open transaction group receives dirty data for a first preset time, where the first preset time is set according to actual conditions, such as 10 seconds), the dirty data in the open transaction group is transferred to the intermediate transaction group, and a new open transaction group is created at the same time (a number is set for each open transaction group, and the number increases by one each time a new open transaction group is created) to receive the dirty data newly written to the storage until the end.

[0051] S3: Transfer the dirty data received by the intermediate transaction group to the synchronous transaction group, and then write the dirty data received in the synchronous transaction group to the backend disk;

[0052] Among them, when the intermediate transaction group receives all the current dirty data in the open transaction group, the intermediate transaction group transfers the received data to the synchronization transaction group. After the synchronization transaction group receives all the current dirty data in the intermediate transaction group, the synchronization transaction group writes the received dirty data to the backend disk.

[0053] During the dirty data transfer process, when the current dirty data in the synchronous transaction group is completely written to the backend disk, the dirty data in the intermediate transaction group will be transferred to the synchronous transaction group. Otherwise, the dirty data in the intermediate transaction group will be in a waiting state (when the cached data of the synchronous transaction group is not completely written to the backend disk, the intermediate transaction group will be in a waiting state and its cached data cannot be transferred to the synchronous transaction group);

[0054] When the current dirty data in the intermediate transaction group is completely transferred to the synchronization transaction group, the dirty data in the open transaction group will be written to the intermediate transaction group. Otherwise, the dirty data in the open transaction group will be in a waiting state (the intermediate transaction group still stores cached data and cannot obtain the cached data in the open transaction group at this time. The open transaction group will also be in a waiting state after the dirty data is full).

[0055] When the dirty data in the open transaction group is completely written to the intermediate transaction group, a new open transaction group is created and immediately receives the new data.

[0056] That is, each transaction group will receive the next newly written data only when it is in an idle state.

[0057] S4, in the process of writing the dirty data received in the synchronous transaction group to the backend disk, recording a first average rate at which the dirty data in the current synchronous transaction group is written to the backend disk within a second preset time and a second average rate at which all the dirty data in the current synchronous transaction group is written to the backend disk;

[0058] Among them, the second preset time can be set according to actual needs. For example, in this embodiment, the second preset time is 5 seconds, and the first average rate of writing dirty data in the current synchronization transaction group to the back-end disk within 5 seconds is recorded (that is, the amount of dirty data written to the back-end disk by the current synchronization transaction group within 5 seconds is divided by 5 seconds to obtain the first average rate, wherein 5 seconds is calculated from the time when the current synchronization transaction group starts writing data to the back-end disk), and the second average rate of writing all dirty data in the current synchronization transaction group to the back-end disk is recorded (that is, the amount of all dirty data in the current synchronization transaction group is divided by the time required to write to the back-end disk to obtain the second average rate).

[0059] S5. Compare the first average rate with the second average rate, and dynamically adjust the capacity of the next open transaction or the next interval open transaction group to cache dirty data, specifically including:

[0060] When the second average rate is equal to the first average rate, the capacity of the dirty data cache of the open transaction group is not adjusted;

[0061] When the second average rate is greater than the first average rate, the ratio of the second average rate to the first average rate is calculated, and then the capacity of the dirty data cache of the next open transaction group is adjusted to be reduced by the same ratio;

[0062] If the second average rate is 10% greater than the first average rate, the capacity of the dirty data cache of the next open transaction group is adjusted to be reduced by 10%, thereby reducing the time for the next synchronous transaction group to write dirty data to disk and avoiding long-term blocking of each transaction group (the number of the next open transaction group is increased by one relative to the current open transaction);

[0063] When the second average rate is less than the first average rate, the ratio of the second average rate to the first average rate is calculated, and then the capacity of the dirty data cached in the next interval open transaction group is increased by the same ratio, and the capacity of the dirty data cached in the next open transaction group is not adjusted;

[0064] Among them, if the second average rate is 10% smaller than the first average rate, it means that the write performance of the storage pool has improved, and the storage pool can write more dirty data per unit time. In this case, the capacity of the next interval open transaction group to cache dirty data is increased by 10% (the next interval open transaction group is the next next open transaction group, and the number of the current open transaction increases by two). At this time, the amount of dirty data in the next open transaction group is relatively small, but no adjustment is made, and writing continues to the subsequent transaction group.

[0065] In this embodiment, the data storage write optimization method of adaptive cache also includes opening a shared cache area in the storage pool (the shared cache area is used to temporarily store overflow data of the open transaction group cache). When the second average rate is greater than the first average rate, the capacity of the dirty data cached by the next open transaction group is adjusted to be reduced by the same proportion, and the dirty data in the next open transaction group that exceeds the adjusted cache capacity is transferred to the shared cache area. For the next interval open transaction group, the dirty data in the shared cache area is preferentially written, and then the newly written dirty data is continued to be received until the capacity of the open transaction group is full or the dirty data is received for a first preset time.

[0066] In this embodiment, the data storage write optimization method of the adaptive cache further includes: when the dirty data fails to be written into the open transaction group, an adjustment strategy is set, and the adjustment strategy includes (the reasons for the failure to write the dirty data into the open transaction group are: first, the dirty data in the current open transaction group has reached the latest adjusted capacity; second, the backend disk is a solid state disk and garbage collection is in progress, etc., resulting in an increase in I / O write latency):

[0067] First, determine whether the amount of dirty data in the current open transaction group has reached the latest adjusted capacity. If so, wait for the third preset time (the third preset time is set according to the actual situation and is expressed as timeout), and then retry to write the dirty data to the open transaction group. If the write fails again, the waiting time is extended by twice the last waiting time (that is, the waiting time at this time is 2 times the timeout, and so on. The waiting time for each attempt to rewrite is 2 times the last waiting time). Repeat the operation until the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time (the threshold of the number of rewrites and the fourth preset time are set according to the actual situation, which is the maximum waiting time, expressed as max_timeout); when the number of times that the dirty data cannot be written to the development transaction group reaches the threshold of the number of rewrites or the waiting time reaches max_timeout, it will no longer continue to retry writing to the open transaction group, thereby reducing the overhead of system resources;

[0068] When the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time, wait until all dirty data in the current synchronization transaction group is written to the backend disk before retrying to write the dirty data to the open transaction group to reduce CPU resource overhead. The longer it takes to write dirty data in the cache of the storage pool synchronization transaction group to the backend disk, the longer the business I / O will wait to write to the cache of the open transaction group.

[0069] In another embodiment, Figure 1-Figure 2 As shown, based on the data storage write optimization method based on adaptive cache, a data storage write optimization device based on adaptive cache is proposed, comprising:

[0070] The transaction group module is used to create three independent transaction groups in the storage pool, namely, the open transaction group, the intermediate transaction group, and the synchronous transaction group, and initialize the capacity of the open transaction group to cache dirty data;

[0071] The transaction group module is used to store the newly written dirty data in the storage pool into the open transaction group. When the capacity of the open transaction group is full or the dirty data is received for a first preset time, the dirty data in the open transaction group is transferred to the intermediate transaction group, and a new open transaction group is created to receive the newly written dirty data.

[0072] The transaction group module is used to transfer the dirty data received by the intermediate transaction group to the synchronization transaction group, and then write the dirty data received in the synchronization transaction group to the backend disk;

[0073] A detection module, used for recording a first average rate at which dirty data in the current synchronous transaction group is written to the backend disk and a second average rate at which all dirty data in the current synchronous transaction group is written to the backend disk during a second preset time period when dirty data received in the synchronous transaction group is written to the backend disk;

[0074] The cache adjustment module is used to compare the first average rate with the second average rate and dynamically adjust the capacity of the next open transaction or the next interval open transaction group to cache dirty data.

[0075] In this embodiment, during the dirty data transfer process, the transaction group module is configured as follows:

[0076] When the current dirty data in the synchronous transaction group is completely written to the backend disk, the dirty data in the intermediate transaction group will be transferred to the synchronous transaction group. Otherwise, the dirty data in the intermediate transaction group will be in a waiting state.

[0077] When the current dirty data in the intermediate transaction group is completely transferred to the synchronization transaction group, the dirty data in the open transaction group will be written into the intermediate transaction group. Otherwise, the dirty data in the open transaction group will be in a waiting state.

[0078] In this embodiment, the first average rate is compared with the second average rate, and the capacity of the next open transaction or the next interval open transaction group to cache dirty data is dynamically adjusted. The cache adjustment module is configured as follows:

[0079] When the second average rate is greater than the first average rate, the ratio of the second average rate to the first average rate is calculated, and then the capacity of the dirty data cache of the next open transaction group is adjusted to be reduced by the same ratio;

[0080] When the second average rate is less than the first average rate, the ratio of the second average rate to the first average rate is calculated, and then the capacity of the dirty data cached in the next interval open transaction group is increased by the same ratio, and the capacity of the dirty data cached in the next open transaction group is not adjusted.

[0081] In this embodiment, the data storage write optimization device of adaptive cache further includes opening a shared cache area in the storage pool, and the cache adjustment module is further configured as follows:

[0082] When the second average rate is greater than the first average rate, and the capacity of the next open transaction group to cache dirty data is adjusted to be reduced by the same proportion, the dirty data in the next open transaction group that exceeds the adjusted cache capacity is transferred to the shared cache area, and for the next interval open transaction group, the dirty data in the shared cache area is written first, and then the newly written dirty data continues to be received until the capacity of the open transaction group is full or the dirty data is received for a first preset time.

[0083] In this embodiment, the data storage write optimization device of the adaptive cache is further configured as follows:

[0084] When dirty data fails to be written to an open transaction group, a regulation policy is set and operates as follows:

[0085] First, determine whether the amount of dirty data in the current open transaction group has reached the most recently adjusted capacity. If so, wait for the third preset time before retrying to write the dirty data to the open transaction group. If the write fails again, the waiting time is extended by twice the last waiting time. Repeat the operation until the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time.

[0086] When the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time, wait until all dirty data in the current synchronization transaction group is written to the backend disk before retrying to write the dirty data to the open transaction group.

[0087] The adaptive cache data storage write optimization method and device dynamically adjust the cache capacity of the open transaction group in the storage pool to avoid the slowdown in writing data into the back-end disk in the storage pool, which causes the transaction group's dirty data to be written to the back-end disk for too long, resulting in business I / O timeout interruption, thereby improving reliability and reducing the impact on upper-level business I / O. The adaptive cache data storage write optimization method and device sets an adjustment strategy when the writing of dirty data to the open transaction group fails. When the number of times that the dirty data cannot be written to the development transaction group reaches the threshold of the number of rewrites or the waiting time reaches max_timeout, the writing to the open transaction group will no longer be retried, thereby reducing the overhead of system resources.

[0088] For the specific limitations of the data storage write optimization device based on adaptive cache, please refer to the limitations of the data storage write optimization method based on adaptive cache above, which will not be repeated here. Each module in the above-mentioned data storage write optimization device based on adaptive cache can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0089] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0090] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A data storage write optimization method for adaptive caching, characterized in that: The adaptive cache data storage write optimization method comprises: Create three independent transaction groups in the storage pool, namely the open transaction group, the intermediate transaction group, and the synchronous transaction group, and initialize the capacity of the open transaction group to cache dirty data; For dirty data newly written into the storage pool, the dirty data is first stored in the open transaction group. When the capacity of the open transaction group is full or the dirty data reception reaches the first preset time, the dirty data in the open transaction group is transferred to the intermediate transaction group, and a new open transaction group is created to receive the newly written dirty data. Transfer the dirty data received by the intermediate transaction group to the synchronous transaction group, and then write the dirty data received in the synchronous transaction group to the backend disk; In the process of writing the dirty data received in the synchronous transaction group to the backend disk, recording a first average rate at which the dirty data in the current synchronous transaction group is written to the backend disk within a second preset time and a second average rate at which all the dirty data in the current synchronous transaction group is written to the backend disk; The first average rate is compared with the second average rate, and the capacity of the dirty data cached by the next open transaction or the next interval open transaction group is dynamically adjusted.

2. The data storage write optimization method of adaptive cache according to claim 1, characterized in that: During the dirty data transfer process, when the current dirty data in the synchronous transaction group is completely written to the backend disk, the dirty data in the intermediate transaction group will start to be transferred to the synchronous transaction group. Otherwise, the dirty data in the intermediate transaction group will be in a waiting state. When the current dirty data in the intermediate transaction group is completely transferred to the synchronization transaction group, the dirty data in the open transaction group will be written into the intermediate transaction group. Otherwise, the dirty data in the open transaction group will be in a waiting state.

3. The data storage write optimization method of adaptive cache according to claim 1, characterized in that: The step of comparing the first average rate with the second average rate and dynamically adjusting the capacity of the next open transaction or the next interval open transaction group to cache dirty data includes: When the second average rate is equal to the first average rate, the capacity of the dirty data cache of the open transaction group is not adjusted; When the second average rate is greater than the first average rate, the ratio of the second average rate to the first average rate is calculated, and then the capacity of the dirty data cache of the next open transaction group is adjusted to be reduced by the same ratio; When the second average rate is less than the first average rate, the ratio of the second average rate to the first average rate is calculated, and then the capacity of the dirty data cached in the next interval open transaction group is increased by the same ratio, and the capacity of the dirty data cached in the next open transaction group is not adjusted.

4. The data storage write optimization method of adaptive cache as claimed in claim 3, characterized in that: The adaptive cache data storage write optimization method also includes opening a shared cache area in the storage pool, and when the second average rate is greater than the first average rate, the capacity of the dirty data cached by the next open transaction group is adjusted to be reduced by the same proportion, and the dirty data in the next open transaction group that exceeds the adjusted cache capacity is transferred to the shared cache area, and for the next interval open transaction group, the dirty data in the shared cache area is preferentially written, and then the newly written dirty data continues to be received until the capacity of the open transaction group is full or the dirty data reception reaches a first preset time.

5. The data storage write optimization method of adaptive cache according to claim 1, characterized in that: The adaptive cache data storage write optimization method further includes: when dirty data fails to be written into an open transaction group, setting an adjustment strategy, and the adjustment strategy includes: First, determine whether the amount of dirty data in the current open transaction group has reached the most recently adjusted capacity. If so, wait for the third preset time before retrying to write the dirty data to the open transaction group. If the write fails again, the waiting time is extended by twice the last waiting time. Repeat the operation until the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time. When the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time, wait until all dirty data in the current synchronization transaction group is written to the backend disk before retrying to write the dirty data to the open transaction group.

6. A data storage write optimization device with adaptive caching, characterized in that: The adaptive cache data storage write optimization device comprises: The transaction group module is used to create three independent transaction groups in the storage pool, namely, the open transaction group, the intermediate transaction group, and the synchronous transaction group, and initialize the capacity of the open transaction group to cache dirty data; The transaction group module is used to store the newly written dirty data in the storage pool into the open transaction group. When the capacity of the open transaction group is full or the dirty data is received for a first preset time, the dirty data in the open transaction group is transferred to the intermediate transaction group, and a new open transaction group is created to receive the newly written dirty data. The transaction group module is used to transfer the dirty data received by the intermediate transaction group to the synchronization transaction group, and then write the dirty data received in the synchronization transaction group to the backend disk; A detection module, used for recording a first average rate at which dirty data in the current synchronous transaction group is written to the backend disk and a second average rate at which all dirty data in the current synchronous transaction group is written to the backend disk during a second preset time period when dirty data received in the synchronous transaction group is written to the backend disk; The cache adjustment module is used to compare the first average rate with the second average rate and dynamically adjust the capacity of the next open transaction or the next interval open transaction group to cache dirty data.

7. The data storage write optimization device of the adaptive cache as claimed in claim 6, characterized in that: During the dirty data transfer process, the transaction group module is configured as follows: When the current dirty data in the synchronous transaction group is completely written to the backend disk, the dirty data in the intermediate transaction group will be transferred to the synchronous transaction group. Otherwise, the dirty data in the intermediate transaction group will be in a waiting state. When the current dirty data in the intermediate transaction group is completely transferred to the synchronization transaction group, the dirty data in the open transaction group will be written into the intermediate transaction group. Otherwise, the dirty data in the open transaction group will be in a waiting state.

8. The data storage write optimization device of the adaptive cache as claimed in claim 6, characterized in that: The first average rate is compared with the second average rate, and the capacity of the next open transaction or the next interval open transaction group to cache dirty data is dynamically adjusted. The cache adjustment module is configured as follows: When the second average rate is equal to the first average rate, the capacity of the dirty data cache of the open transaction group is not adjusted; When the second average rate is greater than the first average rate, the ratio of the second average rate to the first average rate is calculated, and then the capacity of the dirty data cache of the next open transaction group is adjusted to be reduced by the same ratio; When the second average rate is less than the first average rate, the ratio of the second average rate to the first average rate is calculated, and then the capacity of the dirty data cached in the next interval open transaction group is increased by the same ratio, and the capacity of the dirty data cached in the next open transaction group is not adjusted.

9. The data storage write optimization device of adaptive cache as claimed in claim 8, characterized in that: The adaptive cache data storage write optimization device further includes opening a shared cache area in the storage pool, and the cache adjustment module is further configured as follows: When the second average rate is greater than the first average rate, and the capacity of the next open transaction group to cache dirty data is adjusted to be reduced by the same proportion, the dirty data in the next open transaction group that exceeds the adjusted cache capacity is transferred to the shared cache area, and for the next interval open transaction group, the dirty data in the shared cache area is written first, and then the newly written dirty data continues to be received until the capacity of the open transaction group is full or the dirty data is received for a first preset time.

10. The data storage write optimization device of the adaptive cache as claimed in claim 6, characterized in that: The data storage write optimization device of the adaptive cache is further configured as follows: When dirty data fails to be written to an open transaction group, a regulation policy is set and operates as follows: First, determine whether the amount of dirty data in the current open transaction group has reached the most recently adjusted capacity. If so, wait for the third preset time before retrying to write the dirty data to the open transaction group. If the write fails again, the waiting time is extended by twice the last waiting time. Repeat the operation until the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time. When the number of rewrites exceeds the set threshold or the total waiting time exceeds the fourth preset time, wait until all dirty data in the current synchronization transaction group is written to the backend disk before retrying to write the dirty data to the open transaction group.

Images