Data storage method, system, device and storage medium of all-flash storage system

Abstract

The application discloses a data storage method and system of a full flash storage system, a device and a storage medium, and applies to the technical field of data storage. The method comprises the following steps: receiving a write request; distributing the write request to a processor in a current first resource pool according to a preset scheduling rule, so that the processor processes the write request according to a preset data aggregation rule; determining the proportion of the number of memory spaces filled and then written into a disk in a cycle to the total number of memory spaces written into the disk as a full data block storage ratio of the cycle every time a cycle is passed; and adjusting the number of processors in the first resource pool based on the full data block storage ratio. The application can effectively complete data storage of the full flash storage system, improve the service life of an SSD, and reduce IO time delay.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and in particular to a data storage method, system, device and storage medium for an all-flash storage system. Background Technology

[0002] All-flash storage systems use SSDs (Solid State Disks) as the backend storage medium. SSDs use logic circuits to read and write data, resulting in low latency. However, SSDs require erasure before data can be written, and the number of times a single physical block can be erased is limited. Therefore, to improve the lifespan of SSDs, typical all-flash software stacks aggregate small data blocks into larger ones, significantly reducing the number of erase / write cycles and extending the lifespan of the SSD.

[0003] However, while improving SSD lifespan, it also comes at the cost of increased IO (Input / Output) latency, because the IO write process adds the step of aggregating small blocks into larger ones.

[0004] Current all-flash storage systems pre-configure a certain number of CPUs (Central Processing Units), with each CPU core allocated a large resource pool. A large block is only written to disk when all smaller blocks are aggregated into a single large block. If the I / O concurrency is low, each CPU core handles fewer concurrent I / O operations, resulting in a lower probability of aggregating into a large block and increased I / O latency. If the I / O concurrency is high, although each CPU core can handle enough concurrent I / O operations to aggregate into a large block, insufficient CPU resources can increase the aggregation time, also leading to increased I / O latency. Furthermore, in real-world applications, the concurrency level frequently fluctuates, causing performance jitter and impacting customer business.

[0005] In summary, how to effectively complete data storage in an all-flash storage system, improve SSD lifespan, and reduce IO latency is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a data storage method, system, device, and storage medium for an all-flash storage system, so as to effectively complete the data storage of the all-flash storage system, improve the lifespan of SSDs, and reduce IO latency.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] A data storage method for an all-flash storage system includes:

[0009] Receive write requests;

[0010] According to the preset scheduling rules, the write request is allocated to the processor in the current first resource pool, so that the processor processes the write request according to the preset data aggregation rules;

[0011] Each time a cycle is completed, the proportion of memory space that is filled and then written to disk within that cycle is determined, and is taken as the full data block storage ratio for that cycle.

[0012] Based on the full data block storage ratio, adjust the number of processors in the first resource pool;

[0013] The first resource pool includes at least one processor; the data aggregation rules include: the processor requests a memory space of a first size to process write requests allocated to itself; when the memory space is full, the processor writes the data in the memory space to the disk; when the memory space is not full but the time elapsed after requesting the memory space reaches the timeout period, the processor writes the data in the memory space to the disk.

[0014] In one implementation, allocating the write request to a processor in the current first resource pool according to a preset scheduling rule includes:

[0015] Determine the count value corresponding to the write request;

[0016] Divide the count value by the current number of processors in the first resource pool to obtain the remainder value a;

[0017] Assign the write request to the processor numbered a;

[0018] The initial value of the counter is 0, and the value of the counter increases by 1 each time a write request is received.

[0019] In one implementation, adjusting the number of processors in the first resource pool based on the full data block storage ratio includes:

[0020] Determine whether the full data block storage ratio is greater than or equal to a preset first threshold;

[0021] If the number of processors in the first resource pool is greater than or equal to the first threshold, then the number of processors in the first resource pool is increased.

[0022] If it is less than the first threshold, then determine whether the full data block storage ratio is less than the preset second threshold;

[0023] If it is less than the second threshold, then reduce the number of processors in the first resource pool;

[0024] If the number of processors in the first resource pool is not less than the second threshold, the number of processors in the first resource pool will remain unchanged.

[0025] In one implementation, it further includes:

[0026] Each time a cycle is completed, the average duration of existence of each memory space within the cycle is determined as the average data block lifetime of the cycle; wherein, for any memory space, the duration of existence of the memory space represents the time elapsed from when the memory space is allocated until the data in the memory space is written to the disk.

[0027] Accordingly, adjusting the number of processors in the first resource pool based on the full data block storage ratio includes:

[0028] The number of processors in the first resource pool is adjusted based on the full data block storage ratio and the average data block lifetime.

[0029] In one implementation, adjusting the number of processors in the first resource pool based on the full data block storage ratio and the average data block lifetime includes:

[0030] Every N preset cycles, the processor number adjustment value for each preset cycle is determined;

[0031] Filter the N processor number adjustment values ​​that have the same value as x, and determine the average value of the remaining processor number adjustment values ​​after filtering.

[0032] The average of the remaining processor quantity adjustment values ​​after filtering is rounded down.

[0033] When the rounded value is different from x, the number of processors in the first resource pool is adjusted to the rounded value.

[0034] Where N is a positive integer not less than 2. For any one of the N preset periods, when the first condition is met, the processor number adjustment value of the period is x+b; x represents the number of processors in the current first resource pool, and b is a positive number; when the second condition is met, the processor number adjustment value of the period is xc; c is a positive number; when neither the first condition nor the second condition is met, the processor number adjustment value of the period is x.

[0035] The first condition includes: the proportion of full data blocks stored in the period is greater than or equal to a preset first threshold, and the average data block lifetime of the period is less than a preset first duration;

[0036] The second condition includes: the proportion of full data blocks stored in the cycle is less than a preset second threshold.

[0037] In one implementation, it further includes:

[0038] Determine whether the number of processor number adjustment values ​​with the same value as x among the N determined processor number adjustment values ​​has reached the first quantity threshold.

[0039] If so, then cancel the current adjustment to the number of processors in the first resource pool and maintain the current number of processors in the first resource pool unchanged.

[0040] In one implementation, it further includes:

[0041] After adjusting the number of processors in the first resource pool to the rounded value, the adjustment to the number of processors in the first resource pool is cancelled within the second time period, and the current number of processors in the first resource pool remains unchanged within the second time period.

[0042] A data storage system for an all-flash storage system, comprising:

[0043] The write request receiving module is used to receive write requests;

[0044] The write request allocation module is used to allocate the write request to the processor in the current first resource pool according to the preset scheduling rules, so that the processor processes the write request according to the preset data aggregation rules.

[0045] The statistics module is used to determine, each time a cycle has elapsed, the proportion of memory space that has been filled and then written to disk within the cycle to the total amount of memory space written to disk, as the full data block storage ratio of the cycle.

[0046] The scheduling module is used to adjust the number of processors in the first resource pool based on the full data block storage ratio;

[0047] The first resource pool includes at least one processor; the data aggregation rules include: the processor requests a memory space of a first size to process write requests allocated to itself; when the memory space is full, the processor writes the data in the memory space to the disk; when the memory space is not full but the time elapsed after requesting the memory space reaches the timeout period, the processor writes the data in the memory space to the disk.

[0048] A data storage device for an all-flash storage system, comprising:

[0049] Memory, used to store computer programs;

[0050] A processor for executing the computer program to implement the steps of the data storage method for the all-flash storage system as described above.

[0051] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the data storage method for the all-flash storage system as described above.

[0052] By applying the technical solution provided in this embodiment of the invention, after receiving a write request, the write request can be allocated to a processor in the current first resource pool according to a preset scheduling rule. The processor then processes the write request according to a preset data aggregation rule. The data aggregation rule includes: the processor requests a memory space of a first size to process the write request allocated to it; when the memory space is full, the processor writes the data in the memory space to the disk; when the memory space is not full but the time elapsed after requesting the memory space reaches the timeout period, the processor writes the data in the memory space to the disk. Therefore, it can be seen that the processor of this application performs a data aggregation process when processing write requests, i.e., when processing I / O, that is, it sets up a process of aggregating small blocks into large blocks, thus effectively ensuring the service life of the all-flash storage system.

[0053] Furthermore, this application considers that traditional solutions cause increased IO latency because the number of IO concurrency changes continuously while the number of CPUs remains fixed. In other words, the fixed number of CPUs in traditional solutions cannot effectively handle varying IO concurrency levels. Therefore, this application proposes that, after each cycle, the proportion of memory space filled and then written to disk within that cycle, relative to the total memory space written to disk, can be determined as the full data block storage ratio for that cycle. The full data block storage ratio effectively reflects the utilization rate of processor resources and indicates the current IO concurrency situation. Therefore, this application's solution adjusts the number of processors in the first resource pool based on the full data block storage ratio, ensuring that the number of processors in the first resource pool is adjusted to an appropriate value to effectively adapt to the current IO concurrency situation. This allows the solution of this application to effectively reduce IO latency.

[0054] In summary, the solution proposed in this application can effectively complete the data storage of the all-flash storage system, improve the lifespan of the SSD, and reduce IO latency. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart illustrating the implementation of a data storage method for an all-flash storage system according to the present invention.

[0057] Figure 2 This is a schematic diagram of the data storage method in a specific embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram of the data storage system of an all-flash storage system according to the present invention;

[0059] Figure 4 This is a schematic diagram of the structure of a data storage device in an all-flash storage system according to the present invention;

[0060] Figure 5 This is a schematic diagram of the structure of a computer-readable storage medium according to the present invention. Detailed Implementation

[0061] The core of this invention is to provide a data storage method for an all-flash storage system, which can effectively complete the data storage of the all-flash storage system, improve the lifespan of SSDs, and reduce IO latency.

[0062] To enable those skilled in the art to better understand the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an implementation of a data storage method for an all-flash storage system according to the present invention. The data storage method for the all-flash storage system may include the following steps:

[0064] Step S101: Receive write request.

[0065] For details, please refer to Figure 2 This is a schematic diagram illustrating the framework of a data storage method in a specific embodiment of the present invention. Figure 2The "upper-layer business" in this context refers to services that can serve as input to the all-flash storage system, providing concurrent I / O. Specifically, this application focuses on the processing of write requests within I / O. Furthermore, it's understood that in practical applications, the I / O concurrency can vary for different services, and even for the same service, it can fluctuate significantly at different times. In other words, the I / O concurrency may continuously change, meaning the total number of write requests received may change over a period of time. Therefore, the solution in this application dynamically adjusts the number of processors in the first resource pool to achieve an appropriate value, thereby effectively reducing I / O latency.

[0066] Step S102: According to the preset scheduling rules, the write request is assigned to the processor in the current first resource pool so that the processor processes the write request according to the preset data aggregation rules.

[0067] After receiving a write request, it needs to be allocated to one processor in the current first resource pool. It's understood that the first resource pool includes at least one processor. The allocation is performed according to a preset scheduling rule. The specific content of the scheduling rule can be set and adjusted according to actual needs without affecting the implementation of this invention. Generally, the scheduling rule should ensure that each processor in the current first resource pool has an equal opportunity to receive a write request, thus achieving an average distribution of write requests.

[0068] For example, in one specific embodiment of the present invention, the step S102, which describes allocating write requests to processors in the current first resource pool according to preset scheduling rules, may specifically include:

[0069] Determine the counter value corresponding to the write request;

[0070] Divide the count value by the number of processors in the current first resource pool to obtain the remainder value a;

[0071] Assign the write request to the processor with the number 'a';

[0072] The initial value of the counter is 0, and the value increases by 1 each time a write request is received.

[0073] In this implementation, a global counter value can be maintained, for example, represented by g_io_dispatch_count. The initial value of this counter value is 0, and the value is incremented by 1 each time a write request is received. In other words, after each write request is received, the value of this counter value can be regarded as the number of that write request.

[0074] After determining the count value corresponding to the current write request, divide the count value by the number of processors in the current first resource pool, and denote the remainder as 'a'. For example, in one scenario, after receiving one write request, the count value becomes 100, so the count value corresponding to the write request is 100. For example, if the number of processors in the current first resource pool is 8, then dividing 100 by 8 will give a remainder of 4, i.e., a = 4 in this example. At this point, the write request can be assigned to the processor numbered 4. It can be seen that in this example, the 8 processors are numbered sequentially from 0 to 7.

[0075] As can be seen, this implementation method can achieve an even distribution of write requests and is relatively simple and convenient to implement.

[0076] The processor processes write requests allocated to itself according to preset data aggregation rules. These rules may include: the processor requests a memory space of a first size to process the write requests allocated to it; when the memory space is full, the processor writes the data in the memory space to the disk; and when the memory space is not full but the timeout period after requesting the memory space has expired, the processor writes the data in the memory space to the disk.

[0077] Figure 2 The memory resource allocation and release in the context refers to the function of allocating and releasing memory resources. Specifically, for each processor in the first resource pool, this function can be used to allocate a certain amount of memory resources to that processor. When a processor processes a write request, it can request a memory space of a first size from its own memory resources to process the write request allocated to it.

[0078] Understandably, the initial memory space can accommodate the data to be written carried by multiple write requests. In other words, for a given processor, after it requests a memory space of the initial size, this space will be continuously filled as write requests are allocated to it. Furthermore, it's understandable that if the current I / O concurrency is high, the memory space requested by the processor will be filled quickly, and when the memory space is full, the processor needs to write the data in that memory space to disk. Conversely, if the I / O concurrency is low, the memory space requested by the processor will take longer to fill. If the timeout period is reached after requesting the memory space, to prevent data from failing to be saved to disk, a save operation needs to be initiated even if the memory space is not full; that is, the processor will write the data in the memory space to disk at this time. Figure 2The data aggregation in this embodiment refers to this process. The data aggregation rules in this embodiment are also commonly used in practical applications. Of course, in other specific situations, the data aggregation rules can be adapted to meet actual needs without affecting the implementation of this invention.

[0079] Step S103: After each cycle, determine the proportion of memory space that is filled and then written to disk within the cycle, and use this proportion as the full data block storage ratio for the cycle.

[0080] The memory space described in this application is written to disk. This refers to the data in the memory space being written to the disk. As can be seen from the above description of the data aggregation rules, there are two situations in which data in the memory space is written to the disk: one is that the first-sized memory space is filled, and the other is that the memory space is not filled.

[0081] This application takes into account that if the proportion of memory space filled and then written to disk within a preset period is high, it indicates that the processor resource utilization is high and the current IO concurrency is high, which may require adding more processors. Conversely, if the proportion of memory space filled and then written to disk within a preset period is low, it indicates that the processors are relatively idle and the current IO concurrency is low, which may require reducing the number of processors in the first resource pool.

[0082] Step S104: Adjust the number of processors in the first resource pool based on the full data block storage ratio.

[0083] In the scheme of this application, the proportion of the amount of memory space that is filled and then written to the disk within the cycle, as determined in the aforementioned steps, to the total amount of memory space written to the disk, is called the full data block storage ratio of that cycle.

[0084] As described above, the determined full data block disk storage ratio for the current period can effectively reflect whether the number of processors in the current first resource pool is appropriate. Therefore, the number of processors in the first resource pool can be adjusted based on the full data block disk storage ratio. That is, the number of processors in the first resource pool can be adjusted to an appropriate value based on the full data block disk storage ratio to reasonably match the current IO concurrency.

[0085] In one specific embodiment of the present invention, step S104 may specifically include:

[0086] Determine whether the percentage of full data blocks stored on disk is greater than or equal to a preset first threshold.

[0087] If the number of processors in the first resource pool is greater than or equal to the first threshold, then the number of processors in the first resource pool is increased.

[0088] If it is less than the first threshold, then determine whether the full data block storage ratio is less than the preset second threshold;

[0089] If it is less than the second threshold, then reduce the number of processors in the first resource pool;

[0090] If the number of processors in the first resource pool is not less than the second threshold, the number of processors in the first resource pool will remain unchanged.

[0091] In this implementation, the number of processors in the first resource pool can be adjusted directly based on the full data block storage ratio, which is relatively convenient to implement.

[0092] Specifically, in this implementation, the full data block storage ratio is directly compared with a preset first threshold. If the full data block storage ratio is greater than the preset first threshold, it indicates that the current IO concurrency is sufficient, so the number of processors in the first resource pool is increased, thereby allowing more processors to participate in IO processing.

[0093] Correspondingly, in this implementation, if the percentage of full data blocks stored on disk is less than the second threshold, it indicates insufficient I / O concurrency. In this case, the current I / O concurrency should be scheduled to fewer processors, making it easier to fill large data blocks, i.e., more easily fill the memory space requested by the processors, thus reducing the number of processors in the first resource pool. Conversely, if the percentage of full data blocks stored on disk is less than the first threshold but not less than the second threshold, it indicates that the current number of processors is reasonable, and therefore, the number of processors in the first resource pool remains unchanged.

[0094] In one specific embodiment of the present invention, it may further include:

[0095] Each time a cycle is completed, the average duration of existence of each memory space within the cycle is determined as the average data block lifetime of the cycle. For any memory space, the duration of existence of the memory space represents the time elapsed from when the memory space is allocated until the data in the memory space is written to the disk.

[0096] Accordingly, step S104 may specifically include:

[0097] Adjust the number of processors in the first resource pool based on the full data block storage ratio and the average data block lifetime.

[0098] Specifically, in this implementation, for any memory space requested by any processor, the current time can be recorded at the time of request, for example, denoted as start_time. When the data in the memory space is subsequently written to the disk, the current time can be recorded, for example, denoted as end_time. Subtracting start_time from end_time gives the duration of existence of the memory space, which is also the aggregate time of the memory space.

[0099] Within a single cycle, data from multiple memory spaces is written to disk. In this implementation, the average duration of these memory spaces is calculated, for example, denoted as avg_merge_time. Figure 2 The document demonstrates the aggregation statistics function, which can be used to calculate the full data block storage ratio and average data block lifetime for each period.

[0100] In this implementation, it is considered that in addition to the full data block disk storage ratio, the average data block lifetime within the period can also effectively reflect whether the IO concurrency is sufficient. Therefore, the number of processors in the first resource pool can be adjusted based on the full data block disk storage ratio and the average data block lifetime.

[0101] Furthermore, in one specific embodiment of the present invention, step S104 may specifically include:

[0102] Every N preset cycles, the processor number adjustment value for each preset cycle is determined;

[0103] Filter the N processor number adjustment values ​​that have the same value as x, and determine the average value of the remaining processor number adjustment values ​​after filtering.

[0104] The average of the remaining processor quantity adjustment values ​​after filtering is rounded down.

[0105] When the rounded value is different from x, adjust the number of processors in the first resource pool to the rounded value.

[0106] Where N is a positive integer not less than 2. For any one of the N preset periods, when the first condition is met, the processor number adjustment value of the period is x+b; x represents the number of processors in the current first resource pool, and b is a positive number; when the second condition is met, the processor number adjustment value of the period is xc; c is a positive number; when neither the first nor the second condition is met, the processor number adjustment value of the period is x.

[0107] The first condition includes: the proportion of full data blocks stored in the period is greater than or equal to a preset first threshold, and the average data block lifetime in the period is less than a preset first duration.

[0108] The second condition includes: the percentage of full data blocks stored on disk during the cycle is less than a preset second threshold.

[0109] Specifically, as described above, the number of processors in the first resource pool can be adjusted based on the full data block disk storage ratio and the average data block lifetime. This implementation method provides a specific example. Figure 2 The document illustrates the aggregation calculation function, which is used to determine the processor number adjustment value, calculate the average value, and perform value calculations in this implementation method. Figure 2 The scheduling function shown refers to adjusting the number of processors in the first resource pool through this function.

[0110] This implementation takes into account that in many situations, the IO concurrency of the business may fluctuate in a short period of time. If the number of processors in the first resource pool is adjusted directly based on the full data block storage ratio and the average data block lifespan determined in a single period, it may lead to frequent adjustments of the number of processors, which is also not conducive to ensuring the stability of system performance. Therefore, in this implementation, the decision on whether to adjust the number of processors is based on the full data block storage ratio and the average data block lifespan of N preset periods.

[0111] Furthermore, since the decision on whether to adjust the number of processors is based on the full data block storage ratio and average data block lifetime over N preset periods, it is also beneficial to obtain a more reasonable number of processors.

[0112] Taking x=8 as an example, and assuming each cycle is set to 1 second, the processor number adjustment value for each preset cycle is determined after 10 preset cycles, i.e., N=10 in this example. Taking the first cycle of these 10 cycles as an example, if the full data block storage ratio in the first cycle is greater than or equal to the preset first threshold, and the average data block lifetime in the first cycle is less than the preset first duration, then the first condition is met, indicating that the IO concurrency is sufficient and more processors are needed to process it. Therefore, the processor number adjustment value for the first cycle is x+b, where b is usually taken as 1. That is, in this example, the processor number adjustment value for the first cycle is 8+1=9.

[0113] For example, if the percentage of full data blocks stored in the disk in the second cycle of these 10 cycles is less than the preset second threshold, then the second condition is met, which indicates that the IO concurrency is insufficient and the number of processors needs to be reduced. Therefore, the number of processors in the second cycle is adjusted to xc, where c is usually 1. In this example, the number of processors in the second cycle is adjusted to 8-1=7.

[0114] Furthermore, for a given cycle, if neither the first nor the second condition is met, it means that there is no need to adjust the number of processors. Therefore, the processor number adjustment value for that cycle is x, which means that the current number remains unchanged.

[0115] After determining N processor quantity adjustment values, it is necessary to filter out the processor quantity adjustment values ​​that have the same value as x from the determined N processor quantity adjustment values, determine the average value of the remaining processor quantity adjustment values ​​after filtering, and then round the average value.

[0116] For example, in one scenario, for the aforementioned 10 preset cycles, three cycles have a processor number adjustment value of 9, one cycle has a processor number adjustment value of 7, and the remaining five cycles have a processor number adjustment value of 8. These five 8s need to be filtered out, and the average of the remaining processor number adjustment values ​​after filtering needs to be determined, i.e., (9+9+9+7) / 4 = 34 / 4 = 8.5. When rounding, it is usually set to the nearest integer. In this example, rounding 8.5 results in a value of 9. Of course, other rounding methods can be set as needed in other scenarios, such as rounding down or rounding up.

[0117] If the rounded value is different from x, the number of processors in the first resource pool needs to be adjusted to the rounded value. That is, the number of processors in the first resource pool needs to be increased or decreased. For example, in the example above, the rounded value is 9, which is different from the current number of 8. Therefore, the number of processors in the first resource pool needs to be increased from 8 to 9.

[0118] As can be seen from the examples above, determining whether to adjust the number of processors based on the full data block storage ratio and average data block lifetime over N preset periods can effectively prevent frequent changes in the number of processors. In the example of 10 preset periods above, the number of processors was adjusted to 9 in 3 periods and to 7 in 1 period. This 7 may have been caused by short-term I / O fluctuations, but it did not ultimately affect the number of processors that needed to be updated after 10 periods. That is, this implementation method can reduce the impact of short-term I / O fluctuations on the results to a certain extent, which is conducive to ensuring the stability of system performance and obtaining a more reasonable number of processors.

[0119] Furthermore, in one specific embodiment of the present invention, it may also include:

[0120] Determine whether the number of processor number adjustment values ​​with the same value as x among the N determined processor number adjustment values ​​has reached the first quantity threshold.

[0121] If so, then cancel the current adjustment to the number of processors in the first resource pool and maintain the current number of processors in the first resource pool unchanged.

[0122] This implementation takes into account that if the number of processor number adjustment values ​​with the same value as x among the obtained N processor number adjustment values ​​reaches the first quantity threshold, it means that most of the N processor number adjustment values ​​are x, which means that the current number of processors is relatively reasonable. Therefore, the adjustment of the number of processors in the first resource pool can be directly cancelled, that is, there is no need to perform subsequent average and rounding calculations, but to directly maintain the current number of processors in the first resource pool unchanged.

[0123] For example, in the 10 cycles mentioned above, if 9 out of the 10 processor number adjustment values ​​are 8, and the first quantity threshold is 7, then 9 exceeds the first quantity threshold of 7, and the number of processors in the current first resource pool can be kept unchanged.

[0124] In one specific embodiment of the present invention, it may further include:

[0125] After adjusting the number of processors in the first resource pool to the rounded value, the adjustment to the number of processors in the first resource pool will be cancelled in the second period, and the current number of processors in the first resource pool will remain unchanged in the second period.

[0126] This implementation takes into account that if the number of processors in the first resource pool is adjusted, the adjustment can be canceled after a period of time. Therefore, in this implementation, the current number of processors in the first resource pool remains unchanged for the subsequent second time period. This implementation also facilitates frequent adjustments to the number of processors, ensuring the stability of system performance.

[0127] By applying the technical solution provided in this embodiment of the invention, after receiving a write request, the write request can be allocated to a processor in the current first resource pool according to a preset scheduling rule. The processor then processes the write request according to a preset data aggregation rule. The data aggregation rule includes: the processor requests a memory space of a first size to process the write request allocated to it; when the memory space is full, the processor writes the data in the memory space to the disk; when the memory space is not full but the time elapsed after requesting the memory space reaches the timeout period, the processor writes the data in the memory space to the disk. Therefore, it can be seen that the processor of this application performs a data aggregation process when processing write requests, i.e., when processing I / O, that is, it sets up a process of aggregating small blocks into large blocks, thus effectively ensuring the service life of the all-flash storage system.

[0128] Furthermore, this application considers that traditional solutions cause increased IO latency because the number of IO concurrency changes continuously while the number of CPUs remains fixed. In other words, the fixed number of CPUs in traditional solutions cannot effectively handle varying IO concurrency levels. Therefore, this application proposes that, after each cycle, the proportion of memory space filled and then written to disk within that cycle, relative to the total memory space written to disk, can be determined as the full data block storage ratio for that cycle. The full data block storage ratio effectively reflects the utilization rate of processor resources and indicates the current IO concurrency situation. Therefore, this application's solution adjusts the number of processors in the first resource pool based on the full data block storage ratio, ensuring that the number of processors in the first resource pool is adjusted to an appropriate value to effectively adapt to the current IO concurrency situation. This allows the solution of this application to effectively reduce IO latency.

[0129] In summary, the solution proposed in this application can effectively complete the data storage of the all-flash storage system, improve the lifespan of the SSD, and reduce IO latency.

[0130] Corresponding to the above method embodiments, this invention also provides a data storage system for an all-flash storage system, which can be referred to in conjunction with the above.

[0131] See Figure 3 The diagram shown is a structural schematic of a data storage system for an all-flash storage system according to the present invention, comprising:

[0132] Write request receiving module 301 is used to receive write requests;

[0133] The write request allocation module 302 is used to allocate write requests to processors in the current first resource pool according to preset scheduling rules, so that the processors process the write requests according to preset data aggregation rules.

[0134] The statistics module 303 is used to determine, each time a cycle has elapsed, the proportion of memory space that is filled and then written to disk within the cycle, and the proportion of the total memory space written to disk, as the full data block storage ratio of the cycle.

[0135] Scheduling module 304 is used to adjust the number of processors in the first resource pool based on the full data block storage ratio;

[0136] The first resource pool includes at least one processor; the data aggregation rules include: the processor requests a memory space of a first size to process write requests allocated to itself; when the memory space is full, the processor writes the data in the memory space to the disk; when the memory space is not full but the time elapsed after requesting the memory space reaches the timeout period, the processor writes the data in the memory space to the disk.

[0137] In one specific embodiment of the present invention, the write request allocation module 302 includes:

[0138] The count value determination unit is used to determine the count value corresponding to the write request;

[0139] The remainder calculation unit is used to divide the count value by the number of processors in the current first resource pool to obtain the remainder value a;

[0140] The allocation unit is used to allocate write requests to the processor numbered a, so that the processor processes the write requests according to the preset data aggregation rules.

[0141] The initial value of the counter is 0, and the value increases by 1 each time a write request is received.

[0142] In one specific embodiment of the present invention, the scheduling module 304 is specifically used for:

[0143] Determine whether the percentage of full data blocks stored on disk is greater than or equal to a preset first threshold.

[0144] If the number of processors in the first resource pool is greater than or equal to the first threshold, then the number of processors in the first resource pool is increased.

[0145] If it is less than the first threshold, then determine whether the full data block storage ratio is less than the preset second threshold;

[0146] If it is less than the second threshold, then reduce the number of processors in the first resource pool;

[0147] If the number of processors in the first resource pool is not less than the second threshold, the number of processors in the first resource pool will remain unchanged.

[0148] In one specific embodiment of the present invention, an average data block lifetime calculation module is further included, for:

[0149] Each time a cycle is completed, the average duration of existence of each memory space within the cycle is determined as the average data block lifetime of the cycle; where, for any memory space, the duration of existence of the memory space represents the time elapsed from when the memory space is allocated to when the data in the memory space is written to the disk.

[0150] Accordingly, the scheduling module 304 is specifically used for:

[0151] Adjust the number of processors in the first resource pool based on the full data block storage ratio and the average data block lifetime.

[0152] In one specific embodiment of the present invention, the scheduling module 304 specifically includes:

[0153] The processor number adjustment value determination unit is used to determine the processor number adjustment value for each preset period after N preset periods.

[0154] The average adjustment value calculation unit is used to filter the processor number adjustment values ​​that have the same value as x from the determined N processor number adjustment values, and determine the average value of the remaining processor number adjustment values ​​after filtering.

[0155] The rounding unit is used to round the average of the remaining processor quantity adjustment values ​​after filtering.

[0156] Adjust the execution unit to adjust the number of processors in the first resource pool to the rounded value when the rounded value is different from x.

[0157] Where N is a positive integer not less than 2. For any one of the N preset periods, when the first condition is met, the processor number adjustment value of the period is x+b; x represents the number of processors in the current first resource pool, and b is a positive number; when the second condition is met, the processor number adjustment value of the period is xc; c is a positive number; when neither the first nor the second condition is met, the processor number adjustment value of the period is x.

[0158] The first condition includes: the proportion of full data blocks stored in the period is greater than or equal to a preset first threshold, and the average data block lifetime in the period is less than a preset first duration.

[0159] The second condition includes: the percentage of full data blocks stored on disk during the cycle is less than a preset second threshold.

[0160] In one specific embodiment of the present invention, a holding module may be further included, for:

[0161] Determine whether the number of processor number adjustment values ​​with the same value as x among the N determined processor number adjustment values ​​has reached the first quantity threshold.

[0162] If so, then cancel the current adjustment to the number of processors in the first resource pool and maintain the current number of processors in the first resource pool unchanged.

[0163] In one specific embodiment of the present invention, the holding module is further configured to:

[0164] After adjusting the number of processors in the first resource pool to the rounded value, the adjustment to the number of processors in the first resource pool will be cancelled in the second period, and the current number of processors in the first resource pool will remain unchanged in the second period.

[0165] Corresponding to the above methods and system embodiments, this invention also provides a data storage device for an all-flash storage system and a computer-readable storage medium, which can be referred to in conjunction with the above.

[0166] See also Figure 4 The data storage device of the all-flash storage system may include:

[0167] Memory 401 is used to store computer programs;

[0168] Processor 402 is configured to execute a computer program to implement the steps of the data storage method of the all-flash storage system as described in any of the above embodiments.

[0169] See also Figure 5 The computer-readable storage medium 50 stores a computer program 51, which, when executed by a processor, implements the steps of the data storage method of the all-flash storage system as described in any of the above embodiments. The computer-readable storage medium 50 referred to herein includes random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0170] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0171] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0172] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the technical solutions and core ideas of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A data storage method for an all-flash storage system, characterized in that, include: Receive write requests; According to the preset scheduling rules, the write request is allocated to the processor in the current first resource pool, so that the processor processes the write request according to the preset data aggregation rules; wherein, through the scheduling rules, each processor in the current first resource pool has an equal opportunity to be allocated a write request. Each time a cycle is completed, the proportion of memory space that is filled and then written to disk within that cycle is determined, and is taken as the full data block storage ratio for that cycle. Based on the full data block storage ratio, adjust the number of processors in the first resource pool; The first resource pool includes at least one processor; the data aggregation rules include: the processor requests a memory space of a first size to process write requests allocated to itself; when the memory space is full, the processor writes the data in the memory space to the disk; when the memory space is not full but the time elapsed after requesting the memory space reaches the timeout period, the processor writes the data in the memory space to the disk.

2. The data storage method of the all-flash storage system according to claim 1, characterized in that, The step of allocating the write request to the processor in the current first resource pool according to the preset scheduling rules includes: Determine the count value corresponding to the write request; Divide the count value by the current number of processors in the first resource pool to obtain the remainder value a; Assign the write request to the processor numbered a; The initial value of the counter is 0, and the value of the counter increases by 1 each time a write request is received.

3. The data storage method of the all-flash storage system according to claim 1, characterized in that, Based on the full data block storage ratio, the number of processors in the first resource pool is adjusted, including: Determine whether the full data block storage ratio is greater than or equal to a preset first threshold; If the number of processors in the first resource pool is greater than or equal to the first threshold, then the number of processors in the first resource pool is increased. If it is less than the first threshold, then determine whether the full data block storage ratio is less than the preset second threshold; If it is less than the second threshold, then reduce the number of processors in the first resource pool; If the number of processors in the first resource pool is not less than the second threshold, the number of processors in the first resource pool will remain unchanged.

4. The data storage method of the all-flash storage system according to any one of claims 1 to 3, characterized in that, Also includes: Each time a cycle is completed, the average duration of existence of each memory space within the cycle is determined as the average data block lifetime of the cycle; wherein, for any memory space, the duration of existence of the memory space represents the time elapsed from when the memory space is allocated until the data in the memory space is written to the disk. Accordingly, adjusting the number of processors in the first resource pool based on the full data block storage ratio includes: The number of processors in the first resource pool is adjusted based on the full data block storage ratio and the average data block lifetime.

5. The data storage method of the all-flash storage system according to claim 4, characterized in that, Based on the full data block storage ratio and the average data block lifetime, the number of processors in the first resource pool is adjusted, including: Every N preset cycles, the processor number adjustment value for each preset cycle is determined; Filter the N processor number adjustment values ​​that have the same value as x, and determine the average value of the remaining processor number adjustment values ​​after filtering. The average of the remaining processor quantity adjustment values ​​after filtering is rounded down. When the rounded value is different from x, the number of processors in the first resource pool is adjusted to the rounded value. Where N is a positive integer not less than 2. For any one of the N preset periods, when the first condition is met, the processor number adjustment value of the period is x+b; x represents the number of processors in the current first resource pool, and b is a positive number; when the second condition is met, the processor number adjustment value of the period is xc; c is a positive number; when neither the first condition nor the second condition is met, the processor number adjustment value of the period is x. The first condition includes: the proportion of full data blocks stored in the period is greater than or equal to a preset first threshold, and the average data block lifetime of the period is less than a preset first duration; The second condition includes: the proportion of full data blocks stored in the cycle is less than a preset second threshold.

6. The data storage method of the all-flash storage system according to claim 5, characterized in that, Also includes: Determine whether the number of processor number adjustment values ​​with the same value as x among the N determined processor number adjustment values ​​has reached the first quantity threshold. If so, then cancel the current adjustment to the number of processors in the first resource pool and maintain the current number of processors in the first resource pool unchanged.

7. The data storage method of the all-flash storage system according to claim 5, characterized in that, Also includes: After adjusting the number of processors in the first resource pool to the rounded value, the adjustment to the number of processors in the first resource pool is cancelled within the second time period, and the current number of processors in the first resource pool remains unchanged within the second time period.

8. A data storage system for an all-flash storage system, characterized in that, include: The write request receiving module is used to receive write requests; The write request allocation module is used to allocate the write request to the processors in the current first resource pool according to the preset scheduling rules, so that the processors process the write request according to the preset data aggregation rules; wherein, the scheduling rules ensure that each processor in the current first resource pool has an equal chance to be allocated a write request. The statistics module is used to determine, each time a cycle has elapsed, the proportion of memory space that has been filled and then written to disk within the cycle to the total amount of memory space written to disk, as the full data block storage ratio of the cycle. The scheduling module is used to adjust the number of processors in the first resource pool based on the full data block storage ratio; The first resource pool includes at least one processor; the data aggregation rules include: the processor requests a memory space of a first size to process write requests allocated to itself; when the memory space is full, the processor writes the data in the memory space to the disk; when the memory space is not full but the time elapsed after requesting the memory space reaches the timeout period, the processor writes the data in the memory space to the disk.

9. A data storage device for an all-flash storage system, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data storage method of the all-flash memory system as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data storage method of the all-flash storage system as described in any one of claims 1 to 7.

Images