Storage space recovery method, electronic device, storage medium and program product

Abstract

The present invention discloses a storage space recovery method, electronic equipment, storage medium and program product, which relate to the field of computer technology. By comprehensively considering the influence of multiple factors on storage space recovery, the recovery score of the data block is determined specifically according to the invalid data rate, data heat and wear degree, and the recovery priority of the data block is determined according to the recovery score. Storage space is recovered for the data blocks according to the recovery priority, and the data blocks with the largest recovery benefits are processed first, thereby improving the rationality of selecting data blocks for storage space recovery. Since data inefficiency and data heat are taken into account, the effective data migration rate of the recovered data blocks can be reduced, thereby improving the recovery efficiency of the storage space, and the considered heat data can reduce the performance impact on the storage system. In addition, the considered wear degree of the data blocks is also extended to a certain extent.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a storage space recovery method, electronic equipment, storage medium, and program product. Background Art

[0002] With the widespread application of all-flash storage systems in modern data centers, cloud computing, and big data analysis platforms, the space recovery efficiency of storage systems has become a key factor affecting overall performance.

[0003] Related technologies mainly use first-in-first-out or random selection strategies for space reclamation, which may result in the need to migrate a large amount of valid data in the data blocks for space reclamation, thereby leading to low space reclamation efficiency. In addition, when the storage system space is tight, related technologies perform real-time reclamation operations on the foreground I / O (Input / Output) path, which will significantly increase I / O latency and thus affect the performance of the storage system. Summary of the Invention

[0004] The present invention provides a storage space recovery method, electronic equipment, storage medium and program product, so as to at least solve the problem in the related art that the space recovery efficiency is low and the performance of the storage system is affected.

[0005] The present invention provides a storage space recovery method, comprising the following steps: identifying multi-dimensional data of at least one data block in a storage system; calculating the invalid data rate, data heat and wear degree of the corresponding data block based on the multi-dimensional data, and calculating a recovery score for the corresponding data block based on the invalid data rate, data heat and wear degree; determining the recovery priority of the data block in the storage system based on the recovery score, and recovering storage space for the data block in the storage system according to the recovery priority.

[0006] The present invention also provides a storage space recovery device, comprising: an identification module for identifying multi-dimensional data of at least one data block in a storage system; a calculation module for calculating the invalid data rate, data heat and wear degree of the corresponding data block based on the multi-dimensional data, and calculating the recovery score of the corresponding data block based on the invalid data rate, data heat and wear degree; a recovery module for determining the recovery priority of the data block in the storage system based on the recovery score, and reclaiming storage space for the data block in the storage system according to the recovery priority.

[0007] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned storage space recovery methods when executing the computer program.

[0008] The present invention also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned storage space recovery methods are implemented.

[0009] The present invention also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned storage space recovery methods when executed by a processor.

[0010] Through the present invention, the impact of multiple factors on storage space recovery can be comprehensively considered. Specifically, the recovery score of the data block can be determined according to the invalid data rate, data heat and wear degree, and the recovery priority of the data block can be determined according to the recovery score. Storage space is recovered for the data blocks according to the recovery priority, and the data blocks with the largest recovery benefits are given priority, which improves the rationality of selecting data blocks for storage space recovery. Since data inefficiency and data heat are taken into consideration, the effective data migration rate of the recovered data blocks can be reduced, thereby improving the recovery efficiency of the storage space, and the considered heat data can reduce the performance impact on the storage system. In addition, the consideration of the wear degree of the data blocks also extends the service life of the storage system to a certain extent. Therefore, the technical problems of low space recovery efficiency and impact on the performance of the storage system in the related technology can be solved, and the technical effect of improving space recovery efficiency and reducing the performance impact on the storage system can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 A schematic diagram of a flow chart of a storage space recovery method according to an embodiment of the present invention;

[0013] Figure 2 A schematic diagram illustrating an execution of a storage space recovery method according to an embodiment of the present invention;

[0014] Figure 3 A block diagram of a storage space recovery device according to an embodiment of the present invention;

[0015] Figure 4 A schematic structural diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0017] It should be noted that, in the description of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. The terms "first," "second," etc., in the present invention are used to distinguish similar objects, and are not used to describe a particular order or precedence.

[0018] Before describing the solution of the present invention, let's first introduce the specific technical problems existing in related space reclamation technologies:

[0019] 1. Low reclamation efficiency: Related technologies often use simple first-in-first-out or random selection strategies when performing space reclamation. They fail to fully consider factors such as the proportion of valid data in the data block and data access popularity. As a result, a large amount of valid data needs to be migrated during the reclamation process, resulting in additional I / O overhead and performance loss.

[0020] 2. Severe foreground I / O interference: When storage system space is tight, the space recovery mechanism of related technologies is often forced to perform real-time recovery operations on the foreground I / O path, which significantly increases I / O latency and causes drastic fluctuations in system response time, making it difficult to meet the service quality requirements of high-performance storage systems.

[0021] 3. Insufficient consideration of wear leveling: When reclaiming space, related technologies fail to fully consider the distribution of the number of erase and write cycles (P / E cycles) of flash memory blocks, which may cause some flash memory blocks to be over-erased while other blocks are underutilized, thereby shortening the service life of the entire storage device.

[0022] 4. Unbalanced resource utilization: Related technologies lack intelligence in recycling thread scheduling and resource allocation, and are unable to dynamically adjust recycling intensity based on the current load of the storage system, which easily leads to waste or insufficient computing resources and I / O bandwidth.

[0023] 5. It is difficult to balance multi-dimensional optimization goals: The space reclamation process needs to consider multiple optimization goals simultaneously, such as reclamation efficiency, performance impact, and wear leveling. Related technologies find it difficult to achieve a good balance between these mutually constrained goals.

[0024] To this end, the present invention proposes a storage space recovery method, which solves at least one of the above technical problems by introducing multi-dimensional evaluation indicators, a dynamic priority adjustment mechanism and an adaptive resource allocation strategy.

[0025] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0026] An embodiment of the present invention provides a storage space reclamation method, which is described in detail in conjunction with the execution flow of the storage space reclamation method.

[0027] like Figure 1 As shown, the storage space recovery method according to the embodiment of the present invention includes the following steps:

[0028] In step S101 , multi-dimensional data of at least one data block in a storage system is identified.

[0029] Among them, the data block is the basic unit for the storage system to logically manage data. It consists of valid data and invalid data. When data is written, deleted or overwritten, the status of the data block (such as the amount of valid data, access frequency, etc.) will be tracked in real time. At the physical level, the data blocks of data will be split and stored in the flash memory blocks. The storage of data blocks depends on the physical space of the flash memory blocks; multi-dimensional data includes the access data of the data block, the amount of valid data, the total capacity, the number of erases and writes, the maximum number of erases and writes, etc.

[0030] The storage system according to the embodiment of the present invention may be an all-flash storage system. The storage system according to the embodiment of the present invention may be a storage system of a server or a storage system of other electronic devices.

[0031] In step S102 , the invalid data rate, data heat and wear degree of the corresponding data block are calculated based on the multi-dimensional data, and the recycling score of the corresponding data block is calculated based on the invalid data rate, data heat and wear degree.

[0032] The recycling score may also be referred to as a recycling priority score.

[0033] It can be understood that the embodiment of the present invention can calculate the invalid data rate, data heat and wear degree of the corresponding data block based on multi-dimensional data, and calculate the recovery score of the corresponding data block based on the invalid data rate, data heat and wear degree, so as to subsequently reclaim the storage space of the storage system.

[0034] In an embodiment of the present invention, the invalid data rate, data heat and wear degree of the corresponding data block are calculated based on the multi-dimensional data, including: extracting the valid data amount, total capacity, number of erased and written times, maximum number of erased and written times and access data in the multi-dimensional data; calculating the invalid data rate based on the valid data amount and total capacity; calculating the data heat based on the access data; and calculating the wear degree based on the number of erased and written times and the maximum number of erased and written times.

[0035] Among them, multi-dimensional data includes the amount of valid data, total capacity, number of erases and writes, maximum number of erases and writes, and access data. Access data includes access frequency, last access time, access mode, etc.

[0036] It can be understood that the embodiments of the present invention can extract specific data from multi-dimensional data, and calculate the invalid data rate, data heat and wear degree of the corresponding data block based on the specific data, specifically: calculate the invalid data rate based on the amount of valid data and the total capacity, calculate the data heat based on the access data, and calculate the wear degree based on the number of erased and written times and the maximum number of erasable times.

[0037] Specifically, the invalid data rate is calculated as follows: invalid data rate = 1 - (valid data volume / total data block capacity);

[0038] Data heat is calculated as follows: data heat = f (access frequency, last access time, access mode);

[0039] The calculation method of wear degree is: wear degree = number of times the data block has been erased and written / maximum number of times it can be erased and written.

[0040] The data popularity evaluation in the embodiment of the present invention comprehensively considers the following factors:

[0041] Short-term access frequency: access frequency in the last minute;

[0042] Long-term access pattern: access distribution in the past 24 hours;

[0043] Temporal locality: Based on exponentially decaying weight distribution, the more recent the access, the higher the weight;

[0044] Spatial locality: Recognition and reward of consecutive access patterns.

[0045] Data heat calculation formula: Data heat = 0.4 × short-term access frequency + 0.3 × long-term access pattern + 0.2 × temporal locality + 0.1 × spatial locality.

[0046] For example, if data block X was accessed 15 times in the last minute and has a maximum access frequency of 30 times, then the short-term access frequency is 15 / 30 = 0.5. It has been accessed 200 times in the last 24 hours, with an average of 400 visits per day. The long-term access pattern is 200 / 400 = 0.5. 80% of the accesses occurred in the last hour, and 20% occurred the rest of the time (exponentially decaying weights). The temporal locality is 0.8. The number of consecutive accesses is 5 (reward coefficient 1.0), and there is no reward for non-consecutive accesses. The spatial locality is 1, and the data popularity of data block X = 0.4×0.5 + 0.3×0.5 + 0.2×0.8 + 0.1×1 = 0.61.

[0047] If the valid data volume of data block X is 10 MB and the total capacity is 16 MB, then the invalid data rate = 1-(10 / 16) = 0.375 (37.5%);

[0048] If data block X has been erased and written 300 times and the maximum number of times it can be erased and written is 3000 times, then the wear degree = 300 times / 3000 times = 0.1 (10%).

[0049] In addition, the embodiment of the present invention can track and record the data block status in real time to obtain multi-dimensional data of the data block, wherein the processing operation on the data block to update the multi-dimensional data includes:

[0050] 1. Write operation processing.

[0051] When a new write request is received, the system first writes the new data to the free area of ​​the flash memory, specifically:

[0052] Update the metadata mapping table to point the logical address to the new physical location; mark the old data location as "invalid"; call the space recovery interface to increase the valid data count of the newly written data block; mark the data heat level based on I / O characteristics (such as access frequency, data size, etc.).

[0053] 2. Data invalidation processing.

[0054] When data becomes invalid due to deletion or overwriting, the valid data count of the corresponding data block is reduced through the space recovery interface; and the invalid data distribution information in the metadata is updated.

[0055] 3. Access pattern tracking.

[0056] The system continuously monitors the access characteristics of each data block, including: access frequency; last access time; access pattern (sequential / random); and data life cycle prediction.

[0057] The present invention can adopt a lightweight monitoring mechanism to record state changes through an efficient data structure (such as a bitmap), ensuring that the monitoring overhead does not exceed 3% of the total I / O resources.

[0058] In an embodiment of the present invention, a recovery score of a corresponding data block is calculated based on the invalid data rate, data heat and wear degree, including: obtaining the corresponding weights of the invalid data rate, data heat and wear degree; and calculating the recovery score of the corresponding data block based on the invalid data rate, data heat and wear degree and the corresponding weights.

[0059] The calculation formula for the recycling score is:

[0060] S=α×A+ β×(1-B) + γ×(1-C);

[0061] Among them, S is the recycling score, α is the weight of the invalid data rate, A is the invalid data rate, β is the weight of the data heat, B is the data heat, γ is the weight of the wear degree, and C is the wear degree.

[0062] It is understandable that the embodiment of the present invention can calculate the recycling score of the corresponding data block according to the invalid data rate, data heat and wear degree and the corresponding weight. Specifically,

[0063] Recovery score = α × invalid data rate + β × (1-data heat) + γ × (1-wear degree).

[0064] In addition, it should be noted that α, β, and γ are all adjustable weight parameters, which can be set to 0.6, 0.3, and 0.1 by default, respectively. The weight parameters can be dynamically adjusted according to the workload characteristics of the storage system.

[0065] For example, the invalid data rate of data block Y is 80%, the data temperature is 0.2 (representing cold data with low access frequency), and the wear degree is 30%.

[0066] The recovery score of data block Y = 0.6 × 0.8 + 0.3 × (1-0.2) + 0.1 × (1-0.3) = 0.79.

[0067] In step S103, the recycling priority of the data blocks in the storage system is determined according to the recycling score, and storage space is recycled for the data blocks in the storage system according to the recycling priority.

[0068] It can be understood that the embodiments of the present invention can determine the recycling priority of data blocks in the storage system based on the recycling score, and reclaim storage space for data blocks in the storage system according to the recycling priority. By comprehensively considering the impact of multiple factors on storage space recycling, the recycling score of the data block is determined specifically according to the invalid data rate, data heat and wear level, and the recycling priority of the data block is determined according to the recycling score. Storage space is reclaimed for the data block according to the recycling priority, and data blocks with the largest recycling benefits are given priority, which improves the rationality of selecting data blocks for storage space recycling. Since data inefficiency and data heat are taken into consideration, the effective data migration rate of the recycled data blocks can be reduced, thereby improving the recycling efficiency of the storage space, and the considered heat data can reduce the performance impact on the storage system. In addition, the wear level of the data blocks is taken into consideration, which to a certain extent extends the service life of the storage system.

[0069] In an embodiment of the present invention, determining the recycling priority of a data block in a storage system according to the recycling score includes: dividing the recycling score into multiple recycling priorities, wherein each recycling priority corresponds to a bidirectional linked list.

[0070] Among them, a doubly linked list is a data structure in which each node contains a predecessor pointer and a successor pointer, allowing bidirectional traversal (forward or backward) starting from any node. Compared with a one-way linked list, it has more flexible node operation capabilities. Each recycling priority corresponds to an independent doubly linked list. The linked list node stores the metadata of the data block in the gear, such as physical address, recycling score, valid data volume, etc. The doubly linked list can directly locate data blocks with different recycling priorities, thereby improving the efficiency of storage space recycling.

[0071] For example, the present invention can divide the recycling score (i.e., the recycling priority score) into multiple recycling priorities, or it can also be understood as dividing the recycling score into multiple levels (such as levels 0-14). Each level corresponds to a bidirectional linked list, which stores data blocks with similar recycling value. The grading standard is shown in Table 1, which is a grading standard table.

[0072]

[0073] This grading mechanism allows the storage system to prioritize data blocks with the greatest recovery benefits, significantly improving the efficiency of storage space recovery.

[0074] In an embodiment of the present invention, reclaiming storage space of data blocks in a storage system according to a recycling priority includes: determining at least one data block to be recycled in the storage system according to the recycling priority; and reclaiming storage space of the at least one data block to be recycled using a recycling thread.

[0075] Among them, one recycling thread corresponds to one data block to be recycled, that is, each recycling thread processes one data block at a time.

[0076] It is understandable that the embodiment of the present invention can determine at least one data block to be reclaimed in the storage system according to the reclaim priority, and reclaim storage space for the at least one data block to be reclaimed using a reclaim thread.

[0077] In an embodiment of the present invention, a recycling thread is used to reclaim storage space of at least one data block to be recycled, including: reading valid data in at least one data block to be recycled; writing the valid data to a target location of a target flash memory block, and performing an erasing operation on at least one data block to be recycled to achieve storage space recycling of at least one data block to be recycled.

[0078] The target flash memory block may be a flash memory block with a smaller number of erase / write times, so as to avoid selecting a flash memory block with a larger number of erase / write times, thereby preventing some flash memory blocks from being over-erased and increasing the service life of the flash memory block.

[0079] It can be understood that the embodiment of the present invention can read the valid data in the data block to be recovered, and after writing the valid data to the target position of the target flash memory block, perform an erase operation on at least one data block to be recovered to avoid erasing the valid data, thereby realizing the recovery of storage space for the data block to be recovered.

[0080] In the embodiment of the present invention, after writing the valid data to the target location of the target flash memory block, the method further includes: updating the mapping relationship of the valid data.

[0081] It can be understood that the embodiment of the present invention can update the mapping relationship of the valid data after writing the valid data to the target location of the target flash memory block, that is, after completing the migration of the valid data in the data block to be recovered, so that the valid data can be used subsequently to avoid the inability to query the valid data.

[0082] Specifically, the embodiment of the present invention performs data migration and space reclamation operations on the data blocks to be reclaimed.

[0083] 1. Data block selection.

[0084] The data blocks to be recycled are selected starting from the highest priority.

[0085] 2. Effective data migration.

[0086] Read all valid data in the data block to be recycled, write the valid data to the new target location, and update the metadata mapping relationship.

[0087] 3. Space recovery.

[0088] After all valid data in the reclaimed data block is migrated, the data block is marked as "erasable"; the background erasure thread periodically erases these data blocks in batches to make them available space.

[0089] 4. Wear leveling treatment.

[0090] When selecting a target location to write migration data, priority is given to flash memory blocks with fewer erase / write counts; the erase / write history of each flash memory block is recorded to avoid local overload.

[0091] In an embodiment of the present invention, before determining at least one data block to be reclaimed in the storage system according to the recycling priority, it also includes: identifying the current load data of the storage system; determining the number of recycling threads for storage space recycling based on the current load data; and determining the number of data blocks to be reclaimed based on the number of recycling threads and the recycling priority.

[0092] It can be understood that the embodiments of the present invention can identify the current load data of the storage system, and determine the number of recovery threads for storage space recovery based on the current load data, and determine the number of data blocks to be recovered based on the number of recovery threads and the recovery priority, thereby dynamically adjusting the recovery intensity according to the current load situation to avoid wasting computing resources or causing insufficient I / O bandwidth.

[0093] In an embodiment of the present invention, determining the number of recycling threads for storage space recycling based on current load data includes: obtaining thread parameters of the storage system; identifying the free space ratio in the current load data; and calculating the number of recycling threads for storage space recycling based on the thread parameters and the free space ratio.

[0094] The thread parameters include the maximum number of recycling threads and the basic number of threads. The basic number of threads and the maximum number of recycling threads are pre-calibrated, and the basic number of threads is usually set to 2-4.

[0095] It is understandable that the present invention can identify the free space ratio in the current load data and calculate the number of recycling threads for storage space recycling based on the thread parameters and the free space ratio.

[0096] The method for calculating the number of recycling threads in the embodiment of the present invention is:

[0097] Number of recycling threads = min(maximum number of recycling threads, number of basic threads + [(1-current free space ratio) × expansion factor]).

[0098] The expansion factor is determined according to the hardware configuration.

[0099] Specifically, the embodiment of the present invention can increase the number of recycling threads to accelerate space recycling when the storage system is idle, and reduce the number of recycling threads to prioritize foreground I / O performance when the storage system is busy.

[0100] In the embodiment of the present invention, before reclaiming storage space of data blocks in the storage system according to the reclaiming priority, the method further includes: limiting bandwidth usage during storage space reclaiming.

[0101] Since the space recovery mechanism of related technologies is often forced to perform real-time recovery operations on the foreground I / O path, this will significantly increase the I / O delay, causing the system response time to fluctuate violently, making it difficult to meet the service quality requirements of high-performance storage systems. Therefore, the embodiments of the present invention can also limit the bandwidth occupancy during storage space recovery to avoid increasing I / O delays, avoiding violent fluctuations in storage system response time, and meeting the service quality requirements of high-performance storage systems.

[0102] Specifically, the embodiment of the present invention may use a token bucket algorithm to limit the I / O bandwidth occupied by the recycling operation to ensure that it does not exceed 30% of the total bandwidth.

[0103] In an embodiment of the present invention, after reclaiming storage space for data blocks in a storage system according to the recycling priority, it also includes: evaluating the impact data of invalid data rate, data heat and wear degree on storage space recycling after storage space recycling for data blocks in the storage system; and optimizing the corresponding weights of invalid data rate, data heat and wear degree based on the impact data.

[0104] It is understandable that the embodiments of the present invention can also evaluate the impact data of invalid data rate, data heat and wear degree on storage space recovery after storage space recovery of data blocks in the storage system; optimize the corresponding weights of invalid data rate, data heat and wear degree based on the impact data to improve the rationality of future selection of data blocks for storage space recovery.

[0105] In an embodiment of the present invention, the method further includes: identifying a score distribution of recycling scores of the plurality of data blocks; and adjusting the division of the plurality of recycling priorities according to the score distribution.

[0106] It is understood that the embodiment of the present invention may further identify the score distribution of the recycling scores of multiple data blocks and adjust the division of multiple recycling priorities according to the score distribution to ensure a balanced number of data blocks in each gear.

[0107] In summary, the storage space reclamation method of the embodiment of the present invention realizes the reclamation of storage space of a storage system by introducing multi-dimensional evaluation indicators, a dynamic priority adjustment mechanism, and an adaptive resource allocation strategy. The method mainly includes the following contents:

[0108] 1. By introducing multi-dimensional data, the three dimensions of data block inefficiency, data heat and wear degree are comprehensively considered, and a composite score is generated through adjustable weight coefficients to accurately identify the data blocks with the highest recovery benefits.

[0109] Among them, data popularity takes into account the following factors:

[0110] Short-term access frequency: access frequency in the last minute;

[0111] Long-term access pattern: access distribution in the past 24 hours;

[0112] Temporal locality: Based on exponentially decaying weight distribution, the more recent the access, the higher the weight;

[0113] Spatial locality: Recognition and reward of consecutive access patterns.

[0114] Data heat calculation formula:

[0115] Data heat = 0.4 × short-term access frequency + 0.3 × long-term access pattern + 0.2 × temporal locality + 0.1 × spatial locality.

[0116] 2. A hierarchical space recovery mechanism divides data blocks into multiple priority levels based on their scores. Each level is organized in a bidirectional linked list to achieve efficient data block selection and management, ensuring that high-value data blocks are recycled first.

[0117] 3. Dynamically adjust the number of recycling threads and I / O bandwidth usage based on the storage system's free space ratio and current load to minimize the impact on foreground I / O.

[0118] 4. The comprehensive popularity score is obtained through weighted calculation by integrating four factors: short-term access frequency, long-term access pattern, temporal locality, and spatial locality.

[0119] 5. By monitoring the recycling effect and performance impact, the weight and recycling intensity parameters are automatically adjusted using feedback control methods. That is, a closed-loop feedback mechanism is established to continuously optimize the recycling strategy parameters to adapt to different workload characteristics.

[0120] Among them, weight parameters (α, β, γ) are adjusted: the impact of each dimension on recycling performance is evaluated regularly (e.g., every hour); the weight distribution is optimized using the gradient descent method;

[0121] Adaptive adjustment of gear thresholds: Dynamically adjust the gear division criteria based on changes in data block score distribution to ensure a balanced number of data blocks in each gear;

[0122] Recycling intensity regulation: Based on historical performance data, a relationship model between recycling intensity and performance impact is established; control theory methods are used to achieve closed-loop regulation.

[0123] 6. Integrates wear leveling considerations to automatically optimize the erase and write distribution of flash blocks during storage space reclamation.

[0124] The following describes a storage space recovery method according to an embodiment of the present invention through a specific embodiment. The method is first described in a modular form and mainly includes the following components:

[0125] 1. Metadata management module: responsible for maintaining metadata information such as the physical layout of data blocks, effective data volume, access popularity, etc.

[0126] 2. Change recording module: real-time tracking of data block status changes, including the increase and decrease of valid data and changes in access modes.

[0127] 3. Statistical analysis module: calculates the recycling priority score of each data block and sorts and classifies them.

[0128] 4. Strategy decision module: Dynamically adjusts recycling strategy parameters according to system status.

[0129] 5. Task execution module: responsible for specific data migration and space recovery operations.

[0130] 6. Monitoring and feedback module: collects recycling effect data for strategy optimization.

[0131] Specifically, the core functions and roles of each module are shown in Table 2, which is a module function and role description table.

[0132]

[0133] Combined with the above modules, the specific process of the storage space recovery method is explained, such as Figure 2 As shown, including:

[0134] Phase 1: Data status monitoring and recording. This phase mainly completes the real-time tracking and recording of data block status.

[0135] 1. Write operation processing: When receiving a new write request, the storage system first writes the new data to the free area of ​​the flash memory, then updates the metadata mapping table to point the logical address to the new physical location; marks the old data location as "invalid"; calls the space reclamation interface to increase the valid data count of the newly written data block; and marks the data heat level based on I / O characteristics (such as access frequency and data size).

[0136] 2. Data invalidation processing: When data becomes invalid due to deletion or overwriting: reduce the valid data count of the corresponding data block through the space recovery interface; update the invalid data distribution information in the metadata.

[0137] 3. Access pattern tracking: Continuously monitor the access characteristics of each data block, including: access frequency; last access time; access pattern (sequential / random); and data lifecycle prediction.

[0138] Phase 2: Recycling priority calculation and sorting. In this phase, all data blocks in the system are evaluated in multiple dimensions to determine the recycling priority.

[0139] 1. Calculation of basic indicators.

[0140] Invalid data rate = 1 - (valid data volume / total data block capacity);

[0141] Data heat = f (access frequency, last access time, access mode);

[0142] Wear degree = number of times a data block has been erased / maximum number of times it can be erased.

[0143] 2. Composite scoring algorithm.

[0144] Recycling priority score = α × (invalid data rate) + β × (1-data heat) + γ × (1-wear degree);

[0145] α, β, and γ are adjustable weight parameters, with default settings of 0.6, 0.3, and 0.1, respectively. These parameters can be dynamically adjusted based on workload characteristics.

[0146] 3. Hierarchical sorting mechanism.

[0147] The recycling priority scores are divided into multiple levels (for example, 0-14), and each level corresponds to a doubly linked list to store data blocks with similar recycling value.

[0148] Phase 3: Adaptive resource allocation. In this phase, resources are dynamically adjusted and recycled based on the current load of the storage system.

[0149] 1. Calculation of recycling intensity.

[0150] Number of recycling threads = min(maximum number of recycling threads, number of basic threads + [(1-current free space ratio) × expansion factor]);

[0151] The basic number of threads is usually set to 2-4, and the expansion factor is determined according to the hardware configuration.

[0152] 2. Resource scheduling strategy.

[0153] When the storage system is idle, the number of recovery threads is increased to accelerate space recovery; when the storage system is busy, the number of recovery threads is reduced to prioritize foreground I / O performance.

[0154] 3. I / O bandwidth control.

[0155] A token bucket algorithm is used to limit the I / O bandwidth usage of the recovery operation to ensure that it does not exceed 30% of the total bandwidth.

[0156] Phase 4: Reclamation task execution. This phase actually performs data migration and space reclamation operations.

[0157] 1. Data block selection.

[0158] The data blocks to be recycled are selected starting from the highest priority gear; each recycling thread processes one data block at a time.

[0159] 2. Effective data migration.

[0160] Read all valid data in the data block; write the valid data to the new target location; update the metadata mapping relationship.

[0161] 3. Space recovery.

[0162] After all valid data in a data block is migrated, the data block is marked as "erasable"; the background erasure thread periodically erases these data blocks in batches to make them available space.

[0163] 4. Wear leveling treatment.

[0164] When selecting a target location to write migration data, priority is given to flash memory blocks with fewer erase / write counts; the erase / write history of each flash memory block is recorded to avoid local overload.

[0165] The storage space recovery solution of the present invention can produce the following effects:

[0166] 1. Improved recycling efficiency: By accurately identifying high-value recycling targets, the amount of effective data migration is reduced by an average of 40%; the amount of reclaimable space per unit time is increased by 2-3 times.

[0167] 2. Reduced performance impact: The fluctuation range of foreground I / O latency is reduced by more than 60%; the performance drop during peak hours does not exceed 15% of the baseline value.

[0168] 3. Extended flash memory life: Wear leveling is enhanced, and the difference in the number of erase and write cycles between the longest- and shortest-life flash memory blocks is reduced to less than 30%. The overall device life is expected to be extended by 20-30%.

[0169] 4. Optimized resource utilization: Recycled resource usage is reduced by 35%; computing and I / O resources are used more balanced and efficiently.

[0170] 5. Strong adaptability: It can automatically adapt to different workload characteristics; the parameter tuning cycle is shortened from weekly level with manual intervention to hourly level.

[0171] 6. Good scalability: supports storage systems of different scales from TB to PB; linear expansion capability meets future growth needs.

[0172] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0173] The storage space recovery method proposed in an embodiment of the present invention comprehensively considers the impact of multiple factors on storage space recovery, specifically determines the recovery score of the data block based on the invalid data rate, data heat and wear level, and determines the recovery priority of the data block based on the recovery score. Storage space is recovered for the data blocks according to the recovery priority, and data blocks with the greatest recovery benefits are processed first, thereby improving the rationality of selecting data blocks for storage space recovery. Since data inefficiency and data heat are taken into account, the effective data migration rate of the recovered data blocks can be reduced, thereby improving the recovery efficiency of the storage space, and the considered heat data can reduce the performance impact on the storage system. In addition, considering the wear level of the data blocks also extends the service life of the storage system to a certain extent.

[0174] An embodiment of the present invention further provides a storage space recovery device.

[0175] Figure 3 A block diagram of a storage space recovery device provided according to an embodiment of the present invention.

[0176] like Figure 3 As shown, the storage space recovery device 10 includes: an identification module 100 , a calculation module 200 and a recovery module 300 .

[0177] Among them, the identification module 100 is used to identify multi-dimensional data of at least one data block in the storage system; the calculation module 200 is used to calculate the invalid data rate, data heat and wear degree of the corresponding data block based on the multi-dimensional data, and calculate the recovery score of the corresponding data block based on the invalid data rate, data heat and wear degree; the recovery module 300 is used to determine the recovery priority of the data block in the storage system according to the recovery score, and reclaim storage space for the data block in the storage system according to the recovery priority.

[0178] In an embodiment of the present invention, the calculation module 200 is further used to: extract the valid data volume, total capacity, number of erased and written times, maximum number of erased and written times, and access data from multi-dimensional data; calculate the invalid data rate based on the valid data volume and total capacity; calculate the data heat based on the access data; and calculate the degree of wear based on the number of erased and written times and the maximum number of erased and written times.

[0179] In an embodiment of the present invention, the calculation module 200 is further used to: obtain the corresponding weights of the invalid data rate, data heat and wear degree; and calculate the recycling score of the corresponding data block based on the invalid data rate, data heat and wear degree and the corresponding weights.

[0180] In this embodiment of the present invention, the calculation formula for the recovery score is:

[0181] S=α×A+ β×(1-B) + γ×(1-C);

[0182] Among them, S is the recycling score, α is the weight of the invalid data rate, A is the invalid data rate, β is the weight of the data heat, B is the data heat, γ is the weight of the wear degree, and C is the wear degree.

[0183] In the embodiment of the present invention, the recycling module 300 is further configured to divide the recycling scores into a plurality of recycling priorities, wherein each recycling priority corresponds to a bidirectional linked list.

[0184] In the embodiment of the present invention, the recycling module 300 is further configured to: determine at least one data block to be recycled in the storage system according to the recycling priority; and reclaim storage space for the at least one data block to be recycled using a recycling thread.

[0185] In the embodiment of the present invention, the storage space recovery device 10 of the embodiment of the present invention further includes: a determination module.

[0186] Among them, the determination module is used to identify the current load data of the storage system before determining at least one data block to be reclaimed in the storage system according to the recycling priority; determine the number of recycling threads for storage space recycling based on the current load data; and determine the number of data blocks to be reclaimed based on the number of recycling threads and the recycling priority.

[0187] In an embodiment of the present invention, the determination module is further used to: obtain thread parameters of the storage system; identify the free space ratio in the current load data; and calculate the number of recycling threads for storage space recycling based on the thread parameters and the free space ratio.

[0188] In the embodiment of the present invention, one reclaiming thread reclaims the storage space of one data block.

[0189] In an embodiment of the present invention, the recovery module 300 is further used to: read valid data in at least one data block to be recovered; write the valid data to the target position of the target flash memory block, and perform an erase operation on at least one data block to be recovered to realize storage space recovery of at least one data block to be recovered.

[0190] In the embodiment of the present invention, the storage space recovery device 10 of the embodiment of the present invention further includes: a restriction module.

[0191] The limiting module is used to limit bandwidth usage during storage space reclaiming before reclaiming storage space for data blocks in the storage system according to the reclaiming priority.

[0192] In the embodiment of the present invention, the storage space recovery device 10 of the embodiment of the present invention further includes: an optimization module.

[0193] Among them, the optimization module is used to evaluate the impact data of invalid data rate, data heat and wear degree on storage space recovery after the storage space of data blocks in the storage system is recovered according to the recovery priority; and optimize the corresponding weights of invalid data rate, data heat and wear degree based on the impact data.

[0194] It should be noted that the description of the features in the embodiment corresponding to the storage space recovery device can be found in the relevant description of the embodiment corresponding to the storage space recovery method, and will not be repeated here.

[0195] The storage space recovery device proposed in accordance with an embodiment of the present invention comprehensively considers the impact of multiple factors on storage space recovery, specifically determines the recovery score of the data block based on the invalid data rate, data heat and wear level, and determines the recovery priority of the data block based on the recovery score. Storage space is recovered for the data blocks according to the recovery priority, and data blocks with the greatest recovery benefits are processed first, thereby improving the rationality of selecting data blocks for storage space recovery. Since data inefficiency and data heat are taken into account, the effective data migration rate of the recovered data blocks can be reduced, thereby improving the recovery efficiency of the storage space, and the considered heat data can reduce the performance impact on the storage system. In addition, the consideration of the wear level of the data blocks also extends the service life of the storage system to a certain extent.

[0196] An embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, it includes a memory 401 and a processor 402, the memory 401 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above storage space recovery method embodiments.

[0197] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned storage space recovery method embodiments when running.

[0198] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0199] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned storage space recovery method embodiments are implemented.

[0200] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0201] The above is a detailed introduction to a storage space recovery method provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A storage space recovery method, characterized in that: The following steps are involved: Identifying multi-dimensional data of at least one data block in a storage system, wherein the multi-dimensional data includes valid data volume, total capacity, number of erase / write cycles, maximum number of erase / write cycles, and access data, wherein the access data includes short-term access frequency, long-term access pattern, temporal locality, and spatial locality; Calculating an invalid data rate, data heat, and wear degree of a corresponding data block based on the multi-dimensional data, and calculating a recycling score of the corresponding data block based on the invalid data rate, the data heat, and the wear degree, wherein the data heat is obtained by weighted calculation based on the short-term access frequency, the long-term access pattern, the temporal locality, and the spatial locality; calculating the recycling score of the corresponding data block based on the invalid data rate, the data heat, and the wear degree, comprising: obtaining corresponding weights of the invalid data rate, the data heat, and the wear degree; and calculating the recycling score of the corresponding data block based on the invalid data rate, the data heat, the wear degree, and the corresponding weights; Determine a recycling priority of a data block in the storage system based on the recycling score, reclaim storage space for at least one data block to be recycled in the storage system according to the recycling priority, and write valid data in the at least one data block to be recycled to a target flash memory block, wherein the target flash memory block is a flash memory block with a number of erase and write times less than a preset threshold.

2. The storage space recovery method according to claim 1, characterized in that: Calculating the invalid data rate, data heat, and wear degree of the corresponding data block according to the multi-dimensional data includes: Extracting the effective data volume, total capacity, number of erase / write times, maximum number of erase / write times and access data from the multi-dimensional data; Calculating the invalid data rate according to the valid data amount and the total capacity; The wear degree is calculated according to the number of erased and written times and the maximum number of erased and written times.

3. The storage space recovery method according to claim 1, wherein: The calculation formula for the recovery score is: S=α×A+ β×(1-B) + γ×(1-C); Among them, S is the recycling score, α is the weight of the invalid data rate, A is the invalid data rate, β is the weight of the data heat, B is the data heat, γ is the weight of the wear degree, and C is the wear degree.

4. The storage space recovery method according to claim 1, characterized in that: Determining the recycling priority of the data block in the storage system according to the recycling score includes: The recycling score is divided into a plurality of recycling priorities, wherein each recycling priority corresponds to a bidirectional linked list.

5. The storage space recovery method according to claim 1, wherein: The reclaiming storage space of the data blocks in the storage system according to the reclaiming priority includes: At least one to-be-reclaimed data block in the storage system is determined according to the reclaim priority, and storage space is reclaimed for the at least one to-be-reclaimed data block using a reclaim thread.

6. The storage space recovery method according to claim 5, characterized in that: Before determining at least one data block to be reclaimed in the storage system according to the reclaim priority, the method further includes: identifying current load data of the storage system; Determining the number of recycling threads for recycling the storage space based on the current load data; The number of the data blocks to be reclaimed is determined based on the number of reclaim threads and the reclaim priority.

7. The storage space recovery method according to claim 6, characterized in that: The determining the number of reclaiming threads for reclaiming the storage space based on the current load data includes: Obtaining thread parameters of the storage system; identifying a proportion of free space in the current load data; The number of reclaiming threads for reclaiming the storage space is calculated based on the thread parameters and the free space ratio.

8. The storage space recovery method according to claim 7, characterized in that: A recycling thread reclaims the storage space of a data block.

9. The storage space recovery method according to claim 5, characterized in that: The reclaiming storage space of the at least one to-be-reclaimed data block by using a reclaim thread includes: Reading valid data in the at least one data block to be recycled; The valid data is written to a target location of a target flash memory block, and an erasing operation is performed on the at least one data block to be reclaimed, so as to reclaim the storage space of the at least one data block to be reclaimed.

10. The storage space recovery method according to claim 7, characterized in that: Before reclaiming storage space of the data blocks in the storage system according to the reclaiming priority, the method further includes: Limits bandwidth usage during storage reclamation.

11. The storage space recovery method according to claim 3, characterized in that: After reclaiming storage space of the data blocks in the storage system according to the reclaim priority, the method further includes: Evaluate the impact of the invalid data rate, the data heat, and the wear degree on the storage space reclamation after the data blocks in the storage system are reclaimed; The corresponding weights of the invalid data rate, the data heat, and the wear degree are optimized based on the impact data.

12. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the storage space recovery method according to any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the storage space reclaiming method according to any one of claims 1 to 11.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the storage space recovery method according to any one of claims 1 to 11 are implemented.

Images