Storage space recovery method, electronic equipment, storage medium and program product

Abstract

The invention discloses a storage space recovery method, electronic equipment, a storage medium and a program product, and relates to the technical field of computers, the influence of various factors on storage space recovery is comprehensively considered, and the recovery score of a data block is determined specifically according to the invalid data rate, the data heat degree and the wear degree; according to the method and the device, the data blocks are selected to be subjected to storage space recovery, the recovery priorities of the data blocks are determined according to the recovery scores, the data blocks are subjected to storage space recovery according to the recovery priorities, and the data blocks with the maximum recovery benefits are preferentially processed, so that the reasonability of the data blocks subjected to storage space recovery is improved; the effective data migration rate of the recovered data blocks can be reduced, so that the recovery efficiency of the storage space is improved, the influence on the performance of the storage system can be reduced by considering the heat data, in addition, the wear degree of the data blocks is considered, and the service life of the storage system is also prolonged to a certain extent.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and in particular, to a storage space recycling method, an electronic device, a storage medium, and a program product. Background Art

[0002] With the wide application of all-flash storage systems in modern data centers, cloud computing, and big data analysis platforms, the space recycling efficiency of storage systems has become a key factor affecting overall performance.

[0003] In related technologies, the first-in-first-out or random selection strategy is mainly used for space recycling, which may cause a large amount of valid data to be migrated for data blocks undergoing space recycling, resulting in low space recycling efficiency. Moreover, when the storage system space is tight, real-time recycling operations are performed on the foreground I / O (Input / Output) path, which will significantly increase I / O latency and thus affect the performance of the storage system. Summary of the Invention

[0004] The present invention provides a storage space recycling method, an electronic device, a storage medium, and a program product to at least solve the problems of low space recycling efficiency and affecting the performance of the storage system in related technologies.

[0005] The present invention provides a storage space recycling method, including the following steps: identifying multi-dimensional data of at least one data block in a storage system; calculating the invalid data rate, data popularity, and wear degree of the corresponding data block according to the multi-dimensional data, and calculating the recycling score of the corresponding data block according to the invalid data rate, data popularity, and wear degree; determining the recycling priority of data blocks in the storage system according to the recycling score, and performing storage space recycling on the data blocks in the storage system according to the recycling priority.

[0006] The present invention also provides a storage space recycling device, including: an identification module for identifying multi-dimensional data of at least one data block in a storage system; a calculation module for calculating the invalid data rate, data popularity, and wear degree of the corresponding data block according to the multi-dimensional data, and calculating the recycling score of the corresponding data block according to the invalid data rate, data popularity, and wear degree; a recycling module for determining the recycling priority of data blocks in the storage system according to the recycling score, and performing storage space recycling on the data blocks in the storage system according to the recycling priority.

[0007] The present invention also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above storage space recycling methods when executing the computer program.

[0008] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any one of the above storage space recovery methods.

[0009] The present invention also provides a computer program product including a computer program, which, when executed by a processor, implements the steps of any one of the above storage space recovery methods.

[0010] Through the present invention, by comprehensively considering the influence of various factors on storage space recovery, specifically determining the recovery score of data blocks according to the invalid data rate, data heat, and wear degree, and determining the recovery priority of data blocks according to the recovery score, and performing storage space recovery on data blocks according to the recovery priority, preferentially processing the data blocks with the greatest recovery benefit, the rationality of the data blocks selected for storage space recovery is improved. Since the data inefficiency and data heat are considered, the effective data migration rate of the recovered data blocks can be reduced, thereby improving the storage space recovery efficiency. Moreover, the heat data considered can reduce the performance impact on the storage system. In addition, considering the wear degree of data blocks, the service life of the storage system is extended to a certain extent. Therefore, the technical problems of low space recovery efficiency and performance impact on the storage system in the related art can be solved, and the technical effects of improving the space recovery efficiency and reducing the performance impact on the storage system can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a flowchart of the storage space recovery method provided by the embodiment of the present invention; Figure 2 It is an execution diagram of the storage space recovery method provided by the embodiment of the present invention; Figure 3 It is a block diagram of the storage space recovery device provided by the embodiment of the present invention; Figure 4 It is a structural diagram of the electronic device provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0013] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present invention.

[0014] It should be noted that in the description of the present invention, the terms "including", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0015] Before describing the solution of the present invention, first introduce the technical problems specifically existing in the related technologies of space recycling: 1. Low recycling efficiency: When performing space recycling, related technologies often adopt simple first-in-first-out or random selection strategies, without fully considering factors such as the proportion of valid data in data blocks and data access popularity. As a result, a large amount of valid data needs to be migrated during the recycling process, causing additional I / O overhead and performance loss.

[0016] 2. Severe foreground I / O interference: When the storage system is short of space, the space recycling mechanism of related technologies is often forced to perform real-time recycling operations on the foreground I / O path, which will significantly increase the I / O latency, resulting in severe fluctuations in the system response time and making it difficult to meet the service quality requirements of high-performance storage systems.

[0017] 3. Insufficient consideration of wear leveling: When recycling space, related technologies fail to fully consider the distribution of the number of erase / write cycles (P / E cycles) of flash memory blocks, which may cause some flash memory blocks to be overwritten, while other blocks are underutilized, thus shortening the overall service life of the storage device.

[0018] 4. Unbalanced resource utilization: Related technologies lack intelligence in recycling thread scheduling and resource allocation, and cannot dynamically adjust the recycling intensity according to the current load situation of the storage system, easily causing waste or insufficiency of computing resources and I / O bandwidth.

[0019] 5. Difficulty in balancing multi-dimensional optimization goals: The space recycling process needs to consider multiple optimization goals such as recycling efficiency, performance impact, and wear leveling at the same time, and it is difficult for related technologies to achieve a good balance among these mutually restrictive goals.

[0020] To this end, the present invention proposes a storage space recycling method, which introduces multi-dimensional evaluation metrics, a dynamic priority adjustment mechanism, and an adaptive resource allocation strategy to solve at least one of the above technical problems.

[0021] To enable those skilled in the art of this technology to better understand the solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] An embodiment of the present invention provides a storage space recycling method, and the method will be described in detail in combination with the execution process of the storage space recycling method.

[0023] As Figure 1 shown, the storage space recycling method of the embodiment of the present invention includes the following steps: In step S101, multi-dimensional data of at least one data block in the storage system is identified.

[0024] Among them, the data block is the basic unit for the storage system to manage data logically, and is composed of valid data and invalid data. When data is written, deleted, or overwritten, the status of the data block (such as the amount of valid data, access frequency, etc.) will be tracked in real time. At the physical level, the data blocks of the data will be split and stored in flash memory blocks, and the storage of the data blocks depends on the physical space of the flash memory blocks; the multi-dimensional data includes access data, valid data volume, total capacity, number of erase cycles, maximum number of erase cycles, etc. of the data block.

[0025] The storage system of the embodiment of the present invention can be an all-flash storage system, and the storage system of the embodiment of the present invention can be the storage system of a server or the storage system of other electronic devices.

[0026] In step S102, the invalid data rate, data heat, and wear degree of the corresponding data block are calculated according to the multi-dimensional data, and the recycling score of the corresponding data block is calculated according to the invalid data rate, data heat, and wear degree.

[0027] Among them, the recycling score can also be called the recycling priority score.

[0028] It can be understood that the embodiment of the present invention can calculate the invalid data rate, data heat, and wear degree of the corresponding data block according to the multi-dimensional data, and calculate the recycling score of the corresponding data block according to the invalid data rate, data heat, and wear degree, so as to recycle the storage space of the storage system subsequently.

[0029] In an embodiment of the present invention, calculating the invalid data rate, data heat, and wear degree of a corresponding data block based on multi-dimensional data includes: extracting the amount of valid data, total capacity, number of erase / write cycles, maximum number of erase / write cycles, and accessed data from the multi-dimensional data; calculating the invalid data rate based on the amount of valid data and the total capacity; calculating the data heat based on the accessed data; and calculating the wear degree based on the number of erase / write cycles and the maximum number of erase / write cycles.

[0030] Among them, the multi-dimensional data includes the amount of valid data, total capacity, number of erase / write cycles, maximum number of erase / write cycles, and accessed data, and the accessed data includes access frequency, most recent access time, access pattern, etc.

[0031] It can be understood that the embodiment of the present invention can extract specific data from the multi-dimensional data and calculate the invalid data rate, data heat, and wear degree of the corresponding data block according to the specific data. Specifically: calculating the invalid data rate based on the amount of valid data and the total capacity, calculating the data heat based on the accessed data, and calculating the wear degree based on the number of erase / write cycles and the maximum number of erasable cycles.

[0032] Specifically, the calculation method of the invalid data rate is: invalid data rate = 1 - (amount of valid data / total capacity of the data block); The calculation method of the data heat is: data heat = f(access frequency, most recent access time, access pattern); The calculation method of the wear degree is: wear degree = number of erase / write cycles of the data block / maximum number of erase / write cycles.

[0033] The data heat evaluation in the embodiment of the present invention comprehensively considers the following factors: Short-term access frequency: the access frequency within the most recent 1 minute; Long-term access pattern: the access distribution within the past 24 hours; Temporal locality: based on weight assignment with exponential decay, the weight of a closer access is higher; Spatial locality: identification and reward of continuous access patterns.

[0034] Data heat calculation formula: data heat = 0.4 × short-term access frequency + 0.3 × long-term access pattern + 0.2 × temporal locality + 0.1 × spatial locality.

[0035] For example, if data block X is accessed 15 times within the last 1 minute and the maximum access frequency is 30 times, then the short-term access frequency is 15 / 30 = 0.5. If it has been accessed 200 times within the past 24 hours and the average daily access is 400 times, then the long-term access pattern is 200 / 400 = 0.5. If the access proportion within the last 1 hour is 80% and the proportion in the remaining time is 20% (exponentially decaying weight), then the temporal locality is 0.8. If the number of consecutive accesses is 5 times (reward coefficient 1.0) and there is no reward for non-consecutive accesses, then the spatial locality is 1. The data heat of data block X = 0.4×0.5 + 0.3×0.5 + 0.2×0.8 + 0.1×1 = 0.61.

[0036] If the effective data volume of data block X is 10 MB and the total capacity is 16 MB, then the invalid data rate = 1 - (10 / 16) = 0.375 (37.5%); If the number of erase / write cycles of data block X is 300 times and the maximum number of erase / write cycles is 3000 times, then the wear level = 300 times / 3000 times = 0.1 (10%).

[0037] In addition, the embodiments of the present invention can perform real-time tracking and recording of the data block status to obtain multi-dimensional data of the data block. Among them, the processing operations for the data block to update the multi-dimensional data include: 1. Write operation processing.

[0038] When a new write request is received, the system first writes the new data in the free area of the flash memory, specifically including: Updating the metadata mapping table to point the logical address to the new physical location; marking the old data location as "invalid"; calling the space recycling interface to increase the effective data count of the newly written data block; marking the data heat level according to the I / O characteristics (such as access frequency, data size, etc.).

[0039] 2. Data invalidation processing.

[0040] When the data becomes invalid due to deletion or overwriting, the effective data count of the corresponding data block is reduced through the space recycling interface; the invalid data distribution information in the metadata is updated.

[0041] 3. Access pattern tracking.

[0042] The system continuously monitors the access characteristics of each data block, including: access frequency; recent access time; access pattern (sequential / random); data life cycle prediction.

[0043] Among them, the present invention can adopt a lightweight monitoring mechanism to record the status changes through an efficient data structure (such as a bitmap) to ensure that the monitoring overhead does not exceed 3% of the total I / O resources.

[0044] In an embodiment of the present invention, calculating a recycling score for a corresponding data block according to the invalid data rate, data heat, and wear degree includes: obtaining corresponding weights of the invalid data rate, data heat, and wear degree; calculating the recycling score for the corresponding data block based on the invalid data rate, data heat, and wear degree and the corresponding weights.

[0045] Among them, the calculation formula for the recycling score is: S = α×A + β×(1 - B) + γ×(1 - C); where S is the recycling score, α is the weight of the invalid data rate, A is the invalid data rate, β is the weight of the data heat, B is the data heat, γ is the weight of the wear degree, and C is the wear degree.

[0046] It can be understood that the embodiment of the present invention can calculate the recycling score for the corresponding data block according to the invalid data rate, data heat, and wear degree and the corresponding weights. Specifically, Recycling score = α×Invalid data rate + β×(1 - Data heat) + γ×(1 - Wear degree).

[0047] In addition, it should be noted that α, β, and γ are all adjustable weight parameters, which can be default set to 0.6, 0.3, and 0.1 respectively, and the weight parameters can be dynamically adjusted according to the workload characteristics of the storage system.

[0048] For example, the invalid data rate of data block Y is 80%, the data heat is 0.2 (representing cold data with low access frequency), and the wear degree is 30%.

[0049] The recycling score of data block Y = 0.6×0.8 + 0.3×(1 - 0.2) + 0.1×(1 - 0.3) = 0.79.

[0050] In step S103, determining the recycling priority of data blocks in the storage system according to the recycling score, and recycling the storage space of data blocks in the storage system according to the recycling priority.

[0051] It can be understood that in the embodiments of the present invention, the recycling priority of data blocks in the storage system can be determined according to the recycling score, and the storage space of the data blocks in the storage system can be recycled according to the recycling priority. By comprehensively considering the impacts of various factors on the storage space recycling, specifically determining the recycling score of the data blocks according to the invalid data rate, data heat, and wear degree, and determining the recycling priority of the data blocks according to the recycling score, and recycling the storage space of the data blocks according to the recycling priority, giving priority to processing the data blocks with the greatest recycling benefit, the rationality of the data blocks selected for storage space recycling is improved. Since the data inefficiency and data heat are considered, the effective data migration rate of the recycled data blocks can be reduced, thereby improving the recycling efficiency of the storage space. Moreover, the considered heat data can reduce the performance impact on the storage system. In addition, considering the wear degree of the data blocks, to a certain extent, the service life of the storage system is extended.

[0052] In the embodiments of the present invention, determining the recycling priority of data blocks in the storage system according to the recycling score includes: dividing the recycling score into multiple recycling priorities, where each recycling priority corresponds to a doubly linked list.

[0053] Wherein, a doubly linked list is a data structure in which each node contains a predecessor pointer and a successor pointer, allowing bidirectional traversal (forward or backward) starting from any node, and having more flexible node operation capabilities compared to a singly linked list. Each recycling priority corresponds to an independent doubly linked list, and the list nodes store the metadata of the data blocks in this gear, such as physical addresses, recycling scores, valid data amounts, etc. The doubly linked list can directly locate the data blocks with different recycling priorities, and thus can improve the efficiency of storage space recycling.

[0054] For example, the present invention can divide the recycling score (i.e., the recycling priority score) into multiple recycling priorities, or it can also be understood as dividing the recycling score into multiple gears (such as levels 0 - 14), and each gear corresponds to a doubly linked list, storing data blocks with similar recycling values. The grading standard is shown in Table 1, and Table 1 is the grading standard table.

[0055]

[0056] Through this grading mechanism, the storage system can be allowed to give priority to processing the data blocks with the greatest recycling benefit, significantly improving the storage space recycling efficiency.

[0057] In the embodiments of the present invention, recycling the storage space of the data blocks in the storage system according to the recycling priority includes: determining at least one data block to be recycled in the storage system according to the recycling priority; using a recycling thread to recycle the storage space of at least one data block to be recycled.

[0058] Among them, one recycling thread corresponds to one data block to be recycled, that is, each recycling thread processes one data block each time.

[0059] It can be understood that the embodiments of the present invention can determine at least one data block to be recycled in the storage system according to the recycling priority, and use the recycling thread to recycle the storage space of the at least one data block to be recycled.

[0060] In the embodiments of the present invention, using the recycling thread to recycle the storage space of at least one data block to be recycled includes: reading the valid data in the at least one data block to be recycled; writing the valid data to the target position of the target flash block, and performing an erasing operation on the at least one data block to be recycled, so as to realize the recycling of the storage space of the at least one data block to be recycled.

[0061] Among them, the target flash block can be selected as the flash block with fewer erasing times, so as to avoid selecting the flash block with more erasing times, avoid excessive erasing of some flash blocks, and improve the service life of the flash block.

[0062] It can be understood that the embodiments of the present invention can read the valid data in the data block to be recycled, write the valid data to the target position of the target flash block, and then perform an erasing operation on the at least one data block to be recycled, so as to avoid erasing the valid data, and thus realize the recycling of the storage space of the data block to be recycled.

[0063] In the embodiments of the present invention, after writing the valid data to the target position of the target flash block, it further includes: updating the mapping relationship of the valid data.

[0064] It can be understood that the embodiments of the present invention can update the mapping relationship of the valid data after writing the valid data to the target position of the target flash block, that is, after completing the migration of the valid data in the data block to be recycled, so as to be able to use the valid data subsequently and avoid being unable to query the valid data.

[0065] Specifically, the embodiments of the present invention perform data migration and space recycling operations on the data block to be recycled.

[0066] 1. Data block selection.

[0067] Select the data block to be recycled starting from the highest priority.

[0068] 2. Valid data migration.

[0069] Read all the valid data in the data block to be recycled, write the valid data to the new target position, and update the metadata mapping relationship.

[0070] 3. Space recycling.

[0071] After all the valid data in the data block to be recycled has been migrated, mark the data block as "erasable"; the background erasure thread periodically erases these data blocks in batches to make them available space.

[0072] 4. Wear leveling processing.

[0073] When selecting a target location to write the migrated data, preferentially select the flash block with fewer erase / write cycles; record the erase / write history of each flash block to avoid local overload.

[0074] In the embodiment of the present invention, before determining at least one data block to be recycled in the storage system according to the recycling priority, it further includes: identifying the current load data of the storage system; determining the number of recycling threads for storage space recycling based on the current load data; determining the number of data blocks to be recycled based on the number of recycling threads and the recycling priority.

[0075] It can be understood that the embodiment of the present invention can identify the current load data of the storage system, determine the number of recycling threads for storage space recycling based on the current load data, and determine the number of data blocks to be recycled based on the number of recycling threads and the recycling priority, so as to dynamically adjust the recycling intensity according to the current load situation to avoid wasting computing resources or causing insufficient I / O bandwidth.

[0076] In the embodiment of the present invention, determining the number of recycling threads for storage space recycling based on the current load data includes: obtaining the thread parameters of the storage system; identifying the free space ratio in the current load data; calculating the number of recycling threads for storage space recycling based on the thread parameters and the free space ratio.

[0077] Among them, the thread parameters include the maximum number of recycling threads and the basic number of threads. The basic number of threads and the maximum number of recycling threads are pre-calibrated, and the basic number of threads is usually set to 2-4.

[0078] It can be understood that the present invention can identify the free space ratio in the current load data and calculate the number of recycling threads for storage space recycling based on the thread parameters and the free space ratio.

[0079] The calculation method of the number of recycling threads in the embodiment of the present invention is as follows: Number of recycling threads = min (maximum number of recycling threads, basic number of threads + [(1 - current free space ratio) × expansion coefficient]).

[0080] Among them, the expansion coefficient is determined according to the hardware configuration.

[0081] Specifically, in the embodiment of the present invention, when the storage system is idle, the number of recycling threads can be increased to accelerate space recycling, and when the storage system is busy, the number of recycling threads can be reduced to preferentially ensure the foreground I / O performance.

[0082] In an embodiment of the present invention, before reclaiming storage space for data blocks in a storage system according to the reclaim priority, the method further includes: restricting the bandwidth occupation during storage space reclamation.

[0083] Since the space reclamation mechanism in the related art often has to perform real-time reclamation operations on the foreground I / O path, which significantly increases the I / O latency, resulting in drastic fluctuations in the system response time and making it difficult to meet the service quality requirements of high-performance storage systems. Therefore, the embodiment of the present invention can also restrict the bandwidth occupation during storage space reclamation to avoid increasing the I / O latency, avoid drastic fluctuations in the storage system response time, and meet the service quality requirements of high-performance storage systems.

[0084] Specifically, the embodiment of the present invention can use the token bucket algorithm to restrict the I / O bandwidth occupation of the reclamation operation to ensure that it does not exceed 30% of the total bandwidth.

[0085] In an embodiment of the present invention, after reclaiming storage space for data blocks in a storage system according to the reclaim priority, the method further includes: evaluating the impact data of the invalid data rate, data heat, and wear degree on storage space reclamation after the storage space of the data blocks in the storage system is reclaimed; optimizing the corresponding weights of the invalid data rate, data heat, and wear degree based on the impact data.

[0086] It can be understood that the embodiment of the present invention can also evaluate the impact data of the invalid data rate, data heat, and wear degree on storage space reclamation after the storage space of the data blocks in the storage system is reclaimed; optimize the corresponding weights of the invalid data rate, data heat, and wear degree based on the impact data to improve the rationality of the selection of data blocks for future storage space reclamation.

[0087] In an embodiment of the present invention, the method further includes: identifying the score distribution of the reclaim scores of multiple data blocks; adjusting the division of multiple reclaim priorities according to the score distribution.

[0088] It can be understood that the embodiment of the present invention can also identify the score distribution of the reclaim scores of multiple data blocks and adjust the division of multiple reclaim priorities according to the score distribution to ensure the balance of the number of data blocks in each gear.

[0089] In summary, the storage space reclamation method of the embodiment of the present invention realizes the reclamation of the storage space of the storage system by introducing multi-dimensional evaluation indicators, a dynamic priority adjustment mechanism, and an adaptive resource allocation strategy, mainly including the following contents: 1. By introducing multi-dimensional data, comprehensively considering three dimensions of data block inefficiency, data heat, and wear degree, generating a composite score through an adjustable weight coefficient, and accurately identifying the data blocks with the greatest reclaim benefit.

[0090] Among them, the data heat comprehensively considers the following factors: Short-term access frequency: The access frequency within the most recent 1 minute; Long-term access pattern: The access distribution within the past 24 hours; Temporal locality: Based on exponentially decaying weight assignment, the more recent the access, the higher the weight; Spatial locality: Identification and reward of consecutive access patterns.

[0091] Data heat calculation formula: Data heat = 0.4 × Short-term access frequency + 0.3 × Long-term access pattern + 0.2 × Temporal locality + 0.1 × Spatial locality.

[0092] 2. Hierarchical space recycling mechanism: Divide data blocks into multiple priority levels according to scores, and organize each level using a doubly linked list to achieve efficient data block selection and management, ensuring that high-value data blocks are recycled first.

[0093] 3. Dynamically adjust the number of recycling threads and the proportion of I / O bandwidth occupation according to the free space ratio of the storage system and the current load situation, minimizing the impact on foreground I / O.

[0094] 4. Integrate four factors: short-term access frequency, long-term access pattern, temporal locality, and spatial locality, and obtain a comprehensive heat score through weighted calculation.

[0095] 5. By monitoring the recycling effect and performance impact, use the feedback control method to automatically adjust the weight and recycling intensity parameters, that is, establish a closed-loop feedback mechanism, continuously optimize the recycling strategy parameters, and adapt to different workload characteristics.

[0096] Among them, weight parameter (α, β, γ) adjustment: Regularly (such as every hour) evaluate the impact of each dimension on the recycling effect; use the gradient descent method to optimize weight assignment; Threshold adaptive adjustment of levels: Dynamically adjust the level division criteria according to the change in the data block score distribution; ensure the balance of the number of data blocks in each level; Recycling intensity adjustment: Based on historical performance data, establish a relationship model between recycling intensity and performance impact; use control theory methods to achieve closed-loop adjustment.

[0097] 6. Integrate wear leveling considerations and automatically optimize the write distribution of flash blocks during the storage space recycling process.

[0098] Next, a specific embodiment is used to describe the storage space recycling method of the embodiment of the present invention. First, it is described in a modular form, mainly including the following components: 1. Metadata management module: Responsible for maintaining metadata information such as the physical layout of data blocks, the amount of valid data, and access heat.

[0099] 2. Change Record Module: It tracks the status changes of data blocks in real time, including the increase or decrease of valid data and the changes in access patterns.

[0100] 3. Statistical Analysis Module: It calculates the recycling priority scores of each data block and sorts and classifies them.

[0101] 4. Policy Decision Module: It dynamically adjusts the recycling policy parameters according to the system status.

[0102] 5. Task Execution Module: It is responsible for specific data migration and space recycling operations.

[0103] 6. Monitoring and Feedback Module: It collects data on the recycling effect for policy optimization.

[0104] Specifically, the core functions and roles of each module are shown in Table 2, which is a description table of the functions and roles of the modules.

[0105]

[0106] Combined with the above modules, the specific process of the storage space recycling method is described as follows, as Figure 2 shown, including: Phase 1: Data Status Monitoring and Recording. In this phase, the real-time tracking and recording of the data block status are mainly completed.

[0107] 1. Write Operation Processing: When a new write request is received, the storage system first writes the new data in the free area of the flash memory, then updates the metadata mapping table to point the logical address to the new physical location; marks the old data location as "invalid"; calls the space recycling interface to increase the valid data count of the newly written data block; marks the data heat level according to the I / O characteristics (such as access frequency, data size, etc.).

[0108] 2. Data Invalidation Processing: When the data becomes invalid due to deletion or overwriting: reduce the valid data count of the corresponding data block through the space recycling interface; update the invalid data distribution information in the metadata.

[0109] 3. Access Pattern Tracking: Continuously monitor the access characteristics of each data block, including: access frequency; recent access time; access pattern (sequential / random); data life cycle prediction.

[0110] Phase 2: Recycling Priority Calculation and Sorting. In this phase, all data blocks in the system are evaluated multi-dimensionally to determine the recycling priority.

[0111] 1. Basic Index Calculation.

[0112] Invalid Data Rate = 1 - (Amount of Valid Data / Total Capacity of Data Block); Data heat = f(access frequency, recent access time, access pattern); Degree of wear = number of times the data block has been erased / maximum number of erasable times.

[0113] 2. Composite scoring algorithm.

[0114] Recycling priority score = α × (invalid data rate) + β × (1 - data heat) + γ × (1 - degree of wear); Among them, α, β, and γ are adjustable weight parameters, and the default settings are 0.6, 0.3, and 0.1. These parameters can be dynamically adjusted according to the characteristics of the workload.

[0115] 3. Hierarchical sorting mechanism.

[0116] The recycling priority score is divided into multiple levels (such as levels 0 - 14), and each level corresponds to a doubly linked list that stores data blocks with similar recycling values.

[0117] Phase 3: Adaptive resource allocation. In this phase, the recycling resources are dynamically adjusted according to the current load situation of the storage system.

[0118] 1. Recycling intensity calculation.

[0119] Number of recycling threads = min(maximum number of recycling threads, base number of threads + [(1 - current free space ratio) × expansion coefficient]); Among them, the base number of threads is usually set to 2 - 4, and the expansion coefficient is determined according to the hardware configuration.

[0120] 2. Resource scheduling strategy.

[0121] When the storage system is idle, increase the number of recycling threads to accelerate space recycling; when the storage system is busy, reduce the number of recycling threads to give priority to ensuring the foreground I / O performance. 3. I / O bandwidth control.

[0122] The token bucket algorithm is used to limit the I / O bandwidth occupancy of the recycling operation to ensure that it does not exceed 30% of the total bandwidth.

[0123] Phase 4: Recycling task execution. In this phase, data migration and space recycling operations are actually performed.

[0124] 1. Data block selection.

[0125] Select the data blocks to be recycled starting from the highest priority level; each recycling thread processes one data block each time.

[0126] 2. Migration of valid data.

[0127] Read all valid data in the data block; Write the valid data to a new target location; Update the metadata mapping relationship.

[0128] 3. Space recycling.

[0129] After all valid data in the data block has been migrated, mark the data block as "erasable"; The background erasure thread periodically erases these data blocks in batches to make them available space.

[0130] 4. Wear leveling processing.

[0131] When selecting a target location to write the migrated data, preferentially select flash blocks with fewer erase / write cycles; Record the erase / write history of each flash block to avoid local overload.

[0132] Through the storage space recycling solution of the present invention, the following effects can be achieved: 1. Improved recycling efficiency: By accurately identifying high-value recycling targets, the average amount of valid data migration is reduced by 40%; The amount of recyclable space per unit time is increased by 2-3 times.

[0133] 2. Reduced performance impact: The fluctuation range of foreground I / O latency is reduced by more than 60%; The performance degradation during peak periods does not exceed 15% of the baseline value.

[0134] 3. Extended flash memory lifespan: The wear leveling effect is improved, and the difference in the number of erase / write cycles between the longest and shortest lifespan flash blocks is reduced to within 30%; The overall device lifespan is expected to be extended by 20-30%.

[0135] 4. Optimized resource utilization: The occupancy rate of recycled resources is reduced by 35%; The use of computing and I / O resources is more balanced and efficient.

[0136] 5. Strong adaptability: It can automatically adapt to different workload characteristics; The parameter tuning cycle is shortened from the weekly level of manual intervention to the hourly level.

[0137] 6. Good scalability: It supports storage systems of different scales from TB level to PB level; The linear expansion ability meets future growth needs.

[0138] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0139] The storage space recycling method proposed according to the embodiments of the present invention comprehensively considers the impacts of various factors on storage space recycling. Specifically, it determines the recycling scores of data blocks based on the invalid data rate, data popularity, and wear level, and determines the recycling priorities of data blocks according to the recycling scores. It performs storage space recycling on data blocks according to the recycling priorities, preferentially processes the data blocks with the greatest recycling benefits, improves the rationality of the data blocks selected for storage space recycling. Since it takes into account the data inefficiency and data popularity, it can reduce the effective data migration rate of the recycled data blocks, thereby improving the recycling efficiency of the storage space. Moreover, the considered popularity data can reduce the performance impact on the storage system. In addition, considering the wear level of the data blocks, it also extends the service life of the storage system to a certain extent.

[0140] An embodiment of the present invention also provides a storage space recycling device.

[0141] Figure 3 It is a block diagram of the storage space recycling device provided according to the embodiments of the present invention.

[0142] As Figure 3 shown, the storage space recycling device 10 includes: an identification module 100, a calculation module 200, and a recycling module 300.

[0143] Among them, the identification module 100 is used to identify multi-dimensional data of at least one data block in the storage system; the calculation module 200 is used to calculate the invalid data rate, data popularity, and wear level of the corresponding data block according to the multi-dimensional data, and calculate the recycling score of the corresponding data block according to the invalid data rate, data popularity, and wear level; the recycling module 300 is used to determine the recycling priority of the data blocks in the storage system according to the recycling score, and perform storage space recycling on the data blocks in the storage system according to the recycling priority.

[0144] In the embodiments of the present invention, the calculation module 200 is further used to: extract the amount of valid data, total capacity, number of erased times, maximum number of erased times, and accessed data in the multi-dimensional data; calculate the invalid data rate according to the amount of valid data and the total capacity; calculate the data popularity according to the accessed data; calculate the wear level according to the number of erased times and the maximum number of erased times.

[0145] In the embodiments of the present invention, the calculation module 200 is further used to: obtain the corresponding weights of the invalid data rate, data popularity, and wear level; calculate the recycling score of the corresponding data block based on the invalid data rate, data popularity, wear level, and the corresponding weights.

[0146] In the embodiments of the present invention, the calculation formula of the recycling score is: S = α×A + β×(1 - B) + γ×(1 - C); Wherein, S is the recycling score, α is the weight of the invalid data rate, A is the invalid data rate, β is the weight of the data popularity, B is the data popularity, γ is the weight of the wear degree, and C is the wear degree.

[0147] In an embodiment of the present invention, the recycling module 300 is further configured to: divide the recycling score into multiple recycling priorities, wherein each recycling priority corresponds to a doubly linked list.

[0148] In an embodiment of the present invention, the recycling module 300 is further configured to: determine at least one data block to be recycled in the storage system according to the recycling priority; use a recycling thread to recycle the storage space of the at least one data block to be recycled.

[0149] In an embodiment of the present invention, the storage space recycling device 10 of the present invention embodiment further includes: a determination module.

[0150] Wherein, the determination module is configured to identify the current load data of the storage system before determining at least one data block to be recycled in the storage system according to the recycling priority; determine the number of recycling threads for storage space recycling based on the current load data; determine the number of data blocks to be recycled based on the number of recycling threads and the recycling priority.

[0151] In an embodiment of the present invention, the determination module is further configured to: obtain the thread parameters of the storage system; identify the free space ratio in the current load data; calculate the number of recycling threads for storage space recycling based on the thread parameters and the free space ratio.

[0152] In an embodiment of the present invention, one recycling thread recycles the storage space of one data block.

[0153] In an embodiment of the present invention, the recycling module 300 is further configured to: read the valid data in at least one data block to be recycled; write the valid data to the target position of the target flash block, and perform an erase operation on the at least one data block to be recycled to implement the recycling of the storage space of the at least one data block to be recycled.

[0154] In an embodiment of the present invention, the storage space recycling device 10 of the present invention embodiment further includes: a restriction module.

[0155] Wherein, the restriction module is configured to restrict the bandwidth occupancy during storage space recycling before recycling the storage space of the data blocks in the storage system according to the recycling priority.

[0156] In an embodiment of the present invention, the storage space recycling device 10 of the present invention embodiment further includes: an optimization module.

[0157] Among them, the optimization module is used to evaluate the impact data of the invalid data rate, data heat, and wear degree on the storage space recovery after the storage space of the data blocks in the storage system is recovered according to the recovery priority; and optimize the corresponding weights of the invalid data rate, data heat, and wear degree based on the impact data.

[0158] It should be noted that for the description of the features in the embodiments corresponding to the storage space recovery device, reference can be made to the relevant descriptions in the embodiments corresponding to the storage space recovery method, which will not be elaborated here one by one.

[0159] According to the storage space recovery device provided by the embodiment of the present invention, by comprehensively considering the influence of various factors on the storage space recovery, specifically determining the recovery score of the data block according to the invalid data rate, data heat, and wear degree, and determining the recovery priority of the data block according to the recovery score, and recovering the storage space of the data block according to the recovery priority, giving priority to processing the data block with the largest recovery benefit, improving the rationality of the data block selected for storage space recovery. Since the data inefficiency and data heat are considered, the effective data migration rate of the recovered data block can be reduced, thereby improving the recovery efficiency of the storage space, and the heat data considered can reduce the performance impact on the storage system. In addition, considering the wear degree of the data block, the service life of the storage system is extended to a certain extent.

[0160] An embodiment of the present invention also provides an electronic device, as Figure 4 shown, including a memory 401 and a processor 402. A computer program is stored in the memory 401, and the processor 402 is configured to run the computer program to execute the steps in any of the above embodiments of the storage space recovery method.

[0161] An embodiment of the present invention also provides a computer-readable storage medium, in which a computer program is stored. Among them, the computer program is configured to execute the steps in any of the above embodiments of the storage space recovery method when running.

[0162] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (abbreviated as ROM), random access memory (abbreviated as RAM), mobile hard disk, magnetic disk, or optical disc and other various media that can store computer programs.

[0163] An embodiment of the present invention also provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the storage space recovery method.

[0164] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered as exceeding the scope of the present invention.

[0165] The above has introduced in detail a method for reclaiming storage space provided by the present invention. Specific examples have been used herein to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present invention, several improvements and modifications can still be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A storage space recycling method, characterized in that, Including the following steps: Identifying multi-dimensional data of at least one data block in the storage system; Calculating the invalid data rate, data heat, and wear degree of the corresponding data block according to the multi-dimensional data, and calculating the recycling score of the corresponding data block according to the invalid data rate, the data heat, and the wear degree; Determining the recycling priority of the data blocks in the storage system according to the recycling score, and reclaiming the storage space of the data blocks in the storage system according to the recycling priority.

2. The storage space recycling method according to claim 1, wherein The calculating the invalid data rate, data heat, and wear degree of the corresponding data block according to the multi-dimensional data includes: Extracting the valid data volume, total capacity, number of erasure times, maximum number of erasure times, and accessed data in the multi-dimensional data; Calculating the invalid data rate according to the valid data volume and the total capacity; Calculating the data heat according to the accessed data; Calculating the wear degree according to the number of erasure times and the maximum number of erasure times.

3. The storage space recycling method according to claim 1, wherein The calculating the recycling score of the corresponding data block according to the invalid data rate, the data heat, and the wear degree includes: Obtaining the corresponding weights of the invalid data rate, the data heat, and the wear degree; Calculating the recycling score of the corresponding data block based on the invalid data rate, the data heat, the wear degree, and the corresponding weights.

4. The storage space recycling method according to claim 1 or 3, characterized in that The calculation formula of the recycling score is: S = α×A + β×(1 - B) + γ×(1 - C); Wherein, S is the recycling score, α is the weight of the invalid data rate, A is the invalid data rate, β is the weight of the data heat, B is the data heat, γ is the weight of the wear degree, and C is the wear degree.

5. The storage space recycling method according to claim 1, wherein The determining the recycling priority of the data blocks in the storage system according to the recycling score includes: Dividing the recycling score into multiple recycling priorities, wherein each recycling priority corresponds to a doubly linked list.

6. The storage space recycling method according to claim 1, wherein The reclaiming the storage space of the data blocks in the storage system according to the recycling priority includes: Determining at least one data block to be recycled in the storage system according to the recycling priority, and reclaiming the storage space of the at least one data block to be recycled by using a recycling thread.

7. The storage space recycling method according to claim 6, wherein Before determining at least one data block to be recycled in the storage system according to the recycling priority, it further includes: Identifying the current load data of the storage system; Determining the number of recycling threads for the storage space recycling based on the current load data; Determining the number of data blocks to be recycled based on the number of recycling threads and the recycling priority.

8. The storage space recycling method according to claim 7, wherein The determining the number of recycling threads for the storage space recycling based on the current load data includes: Obtaining the thread parameters of the storage system; Identifying the free space ratio in the current load data; Calculating the number of recycling threads for the storage space recycling based on the thread parameters and the free space ratio.

9. The storage space recycling method according to claim 8, characterized in that, One recycling thread reclaims the storage space of one data block.

10. The storage space recycling method according to claim 6, wherein The reclaiming the storage space of the at least one data block to be recycled by using a recycling thread includes: Reading the valid data in the at least one data block to be recycled; Write the valid data to the target location of the target flash block, and perform an erase operation on the at least one data block to be recycled, so as to realize the storage space recycling of the at least one data block to be recycled.

11. The storage space recovery method according to claim 8, wherein Before performing storage space recycling on the data blocks in the storage system according to the recycling priority, it further includes: Restrict the bandwidth occupancy during storage space recycling.

12. The storage space recycling method according to claim 4, wherein After performing storage space recycling on the data blocks in the storage system according to the recycling priority, it further includes: Evaluate the impact data of the invalid data rate, the data heat, and the wear degree on the storage space recycling after the data blocks in the storage system are recycled; Optimize the corresponding weights of the invalid data rate, the data heat, and the wear degree based on the impact data.

13. An electronic device, characterized in that, It includes: A memory for storing a computer program; A processor for implementing the steps of the storage space recycling method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of the storage space recycling method according to any one of claims 1 to 12 when executed by a processor.

15. A computer program product, comprising a computer program, characterized in that, The computer program implements the steps of the storage space recycling method according to any one of claims 1 to 12 when executed by a processor.

Images