Data recovery method and electronic device

Abstract

The application discloses a data recycling method and an electronic device, and relates to the technical field of storage, which comprises marking reference levels of data blocks according to reference counting and identifying sharing degrees of the data blocks; generating level distribution by counting the number and / or space occupation of the data blocks in each reference level in a group to obtain the sharing degree composition in the group; determining a comprehensive score based on the level distribution, space usage rate and running load, wherein the comprehensive score is positively correlated with the space usage rate, negatively correlated with the load, positively correlated with the proportion of low levels and negatively correlated with the proportion of high levels, so that the score of the group with low levels, tight space and low load is higher; and determining whether to recycle by comparing the comprehensive score with a score threshold, so that the data blocks with high scores are recycled preferentially, unnecessary migration of high sharing hotspot blocks is avoided, the probability of conflict between recycling and host access is reduced, the increase of time delay and the risk of data damage and loss are reduced, and the stability and data security of the storage system are improved.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a data retrieval method and electronic device. Background Technology

[0002] In scenarios such as deduplication storage, multiple data objects in a storage system can share the same physical data block. The system uses reference counting to record the number of times each data block is referenced by data objects in order to track the sharing of each data block. Data blocks whose reference counts reach zero are no longer referenced by any data objects and become garbage data blocks to be reclaimed. The space they occupy needs to be released through garbage collection operations.

[0003] Data blocks are typically managed in groups. In related technologies, the priority for reclamation is usually determined by the number of valid data blocks in a group, prioritizing the reclamation of groups with fewer valid data blocks. However, this approach treats data blocks with different reference counts within a group the same way, without distinguishing between highly and low-referenced blocks. This leads to highly referenced blocks being repeatedly migrated during the reclamation process. Since highly referenced blocks are accessed by multiple data objects, their access frequency is usually higher. When migration operations and host access requests occur simultaneously, access conflicts can easily arise, increasing access latency and potentially leading to data corruption or loss, which is detrimental to data security. Summary of the Invention

[0004] This application provides a data recycling method and electronic device to at least solve the problems in related technologies where garbage collection does not distinguish the degree of data block sharing, and the same migration strategy is used for highly referenced data blocks and low-referenced data blocks, resulting in repeated migration of highly referenced data blocks, conflicts with host access, increased access latency, and thus endangering data reliability.

[0005] This application provides a data recycling method, including: Obtain the reference count of each data block in the storage system that is referenced by data objects, and set a reference level identifier for each data block based on the reference count. The reference count is used to characterize the degree to which a data block is shared by data objects, and the reference level identifier is used to indicate the reference level to which the data block belongs. Divide all data blocks into multiple data block groups, count the number and / or space occupancy of data blocks at each reference level in the data block group, and generate level distribution statistics. Based on hierarchical distribution statistics, storage system space utilization, and storage system operating load parameters, a comprehensive score for data block groups is determined. The comprehensive score is positively correlated with space utilization, negatively correlated with operating load parameters, positively correlated with the proportion of data blocks with lower reference levels in hierarchical distribution statistics, and negatively correlated with the proportion of data blocks with higher reference levels. Based on the comparison between the comprehensive score and the score threshold, it is determined whether to perform a data block group reclamation operation.

[0006] This application also provides a data recovery device, comprising: The configuration module is used to obtain the reference count of each data block in the storage system that is referenced by data objects, and to set a reference level identifier for each data block according to the reference count. The reference count is used to characterize the degree to which the data block is shared by data objects, and the reference level identifier is used to indicate the reference level to which the data block belongs. The generation module is used to divide all data blocks into multiple data block groups, count the number and / or space occupancy of data blocks of each reference level in the data block group, and generate level distribution statistics. The scoring module is used to determine the comprehensive score of data block groups based on level distribution statistics, storage system space utilization, and storage system operating load parameters. The comprehensive score is positively correlated with space utilization, negatively correlated with operating load parameters, positively correlated with the proportion of data blocks with lower reference levels in the level distribution statistics, and negatively correlated with the proportion of data blocks with higher reference levels. The decision module is used to determine whether to perform a data block group reclamation operation based on the comparison result of the comprehensive score and the score threshold.

[0007] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing any of the above-described data recycling methods when executing the computer program.

[0008] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any of the above-described data reclamation methods.

[0009] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described data recycling methods.

[0010] The data reclamation method and electronic device provided in this application mark reference levels of data blocks according to reference counts to identify the degree of data block sharing; generate a level distribution by statistically analyzing the number of data blocks of each reference level within a group and / or space occupancy to understand the composition of the degree of sharing within the group; determine a comprehensive score based on this level distribution, space utilization, and operating load, and the comprehensive score is positively correlated with space utilization, negatively correlated with load, positively correlated with the proportion of low-level blocks, and negatively correlated with the proportion of high-level blocks, so that groups with low-level blocks, limited space, and low load have higher scores; determine whether to reclaim data blocks by comparing the comprehensive score with a score threshold, so that high-scoring data blocks are reclaimed first, avoiding unnecessary migration of highly shared hotspot blocks, reducing the probability of reclamation and host access conflicts, reducing the risk of increased latency and data corruption and loss, and improving the stability and data security of the storage system. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a data recovery method provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of a data recovery device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0014] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0015] In the field of storage technology, thin provisioning is a widely used technology and a fundamental function of storage systems. Thin provisioning allows storage systems to allocate storage space on demand, rather than pre-allocating all required capacity, thereby improving the utilization efficiency of storage resources. The implementation of thin provisioning relies on thin metadata, which is currently mostly managed based on the mapping relationship between logical addresses and physical addresses of managed data. Through this mapping relationship, the storage system can perform functions such as data aggregation, deduplication, and data compression, further optimizing the efficiency of storage space utilization. However, while storage devices supporting thin provisioning allow data to be written sequentially to the backend hard disk space, compared to traditional storage devices, their storage space requires more granular reclamation and organization to avoid space fragmentation and capacity exhaustion.

[0016] In storage systems employing technologies such as data deduplication, snapshots, cloning, and copy-on-write (COW), multiple logical objects (such as multiple virtual machines, files, or snapshots) may share the same physical data block. To track data block usage, the system typically uses a reference counting mechanism, which records how many logical objects currently reference each physical data block. When a data block's reference count reaches zero, that data block is no longer used by any logical object, becoming garbage data and needing to be reclaimed to free up storage space. A data block refers to the smallest unit of storage management in a storage system; it can be a fixed-size physical storage block (such as a 4KB block) or a deduplicated block based on content hashing.

[0017] One relevant technology involves a data reclamation and release scheme that primarily relies on determining the number of valid data blocks within a data block group (a storage management unit composed of multiple data blocks). When the number of valid data blocks in a particular data block group is low, that group is prioritized for reclamation. Furthermore, this scheme dynamically adjusts the reclamation intensity by setting high and low watermark thresholds for storage system utilization, thereby controlling the amount of data reclaimed per unit time. However, the aforementioned data reclamation and release scheme has the following shortcomings: First, all data blocks are processed using a uniform strategy, without considering the differences in the degree of sharing between different data blocks. Data blocks with high reference counts (i.e., data blocks shared by multiple logical objects) have the same data migration frequency as data blocks with low reference counts. Repeatedly migrating data blocks with high reference counts increases the probability of them conflicting with host access requests, which in turn seriously affects the host's access latency and the overall performance of the storage system.

[0018] Second, this scheme prioritizes data reclamation solely based on the number of valid data blocks in a block group. While this may quickly free up storage space in the short term, it is not optimal for the long-term operation of the storage system. An ideal data reclamation scheme should minimize the number of migrations between data generation and when it is no longer in use, thereby reducing performance overhead and data security risks associated with migration. Due to the lack of differentiated consideration of the degree of data block sharing, frequently shared data blocks are migrated frequently, increasing unnecessary host access overhead and shortening the lifespan of the storage media.

[0019] Third, data blocks with high reference counts carry the access needs of more logical objects, and their data security requirements are higher. Frequent migration increases the risk of data corruption or loss, which is not conducive to ensuring the data reliability of the storage system.

[0020] Based on this, the technical solution proposed in this application is proposed to effectively release storage space while taking into account factors such as data access performance, migration overhead and data security.

[0021] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Figure 1 A flowchart of a data recovery method provided in this application embodiment is shown below. Figure 1 As shown, the method includes the following steps: Step S101: Obtain the reference count of each data block in the storage system that is referenced by data objects, and set a reference level identifier for each data block according to the reference count. The reference count is used to characterize the degree to which the data block is shared by data objects, and the reference level identifier is used to indicate the reference level to which the data block belongs.

[0023] This embodiment describes the execution entity as a terminal device, which includes a storage system. In other embodiments, the execution entity may also be a cloud server, a client, or a smart terminal.

[0024] Specifically, the terminal device obtains the reference count of each data block in its storage system, which is referenced by data objects. This reference count is used to characterize the degree to which the data block is shared by data objects: the higher the reference count, the more data objects share the data block; the lower the reference count, the fewer data objects share the data block.

[0025] The terminal device assigns a corresponding reference level identifier to each data block based on its current reference count. This reference level identifier indicates the reference level to which the data block belongs. The terminal device has multiple pre-defined reference levels, arranged from lowest to highest reference count, each corresponding to a different sharing level range. The terminal device matches the current reference count of a data block with the pre-defined reference levels to determine the reference level to which the current reference count belongs, and sets the identifier of that reference level as the reference level identifier for that data block.

[0026] In this way, each data block in the storage system is assigned a corresponding reference level identifier. Data blocks with lower reference counts are assigned a lower reference level identifier, indicating that the data block is shared by fewer data objects; data blocks with higher reference counts are assigned a higher reference level identifier, indicating that the data block is shared by more data objects.

[0027] Step S102: Divide all data blocks into multiple data block groups, count the number and / or space occupancy of data blocks of each reference level in the data block group, and generate level distribution statistics.

[0028] In this embodiment, the terminal device divides all data blocks into multiple data block groups, each containing several data blocks, serving as the basic management unit for garbage collection operations. This division of data block groups can be based on the physical storage location of the data blocks, the logical relationships between the data blocks, or a combination of the above methods.

[0029] After completing the data block group division, the terminal device performs level distribution statistics for each data block group. Specifically, for each data block group, the terminal device obtains the current reference level of each data block within the data block group, calculates the number of data blocks and / or space occupancy value corresponding to each reference level within the data block group, and generates level distribution statistics for the data block group.

[0030] The level distribution statistics include at least the distribution of the number of data blocks at each reference level and / or the distribution of the space occupied by data blocks at each reference level within the data block group. This level distribution statistics reflect the compositional characteristics of the data blocks within the data block group in terms of their degree of sharing: a high proportion of low-reference-level data blocks indicates that the data block group is dominated by low-shared data blocks; a high proportion of high-reference-level data blocks indicates that the data block group is dominated by high-shared data blocks.

[0031] Step S103: Based on the hierarchical distribution statistics, the storage system's space utilization rate, and the storage system's operating load parameters, determine the comprehensive score of the data block group. The comprehensive score is positively correlated with the space utilization rate, negatively correlated with the operating load parameters, positively correlated with the proportion of data blocks with lower reference levels in the hierarchical distribution statistics, and negatively correlated with the proportion of data blocks with higher reference levels.

[0032] In this embodiment, the terminal device obtains the level distribution statistics of each data block group, the space utilization rate of the storage system, and the operating load parameters of the storage system. Using these three types of information as input, the comprehensive score of each data block group is determined.

[0033] In determining the overall score, the three dimensions mentioned above influence the overall score in the following ways: A higher storage system space utilization rate indicates tighter storage capacity and a greater urgency for reclamation operations, resulting in a higher overall score. Higher storage system operating load parameters indicate more current business requests being processed, leading to a greater potential impact of reclamation operations on business performance, resulting in a lower overall score. A higher proportion of low-reference-level data blocks in a data block group indicates that the group is dominated by low-shared data blocks, minimizing the impact of reclamation migration, resulting in a higher overall score. Conversely, a higher proportion of high-reference-level data blocks in a data block group indicates that the group is dominated by high-shared data blocks, and reclamation migration may affect access to more data objects, resulting in a lower overall score.

[0034] Through the combined effect of the above three dimensions, each data block group is assigned a comprehensive score. The score reflects the suitability of prioritizing the reclamation operation for that data block group at the current moment. The higher the comprehensive score, the more worthy the data block group is of priority reclamation; the lower the comprehensive score, the less suitable the data block group is for priority reclamation.

[0035] Step S104: Based on the comparison result between the comprehensive score and the score threshold, determine whether to perform the data block group recycling operation.

[0036] In this embodiment, the terminal device obtains the comprehensive score and score threshold of each data block group, compares the comprehensive score of each data block group with the score threshold, and determines whether to perform a recycling operation on the data block group based on the comparison result.

[0037] Specifically, for any data block group, if the overall score of the data block group reaches or exceeds the scoring threshold, the terminal device determines that the data block group meets the recycling conditions, identifies it as a data block group to be recycled, and triggers the recycling operation for the data block group; if the overall score of the data block group is lower than the scoring threshold, the terminal device determines that the data block group does not meet the recycling conditions and does not perform the recycling operation for the data block group for the time being.

[0038] Based on the above comparison and determination, the terminal device selects data block groups whose comprehensive score reaches or exceeds the scoring threshold from all data block groups as objects to be recycled, so that the recycling operation is concentrated on data block groups with higher comprehensive scores, while data block groups with lower comprehensive scores are excluded from the scope of this recycling.

[0039] The data reclamation method provided in this application marks data blocks with reference levels based on reference counts to identify the degree of data block sharing; it generates a level distribution by statistically analyzing the number of data blocks at each reference level within a group and / or space occupancy, thus revealing the composition of the sharing degree within the group; it determines a comprehensive score based on this level distribution, space utilization, and operating load, with the comprehensive score being positively correlated with space utilization, negatively correlated with load, positively correlated with the proportion of low-level blocks, and negatively correlated with the proportion of high-level blocks, thus giving higher scores to groups with low-level blocks, limited space, and low load; it determines whether to reclaim data blocks by comparing the comprehensive score with a score threshold, prioritizing the reclamation of high-scoring data blocks, avoiding unnecessary migration of highly shared hotspot blocks, reducing the probability of reclamation and host access conflicts, reducing latency increases and the risk of data corruption and loss, and improving the stability and data security of the storage system.

[0040] In one possible implementation, after generating the level distribution statistics, the following embodiment is also provided: In response to a change in the reference count of a data block, the data block whose reference count has changed is identified as a candidate data block; it is determined whether the candidate data block meets the level change condition, which is that the reference level to which the data block belongs before the reference count changes is different from the reference level to which it belongs after the change; if the level change condition is met, the level distribution statistics of the data block group to which the candidate data block belongs are updated.

[0041] In this embodiment, the terminal device monitors the reference count of each data block in the storage system. When the reference count of any data block changes, the terminal device identifies the data block with the changed reference count as a candidate data block. Changes in the reference count can be triggered by various operations, such as write operations or deletion operations on data blocks by data objects, or the establishment or dissolution of reference relationships during data deduplication operations.

[0042] Subsequently, the terminal device obtains the reference level of the candidate data block before the reference count change and the reference level after the reference count change. The reference level is determined based on the matching relationship between the reference count and multiple preset reference level ranges; the terminal device determines whether the reference level before the change is different from the reference level after the change.

[0043] If the reference level before and after the change is the same, the candidate data block does not meet the reference level change conditions, and there is no need to update the reference level distribution statistics of the data block group to which the candidate data block belongs. If the reference level before and after the change is different, the candidate data block meets the reference level change conditions, and the terminal device updates the reference level distribution statistics of the data block group to which the candidate data block belongs.

[0044] When updating the level distribution statistics, the terminal device obtains the data block group to which the candidate data block belongs, and adjusts the statistics corresponding to the reference level before the change and the statistics corresponding to the reference level after the change in that data block group. For example, it decrements the data block count corresponding to the reference level before the change by one, and increments the data block count corresponding to the reference level after the change by one; and / or, it subtracts the space occupied by the data block from the space occupied by the reference level before the change, and adds the space occupied by the data block to the space occupied by the reference level after the change.

[0045] In this way, the level distribution statistics are updated as the reference level of the data block changes, ensuring that they reflect the distribution of data blocks at each reference level within the data block group.

[0046] In one alternative approach, an example of updating the hierarchical distribution statistics of the data block group to which the candidate data block belongs is as follows: Generate a reference count change event and add it to the event queue; asynchronously retrieve the reference count change events from the event queue and update the level distribution statistics of the data block group to which the candidate data block belongs based on the reference count change events.

[0047] In some embodiments, when the terminal device detects a change in the reference level of a data block, it does not immediately perform an update operation on the level distribution statistics, but instead uses an asynchronous update method.

[0048] Specifically, the terminal device generates a reference count change event, which includes at least the following information: the identifier of the data block that changed, the identifier of the data block group to which the data block belongs, the old reference level of the data block before the reference count change, the new reference level after the change, and the storage space occupied by the data block. The terminal device adds the reference count change event to an event queue, which is used to temporarily store reference count change events to be processed.

[0049] The terminal device is configured with one or more event processing threads, which run independently of the business access path. The event processing threads continuously monitor the aforementioned event queue. When there are pending reference count change events in the event queue, the event processing threads retrieve the reference count change events sequentially from the event queue and update the level distribution statistics of the data block group to which the candidate data block belongs based on the information carried in the reference count change events.

[0050] When updating the reference count distribution statistics, the event processing thread, based on the old and new reference levels recorded in the reference count change event, decrements the block count of the data block corresponding to the old reference level in the block group to which the candidate data block belongs by one and reduces the corresponding space usage; conversely, it increments the block count of the data block corresponding to the new reference level and increases the corresponding space usage. After the update is complete, the reference count change event is marked as processed and removed from the queue.

[0051] Through the aforementioned asynchronous update mechanism, the generation of reference count change events (located in the business access path) and the actual update of level distribution statistics (located in an independent event processing thread) are decoupled in terms of time and thread context, thereby preventing update operations from blocking the business access path. The event queue adopts a first-in-first-out or priority-based dequeue strategy to ensure that the level distribution statistics of each data block group are updated sequentially according to the order of occurrence of reference count change events or priority order.

[0052] Furthermore, the terminal device employs a batch aggregation optimization mechanism. Within a preset time window, if multiple reference count changes occur within the same data block group, the terminal device aggregates these reference count changes into a single update operation. For example, if multiple data blocks in the data block group to which the candidate data block belongs undergo reference level changes within the time window, the terminal device summarizes the changes in reference counts and space occupancy of each data block and updates the level distribution statistics of the data block group to which the candidate data block belongs in one go, thereby reducing the number of update operations and overhead.

[0053] In one possible implementation, an example of determining the comprehensive score of a data block group based on hierarchical distribution statistics, storage system space utilization, and storage system operating load parameters is as follows: The first reference score of the data block group is determined based on the level distribution statistics; the second space score of the data block group is determined based on the space utilization rate of the storage system, and the second space score applies to different data blocks in the same storage system; the third load score of the data block group is determined based on the operating load parameters of the storage system; and the comprehensive score of the data block group is determined based on the first reference score, the second space score, and the third load score.

[0054] In some embodiments, the terminal device calculates scores based on three independent scoring dimensions and then performs a fusion calculation to obtain a comprehensive score.

[0055] First, the terminal device determines the first reference score of the data block group based on the hierarchical distribution statistics. Specifically, the terminal device obtains the number of data blocks corresponding to each reference level within the data block group from the hierarchical distribution statistics, and calculates a quantitative score according to the preset scoring rules for each level. This quantitative score reflects the appropriateness of the data block group in terms of priority for recycling in terms of sharing degree.

[0056] Secondly, the terminal device obtains the overall space utilization rate of the storage system and determines a second space score based on this utilization rate. The second space score reflects the urgency of current storage capacity reclamation in the spatial dimension. This second space score applies to all data block groups within the same storage system; that is, different data block groups within the same storage system have the same second space score.

[0057] Next, the terminal device obtains the current operating load parameters of the storage system and determines a third load score based on these parameters. The third load score reflects the storage system's tolerance for reclamation operations under its current load condition.

[0058] Then, the terminal device obtains a comprehensive score for the data block group based on the first reference score, the second spatial score, and the third load score. In one example, the terminal device obtains the preset weight coefficients corresponding to the first reference score, the second spatial score, and the third load score, multiplies each score by its corresponding weight coefficient, and then sums the results to obtain the comprehensive score for the data block group. In another example, the terminal device multiplies the first reference score, the second spatial score, and the third load score by their respective preset weight coefficients and then takes a weighted average to obtain the comprehensive score for the data block group. In yet another example, the terminal device uses the first reference score as the main score and the second spatial score and the third load score as correction factors, adjusting the first reference score according to preset correction rules to obtain the comprehensive score for the data block group. The preset weight coefficients or correction rules are used to adjust the contribution ratio of the three scoring dimensions in the comprehensive score and can be configured and adjusted according to the actual business scenario.

[0059] In this way, the overall score of each data block group is a weighted fusion of scores from three dimensions: citation level distribution, space utilization, and operational load, which comprehensively reflects the combined impact of each dimension on the recycling decision.

[0060] In one alternative approach, an example of determining the first reference score of a data block group based on hierarchical distribution statistics is as follows: The first citation score is determined based on the proportion of data blocks at each citation level in the data block group and the weight coefficient corresponding to each citation level. The weight coefficient is negatively correlated with the citation count.

[0061] In some embodiments, the terminal device obtains the number of data blocks corresponding to each reference level in the data block group from the level distribution statistics of the data block group, and calculates the proportion of the number of data blocks at each reference level to the total number of data blocks in the data block group.

[0062] For example, citation levels are ranked from lowest to highest based on citation count: Level 0, Level 1, Level 2, and Level 3, with the weight coefficients corresponding to each level decreasing sequentially. The proportions of data blocks at each level within a data block group are the proportions of Level 0, Level 1, Level 2, and Level 3, respectively. Multiplying each level's proportion by its corresponding weight coefficient and summing the results yields the first citation score for that data block group.

[0063] The terminal device pre-configures a corresponding weight coefficient for each citation level. The configuration rule is: the higher the citation count of a level, the lower its corresponding weight coefficient; the lower the citation count of a level, the higher its corresponding weight coefficient. That is, there is a negative correlation between citation level and weight coefficient.

[0064] The terminal device multiplies the proportion of data blocks in each citation level by their respective weight coefficients to obtain the weighted contribution value for each citation level. Then, it sums the weighted contribution values ​​of all citation levels to obtain the first citation score for the data block group.

[0065] Using the above calculation method, the higher the proportion of data blocks with lower citation levels, the higher the first citation score; conversely, the higher the proportion of data blocks with higher citation levels, the lower the first citation score. This score reflects the reclamation suitability of data block groups in terms of sharing degree dimension; that is, data block groups with a high proportion of low-shared data blocks receive higher citation scores, while data block groups with a high proportion of high-shared data blocks receive lower citation scores.

[0066] In one alternative approach, an embodiment of determining the second space score of a data block group based on the space utilization of the storage system is as follows: Obtain the space utilization rate of the storage system; based on the monotonically increasing correspondence between the preset space utilization rate and the second space score, map the space utilization rate to the second space score.

[0067] In some embodiments, the terminal device obtains the current space utilization rate of the storage system by periodic collection or real-time monitoring. The space utilization rate reflects the proportion of the storage system's occupied capacity to the total capacity.

[0068] The terminal device determines a second space score based on the obtained space utilization rate. The determination rule is: the higher the space utilization rate, the higher the second space score; the lower the space utilization rate, the lower the second space score. The second space score is used to quantify the current capacity strain of the storage system and serves as a dimension factor reflecting the urgency of space reclamation in the comprehensive score.

[0069] All data block groups within the same storage system share the same space utilization rate and therefore have the same second space score. When the space utilization rate of the storage system changes, the terminal device recalculates the second space score, and this change affects the overall score of all data block groups in that storage system.

[0070] In one example, the terminal device divides the space utilization rate into multiple preset intervals, each interval corresponding to a second space score value. The terminal device determines the corresponding second space score based on the interval in which the current space utilization rate falls. In another example, the terminal device uses a continuous function to map the space utilization rate to the second space score; the higher the space utilization rate, the higher the mapped score, and the two have a monotonically increasing relationship.

[0071] For example, when the space utilization rate is below 80%, the second space score is zero; when the space utilization rate is between 80% and 90%, the second space score increases linearly from zero to 80%; and when the space utilization rate exceeds 90%, the second space score reaches 100.

[0072] In one alternative approach, an example of determining the third load score for a data block group based on the storage system's operating load parameters is as follows: Obtain the operating load parameters of the storage system, which include at least one or more of the following: operation rate, access response latency, and request queue length; determine the operating load score based on one or more of the following: operation rate, access response latency, and request queue length; and map the operating load score to the third load score based on a preset monotonically decreasing correspondence between the operating load score and the third load score.

[0073] In some embodiments, the terminal device acquires the current operating load parameters of the storage system through periodic data collection or real-time monitoring. The operating load parameters include at least one or more of the following: operation rate, access response latency, and request queue length. Specifically, the operation rate reflects the number of input / output operations processed by the storage system per unit time, the access response latency reflects the average time required for the storage system to process a single operation, and the request queue length reflects the number of operations currently waiting to be processed by the storage system. These operating load parameters reflect the current level of business pressure on the storage system from different perspectives.

[0074] The terminal device calculates a runtime load score based on the acquired runtime load parameters. This runtime load score quantifies the current overall load status of the storage system. The calculation logic for the runtime load score is as follows: the higher the runtime load parameter, the greater the current load of the storage system, and the higher the runtime load score; the lower the runtime load parameter, the smaller the current load of the storage system, and the lower the runtime load score.

[0075] Terminal devices determine a third load score based on the operating load score, and the determination rule is negatively correlated with the operating load score: the higher the operating load score, the lower the third load score; conversely, the lower the operating load score, the higher the third load score. The third load score quantifies the tolerance level for recycling operations under the current storage system load state. Specifically, the lighter the storage system load, the less impact recycling operations have on business operations, resulting in a higher third load score and a correspondingly higher overall score, making it easier to trigger recycling. Conversely, the heavier the storage system load, the greater the impact of recycling operations on business operations, resulting in a lower third load score and a correspondingly lower overall score, suppressing recycling operations to ensure business performance.

[0076] In one example, the load score is a weighted sum of normalized operation rate, access response latency, and request queue length, while the third load score is a preset maximum value minus the load score. That is, the third load score equals one hundred minus the load score; when the load score is zero, the third load score is one hundred, and when the load score is one hundred, the third load score is zero. In another example, the load score can be calculated based on only one of the various load parameters mentioned above, such as determining the load status solely based on operation rate or solely based on request queue length.

[0077] In one possible implementation, before determining whether to perform a data block group reclamation operation based on a comparison between the comprehensive score and the score threshold, the following embodiment is also provided: Obtain the initial scoring threshold and the trend of the storage system's space utilization within a preset time window; adjust the initial scoring threshold according to the trend to obtain the scoring threshold, which shall not exceed the highest comprehensive score of each data block group in the storage system.

[0078] In some embodiments, before performing a reclamation decision, the terminal device first obtains an initial scoring threshold. The terminal device continuously monitors changes in the storage system's space utilization and adjusts the initial scoring threshold based on the trend of space utilization over time to obtain an adjusted scoring threshold. The adjustment rule is as follows: when the space utilization shows a continuous upward trend, the terminal device adjusts the initial scoring threshold downward to make more data block groups meet the reclamation conditions, expanding the reclamation scope to accelerate space release; when the space utilization shows a continuous downward trend or a rapid downward rate, the terminal device adjusts the initial scoring threshold upward to make fewer data block groups meet the reclamation conditions, narrowing the reclamation scope to control the impact of the reclamation operation on services.

[0079] The above adjustment operations can be performed periodically during the operation of the terminal device, or triggered when the space utilization rate changes to a preset level.

[0080] The scoring threshold is limited to a range from zero to the highest overall score among all current data block groups. When the adjusted scoring threshold is below zero, the terminal device sets the scoring threshold to zero; when the adjusted scoring threshold is above the highest overall score among all current data block groups, the terminal device sets the scoring threshold to that highest overall score. This range limitation ensures that the scoring threshold always remains within a valid range, preventing situations where all data block groups meet the recycling criteria or all data block groups fail to meet the recycling criteria due to improper scoring threshold settings.

[0081] In one alternative approach, the initial scoring threshold is adjusted based on the changing trend, resulting in the following example of obtaining the scoring threshold: When the rate of increase or decrease in space utilization is less than the preset rate, the initial scoring threshold is lowered to expand the scope of data block reclamation; when the rate of decrease in space utilization is greater than or equal to the preset rate, the initial scoring threshold is raised to narrow the scope of data block reclamation.

[0082] In some embodiments, the terminal device monitors the space utilization rate of the storage system according to a preset period and calculates the trend of space utilization rate changes. Specifically, the terminal device records the change in space utilization rate between the current period and the previous period, divides the change by the preset period duration, and obtains the rate of increase or decrease of space utilization rate.

[0083] When the space utilization rate continues to rise, or when the space utilization rate decreases but the rate of decrease is less than the preset rate, it indicates that the space release speed is insufficient and the space pressure is high. The terminal device will lower the initial scoring threshold (for example, by 10% to 20% of the current value). The higher the comprehensive score, the higher the recycling value of the corresponding data block group. After the threshold is lowered, the number of data block groups with a comprehensive score greater than or equal to the scoring threshold increases, thereby expanding the recycling range of data block groups and accelerating the space release speed.

[0084] When the rate of decrease in space utilization exceeds the preset rate, it indicates that space pressure is rapidly easing. The terminal device will raise the scoring threshold (for example, by 5% to 10% of the current value). After the threshold is raised, the number of data block groups that meet the recycling conditions will decrease, thereby narrowing the scope of data block group recycling and reducing the impact of recycling operations on business performance.

[0085] The adjustment range can be set as a preset percentage of the current scoring threshold. The terminal device continuously repeats the above monitoring and adjustment process, so that the scoring threshold fluctuates adaptively with the dynamic changes in space utilization, maintaining a dynamic balance between recycling intensity and space consumption rate.

[0086] This application's embodiments achieve the following beneficial effects through the synergistic effect of reference counting hierarchy, data block group level distribution statistics, reference distribution, space utilization, and runtime load three-dimensional fusion scoring, and dynamic adjustment of scoring thresholds: Prioritizing the reclamation of data block groups containing low-shared data blocks reduces the migration frequency of high-shared hotspot data blocks, thereby mitigating write amplification effects and slowing down storage media wear, extending the lifespan of storage devices. Simultaneously, it reduces the probability of conflicts between reclamation migration operations and host service access, minimizing the increase in access latency. By reducing the migration frequency of high-shared data blocks, it avoids the risk of data corruption or loss during the transfer process, thus enhancing data reliability. A three-dimensional fusion and dynamic adjustment mechanism for scoring thresholds enables the reclamation strategy to achieve a dynamic balance between space release efficiency and business performance assurance, reducing manual intervention. Furthermore, by leveraging an asynchronous event queue mechanism, metadata update operations are removed from the main access path, further ensuring host access performance.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0088] Embodiments of this application also provide a data recycling device. Figure 2 This is a schematic diagram of a data recovery device provided in an embodiment of this application. The device includes: The setting module 201 is used to obtain the reference count of each data block in the storage system that is referenced by data objects, and to set a reference level identifier for each data block according to the reference count. The reference count is used to characterize the degree to which the data block is shared by data objects, and the reference level identifier is used to indicate the reference level to which the data block belongs. The generation module 202 is used to divide all data blocks into multiple data block groups, count the number and / or space occupancy of data blocks of each reference level in the data block group, and generate level distribution statistics. The scoring module 203 is used to determine the comprehensive score of the data block group based on the level distribution statistics, the space utilization rate of the storage system, and the operating load parameters of the storage system. The comprehensive score is positively correlated with the space utilization rate, negatively correlated with the operating load parameters, positively correlated with the proportion of data blocks with lower reference levels in the level distribution statistics, and negatively correlated with the proportion of data blocks with higher reference levels. The determination module 204 is used to determine whether to perform a data block group recycling operation based on the comparison result of the comprehensive score and the score threshold.

[0089] For a description of the features in the embodiment corresponding to the data recovery device, please refer to the relevant description of the embodiment corresponding to the data recovery method, which will not be repeated here.

[0090] Embodiments of this application also provide an electronic device. Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device includes a memory 301 and a processor 302. The memory 301 stores a computer program, and the processor 302 is configured to run the computer program to perform the steps in any of the above-described data recovery method embodiments.

[0091] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described data recycling method embodiments when running.

[0092] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0093] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data recycling method embodiments.

[0094] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data reclamation method embodiments.

[0095] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0096] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0097] The data retrieval method and electronic device provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data retrieval method, characterized in that, include: Obtain the reference count of each data block in the storage system that is referenced by data objects, and set a reference level identifier for each data block according to the reference count. The reference count is used to characterize the degree to which the data block is shared by data objects, and the reference level identifier is used to indicate the reference level to which the data block belongs. Divide all data blocks into multiple data block groups, and count the proportion of data blocks of each reference level and / or space occupancy value in the data block group to generate level distribution statistics. Based on the level distribution statistics, the space utilization rate of the storage system, and the operating load parameters of the storage system, a comprehensive score for the data block group is determined. The comprehensive score is positively correlated with the space utilization rate, negatively correlated with the operating load parameters, positively correlated with the proportion of data blocks with lower reference levels in the level distribution statistics, and negatively correlated with the proportion of data blocks with higher reference levels. Based on the comparison between the comprehensive score and the score threshold, it is determined whether to perform a recycling operation on the data block group.

2. The data recovery method according to claim 1, characterized in that, After generating the level distribution statistics, the method further includes: In response to a change in the reference count of the data block, the data block whose reference count has changed is identified as a candidate data block; Determine whether the candidate data block meets the level change condition, wherein the level change condition is that the reference level to which the data block belongs before the reference count changes is different from the reference level to which it belongs after the change; If the level change conditions are met, then update the level distribution statistics of the data block group to which the candidate data block belongs.

3. The data recovery method according to claim 2, characterized in that, The step of updating the level distribution statistics of the data block group to which the candidate data block belongs includes: Generate a reference count change event and add the reference count change event to the event queue; Asynchronously obtain the reference count change event in the event queue, and update the level distribution statistics of the data block group to which the candidate data block belongs based on the reference count change event.

4. The data recovery method according to claim 1, characterized in that, The comprehensive score for the data block group is determined based on the hierarchical distribution statistics, the storage system's space utilization rate, and the storage system's operating load parameters, including: The first reference score of the data block group is determined based on the level distribution statistics. A second space score is determined for the data block group based on the space utilization of the storage system, and the second space score applies to different data blocks in the same storage system. The third load score of the data block group is determined based on the operating load parameters of the storage system. The comprehensive score of the data block group is determined based on the first reference score, the second space score, and the third load score.

5. The data recovery method according to claim 4, characterized in that, Determining the first reference score of the data block group based on the level distribution statistics includes: The first citation score is determined based on the proportion of data blocks of each citation level in the data block group and the weight coefficient corresponding to each citation level. The weight coefficient is negatively correlated with the citation count.

6. The data recovery method according to claim 4, characterized in that, The step of determining the second space score of the data block group based on the space utilization rate of the storage system includes: Obtain the space utilization rate of the storage system; Based on the preset monotonically increasing correspondence between the space utilization rate and the second space score, the space utilization rate is mapped to the second space score.

7. The data recovery method according to claim 4, characterized in that, The third load score of the data block group is determined based on the operating load parameters of the storage system, including: Obtain the operating load parameters of the storage system, wherein the operating load parameters include at least one or more of the following: operation rate, access response latency, and request queue length; The runtime load score is determined based on one or more of the following: operation rate, access response latency, and request queue length. Based on the preset monotonically decreasing correspondence between the operating load score and the third load score, the operating load score is mapped to the third load score.

8. The data recovery method according to claim 1, characterized in that, Before determining whether to perform a reclamation operation on the data block group based on the comparison result of the comprehensive score and the score threshold, the method further includes: Obtain the initial scoring threshold and the trend of the space utilization rate of the storage system within a preset time window; The initial scoring threshold is adjusted according to the changing trend to obtain the scoring threshold, which does not exceed the highest comprehensive score of each data block group in the storage system.

9. The data recovery method according to claim 8, characterized in that, The step of adjusting the initial scoring threshold according to the changing trend to obtain the scoring threshold includes: When the rate of increase or decrease in space utilization is less than a preset rate, the initial scoring threshold is lowered to expand the scope of data block recycling; When the rate of decrease in space utilization is greater than or equal to the preset rate, the initial scoring threshold is increased to narrow the scope of data block recycling.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the data recovery method as described in any one of claims 1 to 9.

Images