Resource processing method and device

Abstract

The present disclosure provides a resource processing method and device, which has the technical feature of releasing the space of the source partition by determining the partition with a medium proportion of valid data as the source partition when the write speed is less than the write threshold, so that the valid data proportion of most partitions in the recording medium disk is low (such as less than 20%) or high (such as greater than 90%), thereby realizing resource release processing of the source partition at a relatively low cost, that is, realizing the technical effect of saving resource processing resources, improving resource processing efficiency, and improving the flexibility and reliability of resource processing.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a resource processing method and device. Background Art

[0002] The computer includes a disk, and a user can write data to the disk through a client, or the user can also read data from the disk.

[0003] Disks can include conventional magnetic recording disks and recording medium disks. In conventional magnetic recording disks, resource processing (such as garbage collection) is implemented by inherent components within the conventional magnetic recording disks. Therefore, how to implement resource processing for recording medium disks has become a pressing issue. Summary of the Invention

[0004] The present disclosure provides a resource processing method and apparatus to reduce the problem of resource loss in resource processing.

[0005] In a first aspect, an embodiment of the present disclosure provides a resource processing method, including:

[0006] Acquiring a writing speed of data to a recording medium disk, wherein the recording medium disk is a disk supporting a partition namespace interface;

[0007] If the write speed is less than a preset write threshold, a partition in the recording medium disk whose valid data ratio falls within a preset medium ratio interval is determined as a source partition, wherein the ratio in the medium ratio interval is greater than a preset minimum ratio threshold and less than a preset maximum ratio threshold, and the partition is the minimum management unit of the recording medium disk;

[0008] A resource release process is performed on the source partition.

[0009] In one embodiment of the present disclosure, after determining the partition in the recording medium disk whose valid data ratio falls within a preset medium ratio range as the source partition, the method further includes:

[0010] Obtaining the storage duration of the data in the source partition;

[0011] Furthermore, the performing resource release processing on the source partition includes: if the storage duration is greater than a preset first threshold, performing resource release processing on the source partition.

[0012] In one embodiment of the present disclosure, before performing resource release processing on the source partition, the method further includes:

[0013] If the storage duration is greater than the first threshold, obtaining a recycle bin data ratio of valid recycle bin data in the source partition, wherein the valid data includes valid recycle bin data;

[0014] Furthermore, the resource release processing for the source partition includes: if the proportion of the recycle bin data is less than a preset second threshold, releasing the storage resources of the source partition.

[0015] In one embodiment of the present disclosure, determining a partition in the recording medium disk, whose valid data ratio falls within a preset medium ratio range, as a source partition includes:

[0016] A partition in a closed state is acquired from the recording medium disc, and a source partition is determined from the partition in the closed state.

[0017] In one embodiment of the present disclosure, before acquiring the writing speed of writing data to the recording medium disk, the method further includes:

[0018] Calculating a valid data ratio distribution of partitions of the recording medium disk, wherein the valid data ratio distribution of partitions is a number distribution of partitions with different valid data ratios;

[0019] And, the acquiring the writing speed of the data to the recording medium disk includes: if the effective data ratio distribution of the partition is normal, acquiring the writing speed of the data to the recording medium disk;

[0020] Among them, the normal distribution of the effective data proportion of the partition is characterized by: the sum of the number of partitions whose resource utilization is greater than the maximum utilization threshold and the number of partitions whose resource utilization is less than the minimum utilization threshold is greater than a preset third threshold, and the third threshold is greater than half of the total number of partitions.

[0021] In one embodiment of the present disclosure, the performing resource release processing on the source partition includes:

[0022] Acquire a free partition from the recording medium disk;

[0023] Migrate valid data in the source partition to the free partition.

[0024] In one embodiment of the present disclosure, the method further includes:

[0025] A partition whose effective data ratio is less than the minimum ratio threshold is obtained from the recording medium disk, and the obtained partition is erased.

[0026] In one embodiment of the present disclosure, one partition corresponds to one data management unit, and the method further includes:

[0027] If the data in any data management unit is invalid data, the partition corresponding to the data management unit is erased.

[0028] In a second aspect, an embodiment of the present disclosure provides a resource processing device, including:

[0029] a first acquiring unit, configured to acquire a writing speed of data to a recording medium disk, wherein the recording medium disk is a disk supporting a partition namespace interface;

[0030] a determining unit configured to, if the write speed is less than a preset write threshold, determine a partition in the recording medium disk whose valid data ratio falls within a preset medium ratio interval as a source partition, wherein if the ratio in the medium ratio interval is greater than a preset minimum ratio threshold and less than a preset maximum ratio threshold, the partition is a minimum management unit of the recording medium disk;

[0031] A processing unit is configured to perform resource release processing on the source partition.

[0032] In one embodiment of the present disclosure, the apparatus further comprises:

[0033] A second acquiring unit, configured to acquire a storage duration of the data in the source partition;

[0034] Furthermore, the processing unit is configured to release resources of the source partition if the storage duration is greater than a preset first threshold.

[0035] In one embodiment of the present disclosure, the apparatus further comprises:

[0036] a third acquiring unit, configured to acquire a recycle bin data ratio of valid recycle bin data in the source partition if the storage duration is greater than the first threshold, wherein the valid data includes valid recycle bin data;

[0037] Furthermore, the processing unit is configured to release storage resources of the source partition if the proportion of the recycle bin data is less than a preset second threshold.

[0038] In one embodiment of the present disclosure, the determining unit includes:

[0039] A first acquiring subunit, configured to acquire a partition in a closed state from the recording medium disk;

[0040] The determination subunit is used to determine the source partition from the partitions in the closed state.

[0041] In one embodiment of the present disclosure, the apparatus further comprises:

[0042] a calculation unit, configured to calculate a distribution of valid data proportions of partitions of the recording medium disk, wherein the distribution of valid data proportions of partitions is a distribution of the number of partitions with different valid data proportions;

[0043] Furthermore, the first acquiring unit is configured to acquire a writing speed of data to the recording medium disk if the effective data ratio distribution of the partition is normal;

[0044] Among them, the normal distribution of the effective data proportion of the partition is characterized by: the sum of the number of partitions whose resource utilization is greater than the maximum utilization threshold and the number of partitions whose resource utilization is less than the minimum utilization threshold is greater than a preset third threshold, and the third threshold is greater than half of the total number of partitions.

[0045] In one embodiment of the present disclosure, the processing unit includes:

[0046] A second acquiring subunit, configured to acquire a free partition from the recording medium disk;

[0047] The migration subunit is used to migrate the valid data in the source partition to the free partition.

[0048] In one embodiment of the present disclosure, the apparatus further comprises:

[0049] A fourth acquiring unit, configured to acquire, from the recording medium disk, a partition whose valid data ratio is less than the minimum ratio threshold;

[0050] The first erasing unit is used to erase the acquired partition.

[0051] In one embodiment of the present disclosure, one partition corresponds to one data management unit, and the apparatus further includes:

[0052] The second erasing unit erases the partition corresponding to any data management unit if the data in the data management unit is invalid data.

[0053] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0054] at least one processor; and

[0055] a memory communicatively connected to at least one processor; wherein,

[0056] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to perform any one of the methods in the first aspect of the present disclosure.

[0057] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements any one of the methods described in the first aspect of the present disclosure when the computer program is executed by a processor.

[0058] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, comprising a computer program, which, when executed by a processor, implements any one of the methods described in the first aspect of the present disclosure.

[0059] The embodiments of the present disclosure provide a resource processing method and device, which has the technical feature of releasing the space of the source partition by determining the partition with a medium proportion of valid data as the source partition when the write speed is less than the write threshold, so that the valid data proportion of most partitions in the recording medium disk is low (such as less than 20%) or high (such as greater than 90%), thereby realizing resource release processing of the source partition at a relatively low cost, that is, realizing the technical effect of saving resource processing resources, improving resource processing efficiency, and improving the flexibility and reliability of resource processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0061] Figure 1 Schematic diagram of zone usage of a recording medium disk according to an embodiment of the present disclosure;

[0062] Figure 2 is a schematic diagram of the occupied capacity of a recording medium disk according to an embodiment of the present disclosure;

[0063] Figure 3 is a schematic diagram of the throughput of a recording medium disk according to an embodiment of the present disclosure;

[0064] Figure 4 This is a schematic diagram of the number of zones where the effective data ratio is greater than 90% according to an embodiment of the present disclosure;

[0065] Figure 5 This is a flowchart of a resource processing method according to an embodiment of the present disclosure;

[0066] Figure 6 This is a schematic diagram of the structure of a Zone according to an embodiment of the present disclosure;

[0067] Figure 7 This is a flowchart of a resource processing method according to another embodiment of the present disclosure;

[0068] Figure 8 This is a schematic structural diagram of a Zone according to another embodiment of the present disclosure;

[0069] Figure 9 A schematic diagram of a processing device according to an embodiment of the present disclosure;

[0070] Figure 10 is a schematic diagram of a resource processing device according to another embodiment of the present disclosure;

[0071] Figure 11 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure.

[0072] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0073] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0074] The terms "first," "second," "third," and the like in the specification and claims of the present disclosure and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present disclosure described herein can be practiced in orders other than those illustrated or described herein.

[0075] In addition, the terms "comprises" and "having" and any variations thereof are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product or apparatus.

[0076] To facilitate reading and understanding of the technical solutions of the present disclosure, at least some of the terms involved in the present disclosure are explained as follows:

[0077] Recording media disks, in contrast to traditional magnetic recording disks, refer to high-capacity disks that utilize novel magnetic storage technologies, such as those supporting the Zoned Namespace (ZNS) interface. Recording media disks include ZNS-supported Shingled Magnetic Recording (SMR) and ZNS-supported solid-state drives (ZNS SSDs), and are not listed here.

[0078] SMR disks partially overlap the data tracks on the disk, like tiles on a roof. This technology is called shingled magnetic recording technology.

[0079] A zone is the smallest management unit on a recording medium disk. A zone can have a capacity of 256 megabytes (MB). A 16-terabyte (TB) recording medium disk can have approximately 70,000 zones.

[0080] Zones support read and write operations including append writes and random reads. Zones do not support random writes and overwrites. For example, after a zone is opened, append writes and random reads are possible. After a zone is closed, neither reading nor writing is possible. If a zone is full or unexpectedly exits and restarts, the zone is closed. Zones support global erase, but not partial erase.

[0081] That is, the Zone has a status attribute, and the status attribute includes an open state and a closed state. If the status attribute of the Zone is open, additional write and random read operations can be performed on the Zone. If the status attribute of the Zone is closed, additional write and random read operations cannot be performed on the Zone.

[0082] The data types include valid data and invalid data, and the valid data includes recycle bin valid data and non-recycle bin valid data.

[0083] After the user deletes the non-recycle bin valid data, the non-recycle bin valid data is converted to the recycle bin valid data. After the recycle bin valid data is recycled, that is, after the recycle bin valid data is cleared, the recycle bin valid data is converted to invalid data.

[0084] A standalone storage engine is a functional node in a distributed storage system that manages the storage of user data on storage media. Typically, a storage server has one standalone storage engine process, which manages the server's multiple recording media disks.

[0085] Garbage collection refers to the operation of reading valid data from the first zone, writing it to the second zone, and freeing up the space in the first zone.

[0086] Zone utilization rate refers to the percentage of zones in the closed state.

[0087] Recording media disks provide users with a ZNS interface. While using this interface to read and write data, users also need to manage the disk's zone resources and perform garbage collection. The greater the amount of garbage collected data, the greater the write amplification on the disk. For recording media disks using mechanical hard disk drives (HDDs), resource management is a pressing issue.

[0088] In some embodiments, the resources of the recording medium disk can be recycled by random deletion, such as by setting a valid data watermark to process the resources of the recording medium disk, such as recycling the resources of the recording medium disk.

[0089] Exemplarily, the valid data water level is set to 90%. For example, when the valid data ratio in the recording medium disk reaches 90%, garbage collection is performed on the Zone in the recording medium disk.

[0090] However, when the valid data level is 90%, Figure 1 As shown in Figure 2, the Zone usage rate of the recording medium disk reaches 98%. Figure 2 As shown, the occupied capacity of the recording medium disk reaches 250T. Figure 3 As shown in FIG. 1 , the throughput of the recording medium disk (ie, the amount of data successfully transferred per unit event) has fallen below 1 Gigabyte (GB) per second (s).

[0091] And combined Figure 4 It can be seen that when the valid data water level is 90%, the number of zones with valid data greater than 90% is small, and a large number of zones have a certain proportion of invalid data. As a result, when the water level is high (that is, the valid data water level is 90%), the cost of garbage collection is very high, and more recording medium disk resources are occupied. It is necessary to limit the front-end throughput, that is, it is necessary to limit the speed of reading data from the recording medium disk or writing data to the recording medium disk.

[0092] In other embodiments, the resource recycling of the recording medium disk may be triggered by a "greedy strategy approach," such as when the number of free zones drops to a certain ratio or number, the resource recycling of the recording medium disk is triggered.

[0093] For example, an idle zone refers to a zone without data. By obtaining the number of idle zones on the recording medium disk, garbage collection is performed on non-idle zones on the recording medium disk when the number of idle zones exceeds a preset threshold. The threshold can be determined based on demand, historical records, and experiments, and is not limited in this embodiment.

[0094] Alternatively, after obtaining the number of free zones, the ratio of the number of free zones to the total number of zones on the recording medium disk is calculated. If the ratio is less than a preset ratio threshold, garbage collection is performed on the non-free zones on the recording medium disk. Similarly, the ratio threshold can be determined based on demand, historical records, and experiments, and is not limited in this embodiment.

[0095] In some embodiments, the efficiency of garbage collection can be ensured by using “front-end pressure limiting”, which means reducing the speed of data read and write operations on the recording medium disk.

[0096] Among them, you can select the zone with the lowest proportion of valid data for garbage collection, that is, migrate the valid data in the zone with the lowest proportion of valid data to other zones to release the storage space of the zone with the lowest proportion of valid data, making the cost of garbage collection relatively low.

[0097] However, if the garbage collection operation is triggered too late, such as when the number of free zones is too small, the storage space of the zones may not be released in time.

[0098] For example, the peak random write throughput of the recording medium disk is around 100 MB / s. If garbage collection is performed at a speed of 50 MB / s, it takes about 6 seconds to reclaim a zone with 60% valid data (3 seconds for reading and writing each). If data is written to the recording medium disk at a speed of 50 MB / s, a zone can be consumed in 5 seconds, which is already faster than the recovery speed.

[0099] Especially when random deletions occur at the same time (such as garbage collection when the valid data water level is 90% in the above example), causing more zones to need to be garbage collected, the zone storage space will be consumed faster.

[0100] In order to avoid the above technical problems, the inventors of the present disclosure have obtained the inventive concept of the present disclosure through creative work: if the data writing speed of the recording medium disk is less than the writing threshold, the area where the proportion of valid data is in the medium proportion interval (that is, the proportion of this interval is greater than the minimum proportion threshold and less than the maximum proportion threshold) is selected as the source partition to release resources of the source partition.

[0101] The technical solution of the present disclosure is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0102] Figure 5 This is a flow chart of a resource processing method according to an embodiment of the present disclosure. Figure 5 As shown, the method includes:

[0103] S501: Acquire a writing speed of data to a recording medium disk, wherein the recording medium disk is a disk supporting a partition namespace interface.

[0104] Illustratively, the execution subject of this embodiment may be a resource processing device, which may be a computer, a terminal device, a server, a processor, a chip, etc., which is not limited in this embodiment.

[0105] This embodiment does not limit the type of recording medium disk. For example, the recording medium disk may be an SMR disk as described in the above embodiment, or a ZNS SDD as described in the above embodiment. Furthermore, this embodiment does not limit the method for obtaining the write speed. For example, the write speed may be obtained through monitoring.

[0106] S502: If the writing speed is less than a preset writing threshold, a zone in which the proportion of valid data in the recording medium disk falls within a preset medium proportion range is determined as a source zone.

[0107] Among them, the proportion in the medium proportion interval is greater than the preset minimum proportion threshold and less than the preset maximum proportion threshold, and Zone is the smallest management unit of the recording medium disk.

[0108] For example, in combination with the above analysis, the recording medium disk includes multiple zones, and the data types stored in different zones are different. Moreover, when the data types of the stored data are the same, the proportions of the same type of data may also be different.

[0109] For example, Figure 6 As shown, Zone1, Zone2 and Zone3 are three zones in the recording medium disk, and the three zones include valid data and invalid data respectively. The valid data accounts for 80% in Zone1 and the invalid data accounts for 20%. The valid data and invalid data in Zone2 are the same, both 50%. The valid data accounts for 10% in Zone3 and the invalid data accounts for 90%.

[0110] The percentage of valid data is 1 minus invalid data / single zone capacity. The percentage of invalid data is 1 minus valid data / single zone capacity = 1 minus (valid data outside the recycle bin + valid data in the recycle bin) / single zone capacity.

[0111] Similarly, this embodiment does not limit the write threshold, minimum share threshold, and maximum share threshold, which can be determined based on demand, historical records, and experiments.

[0112] In this embodiment, the minimum proportion threshold is less than the valid data proportion of the source zone and less than the maximum proportion threshold, for example, 30% is less than the valid data proportion of the source zone and less than 80%.

[0113] S503: Release resources from the source zone.

[0114] Based on the above analysis, it can be seen that the present disclosure provides a resource processing method, which includes: obtaining the writing speed of data written to the recording medium disk, if the writing speed is less than the preset writing threshold, then determining the zone in the recording medium disk whose valid data ratio belongs to the preset medium ratio interval as the source zone, wherein the ratio in the medium ratio interval is greater than the preset minimum ratio threshold and less than the preset maximum ratio threshold, the zone is the smallest management unit of the recording medium disk, and resource release processing is performed on the source zone. In this embodiment, when the writing speed is less than the writing threshold, the zone in the medium ratio area of ​​valid data is determined as the source zone, so as to release the space of the source zone with the technical feature, so that the valid data ratio of most zones in the recording medium disk is low (such as less than 20%) or high (such as greater than 90%), thereby realizing resource release processing of the source zone at a lower cost, that is, realizing the technical effect of saving resource processing resources, improving resource processing efficiency, and improving the flexibility and reliability of resource processing.

[0115] Figure 7 This is a flow chart of a resource processing method according to another embodiment of the present disclosure. Figure 7 As shown, the method includes:

[0116] S701: Calculate the effective data percentage distribution of zones on the recording medium disk, wherein the effective data percentage distribution of zones is the number distribution of zones with different effective data percentages.

[0117] It should be understood that in order to avoid cumbersome descriptions, the technical features of this embodiment that are the same as those in the above embodiments will not be described in detail in this embodiment.

[0118] The calculation of the effective data ratio distribution of each zone can be achieved in the following way: for each zone in the recording medium disk, the effective data ratio of the zone is calculated, and the effective data ratio distribution of the zone is obtained based on the effective data ratio corresponding to each zone.

[0119] For example, a recording medium disk has 10 zones, of which 2 zones have a valid data percentage of 5%, 5 zones have a valid data percentage of 50%, and 3 zones have a valid data percentage of 90%. The valid data percentage distribution is:

[0120] The number of zones with valid data less than or equal to 5% accounts for 20%; the number of zones with valid data less than 5% and less than or equal to 50% accounts for 50%; the number of zones with valid data greater than 50% and less than or equal to 90% accounts for 30%.

[0121] S702: Determine whether the distribution of the Zone's valid data ratio is normal. If so, execute S703; if not, the resource processing ends.

[0122] Among them, the normal distribution of the proportion of valid data in the zones is characterized by: the sum of the number of zones whose resource utilization is greater than the maximum utilization threshold and the number of zones whose resource utilization is less than the minimum utilization threshold is greater than the preset third threshold, and the third threshold is greater than half of the total number of zones.

[0123] For example, if there are 1000 zones on a recording medium disk, the minimum utilization threshold is 10%, the maximum utilization threshold is 90%, and the sum of the number of zones with resource utilization less than 10% and the number of zones with resource utilization greater than 90% is 990, indicating that the distribution of valid data in zones is normal.

[0124] Similarly, the maximum utilization threshold, the minimum utilization threshold, and the third threshold can be determined based on demand, historical records, experiments, etc., and this embodiment does not limit this.

[0125] If the zone effective data ratio distribution is normal, then relatively speaking, there are fewer zones with average resource utilization, and correspondingly, there are more zones with high resource utilization, or more zones with low resource utilization. Conversely, if the zone effective data ratio distribution is abnormal, relatively speaking, there are more zones with average resource utilization, and correspondingly, there are fewer zones with high resource utilization, and also fewer zones with low resource utilization.

[0126] S703: Acquire the writing speed of data to the recording medium disk.

[0127] In this embodiment, by performing subsequent resource processing when the distribution of the effective data ratio of the zones is normal, that is, when the number of zones with average resource utilization is small, the cost of resource processing can be reduced, the resources for resource processing can be saved, and the efficiency of resource processing can be improved.

[0128] S704: Determine whether the write speed is less than the write threshold. If so, execute S705; if not, the resource processing ends.

[0129] S705: Traverse all closed zones.

[0130] Exemplarily, the status attribute corresponding to each zone in the recording medium may be obtained, and the zones whose status attribute is closed may be traversed.

[0131] S706: Calculate the effective data ratio of the current zone.

[0132] S707: Determine whether the proportion of valid data belongs to the medium proportion range. If so, execute S708; if not, return to S705.

[0133] Among them, the proportion in the medium proportion interval is greater than the preset minimum proportion threshold and less than the preset maximum proportion threshold.

[0134] Exemplarily, S705-S707 can be understood as: there are multiple zones in closed state, a closed zone is selected from the closed zones, and the zone is called the current zone, and the effective data ratio of the current zone is calculated. If the effective data ratio belongs to the medium ratio interval, then the subsequent resource processing is continued. If the effective data ratio does not belong to the medium ratio interval, such as the effective data ratio is less than or equal to the minimum ratio threshold, or the effective data ratio is greater than or equal to the maximum ratio threshold, then a closed zone is reselected from each closed zone, and the reselected closed zone is called the current zone, and so on. They are not listed one by one here.

[0135] S708: Obtain the storage duration of the data in the current zone.

[0136] S709: Determine whether the storage duration is greater than a first threshold. If yes, execute S710; if not, return to S705.

[0137] Similarly, the first threshold value may be determined based on demand, historical records, experiments, etc., and this embodiment does not limit this.

[0138] In some embodiments, it can be determined whether the data in the current zone is cold data. If the data in the current zone is cold data, the storage duration is determined to be greater than a first threshold. Conversely, if the data in the current zone is hot data, the storage duration is determined to be less than the first threshold.

[0139] Here, cold data and hot data are relative concepts. Cold data can be understood as data that will not be deleted within a first threshold time, and hot data can be understood as data that will be deleted within a first threshold time.

[0140] In other embodiments, it can be determined whether the data in the current zone is long-life cycle data (such as a life cycle greater than a first threshold). If so, the storage duration is greater than the first threshold; if not, the storage duration is less than or equal to the first threshold.

[0141] S710: Obtain the percentage of valid data in the recycle bin in the current zone, wherein the valid data includes valid data in the recycle bin and non-valid data in the recycle bin.

[0142] Exemplarily, the amount of valid data in the recycle bin in the current zone is obtained, and the ratio of the obtained data amount to the total capacity of the current zone is calculated to obtain the recycle bin data ratio.

[0143] S711: Determine whether the proportion of recycle bin data is less than a second threshold. If so, execute S712; if not, return to S705.

[0144] Similarly, the second threshold value can be determined based on demand, historical records, experiments, etc., and this embodiment does not limit this.

[0145] S712: Determine the current zone as the source zone.

[0146] S713: Migrate valid data in the source zone to the idle zone, and erase invalid data in the source zone.

[0147] In this embodiment, on the one hand, by determining the zone with a storage time greater than the first threshold as the source zone, that is, determining the zone in which the data stored has a longer storage time as the source zone, the storage resources of the source zone can be released, avoiding the storage space of the source zone from being occupied for a long time, and improving the storage resource utilization of the source zone.

[0148] On the other hand, by determining the zone whose recycling bin data ratio is less than the second threshold as the source zone, that is, determining the zone with relatively less valid recycling bin data as the source zone, the migration of valid recycling bin data can be avoided, thereby improving the effectiveness and reliability of resource release processing.

[0149] For example, based on the above analysis, the source zone can be: Figure 6 As shown in Zone2, 50% of the valid data in the source zone can be migrated to Figure 8 Zone4 shown in , so that Zone4 includes 50% of the valid data obtained from Zone2, and as Figure 8 As shown in , Zone 4 can also include 50% of the valid data obtained from other zones.

[0150] Through Figure 7 In the solution shown, after resource processing is performed on the recording medium disk, the recording medium disk after resource processing mainly includes three types of zones: one type is a zone with a high proportion of valid data, such as a zone with valid data greater than 80%; another type is a zone with a low proportion of valid data, such as a zone with valid data of 20% (invalid data of 80%); and the third type is an idle zone.

[0151] Among them, for a zone with a high proportion of valid data, the storage space of this type of zone can be reclaimed in the high watermark method as described in the above embodiment to reduce recycling resources and costs.

[0152] In some embodiments, a zone with a valid data percentage less than a minimum percentage threshold can be erased. This means that there is no need to perform garbage collection on the valid data in the zone, reducing garbage collection costs and saving garbage collection resources.

[0153] In some embodiments, a zone corresponds to a data management unit (chunk file), that is, the capacity of a zone is the same as the size of a chunk file, and a chunk file can be stored in a zone.

[0154] If the data in the chunk file is invalid data, the zone corresponding to the chunk file can be erased to release the storage space of the zone without performing garbage collection on the zone.

[0155] For example, during the data writing process, the stand-alone storage engine may monitor the remaining capacity of the chunk file. If it is detected that the remaining capacity is less than a preset capacity threshold, the chunk file to which the data is written is switched.

[0156] Figure 9 This is a schematic diagram of a resource processing device according to an embodiment of the present disclosure. Figure 9 As shown, the apparatus 900 includes:

[0157] The first acquiring unit 901 is configured to acquire a writing speed of data to a recording medium disk, wherein the recording medium disk is a disk supporting a partition namespace interface.

[0158] Determination unit 902 is used to determine, if the write speed is less than a preset write threshold, a partition in the recording medium disk whose valid data ratio belongs to a preset medium ratio interval as a source partition, wherein the ratio in the medium ratio interval is greater than a preset minimum ratio threshold and less than a preset maximum ratio threshold, and the partition is the minimum management unit of the recording medium disk.

[0159] The processing unit 903 is configured to perform resource release processing on the source partition.

[0160] Figure 10 FIG. 1 is a schematic diagram of a resource processing device according to another embodiment of the present disclosure. Figure 10 As shown, the device 1000 includes:

[0161] The calculation unit 1001 is used to calculate the effective data ratio distribution of the partitions of the recording medium disk, wherein the recording medium disk is a disk that supports the partition namespace interface, and the effective data ratio distribution of the partitions is the number distribution of partitions with different effective data ratios.

[0162] The first acquiring unit 1002 is configured to acquire a writing speed of data to a recording medium disk if the effective data ratio distribution of the partition is normal.

[0163] Among them, the normal distribution of the effective data proportion of the partition is characterized by: the sum of the number of partitions whose resource utilization is greater than the maximum utilization threshold and the number of partitions whose resource utilization is less than the minimum utilization threshold is greater than a preset third threshold, and the third threshold is greater than half of the total number of partitions.

[0164] Determination unit 1003 is used to determine, if the write speed is less than a preset write threshold, a partition in the recording medium disk whose valid data ratio belongs to a preset medium ratio interval as a source partition, wherein the ratio in the medium ratio interval is greater than a preset minimum ratio threshold and less than a preset maximum ratio threshold, and the partition is the minimum management unit of the recording medium disk.

[0165] Combine Figure 10 It can be seen that, in some embodiments, the determining unit 1003 includes:

[0166] The first acquiring subunit 10031 is configured to acquire a partition in a closed state from the recording medium disk.

[0167] The determining subunit 10032 is configured to determine a source partition from partitions in a closed state.

[0168] The second acquiring unit 1004 is configured to acquire the storage duration of the data in the source partition.

[0169] The third obtaining unit 1005 is configured to obtain a recycle bin data ratio of valid recycle bin data in the source partition if the storage duration is greater than the first threshold, wherein the valid data includes valid recycle bin data.

[0170] The processing unit 1006 is configured to release the storage resources of the source partition if the proportion of the recycle bin data is less than a preset second threshold.

[0171] Combine Figure 10 It can be seen that, in some embodiments, the processing unit 1006 includes:

[0172] The second acquiring subunit 10061 is configured to acquire a free partition from the recording medium disk.

[0173] The migration subunit 10062 is configured to migrate valid data in the source partition to the free partition.

[0174] The fourth obtaining unit 1007 is configured to obtain, from the recording medium disk, a partition whose valid data ratio is less than the minimum ratio threshold.

[0175] The first erasing unit 1008 is used to erase the acquired partition.

[0176] One partition corresponds to one data management unit. If the data in any data management unit is invalid, the second erasing unit 1009 erases the partition corresponding to the data management unit.

[0177] Figure 11 Schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present disclosure. Figure 11 As shown, the electronic device 1100 of the embodiment of the present disclosure may include: at least one processor 1101 ( Figure 11 Only one processor is shown); and a memory 1102 communicatively connected to the at least one processor. Memory 1102 stores instructions executable by at least one processor 1101. The instructions are executed by at least one processor 1101 to enable electronic device 1100 to implement the technical solution of any of the aforementioned method embodiments.

[0178] Optionally, the memory 1102 may be independent or integrated with the processor 1101 .

[0179] When the memory 1102 is a device independent of the processor 1101 , the electronic device 1100 further includes a bus 1103 for connecting the memory 1102 and the processor 1101 .

[0180] The electronic device provided by the embodiments of the present disclosure can execute the technical solution of any of the aforementioned method embodiments, and its implementation principles and technical effects are similar, which will not be repeated here.

[0181] The embodiments of the present disclosure further provide a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the technical solution in any of the aforementioned method embodiments.

[0182] An embodiment of the present disclosure provides a computer program product, including a computer program, which implements the technical solution of any of the aforementioned method embodiments when executed by a processor.

[0183] The embodiment of the present disclosure further provides a chip, including: a processing module and a communication interface, wherein the processing module can execute the technical solution in the aforementioned method embodiment.

[0184] Furthermore, the chip also includes a storage module (such as a memory), the storage module is used to store instructions, the processing module is used to execute the instructions stored in the storage module, and the execution of the instructions stored in the storage module enables the processing module to execute the technical solution in the aforementioned method embodiment.

[0185] It should be understood that the processor described above may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASICs). A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0186] The memory may include a high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk.

[0187] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the figures of this disclosure are not limited to just one bus or just one type of bus.

[0188] The storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0189] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an application-specific integrated circuit (ASIC). Of course, the processor and storage medium can also exist as discrete components in an electronic device.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A resource processing method, characterized in that: include: Acquiring a writing speed of data to a recording medium disk, wherein the recording medium disk is a disk supporting a partition namespace interface; If the write speed is less than a preset write threshold, a partition in the recording medium disk whose valid data ratio falls within a preset medium ratio interval is determined as a source partition, wherein the ratio in the medium ratio interval is greater than a preset minimum ratio threshold and less than a preset maximum ratio threshold, and the partition is the minimum management unit of the recording medium disk; A resource release process is performed on the source partition.

2. The method according to claim 1, characterized in that After determining the partition in the recording medium disk, whose valid data ratio falls within a preset medium ratio range, as the source partition, the method further includes: Obtaining the storage duration of the data in the source partition; Furthermore, the performing resource release processing on the source partition includes: if the storage duration is greater than a preset first threshold, performing resource release processing on the source partition.

3. The method according to claim 2, characterized in that Before performing resource release processing on the source partition, the method further includes: If the storage duration is greater than the first threshold, obtaining a ratio of valid data in the recycle bin in the source partition, wherein the valid data includes valid data in the recycle bin; Furthermore, the resource release processing for the source partition includes: if the proportion of valid data in the recycle bin is less than a preset second threshold, releasing the storage resources of the source partition.

4. The method according to any one of claims 1 to 3, characterized in that The step of determining a partition in the recording medium disk, in which the effective data ratio falls within a preset medium ratio range, as a source partition includes: A partition in a closed state is acquired from the recording medium disc, and a source partition is determined from the partition in the closed state.

5. The method according to any one of claims 1 to 3, characterized in that Before acquiring the writing speed of writing data to the recording medium disk, the method further includes: Calculating a valid data ratio distribution of partitions of the recording medium disk, wherein the valid data ratio distribution of partitions is a number distribution of partitions with different valid data ratios; And, the acquiring the writing speed of the data to the recording medium disk includes: if the effective data ratio distribution of the partition is normal, acquiring the writing speed of the data to the recording medium disk; Among them, the normal distribution of the effective data proportion of the partition is characterized by: the sum of the number of partitions whose resource utilization is greater than the maximum utilization threshold and the number of partitions whose resource utilization is less than the minimum utilization threshold is greater than a preset third threshold, and the third threshold is greater than half of the total number of partitions.

6. The method according to any one of claims 1 to 3, characterized in that The performing resource release processing on the source partition includes: Acquire a free partition from the recording medium disk; Migrate valid data in the source partition to the free partition.

7. The method according to any one of claims 1 to 3, characterized in that The method further comprises: A partition whose effective data ratio is less than the minimum ratio threshold is obtained from the recording medium disk, and the obtained partition is erased.

8. The method according to any one of claims 1 to 3, characterized in that One partition corresponds to one data management unit, and the method further includes: If the data in any data management unit is invalid data, the partition corresponding to the data management unit is erased.

9. A resource processing device, characterized in that: include: a first acquiring unit, configured to acquire a writing speed of data to a recording medium disk, wherein the recording medium disk is a disk supporting a partition namespace interface; a determining unit configured to, if the write speed is less than a preset write threshold, determine a partition in the recording medium disk whose valid data ratio falls within a preset medium ratio interval as a source partition, wherein if the ratio in the medium ratio interval is greater than a preset minimum ratio threshold and less than a preset maximum ratio threshold, the partition is a minimum management unit of the recording medium disk; A processing unit is configured to perform resource release processing on the source partition.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to execute the resource processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the resource processing method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the resource processing method according to any one of claims 1 to 8 is implemented.

Images