A method, apparatus and device for disk space management

Abstract

The application relates to the technical field of disk management, and discloses a disk space management method, device and equipment, the method comprising the following steps: dividing disk space into a data space, a metadata space and a configuration space; in response to a write request of target data, writing the target data into the data space; during the writing process of the target data, continuously calculating the area utilization of the data space; when the area utilization of the data space is greater than a preset threshold, adopting a recovery strategy to recover data in the data space until a recovery stop condition is reached, and continuing to write the target data into the data space. According to the application, the disk space is divided into different functional areas, fine management and optimization are realized, the area utilization is continuously calculated during the writing process of the target data, the use of the disk space is monitored in real time, the recovery strategy is adopted to recover data when the area utilization is greater than the preset threshold, data fragmentation is prevented, and efficient and intelligent management of the disk space is realized.

Description

Technical Field

[0001] This invention relates to the field of disk management technology, and more specifically to a disk space management method, apparatus, and device. Background Technology

[0002] With the rapid development of information technology, the demand for data storage is increasing day by day, and efficient disk space management methods have become one of the key factors in improving system performance and reliability. Traditional disk space management methods are often static, and data reclamation strategies are usually based on fixed rules, resulting in problems such as low resource utilization, severe data fragmentation, and poor flexibility. These problems not only affect the storage efficiency of the system, but may also lead to a decline in system performance and an increase in maintenance costs. Summary of the Invention

[0003] In view of this, the present invention provides a disk space management method, apparatus and device to solve the problem of disk space management.

[0004] In a first aspect, the present invention provides a disk space management method, the method comprising:

[0005] Divide the disk space into data space, metadata space, and configuration space;

[0006] In response to a write request for the target data, the target data is written to the data space;

[0007] During the writing of target data, the area utilization rate of the data space is continuously calculated;

[0008] If the utilization rate of the data space exceeds a preset threshold, a recycling strategy is adopted to reclaim the data space until the recycling stop condition is met, and then the target data is written into the data space after the data is reclaimed.

[0009] The disk space management method provided in this invention divides the disk space into different functional areas, enabling more refined management and optimization. When a write request for target data is received, the target data is written to the data space, clarifying the data write path and ensuring that data can be written to the designated area quickly and accurately, thus improving the efficiency of data writing. During the writing process of target data, the area utilization rate of the data space is continuously calculated, and the disk space usage is monitored in real time. When the area utilization rate exceeds a preset threshold, a recycling strategy is adopted to reclaim data, avoiding data write failures due to insufficient space, preventing data fragmentation, ensuring efficient utilization of disk space, and achieving efficient and intelligent management of disk space.

[0010] In one alternative implementation, the data space includes an empty data area, a filled data area, a written data area, a periodic data area, and a recycled data area.

[0011] Write the target data into the data space, including:

[0012] If the target data has a lifecycle, determine whether the free space in the periodic data region meets the target data requirements;

[0013] If the available space in the periodic data region is insufficient to meet the target data requirement, move any first available data group from the empty data region to the periodic data region, and store the target data in the first available data group; or...

[0014] If the available space in the periodic data region is sufficient to support the target data, the target data is stored in the data group of the periodic data region based on the time range of the target data's lifecycle.

[0015] The disk space management method provided in this invention divides the data space into multiple functional areas, enabling more refined management and optimization. Each area has a specific purpose, which helps improve data access efficiency, reduce data fragmentation, and facilitates maintenance and management. When writing target data, data with a lifecycle is preferentially stored in the periodic data area. When the free space in the periodic data area is insufficient, the space in the empty data area is dynamically adjusted to ensure smooth data writing, avoid write failures due to insufficient space, reduce the possibility of data fragmentation, and improve the overall utilization of disk space.

[0016] In one optional implementation, writing the target data into the data space further includes:

[0017] If the target data has no lifecycle, determine whether the free space in the write data area is sufficient for the target data.

[0018] If the available space in the write data area is insufficient to meet the target data requirement, move any second available data group from the empty data area to the write data area, and store the target data in the second available data group; or,

[0019] If the available space in the write data area is sufficient to store the target data, the target data will be stored in the data group of the write data area.

[0020] The disk space management method provided in this embodiment of the invention first determines whether the free space of the write data area meets the needs of the target data when the target data has no life cycle, and prioritizes the use of the write data area. When the free space of the write data area is insufficient, the space of the empty data area is dynamically adjusted to ensure the smooth writing of data and avoid write failure due to insufficient space, thereby improving the overall utilization of disk space.

[0021] In one alternative implementation, the method further includes:

[0022] If any data group is full, move the data group to the area where the data is full.

[0023] The disk space management method provided in this embodiment of the invention improves space utilization by promptly moving full data groups to full data areas, thus preventing these data groups from occupying space in other areas.

[0024] In one optional implementation, when the area utilization rate of the data space exceeds a preset threshold, a data reclamation strategy is employed to reclaim data from the data space until a reclamation stop condition is met, including:

[0025] For each data group that fills the data area, determine the valid data blocks in the data group;

[0026] Determine the lifespan of the valid data block based on the current date, the write time of the valid data block, and its lifespan.

[0027] The lifecycle duration of the data group is determined based on the lifecycle duration of all valid data blocks in the data group;

[0028] The recycling priority is determined based on the lifecycle duration of all data groups in the data area that is filled with data, and the data groups to be recycled are determined based on the recycling priority.

[0029] Identify the target data group from all data groups in the recycled data area;

[0030] Data recycling is performed based on the recycled data set and the target data set. The above data recycling process is repeated until the recycling stop condition is met.

[0031] The disk space management method provided in this invention ensures efficient use of disk space by initiating a recycling strategy when the area utilization of the data space exceeds a preset threshold. By identifying the effective data blocks of each data group filled with data in the data area, the lifecycle of each effective data block is determined, and the recycling priority of the data group is calculated accordingly. This ensures that the data groups with the fewest effective data blocks and the least likely to be deleted are recycled first. By repeatedly executing the data recycling process until the recycling stop condition is met, efficient and intelligent management of disk space is achieved.

[0032] In one alternative implementation, determining the target data group from all data groups in the reclaimed data area includes:

[0033] All data groups in the reclaimed data area are classified according to their write time to obtain multiple target data group sets;

[0034] Based on the lifecycle duration of the recycled data group, determine the target data set to which the lifecycle duration belongs, and then determine the target data group from the target data group set.

[0035] The disk space management method provided in this invention classifies all data groups in the reclaimed data area according to their write time, which can group data groups with similar write times into the same set, simplifying the reclamation process and improving reclamation efficiency. Based on the life cycle duration of the reclaimed data group, the target data group is determined from the corresponding target data group set to ensure the accuracy of reclamation and further improve space utilization.

[0036] In one alternative implementation, data recycling is performed based on a recycled data set and a target data set, including:

[0037] Retrieve the reclamation data for all valid data blocks included in the reclamation data group;

[0038] Determine if the available space in the target data group is sufficient for data recycling.

[0039] If there is enough free space in the target data group to collect the data, move the collected data to the target data group.

[0040] The disk space management method provided in this invention obtains the reclamation data of all data blocks in the reclamation data group to ensure that all data that needs to be reclaimed is accurately collected. By determining whether the free space of the target data group meets the requirements of the reclamation data, it can ensure that data movement is only performed when the target data group has sufficient space, thereby avoiding data conflicts caused by insufficient space and ensuring the smooth progress of data movement. When the free space of the target data group meets the requirements of the reclamation data, the reclamation data is moved to the target data group, ensuring the efficiency and reliability of data migration.

[0041] In one optional implementation, after determining whether the available space in the target data group is sufficient for data reclamation, the method further includes:

[0042] If the available space in the target data group is insufficient to reclaim data, any third available data group in the empty data area is added to the target data group, and the reclaimed data is moved to the third available data group.

[0043] The disk space management method provided in this embodiment of the invention ensures data reclamation by adding selected free data groups to the target data group when the free space of the target data group is insufficient, and moving the reclaimed data to the expanded target data group.

[0044] In a second aspect, the present invention provides a disk space management device, the device comprising:

[0045] The partitioning module is used to divide disk space into data space, metadata space, and configuration space;

[0046] The write module is used to write the target data into the data space in response to a write request for the target data;

[0047] The calculation module is used to continuously calculate the area utilization rate of the data space during the writing process of the target data;

[0048] The recycling module is used to reclaim data from the data space when the area utilization rate of the data space exceeds a preset threshold, and then continue to write the target data into the data space after the data is reclaimed.

[0049] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the disk space management method of the first aspect or any corresponding embodiment described above.

[0050] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the disk space management method of the first aspect or any corresponding embodiment thereof.

[0051] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the disk space management method of the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0052] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0053] Figure 1 This is a flowchart of a disk space management method according to an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of disk space according to an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of data group type conversion according to an embodiment of the present invention;

[0056] Figure 4This is a schematic diagram of another data group type conversion according to an embodiment of the present invention;

[0057] Figure 5 This is a structural block diagram of a disk space management device according to an embodiment of the present invention;

[0058] Figure 6 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Traditional disk space management methods are often static, and data reclamation strategies are usually based on fixed rules, resulting in low resource utilization, severe data fragmentation, and poor flexibility. These problems not only affect system storage efficiency but may also lead to system performance degradation and increased maintenance costs. The disk space management method provided in this invention divides disk space into different functional areas, enabling more refined management and optimization. When a write request for target data is received, the target data is written to the data space, clearly defining the data write path and ensuring that data can be written to the designated area quickly and accurately, improving data write efficiency. During the target data writing process, the area utilization rate of the data space is continuously calculated, and the disk space usage is monitored in real time. When the area utilization rate exceeds a preset threshold, a reclamation strategy is used to reclaim data, avoiding data write failures due to insufficient space, preventing data fragmentation, ensuring efficient use of disk space, and achieving efficient and intelligent management of disk space.

[0061] According to an embodiment of the present invention, a disk space management method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0062] This embodiment provides a disk space management method that can be used in a storage system, including a Key-Value storage engine. Figure 1 This is a flowchart of a disk space management method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0063] Step S101 involves dividing the disk space into a data space, a metadata space, and a configuration space. Specifically, the disk can be an SMR (Shingled Magnetic Recording) disk. Figure 2 This is a schematic diagram of disk space according to an embodiment of the present invention, such as... Figure 2 As shown, disk formatting divides disk space into three areas: data space, metadata space, and configuration space. Each space includes one or more data groups of the same size, and each data group includes several data blocks. The configuration space stores space configuration information, such as the user-specified data block size, the list of data groups occupied by the metadata space, and formatting parameters. The metadata space stores log and index information, with content stored in record units, including name, write time, data storage location, length, modification time, data verification, and the hash value of the record. Data verification verifies data validity, and the record's hash value verifies the record's integrity. The metadata space also includes an index stored using an LSM-tree (Log-Structured Merge Tree), sorted by name and position, facilitating record retrieval and space reclamation. The data space stores user-written data. If the disk is an SMR disk, the data group size is consistent with the physical data group size of the SMR disk, which can be obtained through the SMR disk management interface. For ordinary disks, the size can be defined manually. The data block size can be set during formatting according to business needs, for example, 4KB. The size of a data block must be divisible by the size of the data group. Since the outer tracks of a disk offer better performance than the inner tracks, 1% of the outer tracks can be reserved as metadata space. Use the first data group on the disk as configuration space, and the last data group as a backup for fault recovery. Dividing the disk space into different functional areas allows for finer-grained management and optimization.

[0064] Step S102: In response to the write request for the target data, the target data is written to the data space. Specifically, when a user wants to store the target data on the disk, the storage system receives the write request for the target data and writes the target data to the data space according to the functional partition of the disk space.

[0065] Step S103: During the writing of the target data, the region utilization rate of the data space is continuously calculated. Specifically, the region utilization rate represents the proportion of used data blocks to all data blocks in the data space. Continuously calculating the region utilization rate during the writing process of the target data allows for continuous monitoring of the writing process and disk usage, ensuring that data can be written to the designated region quickly and accurately, thus improving the efficiency of data writing.

[0066] Step S104: If the area utilization rate of the data space exceeds a preset threshold, a data reclamation strategy is employed to reclaim data from the data space until the reclamation stop condition is met. Then, the target data is written to the reclaimed data space. Specifically, the preset threshold is a value pre-stored in the metadata space. When the area utilization rate of the data space exceeds this preset threshold, a data reclamation strategy is initiated to reorganize and optimize the data space until the reclamation stop condition is met. After the data reclamation operation is completed, the target data is written to the reclaimed data space. At this point, the area utilization rate of the data space has decreased, allowing it to accommodate more data, avoiding data write failures due to insufficient space, preventing data fragmentation, and ensuring efficient use of disk space.

[0067] The disk space management method provided in this invention divides the disk space into different functional areas, enabling more refined management and optimization. When a write request for target data is received, the target data is written to the data space, clarifying the data write path and ensuring that data can be written to the designated area quickly and accurately, thus improving the efficiency of data writing. During the writing process of target data, the area utilization rate of the data space is continuously calculated, and the disk space usage is monitored in real time. When the area utilization rate exceeds a preset threshold, a recycling strategy is adopted to reclaim data, avoiding data write failures due to insufficient space, preventing data fragmentation, ensuring efficient utilization of disk space, and achieving efficient and intelligent management of disk space.

[0068] This embodiment provides a disk space management method that can be used in the aforementioned storage system. The method specifically includes the following steps:

[0069] Step S201 involves dividing the disk space into data space, metadata space, and configuration space. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0070] Step S202: In response to the write request for target data, the target data is written to the data space, which includes an empty data area, a full data area, a write data area, a periodic data area, and a reclaimed data area.

[0071] Specifically, step S202 includes:

[0072] Step S2021: If the target data has a lifecycle, determine whether the available space in the periodic data area is sufficient for the target data. Specifically, the empty data area contains unused data groups; the full data area contains data groups that have been completely written to, and no further writing is allowed even if records are deleted; the write data area is used to write data that does not have a lifecycle; the periodic data area is used to store data with a lifecycle; and the reclaimable data area is used to store data that has been marked as reclaimable. When a user wants to write the target data, they can set a lifecycle for the target data, meaning the target data is only valid within that lifecycle. For target data with a lifecycle, priority is given to writing the target data to the periodic data area; therefore, it is necessary to determine whether the available space in the periodic data area is sufficient to write the target data.

[0073] In step S2022, if the available space in the periodic data area is insufficient to support the target data, any first available data group from the empty data area is moved to the periodic data area, and the target data is stored in the first available data group. Specifically, if the available space in the periodic data area is insufficient to write the target data, an empty data group, i.e., the first available data group, is requested from the empty data area and moved to the periodic data area to ensure that the target data is successfully written to the periodic data area. This ensures smooth data writing, avoids write failures due to insufficient space, reduces the possibility of data fragmentation, and improves the overall utilization of disk space.

[0074] Alternatively, in step S2023, if the available space in the periodic data area is sufficient for the target data, the target data is stored in a data group within the periodic data area based on the time range of the target data's lifecycle. Specifically, if the periodic data area can hold the target data, the writing location is determined according to the target data's lifecycle. Assuming the current date is used as the baseline, the periodic data area is grouped according to the expiration time from the current date, resulting in four data groups: 0-1 month ago, 1 month ago-3 months ago, and 3 months ago-6 months ago. If the target data's lifecycle is 2 months, then the target data is written to the 1 month ago-3 months ago data group. Optionally, the above grouping method for the periodic data area is merely an example and is not intended to be limiting.

[0075] Step S2024: If the target data has no lifecycle, determine whether the available space in the write data area is sufficient for the target data. Specifically, when the target data has no lifecycle, the target data is written to the corresponding write data area; therefore, it is necessary to first determine whether the write data area has enough space.

[0076] Step S2025: If the free space in the write data area is insufficient to support the target data, any second free data group from the empty data area is moved to the write data area, and the target data is stored in the second free data group. Specifically, if the free space in the write data area is insufficient to write the target data, an empty data group, i.e., a second free data group, is requested from the empty data area and moved to the write data area to ensure that the target data is successfully written to the write data area. This ensures smooth data writing, avoids write failures due to insufficient space, reduces the possibility of data fragmentation, and improves the overall utilization of disk space.

[0077] Alternatively, in step S2026, if the available space in the write data area is sufficient to store the target data, the target data is stored in a data group within the write data area. Specifically, if the write data area can hold the target data, the target data can be preferentially written to a data group with less available space.

[0078] Step S203: If any data group is full, move the data group to the full data area. Specifically, during the writing process of the target data, if any data group is full, move the data group to the corresponding full data area to avoid these full data groups occupying space in other areas and improve space utilization.

[0079] Step S204: During the writing of the target data, the area utilization rate of the data space is continuously calculated. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0080] Step S205: If the regional utilization rate of the data space is greater than a preset threshold, a recycling strategy is adopted to recycle the data space until the recycling stop condition is reached, and then the target data is written into the data space after the data recycling.

[0081] Specifically, step S205 includes:

[0082] Step S2051: For each data group in the filled data area, determine the valid data blocks within that data group. Specifically, the filled data area includes multiple data groups, and each data group includes multiple data blocks. Examine the data blocks in each data group to identify valid data blocks. Valid data blocks with earlier write times are considered "colder" (less active). For valid data blocks with known lifecycles, those closer to the specified deletion date are considered "hotter" (more active). By identifying valid data blocks, they can be reclaimed to free up disk space.

[0083] Step S2052: Determine the life cycle duration of the valid data block based on the current date, the write full time of the valid data block, and the life cycle. Specifically, if the valid data block does not have a life cycle, the life cycle duration of the valid data block is calculated by the following formula (1). If the valid data block has a life cycle, the life cycle duration of the valid data block is calculated by the following formula (2).

[0084] day = currentDate - writeDate (1)

[0085] Where day represents the life cycle duration; currentDate represents the current date; writeDate represents the date when the valid data block is written full.

[0086] day = |currentDate - writeDate – Max(lifeCycleDays)| (2)

[0087] Where day represents the life cycle duration; currentDate represents the current date; writeDate represents the date when the valid data block is written full; lifeCycleDays represents the life cycle. Since the life cycle is a time range, Max(lifeCycleDays) is the upper limit of the life cycle time range.

[0088] Step S2053: Determine the life cycle duration of the data group based on the life cycle durations of all valid data blocks in the data group. Specifically, since a data group usually includes at least one valid data block, the life cycle durations of different valid data blocks may be different. For any data group, the longest life cycle duration among all valid data blocks included in the data group is used as the life cycle duration of the data group.

[0089] Step S2054: Determine the recycling priority based on the life cycle durations of all data groups in the written full data area, and determine the recycling data group based on the recycling priority. Specifically, assume there are multiple time classifications: 0 days - day1 days, day1 days - day2 days, day2 days - day3 days,..., dayn days and above, corresponding to recycling coefficients t1, t2, t3,..., tn respectively, and 0 < t1 < t2 < t3 <... < tn < 1. For any data group, determine the recycling coefficient of the data group according to the time classification to which the life cycle duration of the data group belongs. Then, determine the recycling priority of the data group through the following formula (3).

[0090]

[0091] Where x represents the reclamation priority of any data group; k represents the number of valid data blocks in the data group; K represents the total number of data blocks in the data group; t final This represents the recovery coefficient for this data set.

[0092] The above formula (3) can be used to obtain the recycling priority of all data groups filling the data area. The larger the value of x, the higher the recycling priority, that is, the earlier it will be recycled. Generally, the data group with the largest x value, that is, the data group with the highest recycling priority, is taken as the recycling data group for subsequent data recycling. By determining the recycling data group based on the recycling priority, it is ensured that the data groups with the fewest valid data blocks and the least likely to be deleted are recycled first, so as to achieve efficient management of disk space.

[0093] Step S2055: Determine the target data group from all data groups in the reclaimed data area.

[0094] In some optional implementations, step S2055 above includes:

[0095] Step a1 involves classifying all data groups in the reclaimed data area according to their write time, resulting in multiple target data group sets. Specifically, when the storage system performs data reclamation, it selects data blocks from the reclaimed data area for data reclamation. Since the reclaimed data group is located in a filled data area, the data to be reclaimed needs to be transferred to the reclaimed data area to achieve data reclamation. Classifying all data groups in the reclaimed data area according to their write time can, for example, result in five target data group sets: 0-4 days, 5-16 days, 17-64 days, 65-256 days, and 257 days and above. Optionally, when calculating the write time of any data group, since the data group may include multiple data blocks, the earliest write time, latest write time, or average write time among all data blocks can be used as the write time of the data group. This embodiment of the invention does not impose any limitations on this.

[0096] Step a2: Based on the lifecycle duration of the recycled data group, determine the target data set to which the lifecycle duration belongs, and then select the target data group from the target data group set. Specifically, assuming the lifecycle duration of the recycled data group is 20 days, it corresponds to the target data group set of 17-64 days. Select the target data group from all data groups included in this 17-64 day target data group set. Optionally, the target data group can be selected based on the write time and lifecycle duration, or the target data group with the longest write time in the 17-64 day target data group set can be selected as the target data group. Other methods can also be used to determine the target data group, and this embodiment of the invention does not limit this. By classifying all data groups in the recycled data area according to their write time, data groups with similar write times can be grouped into the same set, simplifying the recycling process and improving recycling efficiency. Determining the target data group from the corresponding target data group set based on the lifecycle duration of the recycled data group ensures the accuracy of recycling and further improves space utilization.

[0097] Step S2056: Perform data recycling based on the recycled data group and the target data group, repeating the above data recycling process until the recycling stop condition is met.

[0098] In some optional implementations, step S2056 above performs data recovery based on the recovered data set and the target data set, including:

[0099] Step b1: Obtain the reclamation data for all valid data blocks included in the reclamation data group. Specifically, retrieve all reclamation data to be reclaimed from the reclamation data group.

[0100] Step b2: Determine if the available space in the target data group is sufficient for data reclamation. Specifically, for data reclamation to proceed, the data to be reclaimed needs to be moved from the filled data area to the reclamation data area. Therefore, it is necessary to determine whether the target data group in the reclamation data area is sufficient to write the reclaimed data.

[0101] Step b3: If the available space in the target data group is sufficient to write the reclaimed data, move the reclaimed data to the target data group. Specifically, if the space in the target data group is sufficient to write the reclaimed data, move the reclaimed data to the target data group so that the data can be reclaimed when the storage system performs data reclamation.

[0102] Step b4: If the available space in the target data group is insufficient to accommodate the reclaimed data, any third available data group from the empty data area is added to the target data group, and the reclaimed data is moved to the third available data group. Specifically, if the available space in the target data group is insufficient to write the reclaimed data, an empty data group, i.e., a third available data group, is requested from the empty data area and added to the target data group to ensure that the reclaimed data can be successfully written to the reclaimed data area, thus ensuring the data reclamation process.

[0103] In some alternative implementations, after the data movement is complete, the data is written to disk, the data location pointed to by the record stored in the metadata area is modified to the new location, and the metadata record is written to disk.

[0104] In some optional implementations, after data reclamation based on the above steps, it is determined whether the reclamation stop condition has been met. If the reclamation stop condition has not been met, the process returns to step S2054 to continue determining new data groups to be reclaimed and their corresponding target data groups, so as to continue data reclamation until the reclamation stop condition is met. The reclamation stop condition can be that the sum of the area utilization rate and the busy factor is greater than the effective data rate. The area utilization rate is the area utilization rate of the data area, that is, the proportion of used data blocks to all data blocks in the data space. The busy factor is positive when the disk is idle and negative when the disk is busy. By dynamically adjusting the busy factor, more space is reclaimed when the disk is idle. The effective data rate is calculated using the following formula (4). Optionally, the reclamation stop condition can also be that the area utilization rate is not greater than a user-preset data reclamation threshold.

[0105]

[0106] Where p represents the effective data rate; totalUsedBlockSize represents the total length of the used data blocks; fullZoneCount represents the number of data groups that are full; and ZoneSize represents the size of the data group.

[0107] In some optional implementations, embodiments of the present invention may also reclaim data groups in the data area in order of the amount of free space from most to least, until the target data is successfully written into the data space.

[0108] In some alternative implementations, Figure 3 This is a schematic diagram of data group type conversion according to an embodiment of the present invention, such as... Figure 3 As shown, if the metadata space is insufficient, empty data areas in the data space can be moved to the metadata space. Conversely, if the reserved metadata space is too large, data groups in the metadata space can be dynamically moved to empty data areas in the data space. Figure 4This is a schematic diagram of another data group type conversion according to an embodiment of the present invention, such as... Figure 4 As shown, when the data groups in the periodic data area and the write data area are full, the full data groups can be transferred to the full data area. Specifically, when the valid data groups in the reclaimed data area are full, they can also be transferred to the full data area. When reclaimed data from a full data area needs to be written to the reclaimed data area, if the reclaimed data area has insufficient space, empty data groups can be transferred from the empty data area to the reclaimed data area. When data needs to be written to the periodic data area or the write data area, if the free space in these two areas is insufficient to write the data, free data groups can be transferred from the empty data area to these two areas. When data in a full data area is transferred to the reclaimed data area for data reclamation, empty data groups may exist after the data transfer; these empty data groups can be transferred to the empty data area.

[0109] The disk space management method provided in this invention divides the disk space into different functional areas, enabling more refined management and optimization. When a write request for target data is received, the target data is written to the data space, clarifying the data write path and ensuring that data can be written to the designated area quickly and accurately, thus improving the efficiency of data writing. During the writing process of target data, the area utilization rate of the data space is continuously calculated, and the disk space usage is monitored in real time. When the area utilization rate exceeds a preset threshold, a recycling strategy is adopted to reclaim data, avoiding data write failures due to insufficient space, preventing data fragmentation, ensuring efficient utilization of disk space, and achieving efficient and intelligent management of disk space.

[0110] This embodiment also provides a disk space management device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0111] This embodiment provides a disk space management device, such as... Figure 5 As shown, it includes:

[0112] The partitioning module 501 is used to divide the disk space into data space, metadata space and configuration space.

[0113] The write module 502 is used to write the target data into the data space in response to a write request for the target data.

[0114] The calculation module 503 is used to continuously calculate the area utilization rate of the data space during the writing process of the target data.

[0115] The recycling module 504 is used to recycle data in the data space when the area utilization rate of the data space is greater than a preset threshold, and then continue to write the target data into the data space after data recycling.

[0116] In some alternative implementations, the data space includes an empty data area, a filled data area, a written data area, a periodic data area, and a recycled data area;

[0117] The write module 502 includes:

[0118] The first judgment unit is used to determine whether the free space in the periodic data area meets the target data requirements, given that the target data has a lifecycle.

[0119] The first storage unit is used to move any first free data group in the empty data area to the periodic data area and store the target data in the first free data group when the free space in the periodic data area does not meet the target data requirements.

[0120] Alternatively, a second storage unit is used to store the target data into a data group in the periodic data area, based on the time range of the target data's lifecycle, provided that the available space in the periodic data area meets the target data requirements.

[0121] In some alternative implementations, the writing module 502 further includes:

[0122] The second judgment unit is used to determine whether the free space in the data area to be written meets the requirements of the target data when the target data does not have a lifecycle.

[0123] The third storage unit is used to move any second spare data group in the empty data area to the write data area and store the target data in the second spare data group when the spare space in the write data area does not meet the target data requirements.

[0124] Alternatively, a fourth storage unit is used to store the target data into a data group in the write data area, provided that the available space in the write data area is sufficient for the target data.

[0125] In some alternative embodiments, the device further includes:

[0126] The first moving module is used to move a data group to a full data area when any data group is already full.

[0127] In some alternative implementations, the recycling module 504 includes:

[0128] The first determining unit is used to determine the valid data blocks in each data group that fills the data area.

[0129] The second determining unit is used to determine the lifecycle duration of a valid data block based on the current date, the write-full time of the valid data block, and its lifecycle.

[0130] The third determining unit is used to determine the lifecycle duration of the data group based on the lifecycle duration of all valid data blocks in the data group.

[0131] The fourth determining unit is used to determine the recycling priority based on the lifecycle duration of all data groups in the data area, and to determine the data groups to be recycled based on the recycling priority.

[0132] The fifth determining unit is used to determine the target data group from all data groups in the data recovery area.

[0133] The recycling unit is used to recycle data based on the recycled data set and the target data set, repeating the above data recycling process until the recycling stop condition is met.

[0134] In some optional implementations, the fifth determining unit includes:

[0135] The first determining sub-unit is used to classify all data groups in the reclaimed data area according to their write time, thereby obtaining multiple target data group sets.

[0136] The second determining subunit is used to determine the target data set to which the lifecycle duration belongs based on the lifecycle duration of the recycled data set, and to determine the target data set from the target data set.

[0137] In some alternative implementations, the recycling unit includes:

[0138] The acquisition sub-unit is used to acquire the reclaimed data of all valid data blocks included in the reclaimed data group.

[0139] The judgment sub-unit is used to determine whether the available space in the target data group is sufficient for data recycling.

[0140] The moving sub-unit is used to move the recycled data to the target data group when there is enough free space in the target data group.

[0141] In some alternative implementations, after determining the subunit, the device further includes:

[0142] The second moving module is used to add any third empty data group in the empty data area to the target data group and move the recycled data to the third empty data group when the empty space of the target data group is not sufficient to recycle the data.

[0143] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0144] In this embodiment, the disk space management device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0145] This invention also provides a computer device having the above-described features. Figure 5 The disk space management device shown.

[0146] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 6 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 6 Take a processor 10 as an example.

[0147] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GPA), or any combination thereof.

[0148] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0149] The memory 20 may include a program storage area and a data space, wherein the program storage area may store the operating system and applications required for at least one function; the data space may store data created based on the use of the computer device, etc. Furthermore, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0150] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0151] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0152] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0153] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0154] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method of managing disk space, characterized by, The method comprises: dividing disk space into data space, metadata space and configuration space; in response to a write request of target data, writing the target data into the data space, the data space comprising full data area, recycled data area and empty data area; during the writing of the target data, continuously calculating the area utilization of the data space; in the case where the area utilization of the data space is greater than a preset threshold, adopting a recycling strategy to recycle data in the data space until a recycling stop condition is reached, and continuing to write the target data into the data space after recycling; wherein, in the case where the area utilization of the data space is greater than a preset threshold, adopting a recycling strategy to recycle data in the data space until a recycling stop condition is reached, comprises: for each data group in the full data area, determining valid data blocks in the data group; based on the current date, the full-time of the valid data blocks and the life cycle, determining the life cycle duration of the valid data blocks; based on the life cycle duration of all valid data blocks in the data group, determining the life cycle duration of the data group; based on the life cycle duration of all data groups in the full data area, determining a recycling priority, and determining a recycled data group based on the recycling priority; determining a target data group from all data groups in the recycled data area; based on the recycled data group and the target data group, recycling data, and repeating the above data recycling process until the recycling stop condition is reached.

2. The method of claim 1, wherein, The data space comprises empty data area, full data area, write data area, periodic data area and recycled data area; The writing of the target data into the data space comprises: in the case where the target data has a life cycle, judging whether the free space of the periodic data area meets the target data; in the case where the free space of the periodic data area does not meet the target data, moving any first free data group in the empty data area to the periodic data area, and storing the target data into the first free data group; or in the case where the free space of the periodic data area meets the target data, based on the time range in which the life cycle of the target data is located, storing the target data into a data group of the periodic data area.

3. The method of claim 2, wherein, The writing of the target data into the data space further comprises: in the case where the target data has no life cycle, judging whether the free space of the write data area meets the target data; in the case where the free space of the write data area does not meet the target data, moving any second free data group in the empty data area to the write data area, and storing the target data into the second free data group; or in the case where the free space of the write data area meets the target data, storing the target data into a data group of the write data area.

4. The method according to any of claims 2 or 3, characterized in that, The method further comprises: in the case where any data group has been filled, moving the data group to the full data area.

5. The method of claim 4, wherein, The method comprises: determining a target data group from all data groups in the recycle data area, comprising: sorting all data groups in the recycle data area according to writing time to obtain a plurality of target data group sets; 6. The method of claim 1, wherein, determining a target data set to which a life cycle length of the recycle data group belongs based on the life cycle length, and determining the target data group from the target data group set. The data recycling based on the recycle data group and the target data group comprises: obtaining recycle data of all valid data blocks included in the recycle data group; judging whether a free space of the target data group satisfies the recycle data; 7. The method of claim 6, wherein, in a case where the free space of the target data group satisfies the recycle data, moving the recycle data to the target data group. After judging whether the free space of the target data group satisfies the recycle data, the method further comprises:

8. A disk space management apparatus characterized by comprising: in a case where the free space of the target data group does not satisfy the recycle data, adding any third free data group in the empty data area to the target data group, and moving the recycle data to the third free data group. The device comprises: a division module for dividing disk space into data space, metadata space and configuration space; a writing module for writing target data into the data space in response to a writing request of the target data, the data space comprising full data area and recycle data area; a calculation module for continuously calculating area utilization of the data space during writing of the target data; a recycling module for recycling data of the data space by using a recycling strategy until a recycling stop condition is reached in a case where the area utilization of the data space is greater than a preset threshold, and continuing to write the target data into the data space after recycling; wherein the recycling module is specifically configured to: determine valid data blocks in each data group in the full data area; determine a life cycle length of the valid data blocks based on a current date, full writing time and life cycle of the valid data blocks; determine a life cycle length of the data group based on life cycle lengths of all valid data blocks in the data group; determine a recycling priority based on the life cycle lengths of all data groups in the full data area, and determine a recycle data group based on the recycling priority; determine a target data group from all data groups in the recycle data area; 9. A computer device, comprising: recycle data based on the recycle data group and the target data group, and repeat the above data recycling process until the recycling stop condition is reached. comprise: a memory and a processor, which are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the disk space management method in any one of claims 1 to 7.

Images