Disk space management method, device and equipment

Abstract

The invention relates to the technical field of disk management, and discloses a disk space management method, device and equipment, and the method comprises the steps: dividing a disk space into a data space, a metadata space and a configuration space; in response to a write-in request of the target data, writing the target data into the data space; in the writing process of the target data, continuously calculating the regional utilization rate of the data space; and when the regional utilization rate of the data space is greater than a preset threshold value, performing data recovery on the data space by adopting a recovery strategy until a recovery stop condition is met, and continuing to write the target data into the data space. The disk space is divided into different functional areas, finer management and optimization are achieved, the area utilization rate is continuously calculated in the write-in process of the target data, the use condition of the disk space is monitored in real time, when the use condition is larger than the preset threshold value, a recovery strategy is adopted for data recovery, data fragmentation is prevented, and the use efficiency of the disk space is improved. And efficient and intelligent management of the disk space is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of disk management, and in particular to a disk space management method, device and equipment. Background Art

[0002] With the rapid development of information technology, the demand for data storage is growing, and efficient disk space management methods have become one of the key factors in improving system performance and reliability. Traditional disk space management methods are often static, and data recovery strategies are usually based on fixed rules, resulting in low resource utilization, severe data fragmentation, and poor flexibility. These problems not only affect the storage efficiency of the system, but may also lead to reduced system performance and increased maintenance costs. Summary of the invention

[0003] In view of this, the present invention provides a disk space management method, device and equipment to solve the problem of disk space management.

[0004] In a first aspect, the present invention provides a disk space management method, the method comprising:

[0005] Divide the disk space into data space, metadata space and configuration space;

[0006] In response to a write request for target data, writing the target data into the data space;

[0007] During the writing process of the target data, the regional utilization rate of the data space is continuously calculated;

[0008] When the area utilization rate of the data space is greater than a preset threshold, a recycling strategy is used to recycle data from the data space until a recycling stop condition is reached, and the target data continues to be written into the data space after data recycling.

[0009] The disk space management method provided by the embodiment of the present invention can achieve more refined management and optimization by dividing the disk space into different functional areas. When there is a request to write target data, the target data is written into the data space, the data writing path is clarified, and it is ensured that the data can be written to the specified area quickly and accurately, thereby improving the efficiency of data writing. During the writing process of the target data, the regional utilization rate of the data space is continuously calculated, and the usage of the disk space is monitored in real time. When the regional utilization rate is greater than a preset threshold, a recovery strategy is adopted to recover data, thereby avoiding data writing failures due to insufficient space, preventing data fragmentation, ensuring efficient utilization of the disk space, and realizing efficient and intelligent management of the disk space.

[0010] In an optional implementation, the data space includes an empty data area, a full data area, a written data area, a periodic data area, and a recycled data area;

[0011] Write the target data into the data space, including:

[0012] In the case where the target data has a life cycle, determining whether the free space in the periodic data area satisfies the target data;

[0013] When the free space in the periodic data area does not satisfy the target data, any first free data group in the free data area is moved to the periodic data area, and the target data is stored in the first free data group; or

[0014] When the free space in the periodic data area satisfies the target data, the target data is stored in the data group in the periodic data area based on the time range of the life cycle of the target data.

[0015] The disk space management method provided by the embodiment of the present invention can achieve more refined management and optimization by dividing the data space into multiple functional areas. Each area has a clear purpose, which helps to improve data access efficiency and reduce data fragmentation. It is also easy to maintain and manage. When writing target data, data with a life cycle is stored in the periodic data area first. When the free space in the periodic data area is insufficient, the space in the empty data area is dynamically adjusted to ensure smooth data writing, avoid writing failures due to insufficient space, reduce the possibility of data fragmentation, and improve the overall utilization of disk space.

[0016] In an optional implementation, writing the target data into the data space further includes:

[0017] In the case that the target data does not have a life cycle, determining whether the free space in the write data area satisfies the target data;

[0018] When the free space in the write data area does not satisfy the target data, any second free data group in the free data area is moved to the write data area, and the target data is stored in the second free data group; or

[0019] When the free space in the write data area satisfies the target data, the target data is stored in the data group of the write data area.

[0020] The disk space management method provided by the embodiment of the present invention first determines whether the free space in the write data area meets the needs of the target data when the target data does not have a life cycle, and gives priority to using the write data area. When the free space in the write data area is insufficient, the space of the empty data area is dynamically adjusted to ensure smooth data writing, avoid writing failures due to insufficient space, and improve the overall utilization of the disk space.

[0021] In an optional embodiment, the method further includes:

[0022] When any data group is full, the data group is moved to the full data area.

[0023] The disk space management method provided by the embodiment of the present invention avoids these data groups occupying space in other areas by timely moving the full data groups to the full data area, thereby improving space utilization.

[0024] In an optional implementation, when the area utilization of the data space is greater than a preset threshold, a recycling strategy is used to recycle data from the data space until a recycling stop condition is reached, including:

[0025] For each data group in the filled data area, determining a valid data block in the data group;

[0026] Determine the life cycle of a valid data block based on the current date, the full write time of the valid data block, and the life cycle;

[0027] Determine the life cycle of the data group based on the life cycle of all valid data blocks in the data group;

[0028] Determine a recycling priority based on the life cycle duration of all data groups in the fully written data area, and determine a recycled data group based on the recycling priority;

[0029] determining a target data group from all data groups in the recovered data area;

[0030] Data recovery is performed based on the recovered data group and the target data group, and the above data recovery process is repeated until the recovery stop condition is reached.

[0031] The disk space management method provided by the embodiment of the present invention ensures that the disk space is effectively utilized by initiating a recycling strategy when the regional utilization rate of the data space exceeds a preset threshold value, determines the valid data blocks of each data group in the fully written data area, thereby determining the life cycle length of each valid data block, and calculates the recycling priority of the data group accordingly, ensuring that the data groups with the least valid data blocks and the least likely to be deleted are recycled first, and realizes efficient and intelligent management of the disk space by repeatedly executing the data recycling process until the recycling stop condition is reached.

[0032] In an optional implementation, determining the target data group from all data groups in the recovered data area includes:

[0033] Classify all data groups in the recovered data area according to the writing time to obtain multiple target data group sets;

[0034] Based on the life cycle duration of the recycled data group, a target data set to which the life cycle duration belongs is determined, and the target data group is determined from the target data group set.

[0035] The disk space management method provided by the embodiment of the present invention can classify all data groups in the recovered data area according to the write time, so that data groups with similar write times can be grouped into the same set, thereby simplifying the recovery process, improving the recovery efficiency, and determining the target data group from the corresponding target data group set based on the life cycle duration of the recovered data group, thereby ensuring the accuracy of recovery and further improving space utilization.

[0036] In an optional implementation, data recovery is performed based on the recovery data group and the target data group, including:

[0037] Obtaining the recovery data of all valid data blocks included in the recovery data group;

[0038] Determine whether the free space of the target data group is sufficient to recycle the data;

[0039] When the free space of the target data group is sufficient for the reclaimed data, the reclaimed data is moved to the target data group.

[0040] The disk space management method provided by the embodiment of the present invention ensures that all data that need to be recovered are accurately collected by obtaining the recovered data of all data blocks in the recovered data group, and by judging whether the free space of the target data group meets the demand for recovered data, it can ensure that data movement is only performed when there is enough space in the target data group, and data conflicts caused by insufficient space can be avoided, ensuring the smooth progress of data movement. When the free space of the target data group meets the demand for recovered data, the recovered data is moved to the target data group, ensuring the efficiency and reliability of data migration.

[0041] In an optional implementation, after determining whether the free space of the target data group satisfies the need to recycle data, the method further includes:

[0042] When the free space of the target data group does not satisfy the amount of recycled data, any third free data group in the free data area is added to the target data group, and the recycled data is moved to the third free data group.

[0043] The disk space management method provided by the embodiment of the present invention ensures data recovery by adding the selected free data group to the target data group when the free space of the target data group is insufficient, and moving the recovered data to the expanded target data group.

[0044] In a second aspect, the present invention provides a disk space management device, the device comprising:

[0045] A partitioning module is used to partition the disk space into data space, metadata space and configuration space;

[0046] A writing module, used for writing the target data into the data space in response to a writing request of the target data;

[0047] A calculation module, used for continuously calculating the area utilization rate of the data space during the writing process of the target data;

[0048] The recycling module is used to adopt a recycling strategy to recycle data from the data space when the area utilization rate of the data space is greater than a preset threshold until the recycling stop condition is reached, and continue to write the target data into the data space after the data is recycled.

[0049] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the disk space management method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0050] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the disk space management method of the first aspect or any corresponding embodiment thereof.

[0051] In a fifth aspect, the present invention provides a computer program product, including computer instructions, which are used to enable a computer to execute the disk space management method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 is a flow chart of a disk space management method according to an embodiment of the present invention;

[0054] Figure 2 is a schematic diagram of a disk space according to an embodiment of the present invention;

[0055] Figure 3 is a schematic diagram of data group type conversion according to an embodiment of the present invention;

[0056] Figure 4is a schematic diagram of another data group type conversion according to an embodiment of the present invention;

[0057] Figure 5 is a structural block diagram of a disk space management device according to an embodiment of the present invention;

[0058] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0060] Traditional disk space management methods are often static, and data recovery strategies are usually based on fixed rules, resulting in low resource utilization, severe data fragmentation, and poor flexibility. These problems not only affect the storage efficiency of the system, but may also lead to system performance degradation and increased maintenance costs. The disk space management method provided by the embodiment of the present invention can achieve more sophisticated management and optimization by dividing the disk space into different functional areas. When there is a write request for target data, the target data is written into the data space, the data writing path is clarified, and it is ensured that the data can be written to the specified area quickly and accurately, thereby improving the efficiency of data writing. During the writing process of the target data, the regional utilization of the data space is continuously calculated, and the use of the disk space is monitored in real time. When the regional utilization is greater than the preset threshold, a recovery strategy is used to recover data, thereby avoiding data writing failures due to insufficient space, preventing data fragmentation, ensuring efficient use of the disk space, and realizing efficient and intelligent management of the disk space.

[0061] According to an embodiment of the present invention, a disk space management method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0062] In this embodiment, a disk space management method is provided, which can be used in a storage system including a Key-Value storage engine. Figure 1 is a flow chart of a disk space management method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0063] Step S101, dividing the disk space into data space, metadata space and configuration space. Specifically, the disk may be an SMR (Shingled Magnetic Recording) disk. Figure 2 is a schematic diagram of a disk space according to an embodiment of the present invention, such as Figure 2 As shown, when the disk is formatted, the disk space is divided into three area spaces: data space, metadata space and configuration space. Each space includes one or more data groups of the same size, and each data group includes several data blocks. Among them, the configuration space is used to save space configuration information, such as the data block size specified by the user, the list of data groups occupied by the metadata space, formatting parameters and other information. The metadata space is used to save log and index information. The content saved in the metadata space is in units of records, including name, write time, data storage location, length, modification time, data verification, hash value of this record, etc. Among them, data verification is used to verify the validity of data, and the hash value of the record is used to verify the integrity of this record. The metadata space also includes indexes sorted by name and by position saved using LSM-tree (Log-Structured Merge Tree), which is convenient for record query and space recovery. The data space is used to save data written by the user. If the disk is an SMR disk, the size of the data group is consistent with the physical data group size of the SMR disk, which can be obtained through the management interface of the SMR disk. If it is a normal disk, it can be defined by itself. The data block size can be set according to the business situation during formatting, such as 4KB. The size of the data block must be able to divide the size of the data group. Because the outer track of the disk has better performance than the inner track, 1% of the data groups of the outer track of the disk can be reserved as metadata space. Use the first data group of the disk as the configuration space and the last data group as the backup of the configuration space for failure recovery. By dividing the disk space into different functional areas, more sophisticated management and optimization can be achieved.

[0064] Step S102, in response to a write request for target data, write the target data into the data space. Specifically, when a user wants to store the target data into a disk, the storage system receives the write request for the target data and writes the target data into the data space according to the functional partition of the disk space.

[0065] Step S103, during the writing process of the target data, the regional utilization rate of the data space is continuously calculated. Specifically, the regional utilization rate indicates the proportion of the used data blocks in the data space to all the data blocks in the data space. During the writing process of the target data, the regional utilization rate is continuously calculated to continuously monitor the writing process of the target data and the disk usage, to ensure that the data can be written to the designated area quickly and accurately, and to improve the efficiency of data writing.

[0066] Step S104, when the regional utilization rate of the data space is greater than a preset threshold, a recovery strategy is used to recover data from the data space until the recovery stop condition is reached, and the target data continues to be written into the data space after data recovery. Specifically, the preset threshold is a value pre-stored in the metadata space. When the regional utilization rate of the data space exceeds the preset threshold, the data recovery strategy is started to organize and optimize the data space until the recovery stop condition is reached. After the data recovery operation is completed, the target data continues to be written into the recovered data space. At this point, the regional utilization rate of the data space has been reduced, and more data can be accommodated, thereby avoiding data writing failures due to insufficient space, preventing data fragmentation, and ensuring efficient use of disk space.

[0067] The disk space management method provided by the embodiment of the present invention can achieve more refined management and optimization by dividing the disk space into different functional areas. When there is a request to write target data, the target data is written into the data space, the data writing path is clarified, and it is ensured that the data can be written to the specified area quickly and accurately, thereby improving the efficiency of data writing. During the writing process of the target data, the regional utilization rate of the data space is continuously calculated, and the usage of the disk space is monitored in real time. When the regional utilization rate is greater than a preset threshold, a recovery strategy is adopted to recover data, thereby avoiding data writing failures due to insufficient space, preventing data fragmentation, ensuring efficient utilization of the disk space, and realizing efficient and intelligent management of the disk space.

[0068] In this embodiment, a disk space management method is provided, which can be used in the above storage system. The method specifically includes the following steps:

[0069] Step S201, divide the disk space into data space, metadata space and configuration space. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0070] Step S202, in response to a write request for target data, write the target data into a data space, where the data space includes an empty data area, a full data area, a write data area, a periodic data area, and a reclaimed data area.

[0071] Specifically, the above step S202 includes:

[0072] Step S2021, in the case that the target data has a life cycle, determine whether the free space in the periodic data area satisfies the target data. Specifically, the empty data area contains data groups that have not been used; the full data area contains data groups that have been completely written, and writing cannot continue even if records are deleted; the written data area is used to write data that does not have a life cycle; the periodic data area is used to store data with a life cycle; the recycled data area is used to store data that has been marked as recyclable. When the user wants to write the target data, a life cycle can be set for the target data, that is, the target data is only valid within the life cycle. For target data with a life cycle, the target data is preferentially written to the periodic data area, so it is necessary to determine whether the free space in the periodic data area is sufficient to write the target data.

[0073] Step S2022, when the free space in the periodic data area does not meet the target data, any first free data group in the empty data area is moved to the periodic data area, and the target data is stored in the first free data group. Specifically, if the free space in the periodic data area is insufficient to write the target data, an empty data group, i.e., the first free data group, is requested from the empty data area, and the first free data group is moved to the periodic data area to smoothly write the target data into the periodic data area, thereby ensuring smooth data writing, avoiding writing failures due to insufficient space, reducing the possibility of data fragmentation, and improving the overall utilization of disk space.

[0074] Alternatively, step S2023, when the free space in the periodic data area satisfies the target data, based on the time range of the target data's life cycle, the target data is stored in the data group of the periodic data area. Specifically, if the periodic data area can write the target data, the write location is determined according to the life cycle of the target data. Assuming that the current date is used as a reference, the periodic data area is grouped according to the expiration time from the current date, and four data groups of 0-1 month from now, 1 month from now-3 months from now, and 3 months from now-6 months from now are obtained. If the life cycle of the target data is 2 months, the target data is written to the data group of 1 month from now-3 months from now. Optionally, the above-mentioned grouping method of the periodic data area is only an example and is not intended to be limiting.

[0075] Step S2024, when the target data does not have a life cycle, determine whether the free space in the write data area satisfies the target data. Specifically, when the target data does not have a life cycle, the target data is written into the corresponding write data area, so it is necessary to first determine whether the write data area has enough space.

[0076] Step S2025, when the free space in the write data area does not meet the target data, any second free data group in the empty data area is moved to the write data area, and the target data is stored in the second free data group. Specifically, if the free space in the write data area is insufficient to write the target data, an empty data group, i.e., the second free data group, is requested from the empty data area, and the second free data group is moved to the write data area to smoothly write the target data into the write data area, thereby ensuring smooth data writing, avoiding write failures due to insufficient space, reducing the possibility of data fragmentation, and improving the overall utilization of disk space.

[0077] Alternatively, in step S2026, if the free space in the write data area satisfies the target data, the target data is stored in the data group in the write data area. Specifically, if the write data area can write the target data, the target data may be preferentially written into the data group with less free space.

[0078] Step S203, if any data group is full, move the data group to the full data area. Specifically, during the writing of target data, if any data group is full, move the data group to the corresponding full data area, so as to avoid these full data groups occupying the space of other areas and improve the space utilization.

[0079] Step S204: During the writing process of the target data, the regional utilization rate of the data space is continuously calculated. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0080] Step S205, when the area utilization rate of the data space is greater than a preset threshold, a recovery strategy is used to recover data from the data space until a recovery stop condition is reached, and the target data continues to be written into the data space after data recovery.

[0081] Specifically, the above step S205 includes:

[0082] Step S2051, for each data group in the full data area, determine the valid data blocks in the data group. Specifically, the full data area includes multiple data groups, each data group includes multiple data blocks, and the data blocks in each data group are checked to identify the valid data blocks. The earlier the writing time of the valid data block, the colder it is. For the valid data block with a known life cycle, the closer the current date is to the specified deletion date, the hotter it is. By determining the valid data block, the valid data block can be recycled to release disk space.

[0083] Step S2052: Determine the life cycle duration of the valid data block based on the current date, the write full time of the valid data block, and the life cycle. Specifically, if the valid data block does not have a life cycle, the life cycle duration of the valid data block is calculated by the following formula (1). If the valid data block has a life cycle, the life cycle duration of the valid data block is calculated by the following formula (2).

[0084] day = currentDate - writeDate (1)

[0085] Where day represents the life cycle duration; currentDate represents the current date; writeDate represents the date when the valid data block is written full.

[0086] day = |currentDate - writeDate – Max(lifeCycleDays)| (2)

[0087] Where day represents the life cycle duration; currentDate represents the current date; writeDate represents the date when the valid data block is written full; lifeCycleDays represents the life cycle. Since the life cycle is a time range, Max(lifeCycleDays) is the upper limit of the life cycle time range.

[0088] Step S2053: Determine the life cycle duration of the data group based on the life cycle durations of all valid data blocks in the data group. Specifically, since a data group usually includes at least one valid data block, the life cycle durations of different valid data blocks may be different. For any data group, the longest life cycle duration among all valid data blocks included in the data group is used as the life cycle duration of the data group.

[0089] Step S2054: Determine the recycling priority based on the life cycle durations of all data groups in the written full data area, and determine the data group to be recycled based on the recycling priority. Specifically, assume there are multiple time classifications: 0 days - day1 days, day1 days - day2 days, day2 days - day3 days,..., dayn days and above, corresponding to recycling coefficients t1, t2, t3,..., tn respectively, and 0 < t1 < t2 < t3 <... < tn < 1. For any data group, determine the recycling coefficient of the data group according to the time classification to which the life cycle duration of the data group belongs. Then, determine the recycling priority of the data group through the following formula (3).

[0090]

[0091] Where x represents the recycling priority of any data group; k represents the number of valid data blocks in the data group; K represents the number of all data blocks in the data group; t final Represents the recovery factor for this data set.

[0092] The above formula (3) can be used to obtain the recycling priority of all data groups in the full data area. The larger the value of x, the higher the recycling priority, that is, the earlier it is recycled. Generally, the data group with the largest value of x, that is, the data group with the highest recycling priority, is used as the recycling data group for subsequent data recycling. By determining the recycling data group based on the recycling priority, it is ensured that the data group with the least valid data blocks and the least likely to be deleted is recycled first, thereby achieving efficient management of disk space.

[0093] Step S2055, determining the target data group from all data groups in the recovered data area.

[0094] In some optional implementations, the above step S2055 includes:

[0095] Step a1, classify all data groups in the recovered data area according to the write time, and obtain multiple target data group sets. Specifically, when the storage system is performing data recovery, it can select data blocks from the recovered data area to carry out data recovery, and the recovered data group is located in the full data area, and the data that needs to be recovered in the recovered data group needs to be transferred to the recovered data area to realize data recovery. All data groups in the recovered data area are classified according to the write time, for example, 0 days-4 days, 5 days-16 days, 17 days-64 days, 65 days-256 days, 257 days and above 5 target data group sets can be obtained. Optionally, when calculating the write time of any data group, since the data group may include multiple data blocks, the earliest write time, the latest write time or the average write time in all data blocks can be used as the write time of the data group, and the embodiment of the present invention does not limit this.

[0096] Step a2, based on the life cycle duration of the recovery data group, determine the target data set to which the life cycle duration belongs, and determine the target data set from the target data set. Specifically, assuming that the life cycle duration of the recovery data group is 20 days, corresponding to the target data set of 17 days-64 days, select the target data set from all data groups included in the target data set of 17 days-64 days. Optionally, the write time consistent with the life cycle duration can be selected as the target data group, or the target data set with the longest write time in the target data set of 17 days-64 days can be selected as the target data group, and other methods can also be used to determine the target data group, which is not limited by the embodiment of the present invention. By classifying all data groups in the recovery data area according to the write time, the data groups with similar write times can be classified as the same set, simplifying the recovery process, improving recovery efficiency, determining the target data group from the corresponding target data set based on the life cycle duration of the recovery data group, ensuring the accuracy of recovery, and further improving space utilization.

[0097] Step S2056, performing data recovery based on the recovered data group and the target data group, repeating the above data recovery process until the recovery stop condition is reached.

[0098] In some optional implementations, the above step S2056 performs data recovery based on the recovery data group and the target data group, including:

[0099] Step b1, obtaining the recovered data of all valid data blocks included in the recovered data group. Specifically, obtaining all recovered data to be recovered from the recovered data group.

[0100] Step b2, determining whether the free space of the target data group is sufficient for the recycled data. Specifically, for data recycling, the recycled data in the full data area needs to be transferred to the recycled data area, so it is necessary to determine whether the target data group in the recycled data area is sufficient to write the recycled data.

[0101] Step b3, if the free space of the target data group is sufficient for the recovered data, move the recovered data to the target data group. Specifically, if the space of the target data group can write the recovered data, move the recovered data to the target data group so that the data can be recovered when the storage system performs data recovery.

[0102] Step b4, when the free space of the target data group does not meet the requirements for the recovered data, any third free data group in the free data area is added to the target data group, and the recovered data is moved to the third free data group. Specifically, if the free space of the target data group is insufficient to write the recovered data, an empty data group, i.e., the third free data group, is applied to the free data area, and the third free data group is added to the target data group, so that the recovered data is smoothly written into the recovered data area, thereby ensuring the progress of data recovery.

[0103] In some optional implementations, after the data movement is completed, the data is written to the disk, the data location pointed to by the record stored in the metadata area is modified to a new location, and the metadata record is written to the disk.

[0104] In some optional implementations, after data recovery is performed based on the above steps, it is determined whether the recovery stop condition is reached. If the recovery stop condition is not reached, the process returns to step S2054, and a new recovery data group and its corresponding target data group are continuously determined to continue data recovery until the recovery stop condition is reached. Wherein, the recovery stop condition may be that the sum of the area utilization and the busy factor is greater than the effective data rate. Wherein, the area utilization is the area utilization of the data area, that is, the proportion of the used data blocks to all data blocks in the data space. The busy factor takes a positive value when the disk reads and writes idle, and takes a negative value when the disk reads and writes busy. By dynamically adjusting the busy factor, more space is recovered when the disk is idle. The effective data rate is calculated by the following formula (4). Optionally, the recovery stop condition may also be that the area utilization is not greater than the data recovery threshold preset by the user.

[0105]

[0106] Among them, p represents the effective data rate; totalUsedBlockSize represents the total length of the used data blocks; fullZoneCount represents the number of full data groups; ZoneSize represents the size of the data group.

[0107] In some optional implementations, the embodiments of the present invention may also recycle the data groups in the reclaimed data area in the order of most available space to least available space until the target data is successfully written into the data space.

[0108] In some optional embodiments, Figure 3 is a schematic diagram of data group type conversion according to an embodiment of the present invention, such as Figure 3 As shown, if the metadata space is insufficient, the empty data area of ​​the data space can be transferred to the metadata space. Conversely, if the metadata space is too large, the data group in the metadata space can also be dynamically transferred to the empty data area of ​​the data space. Figure 4is a schematic diagram of another data group type conversion according to an embodiment of the present invention, such as Figure 4 As shown, when the data groups in the periodic data area and the write data area are full, the full data groups can be transferred to the full data area. In particular, when the valid data groups in the reclaimed data area are full, they can also be transferred to the full data area. When the reclaimed data in the full data area is to be written into the reclaimed data area, if the space in the reclaimed data area is insufficient, the empty data area can transfer the empty data groups to the reclaimed data area. When there is data to be written into the periodic data area or the write data area, if the free space in these two areas is insufficient to write the data, the free data groups can be transferred from the empty data area to these two areas. When there is data in the full data area that is transferred to the reclaimed data area for data recovery, there may be empty data groups after the data transfer, and the empty data groups can be transferred to the empty data area.

[0109] The disk space management method provided by the embodiment of the present invention can achieve more refined management and optimization by dividing the disk space into different functional areas. When there is a request to write target data, the target data is written into the data space, the data writing path is clarified, and it is ensured that the data can be written to the specified area quickly and accurately, thereby improving the efficiency of data writing. During the writing process of the target data, the regional utilization rate of the data space is continuously calculated, and the usage of the disk space is monitored in real time. When the regional utilization rate is greater than a preset threshold, a recovery strategy is adopted to recover data, thereby avoiding data writing failures due to insufficient space, preventing data fragmentation, ensuring efficient utilization of the disk space, and realizing efficient and intelligent management of the disk space.

[0110] In this embodiment, a disk space management device is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0111] This embodiment provides a disk space management device, such as Figure 5 As shown, including:

[0112] The partitioning module 501 is used to partition the disk space into data space, metadata space and configuration space.

[0113] The writing module 502 is used to write the target data into the data space in response to a write request of the target data.

[0114] The calculation module 503 is used to continuously calculate the area utilization rate of the data space during the writing process of the target data.

[0115] The recycling module 504 is used to adopt a recycling strategy to recycle data from the data space when the area utilization rate of the data space is greater than a preset threshold value until a recycling stop condition is reached, and continue to write the target data into the data space after data recycling.

[0116] In some optional implementations, the data space includes an empty data area, a full data area, a written data area, a periodic data area, and a recycled data area;

[0117] The writing module 502 includes:

[0118] The first judgment unit is used to judge whether the free space in the periodic data area satisfies the target data when the target data has a life cycle.

[0119] The first storage unit is used to move any first free data group in the free data area to the periodic data area and store the target data in the first free data group when the free space in the periodic data area does not satisfy the target data.

[0120] Alternatively, the second storage unit is used to store the target data into the data group of the periodic data area based on the time range of the life cycle of the target data when the free space of the periodic data area satisfies the target data.

[0121] In some optional implementations, the writing module 502 further includes:

[0122] The second judgment unit is used to judge whether the free space in the write data area satisfies the target data when the target data does not have a life cycle.

[0123] The third storage unit is used to move any second free data group in the free data area to the write data area and store the target data in the second free data group when the free space in the write data area does not satisfy the target data.

[0124] Alternatively, the fourth storage unit is used to store the target data into the data group of the write data area when the free space of the write data area satisfies the target data.

[0125] In some optional embodiments, the device further comprises:

[0126] The first moving module is used to move the data group to the full data area when any data group is full.

[0127] In some optional embodiments, the recovery module 504 includes:

[0128] The first determining unit is used to determine, for each data group in the fully written data area, a valid data block in the data group.

[0129] The second determining unit is used to determine the life cycle of the valid data block based on the current date, the full writing time of the valid data block and the life cycle.

[0130] The third determination unit is used to determine the life cycle duration of the data group based on the life cycle durations of all valid data blocks in the data group.

[0131] The fourth determining unit is used to determine a recycling priority based on the life cycle duration of all data groups in the fully written data area, and to determine a recycled data group based on the recycling priority.

[0132] The fifth determining unit is used to determine a target data group from all data groups in the recovered data area.

[0133] The recycling unit is used to perform data recycling based on the recycling data group and the target data group, and repeat the above data recycling process until the recycling stop condition is reached.

[0134] In some optional implementations, the fifth determining unit includes:

[0135] The first determination subunit is used to classify all data groups in the recovered data area according to the writing time to obtain multiple target data group sets.

[0136] The second determining subunit is used to determine the target data set to which the life cycle duration belongs based on the life cycle duration of the recycled data group, and determine the target data group from the target data group set.

[0137] In some optional embodiments, the recovery unit comprises:

[0138] The acquisition subunit is used to acquire the recovered data of all valid data blocks included in the recovered data group.

[0139] The judgment subunit is used to judge whether the free space of the target data group is sufficient for data recovery.

[0140] The moving subunit is used to move the recovered data to the target data group when the free space of the target data group is sufficient for the recovered data.

[0141] In some optional implementations, after the determination subunit, the device further includes:

[0142] The second moving module is used to add any third free data group in the free data area to the target data group and move the recovered data to the third free data group when the free space of the target data group does not meet the requirements of the recovered data.

[0143] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0144] The disk space management device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0145] The embodiment of the present invention also provides a computer device having the above Figure 5 The Disk Space Management Appliance shown.

[0146] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0147] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0148] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0149] The memory 20 may include a program storage area and a data space, wherein the program storage area may store an operating system, an application required for at least one function; the data space may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0150] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0151] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0152] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0153] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0154] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A disk space management method, characterized in that: The method comprises: Divide the disk space into data space, metadata space and configuration space; In response to a write request for target data, writing the target data into the data space; During the writing process of the target data, continuously calculating the area utilization rate of the data space; When the area utilization rate of the data space is greater than a preset threshold, a recovery strategy is adopted to recover data from the data space until a recovery stop condition is reached, and the target data continues to be written into the data space after data recovery.

2. The method according to claim 1, characterized in that The data space includes an empty data area, a full data area, a written data area, a periodic data area and a recycled data area; The step of writing the target data into the data space comprises: In the case where the target data has a life cycle, determining whether the free space in the periodic data area satisfies the target data; When the free space in the periodic data area does not satisfy the target data, any first free data group in the free data area is moved to the periodic data area, and the target data is stored in the first free data group; or When the free space of the periodic data area satisfies the target data, the target data is stored in the data group of the periodic data area based on the time range of the life cycle of the target data.

3. The method according to claim 2, characterized in that The step of writing the target data into the data space further includes: In the case that the target data does not have a life cycle, determining whether the free space of the data writing area satisfies the target data; When the free space in the write data area does not satisfy the target data, any second free data group in the free data area is moved to the write data area, and the target data is stored in the second free data group; or When the free space of the data writing area satisfies the target data, the target data is stored in the data group of the data writing area.

4. The method according to any one of claims 2 or 3, characterized in that: The method further comprises: When any data group is full, the data group is moved to the full data area.

5. The method according to claim 3, characterized in that: When the area utilization rate of the data space is greater than a preset threshold, adopting a recycling strategy to recycle data from the data space until a recycling stop condition is reached, including: For each data group in the fully written data area, determining a valid data block in the data group; Determine the life cycle of the valid data block based on the current date, the full writing time and the life cycle of the valid data block; Determining the life cycle duration of the data group based on the life cycle durations of all valid data blocks in the data group; Determine a recycling priority based on the life cycle duration of all data groups in the fully written data area, and determine a recycled data group based on the recycling priority; Determine a target data group from all data groups in the recovered data area; Data recovery is performed based on the recovered data group and the target data group, and the above data recovery process is repeated until the recovery stop condition is reached.

6. The method according to claim 5, characterized in that The step of determining a target data group from all data groups in the recovered data area comprises: Classifying all data groups in the recovered data area according to write time to obtain multiple target data group sets; Based on the life cycle duration of the recycled data group, a target data set to which the life cycle duration belongs is determined, and the target data group is determined from the target data set.

7. The method according to claim 5, characterized in that The performing data recovery based on the recovered data group and the target data group includes: Acquire the recovered data of all valid data blocks included in the recovered data group; Determine whether the free space of the target data group satisfies the reclaimed data; When the free space of the target data group is sufficient for the recycled data, the recycled data is moved to the target data group.

8. The method according to claim 7, characterized in that After determining whether the free space of the target data group satisfies the requirement for recovering data, the method further includes: When the free space of the target data group does not satisfy the recycled data, any third free data group in the free data area is added to the target data group, and the recycled data is moved to the third free data group.

9. A disk space management device, characterized in that: The device comprises: A partitioning module is used to partition the disk space into data space, metadata space and configuration space; A writing module, configured to write the target data into the data space in response to a writing request of the target data; A calculation module, used for continuously calculating the area utilization rate of the data space during the writing process of the target data; The recycling module is used to adopt a recycling strategy to recycle data from the data space when the area utilization rate of the data space is greater than a preset threshold until a recycling stop condition is reached, and continue to write the target data into the data space after data recycling.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the disk space management method according to any one of claims 1 to 8 by executing the computer instructions.

Images