Space management method and device of hard disk, electronic equipment and readable medium

Abstract

The embodiment of the invention provides a space management method and device for a hard disk, electronic equipment and a readable medium, and is applied to the hard disk, the hard disk comprises at least one logic storage area, the logic storage area of the hard disk is emptied, and the storage use state and / or the storage data state of the logic storage area are / is stored by utilizing a preset database; metadata of the logic storage area is a data block; in the process of writing the preset to-be-written data block into the hard disk, judging whether a target logic storage area exists in the hard disk or not; data related to the to-be-written data block is stored in the target logic storage area; if yes, writing the to-be-written data block into the target logic storage area, and updating a storage use state and / or a storage data state of the target logic storage area in the database; on the basis of the database, the junk data blocks in any logic storage area in the hard disk are cleared, and the implementation strategy of the FTL of the zns disk and the automatic implementation of the function of the GC are achieved.

Description

Technical Field

[0001] The present invention relates to the field of hard disk technology, and in particular to a hard disk space management method, a hard disk space management device, an electronic device and a computer-readable medium. Background Art

[0002] A solid-state drive (SSD) usually reserves 15% of the reserved space for GC (Garbage Collection) and bad block replacement of the SSD disk. This reserved space is invisible to users.

[0003] ZNS (Zone Namespace SSD) is a special SSD disk that exposes reserved space to users, who can directly manage this space.

[0004] When users use the reserved space in the ZNS disk, they need to manage the disk space by themselves, implement the FTL (Flash Translate layer) strategy and GC function, and the threshold for use is relatively high. It should be noted that the FTL strategy refers to converting the user's logical lba (Logical Block Addressing) address into the physical location of the disk, that is, finding the location of the data to be operated. Summary of the invention

[0005] The embodiments of the present invention provide a hard disk space management method, device, electronic device and computer-readable storage medium to solve the problem that when users use the reserved space in the ZNS disk, they need to manage the disk space by themselves, implement the FTL (Flash Translate layer) strategy and GC function, and have a high threshold for use.

[0006] An embodiment of the present invention discloses a hard disk space management method, which is applied to a hard disk, wherein the hard disk includes at least one logical storage area, and the method includes:

[0007] Clearing the logical storage area of ​​the hard disk, and using a preset database to store the storage usage status and / or storage data status of the logical storage area; the metadata of the logical storage area is a data block;

[0008] In the process of writing a preset data block to be written into the hard disk, determining whether there is a target logical storage area in the hard disk; the target logical storage area stores data related to the data block to be written;

[0009] If it exists, writing the data block to be written into the target logical storage area, and updating the storage usage status and / or storage data status of the target logical storage area in the database;

[0010] Based on the database, garbage data blocks in any of the logical storage areas in the hard disk are cleared.

[0011] Optionally, the storage usage state includes an idle state; when the logical storage area is cleared, the logical storage area is in an idle state; and the method includes:

[0012] If the target logical storage area does not exist in the hard disk, the data block to be written is written into any of the logical storage areas in the idle state.

[0013] Optionally, the storage usage status further includes an active status; and updating the storage usage status and / or storage data status of the target logical storage area in the database includes:

[0014] In the process of writing the to-be-written data block into the target logical storage area, updating the storage usage state of the target logical storage area from the idle state to the active state;

[0015] After the to-be-written data block is written into the target logical storage area, the storage data state in the target logical storage area is updated.

[0016] Optionally, the storage data state includes a clearing identifier of at least one data block; and clearing the garbage data blocks in any of the logical storage areas in the hard disk based on the database includes:

[0017] Using the logical storage area in the hard disk where the garbage data blocks exist as the logical storage area to be processed;

[0018] By modifying the clearing identifier of the junk data block, the clearing identifier of the junk data block indicates that the junk data block is a data block to be cleared;

[0019] Based on the clearing identifier, obtaining information on the proportion of data blocks to be cleared in the to-be-processed logical storage area;

[0020] Based on the to-be-cleared data block ratio information, it is determined whether to clear the to-be-cleared data block.

[0021] Optionally, judging whether to clear the data blocks to be cleared based on the ratio information of the data blocks to be cleared includes:

[0022] Based on the to-be-cleared data block ratio information, determining whether the ratio of the to-be-cleared data blocks in the to-be-processed logical storage area exceeds a preset ratio threshold;

[0023] If the proportion of the to-be-cleared data blocks in the to-be-processed logical storage area exceeds a preset proportion threshold, the to-be-cleared data blocks are cleared.

[0024] Optionally, clearing the data block to be cleared includes:

[0025] Based on the clearing identifier of the data block, determining a target data block in the to-be-processed logical storage area that is not the to-be-cleared data block;

[0026] Transferring the target data block to other logical storage areas in the hard disk except the to-be-processed logical storage area;

[0027] Clear the logic storage area to be processed.

[0028] Optionally, the method comprises:

[0029] After the target data block is transferred to other logical storage areas in the hard disk, modifying the storage data state of the logical storage area that receives the target data block;

[0030] After the to-be-processed logical storage area is cleared, the storage usage state and / or the storage data state of the to-be-processed logical storage area is modified.

[0031] The embodiment of the present invention further discloses a hard disk space management device, which is applied to a hard disk, wherein the hard disk includes at least one logical storage area, and the device includes:

[0032] a clearing module, used to clear the logical storage area of ​​the hard disk, and store the storage usage status and / or storage data status of the logical storage area using a preset database; the metadata of the logical storage area is a data block;

[0033] A target logical storage area determination module is used to determine whether there is a target logical storage area in the hard disk during the process of writing a preset data block to be written into the hard disk; the target logical storage area stores data related to the data block to be written;

[0034] A first writing module, configured to write the data block to be written into the target logical storage area if it exists, and update the storage usage status and / or storage data status of the target logical storage area in the database;

[0035] A clearing module is used to clear junk data blocks in any of the logical storage areas in the hard disk based on the database.

[0036] Optionally, the storage usage state includes an idle state; when the logical storage area is cleared, the logical storage area is in an idle state; and the device includes:

[0037] The second writing module is used for writing the data block to be written into any of the logical storage areas in the idle state if the target logical storage area does not exist in the hard disk.

[0038] Optionally, the storage usage status further includes an active status; and the first writing module includes:

[0039] A first state updating submodule, configured to update the storage usage state of the target logical storage area from the idle state to the active state during the process of writing the to-be-written data block into the target logical storage area;

[0040] The second status updating submodule is used to update the storage data status in the target logical storage area after the to-be-written data block is written into the target logical storage area.

[0041] Optionally, the storage data state includes a clearing identifier of at least one data block; and the clearing module includes:

[0042] The to-be-processed logical storage area is used as a submodule, and is used to use the logical storage area in the hard disk where the garbage data block exists as the to-be-processed logical storage area;

[0043] A modification submodule, configured to modify the removal identifier of the junk data block so that the removal identifier of the junk data block indicates that the junk data block is a data block to be removed;

[0044] A submodule for obtaining information on the proportion of data blocks to be cleared, used for obtaining information on the proportion of data blocks to be cleared in the to-be-processed logical storage area based on the clearing identifier;

[0045] The clearing judgment submodule is used to judge whether to clear the data blocks to be cleared based on the ratio information of the data blocks to be cleared.

[0046] Optionally, the clearing judgment submodule includes:

[0047] A threshold judgment unit, configured to judge whether the proportion of the data blocks to be cleared in the to-be-cleared logical storage area exceeds a preset proportion threshold based on the proportion information of the data blocks to be cleared;

[0048] The clearing unit is configured to clear the data blocks to be cleared if the proportion of the data blocks to be cleared in the logical storage area to be processed exceeds a preset proportion threshold.

[0049] Optionally, the clearing unit comprises:

[0050] A target data block determination subunit, configured to determine a target data block in the to-be-processed logical storage area that is not the to-be-cleared data block based on the clearing identifier of the data block;

[0051] A transfer subunit, used for transferring the target data block to other logical storage areas in the hard disk except the to-be-processed logical storage area;

[0052] The clearing subunit is used to clear the to-be-processed logical storage area.

[0053] Optionally, the device comprises:

[0054] A first state modification module, configured to modify the storage data state of the logical storage area that receives the target data block after the target data block is transferred to other logical storage areas in the hard disk;

[0055] The second state modification module is used to modify the storage usage state and / or the storage data state of the to-be-processed logical storage area after the to-be-processed logical storage area is cleared.

[0056] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0057] The memory is used to store computer programs;

[0058] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0059] The embodiment of the present invention further discloses one or more computer-readable media on which instructions are stored. When executed by one or more processors, the processors are enabled to execute the method described in the embodiment of the present invention.

[0060] The embodiments of the present invention include the following advantages:

[0061] In the embodiment of the present invention, the hard disk includes at least one logical storage area, the logical storage area of ​​the hard disk is emptied, and the storage usage status and / or storage data status of the logical storage area are stored in a preset database; the metadata of the logical storage area is a data block; in the process of writing the preset data block to be written into the hard disk, it is determined whether there is a target logical storage area in the hard disk; the target logical storage area stores data related to the data block to be written; if so, the data block to be written is written into the target logical storage area, and the storage usage status and / or storage data status of the target logical storage area in the database is updated; based on the database, the garbage data blocks in any logical storage area of ​​the hard disk are cleared. A zns disk space management solution based on rocksdb is provided, which realizes the FTL implementation strategy of the zns disk and the automatic implementation of the GC function. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is a flowchart of the steps of a hard disk space management method provided in an embodiment of the present invention;

[0063] Figure 2 is a schematic diagram of the relationship between a zone and a chunk provided in an embodiment of the present invention;

[0064] Figure 3 is a state diagram of a logical storage area provided in an embodiment of the present invention;

[0065] Figure 4 It is a schematic diagram of writing a data block to be written into a target logical storage area provided in an embodiment of the present invention;

[0066] Figure 5 It is a schematic diagram of a flow chart of a GC function provided in an embodiment of the present invention;

[0067] Figure 6 is a structural block diagram of a hard disk space management device provided in an embodiment of the present invention;

[0068] Figure 7 is a block diagram of an electronic device provided in an embodiment of the present invention;

[0069] Figure 8 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0071] To facilitate understanding of the technical solutions and technical effects of the embodiments of the present invention, the prior art of the present invention is briefly described below.

[0072] Solid-state drives (SSDs) have been widely used in the storage field. SSDs usually reserve 15% of their reserved space for GC (Garbage Collection) and bad block replacement, and this reserved space is invisible to users.

[0073] ZNS (Zone Namespace SSD) is a special SSD disk that exposes reserved space to users, who can directly manage this space.

[0074] The ZNS disk includes multiple zones (logical storage areas). It should be noted that before writing data to a zone, it must be opened first. The space of a standard zone can be more than ten GB (gigabytes) or several hundred MB (megabytes).

[0075] In a ZNS disk, each independent zone allows data to be written in an appended manner, that is, data can only be added to the end of the zone, rather than overwriting existing data. At the same time, the zone supports random read operations, that is, users can randomly read data from any location in the zone. Data writing within the zone must be done in order.

[0076] When users use the reserved space in the ZNS disk, they need to manage the disk space by themselves, implement the FTL (Flash Translate layer) strategy and GC function, and the threshold for use is relatively high. It should be noted that the FTL strategy refers to converting the user's logical lba (Logical Block Addressing) address into the physical location of the disk, that is, finding the location of the data to be operated.

[0077] Reference Figure 1 , shows a flowchart of the steps of a hard disk space management method provided in an embodiment of the present invention, which is applied to a hard disk, wherein the hard disk includes at least one logical storage area, and specifically may include the following steps:

[0078] Step 101, clearing the logical storage area of ​​the hard disk, and using a preset database to store the storage usage status and / or storage data status of the logical storage area; the metadata of the logical storage area is a data block;

[0079] In the embodiment of the present invention, the ZNS hard disk includes at least one logical storage area.

[0080] Reference Figure 2, shows a schematic diagram of the relationship between a zone and a chunk provided in an embodiment of the present invention. The ZNS hard disk manages metadata at the granularity of a zone. A zone can be composed of multiple chunks, so the metadata of a zone is composed of the metadata of multiple chunks; that is, the metadata unique data block of the logical storage area. The details are as follows:

[0081]

[0082]

[0083] Among them, zone_id is the unique identifier of the logical storage area, chunk_id is the unique identifier of a chunk, and chunk_len represents the length of this chunk. offset_of_zone represents the offset position of this chunk in the zone. is_purged represents whether this chunk is cleared (cleared means that the space of this chunk can be released), is_sealed represents whether this chunk is sealed (sealed means that this chunk can no longer be written), empty means empty, and append data usually means adding new content to the end of the file without overwriting the existing data in the file. In an embodiment of the present invention, a preset database can be used to store the storage usage status and / or storage data status of the logical storage area. Among them, the storage usage status refers to whether the logical storage area is in an idle state, an active state, or a full state, and the storage data status refers to the length of the data block in the logical storage area and the clear identifier of the data block.

[0084] In the embodiment of the present invention, all zones are reset (reset operation) to clean up the original data. That is, the zones are formatted. After formatting, the metadata of each zone is empty, and these empty metadata are stored in a database called RocksDB. RocksDB is a high-performance embedded Key-Value storage database used to store and manage the metadata of the zones.

[0085] In a ZNS disk, a zone can only be written by one data block at a time. This is because in a ZNS SSD, data append writes are performed by zone, and a zone cannot be interrupted by other data blocks during the writing process. Only when the chunk is sealed can the zone be written by other chunks. Sealed means that when a chunk completes the write operation, it will be "sealed", which means that the data of this chunk is finalized and will not be modified again.

[0086] Reference Figure 3 , showing a state diagram of a logical storage area provided in an embodiment of the present invention.

[0087] In this solution, the zone status is divided into three types: idle: idle status, indicating that this zone is not currently written by any chunk and can be written by new chunks. Active: active status, indicating that this zone is being written by a chunk and cannot be written by other chunks. Full: full status, indicating that the space of this zone is full and cannot be written by any chunk.

[0088] In the embodiment of the present invention, these empty metadata are stored in a database named RocksDB, that is, the empty metadata is persisted in the database. A new record is created in the RocksDB database, which is used to store metadata related to a data block (such as a chunk), but currently this metadata does not contain any actual data content or information, it is empty or a placeholder.

[0089] Specifically, this process may include the following steps:

[0090] Define the key: First, you need to define a unique key, which will be used to identify this metadata record in RocksDB. The key is usually a string or binary sequence that can uniquely represent the data block associated with it.

[0091] Create an empty value: Then, create an empty value, which may be an empty string, empty array, empty object, or other data structure representing "no content".

[0092] Write to RocksDB: Write this key and null value as a key-value pair to RocksDB. This process is called "persistence" because it means that the data is saved to non-volatile storage and will not be lost even if the system is restarted.

[0093] In a specific example, the process of clearing a zone is as follows: First, call the zns disk interface to obtain a list of all zones. Then call the zone reset interface to clear the zone data. Persist the metadata of each zone in rocksdb, and after formatting, the metadata of each zone is empty. At this time, all zones are added to the idle state.

[0094] Step 102, in the process of writing a preset data block to be written into the hard disk, determining whether there is a target logical storage area in the hard disk; the target logical storage area stores data related to the data block to be written;

[0095] In the embodiment of the present invention, when writing a preset data block to be written into a hard disk, it is determined whether there is a target logical storage area in the hard disk; the target logical storage area stores data related to the data block to be written.

[0096] Step 103: if it exists, write the data block to be written into the target logical storage area, and update the storage usage status and / or storage data status of the target logical storage area in the database;

[0097] In the embodiment of the present invention, if the target logical storage area exists in the hard disk, the data block to be written is written into the target logical storage area, and the storage usage status and / or storage data status of the target logical storage area in the database is updated.

[0098] In some embodiments of the present invention, the storage usage state includes an idle state; when the logical storage area is cleared, the logical storage area is in an idle state; and the method includes:

[0099] If the target logical storage area does not exist in the hard disk, the data block to be written is written into any of the logical storage areas in the idle state.

[0100] Reference Figure 4 , showing a schematic diagram of writing a data block to be written into a target logical storage area provided in an embodiment of the present invention.

[0101] For read and write operations, the business logic is not aware of the specific zone to which the data is written. Two functions, append and read, are defined, which write and read data in units of chunks.

[0102] int append(uint64_t chunk_id,void*data,uint64_t data_length); This function is used to append data to a specified chunk. chunk_id is the unique identifier of the chunk, data is the pointer to the data to be written, and data_length is the length of the data. The function returns an integer that may indicate the success or error code of the operation. uint64_t is a data type that represents an unsigned 64-bit integer. void* is a pointer type that can point to any type of data.

[0103] int read(uint64_t chunk_id,void*data,uint64_t read_length); This function is used to read data from the specified chunk. chunk_id is the unique identifier of the chunk, data is a pointer to the buffer to store the read data, and read_length is the length of the data to be read. The function returns an integer that may indicate the success of the operation or an error code.

[0104] In the embodiment of the present invention, these two functions are prepared for GC. GC is a mechanism for SSD to manage storage space. Since GC operations may move data blocks (chunks) to different zones, a way is needed to abstract and mask the specific physical location of the data. That is, since the location of the chunk may change, the upper-level application or business logic should not depend on the specific physical location of the data block. Instead, the system should provide an abstraction so that the application can access the data through the chunk identifier (such as chunk_id) without knowing where the data is actually stored on the hard disk. This abstraction is called "shielding", which hides the underlying implementation details and makes the upper-level application simpler and more stable. The system provides an abstraction layer for the upper-level application so that the application does not need to care about which zone of the hard disk the data is stored when accessing the data. The application only needs to use the chunk identifier (chunk_id) to request access to the data, and the system will be responsible for finding and returning the correct data based on this identifier.

[0105] When a write request for a chunk is received, the following are the steps to process the write request: Before writing the data, the system checks whether a zone has been assigned to the chunk. If a zone has not been assigned to the chunk, the system gets a new zone from the idle zone list to store the chunk.

[0106] In a specific example, the process of writing data to the hard disk is as follows: the user calls the Append(uint64_tchunk_id,void*data,uint64_t data_len) interface, first querying whether there is already a zone storing the data of this chunk, if so, continue to write to this zone. After the write is successful, update the length of the chunk in this zone, and then persist it to rocksdb. If this chunk is written for the first time, you need to get a zone from the idle list of the zone, then write the chunk to the zone, change the state of this zone to active, and add it to the active list. After the data is written successfully, persist the metadata of this zone to rocksdb. When sealing a chunk, you only need to find the corresponding zone, mark chunk is_sealed=true in the metadata of the zone, and persist it to rocksdb. At this time, determine whether the remaining space of the zone is sufficient. If it is sufficient, change the state of the zone to idle state, if not, change the state of the zone to full state.

[0107] In some embodiments of the present invention, the storage usage status further includes an active status; and updating the storage usage status and / or storage data status of the target logical storage area in the database includes:

[0108] In the process of writing the to-be-written data block into the target logical storage area, updating the storage usage state of the target logical storage area from the idle state to the active state;

[0109] After the to-be-written data block is written into the target logical storage area, the storage data state in the target logical storage area is updated.

[0110] In the embodiment of the present invention, after writing data, the system needs to update metadata. Metadata update mainly includes updating the status of chunks in the database (for example, whether the active status of writing is being performed) and the length of chunks. The database indicates the status and length of chunks in the zone.

[0111] Step 104: based on the database, clear the garbage data blocks in any of the logical storage areas in the hard disk.

[0112] In the embodiment of the present invention, garbage data blocks in any logical storage area in the hard disk can be cleared according to the database.

[0113] In some embodiments of the present invention, the storage data state includes a clearing identifier of at least one data block; and clearing the garbage data blocks in any of the logical storage areas in the hard disk based on the database includes:

[0114] Using the logical storage area in the hard disk where the garbage data blocks exist as the logical storage area to be processed;

[0115] By modifying the clearing identifier of the junk data block, the clearing identifier of the junk data block indicates that the junk data block is a data block to be cleared;

[0116] Based on the clearing identifier, obtaining information on the proportion of data blocks to be cleared in the to-be-processed logical storage area;

[0117] Based on the to-be-cleared data block ratio information, it is determined whether to clear the to-be-cleared data block.

[0118] In the embodiment of the present invention, the storage data state includes a clearing identifier of at least one data block; the logical storage area in the hard disk where the garbage data block exists can be used as the logical storage area to be processed. By modifying the clearing identifier of the garbage data block, the clearing identifier of the garbage data block indicates that the garbage data block is a data block to be cleared; based on the clearing identifier, the proportion information of the data blocks to be cleared in the logical storage area to be processed is obtained; based on the proportion information of the data blocks to be cleared, it is determined whether to clear the data blocks to be cleared.

[0119] In some embodiments of the present invention, the determining whether to clear the data blocks to be cleared based on the ratio information of the data blocks to be cleared includes:

[0120] Based on the to-be-cleared data block ratio information, determining whether the ratio of the to-be-cleared data blocks in the to-be-processed logical storage area exceeds a preset ratio threshold;

[0121] If the proportion of the to-be-cleared data blocks in the to-be-processed logical storage area exceeds a preset proportion threshold, the to-be-cleared data blocks are cleared.

[0122] In the embodiment of the present invention, when determining whether to clear the data blocks to be cleared based on the ratio information of the data blocks to be cleared, it is determined whether the ratio of the data blocks to be cleared in the pending logical storage area exceeds a preset ratio threshold based on the ratio information of the data blocks to be cleared. If the ratio of the data blocks to be cleared in the pending logical storage area exceeds the preset ratio threshold, the data blocks to be cleared are cleared.

[0123] In some embodiments of the present invention, the clearing the to-be-cleared data block includes:

[0124] Based on the clearing identifier of the data block, determining a target data block in the to-be-processed logical storage area that is not the to-be-cleared data block;

[0125] Transferring the target data block to other logical storage areas in the hard disk except the to-be-processed logical storage area;

[0126] Clear the logic storage area to be processed.

[0127] In the embodiment of the present invention, when clearing the data block to be cleared, it is necessary to determine the target data block in the to-be-cleared logical storage area that is not the to-be-cleared data block based on the clearing identifier of the data block, transfer the target data block to other logical storage areas in the hard disk except the to-be-cleared logical storage area, and clear the to-be-cleared logical storage area.

[0128] In some embodiments of the invention, the method comprises:

[0129] After the target data block is transferred to other logical storage areas in the hard disk, modifying the storage data state of the logical storage area that receives the target data block;

[0130] After the to-be-processed logical storage area is cleared, the storage usage state and / or the storage data state of the to-be-processed logical storage area is modified.

[0131] In an embodiment of the present invention, after the target data block is transferred to other logical storage areas in the hard disk, the storage data status of the logical storage area that receives the target data block is modified; after the logical storage area to be processed is cleared, the storage usage status and / or storage data status of the logical storage area to be processed is modified.

[0132] Reference Figure 5 , showing a flow chart of a GC function provided in an embodiment of the present invention.

[0133] When a chunk needs to be deleted, the zone metadata is modified to change the chunk's is_purged to true, which makes deletion very simple and efficient. However, since the data space is not actually deleted, there will be a lot of garbage data in the zone. is_purged is the purge identifier.

[0134] In the embodiment of the present invention, the ratio of valid data to space is calculated through the metadata of the zone. When the valid data in the zone is less than 50%, it is necessary to start GC to move the valid data to the new zone, and then reset the old zone to release space.

[0135] Figure 5There are 3 chunks stored in zone_id=1, of which chunk=1 and chunk=3 have been marked for deletion. GC is started to move the valid chunk=2 to zone_id=2, and then reset zone_id=1, so that the space of zone_id=1 is reclaimed. After the data is moved, the modified metadata of zone_id=1 and zone_id=2 are persisted to rocksdb.

[0136] In a specific example, the process of the GC function is as follows:

[0137] When deleting a chunk, you only need to find the corresponding zone, mark chunks_purged=true in the zone's metadata, and then persist it to rocksdb. At this time, you can recalculate the total effective space. If the effective space is less than 50%, start GC for this zone. Read the valid data of this zone and write it to a new zone. After the write is successful, reset this zone, clear the metadata of this zone and write it to rocksdb. Set the state of this zone to idle state, so that the space of the zone is reclaimed.

[0138] In the embodiment of the present invention, the hard disk includes at least one logical storage area, the logical storage area of ​​the hard disk is emptied, and the storage usage status and / or storage data status of the logical storage area are stored in a preset database; the metadata of the logical storage area is a data block; in the process of writing the preset data block to be written into the hard disk, it is determined whether there is a target logical storage area in the hard disk; the target logical storage area stores data related to the data block to be written; if so, the data block to be written is written into the target logical storage area, and the storage usage status and / or storage data status of the target logical storage area in the database is updated; based on the database, the garbage data blocks in any logical storage area of ​​the hard disk are cleared. A zns disk space management solution based on rocksdb is provided, which realizes the FTL implementation strategy of the zns disk and the automatic implementation of the GC function.

[0139] In the embodiment of the present invention, a simple / efficient management solution for ZNS disk space is provided from the software level. This solution ensures the reliability of metadata by storing metadata in rocksdb. A batch of read and write interfaces for ZNS disks are encapsulated. These interfaces are no longer based on LBA read and write, but directly provide read and write interfaces for chunks.

[0140] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0141] Reference Figure 6 , shows a structural block diagram of a hard disk space management device provided in an embodiment of the present invention, which is applied to a hard disk, wherein the hard disk includes at least one logical storage area, and specifically may include the following modules:

[0142] The clearing module 601 is used to clear the logical storage area of ​​the hard disk and store the storage usage status and / or storage data status of the logical storage area using a preset database; the metadata of the logical storage area is a data block;

[0143] The target logical storage area determination module 602 is used to determine whether there is a target logical storage area in the hard disk during the process of writing the preset data block to be written into the hard disk; the target logical storage area stores data related to the data block to be written;

[0144] A first writing module 603, configured to write the data block to be written into the target logical storage area if it exists, and update the storage usage status and / or storage data status of the target logical storage area in the database;

[0145] The clearing module 604 is used to clear junk data blocks in any of the logical storage areas in the hard disk based on the database.

[0146] In an optional embodiment of the present invention, the storage usage state includes an idle state; when the logical storage area is cleared, the logical storage area is in an idle state; the device includes:

[0147] The second writing module is used for writing the data block to be written into any of the logical storage areas in the idle state if the target logical storage area does not exist in the hard disk.

[0148] In an optional embodiment of the present invention, the storage usage status further includes an active state; and the first writing module includes:

[0149] A first state updating submodule, configured to update the storage usage state of the target logical storage area from the idle state to the active state during the process of writing the to-be-written data block into the target logical storage area;

[0150] The second status updating submodule is used to update the storage data status in the target logical storage area after the to-be-written data block is written into the target logical storage area.

[0151] In an optional embodiment of the present invention, the stored data state includes a clear identifier of at least one data block; and the clear module includes:

[0152] The to-be-processed logical storage area is used as a submodule, and is used to use the logical storage area in the hard disk where the garbage data block exists as the to-be-processed logical storage area;

[0153] A modification submodule, configured to modify the removal identifier of the junk data block so that the removal identifier of the junk data block indicates that the junk data block is a data block to be removed;

[0154] A submodule for obtaining information on the proportion of data blocks to be cleared, used for obtaining information on the proportion of data blocks to be cleared in the to-be-processed logical storage area based on the clearing identifier;

[0155] The clearing judgment submodule is used to judge whether to clear the data blocks to be cleared based on the ratio information of the data blocks to be cleared.

[0156] In an optional embodiment of the present invention, the clearing judgment submodule includes:

[0157] A threshold judgment unit, configured to judge whether the proportion of the data blocks to be cleared in the to-be-cleared logical storage area exceeds a preset proportion threshold based on the proportion information of the data blocks to be cleared;

[0158] The clearing unit is configured to clear the data blocks to be cleared if the proportion of the data blocks to be cleared in the logical storage area to be processed exceeds a preset proportion threshold.

[0159] In an optional embodiment of the present invention, the clearing unit includes:

[0160] A target data block determination subunit, configured to determine a target data block in the to-be-processed logical storage area that is not the to-be-cleared data block based on the clearing identifier of the data block;

[0161] A transfer subunit, used for transferring the target data block to other logical storage areas in the hard disk except the to-be-processed logical storage area;

[0162] The clearing subunit is used to clear the to-be-processed logical storage area.

[0163] In an optional embodiment of the present invention, the device comprises:

[0164] A first state modification module, configured to modify the storage data state of the logical storage area that receives the target data block after the target data block is transferred to other logical storage areas in the hard disk;

[0165] The second state modification module is used to modify the storage usage state and / or the storage data state of the to-be-processed logical storage area after the to-be-processed logical storage area is cleared.

[0166] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0167] In addition, an embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0168] Memory 703, used for storing computer programs;

[0169] The processor 701 is used to execute the program stored in the memory 703 to implement the following steps:

[0170] Clearing the logical storage area of ​​the hard disk, and using a preset database to store the storage usage status and / or storage data status of the logical storage area; the metadata of the logical storage area is a data block;

[0171] In the process of writing a preset data block to be written into the hard disk, determining whether there is a target logical storage area in the hard disk; the target logical storage area stores data related to the data block to be written;

[0172] If it exists, writing the data block to be written into the target logical storage area, and updating the storage usage status and / or storage data status of the target logical storage area in the database;

[0173] Based on the database, garbage data blocks in any of the logical storage areas in the hard disk are cleared.

[0174] In an optional embodiment of the present invention, the storage usage state includes an idle state; when the logical storage area is cleared, the logical storage area is in an idle state; and the method includes:

[0175] If the target logical storage area does not exist in the hard disk, the data block to be written is written into any of the logical storage areas in the idle state.

[0176] In an optional embodiment of the present invention, the storage usage status further includes an active status; and updating the storage usage status and / or storage data status of the target logical storage area in the database includes:

[0177] In the process of writing the to-be-written data block into the target logical storage area, updating the storage usage state of the target logical storage area from the idle state to the active state;

[0178] After the to-be-written data block is written into the target logical storage area, the storage data state in the target logical storage area is updated.

[0179] In an optional embodiment of the present invention, the storage data state includes a clearing identifier of at least one data block; and clearing the garbage data blocks in any of the logical storage areas in the hard disk based on the database includes:

[0180] Using the logical storage area in the hard disk where the garbage data blocks exist as the logical storage area to be processed;

[0181] By modifying the clearing identifier of the junk data block, the clearing identifier of the junk data block indicates that the junk data block is a data block to be cleared;

[0182] Based on the clearing identifier, obtaining information on the proportion of data blocks to be cleared in the to-be-processed logical storage area;

[0183] Based on the to-be-cleared data block ratio information, it is determined whether to clear the to-be-cleared data block.

[0184] In an optional embodiment of the present invention, judging whether to clear the data blocks to be cleared based on the ratio information of the data blocks to be cleared includes:

[0185] Based on the to-be-cleared data block ratio information, determining whether the ratio of the to-be-cleared data blocks in the to-be-processed logical storage area exceeds a preset ratio threshold;

[0186] If the proportion of the to-be-cleared data blocks in the to-be-processed logical storage area exceeds a preset proportion threshold, the to-be-cleared data blocks are cleared.

[0187] In an optional embodiment of the present invention, the clearing the data block to be cleared includes:

[0188] Based on the clearing identifier of the data block, determining a target data block in the to-be-processed logical storage area that is not the to-be-cleared data block;

[0189] Transferring the target data block to other logical storage areas in the hard disk except the to-be-processed logical storage area;

[0190] Clear the logic storage area to be processed.

[0191] In an optional embodiment of the present invention, the method includes:

[0192] After the target data block is transferred to other logical storage areas in the hard disk, modifying the storage data state of the logical storage area that receives the target data block;

[0193] After the to-be-processed logical storage area is cleared, the storage usage state and / or the storage data state of the to-be-processed logical storage area is modified.

[0194] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0195] The communication interface is used for communication between the above terminal and other devices.

[0196] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0197] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0198] like Figure 8 As shown, in another embodiment provided by the present invention, a computer-readable storage medium 801 is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes a hard disk space management method described in the above embodiment.

[0199] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes a hard disk space management method described in the above embodiment.

[0200] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0201] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0202] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0203] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A hard disk space management method, characterized in that: Applied to a hard disk, the hard disk includes at least one logical storage area, the method includes: Clearing the logical storage area of ​​the hard disk, and using a preset database to store the storage usage status and / or storage data status of the logical storage area; the metadata of the logical storage area is a data block; In the process of writing a preset data block to be written into the hard disk, determining whether there is a target logical storage area in the hard disk; the target logical storage area stores data related to the data block to be written; If it exists, writing the data block to be written into the target logical storage area, and updating the storage usage status and / or storage data status of the target logical storage area in the database; Based on the database, garbage data blocks in any of the logical storage areas in the hard disk are cleared.

2. The method according to claim 1, characterized in that The storage usage state includes an idle state; when the logical storage area is cleared, the logical storage area is in an idle state; the method includes: If the target logical storage area does not exist in the hard disk, the data block to be written is written into any of the logical storage areas in the idle state.

3. The method according to claim 2, characterized in that The storage usage status also includes an active status; and updating the storage usage status and / or storage data status of the target logical storage area in the database includes: In the process of writing the to-be-written data block into the target logical storage area, updating the storage usage state of the target logical storage area from the idle state to the active state; After the to-be-written data block is written into the target logical storage area, the storage data state in the target logical storage area is updated.

4. The method according to claim 1, characterized in that: The storage data state includes a clearing identifier of at least one data block; and clearing the garbage data blocks in any of the logical storage areas in the hard disk based on the database includes: Using the logical storage area in the hard disk where the garbage data blocks exist as the logical storage area to be processed; By modifying the clearing identifier of the junk data block, the clearing identifier of the junk data block indicates that the junk data block is a data block to be cleared; Based on the clearing identifier, obtaining information on the proportion of data blocks to be cleared in the to-be-processed logical storage area; Based on the to-be-cleared data block ratio information, it is determined whether to clear the to-be-cleared data block.

5. The method according to claim 4, characterized in that The determining whether to clear the data blocks to be cleared based on the ratio information of the data blocks to be cleared includes: Based on the to-be-cleared data block ratio information, determining whether the ratio of the to-be-cleared data blocks in the to-be-processed logical storage area exceeds a preset ratio threshold; If the proportion of the to-be-cleared data blocks in the to-be-processed logical storage area exceeds a preset proportion threshold, the to-be-cleared data blocks are cleared.

6. The method according to claim 5, characterized in that The clearing of the to-be-cleared data block comprises: Based on the clearing identifier of the data block, determining a target data block in the to-be-processed logical storage area that is not the to-be-cleared data block; Transferring the target data block to other logical storage areas in the hard disk except the to-be-processed logical storage area; Clear the logic storage area to be processed.

7. The method according to claim 6, characterized in that The method comprises: After the target data block is transferred to other logical storage areas in the hard disk, modifying the storage data state of the logical storage area that receives the target data block; After the to-be-processed logical storage area is cleared, the storage usage state and / or the storage data state of the to-be-processed logical storage area is modified.

8. A hard disk space management device, characterized in that: Applied to a hard disk, the hard disk includes at least one logical storage area, and the device includes: A clearing module, used to clear the logical storage area of ​​the hard disk, and store the storage usage status and / or storage data status of the logical storage area using a preset database; the metadata of the logical storage area is a data block; A target logical storage area determination module is used to determine whether there is a target logical storage area in the hard disk during the process of writing a preset data block to be written into the hard disk; the target logical storage area stores data related to the data block to be written; a writing module, configured to write the data block to be written into the target logical storage area if it exists, and update the storage usage status and / or storage data status of the target logical storage area in the database; A clearing module is used to clear junk data blocks in any of the logical storage areas in the hard disk based on the database.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 7 when executing the program stored in the memory.

10. One or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 7.

Images