Method, System, Terminal and Storage Medium for Improving Garbage Collection Precision

Abstract

The present invention relates to the field of all-flash storage technology, and specifically provides a method, system, terminal and storage medium for improving the accuracy of garbage collection, including: obtaining the physical addresses involved in the data in the block to be recycled, and screening out valid data by querying the logical mapping of the physical addresses; after migrating the valid data to a new block, reversely querying the mapping between the logical address and the physical address of the valid data; if the reverse query is successful, verifying the consistency of the valid data before and after migration, and updating the mapping relationship of the valid data after passing the verification. By screening the valid data in the block to be recycled and adopting the method of reverse querying the mapping relationship and MD5 value verification of the data after migrating the valid data to a new block, the present invention ensures the accuracy of the data and improves the garbage collection efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of all-flash storage, and particularly relates to a method, system, terminal and storage medium for improving the accuracy of garbage collection. Background Art

[0002] The principle of garbage collection in an all-flash storage system is as follows: Select a block, copy the valid data in the block to a new block. After all the valid data is copied, erase all the data in the block to make it available again. The current way to determine whether data is valid in the storage system is to query whether there is a mapping relationship between PBA (Physical Block Address) → LBA (Logical Block Address). However, this method has the following problems:

[0003] When selecting a block for recycling, after querying the existence of the PBA→LBA mapping relationship, then copy and migrate the data. Finally, after the data is successfully copied and migrated, modify the PBA→LBA mapping relationship to PBA~→LBA. However, before modifying the mapping relationship, there may be a situation where the host writes the same data (LBA) and modifies the mapping relationship again to PBA~~→LBA, resulting in the non-existence of the PBA→LBA mapping relationship, the data being invalid and not needing to be migrated, and there may also be a situation where the PBA~→LBA mapping relationship overwrites the PBA~~→LBA, causing data inconsistency problems, etc. Summary of the Invention

[0004] In view of the problem of inconsistent migrated data caused by migrating data changes before changing the mapping relationship in the prior art, the present invention provides a method, system, terminal and storage medium for improving the accuracy of garbage collection to solve the above technical problems.

[0005] In a first aspect, the present invention provides a method for improving the accuracy of garbage collection, including:

[0006] Obtain the physical addresses involved in the data in the block to be recycled, and filter out the valid data by querying the logical mapping of the physical addresses;

[0007] After migrating the valid data to a new block, inversely query the mapping between the logical address and the physical address of the valid data;

[0008] If the inverse query is successful, verify the consistency of the valid data before and after migration, and update the mapping relationship of the valid data after passing the verification.

[0009] Further, obtaining the physical addresses involved in the data in the block to be recycled, and filtering out the valid data by querying the logical mapping of the physical addresses, includes:

[0010] Represent the data in the block to be recycled as physical addresses during encoding;

[0011] Query whether there is a logical address with a mapping relationship for the physical address:

[0012] If so, mark the data corresponding to the physical address as valid data;

[0013] If not, mark the data corresponding to the physical address as invalid data.

[0014] Further, after migrating the valid data to a new block, reverse query the mapping between the logical address and the physical address of the valid data, including:

[0015] Copy the valid data in the block to be recycled to the new block, and query the reverse mapping between the logical address and the physical address of the valid data after the copying is completed;

[0016] If the reverse mapping exists, confirm that the data has not been appended and updated;

[0017] If the reverse mapping does not exist, re-screen the valid data in the block to be recycled.

[0018] Further, if the reverse query is successful, verify the consistency of the valid data before and after migration, and update the mapping relationship of the valid data after passing the verification, including:

[0019] Verify the consistency of the MD5 values of the valid data before and after migration. If the verification passes, update the mapping relationship of the valid data, and clear all the data in the block to be recycled;

[0020] If the verification fails, re-execute the screening of the valid data in the block to be recycled and the migration of the valid data.

[0021] In a second aspect, the present invention provides a system for improving the accuracy of garbage collection, including:

[0022] A data screening unit for obtaining the physical addresses involved in the data in the block to be recycled, and screening out valid data by querying the logical mapping of the physical addresses;

[0023] A mapping verification unit for reverse querying the mapping between the logical address and the physical address of the valid data after migrating the valid data to a new block;

[0024] A data verification unit for verifying the consistency of the valid data before and after migration if the reverse query is successful, and updating the mapping relationship of the valid data after passing the verification.

[0025] Further, the data screening unit includes:

[0026] An encoding conversion module for representing the data in the block to be recycled as a physical address in the encoding;

[0027] A mapping query module, used to query whether there is a logical address with a mapping relationship for a physical address;

[0028] A valid marking module, used to mark the data corresponding to the physical address as valid data if there is a logical address with a mapping relationship for the physical address;

[0029] An invalid marking module, used to mark the data corresponding to the physical address as invalid data if there is no logical address with a mapping relationship for the physical address.

[0030] Furthermore, the mapping verification unit includes:

[0031] A reverse query module, used to copy the valid data in the block to be recycled to a new block, and query the reverse mapping between the logical address and the physical address of the valid data after the copying is completed;

[0032] A data confirmation module, used to confirm that the data has not been appended and updated if there is such a reverse mapping;

[0033] A re-screening module, used to re-screen the valid data in the block to be recycled if there is no such reverse mapping.

[0034] Furthermore, the data verification unit includes:

[0035] A data verification module, used to verify the consistency of the MD5 values before and after the migration of valid data. If the verification passes, update the mapping relationship of the valid data and clear all the data in the block to be recycled;

[0036] A re-recycling module, used to re-execute the screening of valid data and the migration of valid data for the block to be recycled if the verification fails.

[0037] In a third aspect, a terminal is provided, including:

[0038] A processor and a memory, wherein,

[0039] The memory is used to store a computer program,

[0040] The processor is used to call and run the computer program from the memory, so that the terminal executes the method of the above-mentioned terminal.

[0041] In a fourth aspect, a computer storage medium is provided. Instructions are stored in the computer-readable storage medium, and when it runs on a computer, it causes the computer to execute the methods described in the above aspects.

[0042] The beneficial effects of the present invention are as follows. The method, system, terminal and storage medium for improving the accuracy of garbage collection provided by the present invention ensure the accuracy of data and improve the garbage collection efficiency by effectively screening the data in the block to be recycled and verifying the data by reverse query mapping relationship and MD5 value after migrating the valid data to the new block.

[0043] In addition, the design principle of the present invention is reliable, the structure is simple, and it has a very wide application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0045] Figure 1 It is a schematic flowchart of the method according to an embodiment of the present invention.

[0046] Figure 2 It is a schematic block diagram of the system according to an embodiment of the present invention.

[0047] Figure 3 It is a schematic structural diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0049] The following explains the key terms in the present invention.

[0050] LBA, short for Logical Block Address, is a common mechanism used on PC data storage devices to indicate the location of data. The device where we most commonly see it used is the hard disk. LBA can refer to the address of a certain data block or the data block pointed to by a certain address. For example, LBA is equivalent to the house number address we usually use (e.g., No. 26, Zhongshan 4th Road, Guangzhou, Guangdong Province, People's Republic of China). And PBA, short for Physics Block Address, compared to LBA, is like the longitude and latitude used in GPS positioning (e.g., the longitude and latitude of the above address are: East longitude: 113°16′40.0621″, North latitude: 23°07′37.6129″).

[0051] On HDDs, since the data on HDDs can be directly overwritten, the relationship between LBA and PBA is a 1:1 correspondence and will not change, that is, LBA = PBA. However, on SSDs, this relationship becomes complicated because the storage medium NAND flash used in SSDs needs to be erased first before writing again, and the reading and writing are in units of pages, and the erasing is in units of blocks (composed of multiple pages). This leads to the relationship between LBA and PBA no longer being fixed. Therefore, SSDs need a layer called FTL for conversion to cooperate with the existing file system.

[0052] Due to the characteristics of NAND flash, the main controller of SSDs uses a mapping table of LBA and PBA to manage the flash memory. When there is data to be updated that needs to be written, the main controller will write the new data into the blank flash memory space (in the erased state), and then update the mapping table data to point the LBA to the new PBA. The original PBA becomes invalid data. If data needs to be written to this PBA again, an erasing operation must be performed first. But here a problem will arise. The lifespan of flash memory particles is calculated by the number of Program / Erase (programming / erasing. Since flash memory cannot be overwritten and must be erased before writing, it is usually called programming, and this process is often called writing) times. The lifespan of MLC (Multi-Level Cell flash memory) is generally 1000 - 10000 times, and that of SLC (Single-Level Cell flash memory) is about 100,000 times. (For details, please refer to the specific data sheet of the flash memory particles).

[0053] Figure 1 It is a schematic flowchart of the method of an embodiment of the present invention. Among them, Figure 1 The execution subject can be a system for improving the accuracy of garbage collection.

[0054] As Figure 1 shown, the method includes:

[0055] Step 110: Obtain the physical addresses involved in the data within the block to be recycled, and filter out the valid data by querying the logical mapping of the physical addresses.

[0056] Step 120: After migrating the valid data to a new block, inversely query the mapping between the logical address and the physical address of the valid data.

[0057] Step 130: If the inverse query is successful, verify the consistency of the valid data before and after migration, and update the mapping relationship of the valid data after passing the verification.

[0058] For the convenience of understanding the present invention, the principle of the method for improving the accuracy of garbage collection in the present invention will be further described below in combination with the process of recycling blocks in the embodiments.

[0059] Specifically, the method for improving the accuracy of garbage collection includes:

[0060] S1. Obtain the physical addresses involved in the data within the block to be recycled, and filter out the valid data by querying the logical mapping of the physical addresses.

[0061] In the encoding, represent the data within the block to be recycled as physical addresses; query whether there is a logical address with a mapping relationship for the physical address: if so, mark the data corresponding to the physical address as valid data; if not, mark the data corresponding to the physical address as invalid data.

[0062] Specifically, obtain all PBAs within the block to be recycled; perform a PBA→LBA query. If the query is valid, execute the copy migration of the valid data. If the query is invalid, just mark it as invalid.

[0063] S2. After migrating the valid data to a new block, inversely query the mapping between the logical address and the physical address of the valid data.

[0064] Copy the valid data within the block to be recycled to a new block, and after the copying is completed, query the inverse mapping between the logical address and the physical address of the valid data; if the inverse mapping exists, confirm that the data has not been appended and updated; if the inverse mapping does not exist, re-screen the valid data in the block to be recycled.

[0065] Specifically, inversely query LBA→PBA. If the query is valid, perform a data MD5 value verification. If the query is invalid, perform a grain recycling retry.

[0066] S3. If the inverse query is successful, verify the consistency of the valid data before and after migration, and update the mapping relationship of the valid data after passing the verification.

[0067] Verify the consistency of the MD5 values of the valid data before and after migration. If the verification passes, update the mapping relationship of the valid data and clear all the data in the to-be-recycled block; if the verification fails, re-execute the screening of the valid data in the to-be-recycled block and the migration of the valid data.

[0068] Perform data MD5 value verification. If the verification passes, execute the modification of the mapping relationships LBA→PBA and PBA→LBA to complete a replication migration of the valid data; if the verification fails, execute the grain recovery retry.

[0069] As Figure 2 shown, the system 200 includes:

[0070] A data screening unit 210, configured to obtain the physical addresses involved in the data in the to-be-recycled block, and screen out the valid data by querying the logical mapping of the physical addresses;

[0071] A mapping verification unit 220, configured to, after migrating the valid data to a new block, reversely query the mapping between the logical address and the physical address of the valid data;

[0072] A data verification unit 230, configured to, if the reverse query is successful, verify the consistency of the valid data before and after migration, and update the mapping relationship of the valid data after passing the verification.

[0073] Optionally, as an embodiment of the present invention, the data screening unit includes:

[0074] An encoding conversion module, configured to represent the data in the to-be-recycled block as a physical address during encoding;

[0075] A mapping query module, configured to query whether there is a logical address with a mapping relationship for the physical address;

[0076] A valid marking module, configured to, if there is a logical address with a mapping relationship for the physical address, mark the data corresponding to the physical address as valid data;

[0077] An invalid marking module, configured to, if there is no logical address with a mapping relationship for the physical address, mark the data corresponding to the physical address as invalid data.

[0078] Optionally, as an embodiment of the present invention, the mapping verification unit includes:

[0079] A reverse query module, configured to copy the valid data in the to-be-recycled block to a new block, and query the reverse mapping between the logical address and the physical address of the valid data after the copying is completed;

[0080] A data confirmation module, configured to, if there is the reverse mapping, confirm that the data has not been appended and updated;

[0081] A re-screening module, configured to re-screen the valid data of the to-be-recycled block if the reverse mapping does not exist.

[0082] Optionally, as an embodiment of the present invention, the data verification unit includes:

[0083] A data verification module, configured to verify the consistency of the MD5 values before and after the migration of the valid data. If the verification passes, update the mapping relationship of the valid data and clear all the data in the to-be-recycled block;

[0084] A re-recycling module, configured to re-execute the screening of the valid data of the to-be-recycled block and the migration of the valid data if the verification fails.

[0085] Figure 3 FIG. 13 is a schematic structural diagram of a terminal 300 provided by an embodiment of the present invention. The terminal 300 can be used to execute the method for improving the accuracy of garbage collection provided by the embodiment of the present invention.

[0086] Wherein, the terminal 300 may include a processor 310, a memory 320, and a communication unit 330. These components communicate through one or more buses. Those skilled in the art can understand that the structure of the server shown in the figure does not constitute a limitation to the present invention. It can be a bus structure, a star structure, and may also include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0087] Wherein, the memory 320 can be used to store the execution instructions of the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 can execute some or all of the steps in the above method embodiments.

[0088] The processor 310 serves as the control center of the storage terminal, connecting various parts of the entire electronic terminal through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 320, and by invoking data stored in the memory, it performs various functions of the electronic terminal and / or processes data. The processor may be composed of an integrated circuit (IC), for example, it may be composed of a single packaged IC, or it may be composed of multiple packaged ICs with the same or different functions connected together. For example, the processor 310 may only include a central processing unit (CPU). In the embodiments of the present invention, the CPU may be a single arithmetic core or may include multiple arithmetic cores.

[0089] The communication unit 330 is used to establish a communication channel so that the storage terminal can communicate with other terminals. It receives user data sent by other terminals or sends user data to other terminals.

[0090] The present invention also provides a computer storage medium. Among them, the computer storage medium can store a program, and when the program is executed, it may include some or all of the steps in the embodiments provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.

[0091] Therefore, by effectively screening the data in the blocks to be recycled and using the method of reverse query mapping relationship and MD5 value verification of the data after migrating the valid data to the new blocks, the present invention ensures the accuracy of the data and improves the garbage collection efficiency. The technical effects that can be achieved in this embodiment can be seen from the description above and will not be elaborated here.

[0092] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be realized by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., which can store program codes, and includes several instructions to enable a computer terminal (which may be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the methods described in the embodiments of the present invention.

[0093] In each embodiment described in this specification, the same or similar parts among the embodiments can be referred to each other. In particular, for the terminal embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the descriptions in the method embodiments.

[0094] In several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the system or unit can be in electrical, mechanical or other forms.

[0095] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0096] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0097] Although the present invention has been described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, those of ordinary skill in the art can make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions should all be within the scope of the present invention. / Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for improving the accuracy of garbage collection, characterized in that, comprising: Obtain the physical addresses involved in the data in the block to be recycled, and filter out the valid data by querying the logical mapping of the physical addresses; After migrating the valid data to a new block, reverse query the mapping between the logical address and the physical address of the valid data, including: Copy the valid data in the block to be recycled to the new block, and query the reverse mapping between the logical address and the physical address of the valid data after the copying is completed; If there is such a reverse mapping, confirm that the data has not been appended and updated; If there is no such reverse mapping, re-screen the valid data in the block to be recycled; Reverse query LBA to PBA, if the query is valid, perform MD5 value verification of the data, if the query is invalid, perform grain recycling retry; If the reverse query is successful, verify the consistency of the valid data before and after migration, and update the mapping relationship of the valid data after passing the verification.

2. The method according to claim 1, characterized in that, Obtain the physical addresses involved in the data in the block to be recycled, and filter out the valid data by querying the logical mapping of the physical addresses, including: Represent the data in the block to be recycled as a physical address during encoding; Query whether there is a logical address with a mapping relationship for the physical address: If so, mark the data corresponding to the physical address as valid data; If not, mark the data corresponding to the physical address as invalid data.

3. The method according to claim 1, characterized in that, If the reverse query is successful, verify the consistency of the valid data before and after migration, and update the mapping relationship of the valid data after passing the verification, including: Verify the MD5 value consistency of the valid data before and after migration, if the verification passes, update the mapping relationship of the valid data, and clear all the data in the block to be recycled; If the verification fails, re-perform the screening of the valid data in the block to be recycled and the migration of the valid data.

4. A system for improving the accuracy of garbage collection, characterized in that, comprising: A data screening unit for obtaining the physical addresses involved in the data in the block to be recycled, and filtering out the valid data by querying the logical mapping of the physical addresses; A mapping verification unit for, after migrating the valid data to a new block, reverse querying the mapping between the logical address and the physical address of the valid data, including: A reverse query module for copying the valid data in the block to be recycled to the new block, and querying the reverse mapping between the logical address and the physical address of the valid data after the copying is completed; A data confirmation module for, if there is such a reverse mapping, confirming that the data has not been appended and updated; A re-screening module for, if there is no such reverse mapping, re-screening the valid data in the block to be recycled; A data verification unit for, if the reverse query is successful, verifying the consistency of the valid data before and after migration, and updating the mapping relationship of the valid data after passing the verification.

5. The system according to claim 4, characterized in that, The data screening unit includes: An encoding conversion module for representing the data in the block to be recycled as a physical address during encoding; A mapping query module for querying whether there is a logical address with a mapping relationship for the physical address; A valid marking module, configured to mark the data corresponding to the physical address as valid data if there is a logical address with a mapping relationship for the physical address; An invalid marking module, configured to mark the data corresponding to the physical address as invalid data if there is no logical address with a mapping relationship for the physical address.

6. The system according to claim 4, wherein, the data verification unit includes: a data verification module, configured to verify the consistency of the MD5 values before and after the migration of valid data, and if the verification passes, update the mapping relationship of the valid data and clear all the data in the to-be-reclaimed block; a reclamation module, configured to, if the verification fails, re-perform the screening of valid data and the migration of valid data for the to-be-reclaimed block.

7. A terminal, wherein, it includes: a processor; a memory for storing the execution instructions of the processor; wherein, the processor is configured to execute the method according to any one of claims 1-3.

8. A computer-readable storage medium storing a computer program, wherein, when the program is executed by a processor, it implements the method according to any one of claims 1-3.

Images