A small system fast file recovery method based on OOB enhanced mapping table

Abstract

The application discloses a kind of small system fast file recovery methods based on OOB enhanced mapping table, the method includes: when data is written into idle physical page, the mapping enhanced field of the physical page OOB area is expanded;When mapping relationship is updated, log record is written into log page;When mapping page is dirty, record the latest position information of mapping page page number and its associated log page;Periodically generate system rebuild point, generate mapping table snapshot based on the mapping page state of this moment, and write mapping table snapshot into special recovery point block;Based on the mapping enhanced field, log page and recovery point block, realize fast file recovery after system restart.The method provided by the application, based on the mapping enhanced field of page OOB structure expansion, combined with log chain structure and system rebuild point mechanism, realizes that small system power failure can be quickly recovered without full disk scanning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of flash memory storage systems and file systems, and particularly relates to a small system fast file recovery method based on an OOB enhanced mapping table. BACKGROUND

[0002] At present, NAND flash file systems usually use a Flash Translation Layer (FTL) to implement mapping of a logical page number (LPN) to a physical page number (PPN). A page-level mapping FTL (such as a DFTL) stores a mapping table in a flash memory and uses a cache to delay updating a mapping page. For example, the DFTL temporarily stores update records in a DRAM cache during a write operation, and reads and updates the mapping table page only when the cache page is evicted, which means that the system needs to scan the mapping table page in the flash memory to rebuild the mapping relationship when the system is restarted, resulting in a long time consumption during the startup process. To improve data consistency, the LazyFTL classifies and manages hot pages and cold pages to ensure that the cache data is not lost after a system crash as much as possible. However, these schemes still need to load a large amount of mapping information from the flash memory when the system is restarted.

[0003] The prior art has the following disadvantages: the mapping table needs to be scanned in the entire flash memory OOB area (i.e. the physical page OOB area) when it is rebuilt, resulting in a long time consumption during the startup process; the mapping recovery speed of hot data pages is slow; and the mapping log needs to be frequently written to the main mapping page, resulting in additional write amplification and consistency management overhead. Therefore, it is necessary to propose a new scheme to quickly rebuild the page mapping without full-disk scanning after the system is powered off and restarted, so as to improve the startup efficiency and data consistency. SUMMARY

[0004] In view of the above-mentioned deficiencies of the existing files, the present application provides a small system fast file recovery method based on an OOB enhanced mapping table, which expands a mapping enhancement field based on a page OOB structure, combines a log chain structure with a system rebuild point mechanism, and realizes fast recovery of files without full-disk scanning after the small system is powered off.

[0005] To achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions:

[0006] A small system fast file recovery method based on an OOB enhanced mapping table, the method comprising:

[0007] When data is written to an idle physical page, the mapping enhancement field of the OOB area of the physical page is expanded;

[0008] When the mapping relationship is updated, a log record is written to a log page;

[0009] When the mapping page is dirty, the latest position information of the mapping page page number and its associated log page is recorded;

[0010] Periodically generate system reconstitution points, generate mapping table snapshots based on the mapping page state at that moment, and write the mapping table snapshots into dedicated recovery point blocks;

[0011] Based on the mapping enhancement field, the log page, and the recovery point block, fast file recovery after system restart is realized.

[0012] According to an aspect of the present application, the mapping enhancement field comprises a validity flag, an access count, a version number, and a hot / cold page flag field.

[0013] According to an aspect of the present application, the mapping enhancement field further comprises a logical page address checksum field or a complete logical page address field.

[0014] According to an aspect of the present application, the small system fast file recovery method further comprises initializing the mapping enhancement field: synchronously initializing the mapping enhancement field when writing user data.

[0015] According to an aspect of the present application, the log record comprises a logical page number, a physical page number, a current version number, and a management flag field; wherein the log record is stored in a log page in a variable-length compressed format.

[0016] According to an aspect of the present application, when writing a log record, if the current log page space is exhausted, a full state is marked at the page tail or OOB area of the log page; at the same time, a new log page is allocated, and the physical address of the new log page is written into the page tail pointer field of the current log page, so that the current log page and the new log page form a chain structure, thereby constituting a log chain.

[0017] According to an aspect of the present application, when periodically generating the system reconstitution point, a recovery point version number is set in the recovery point block, which is used to identify the logical version reference corresponding to the mapping table snapshot.

[0018] According to an aspect of the present application, when periodically generating the system reconstitution point, the start address, version number, and associated log chain start address of the recovery point block are recorded in the Boot page containing the free block list.

[0019] According to an aspect of the present application, the fast file recovery after system restart comprises:

[0020] Reading the Boot page: parsing the free block list, recovery point block address, and log chain start address;

[0021] Recovering the reconstitution point mapping table: reading the mapping table snapshot according to the recovery point address, and loading it into the internal DRAM cache, with the recovery point version number as the initial value of the current mapping version;

[0022] Sequential replay log chain: sequentially traverse the log pages from the log chain start address, check each log record and perform update operation, traverse to the chain end through the page tail pointer field of the log page;

[0023] State synchronization: update block management state based on free block list, start garbage collection.

[0024] According to an aspect of the present application, the checking each log record and performing update operation comprises:

[0025] Based on the log record, read the validity flag field of the mapping enhancement field, if the validity flag is in the valid state and the version number of the log record is greater than the current mapping version, update the mapping relationship and the current mapping version, otherwise discard the log record.

[0026] Advantages of the present application: through the enhancement of OOB field, log chain and periodic reestablishment point, fast recovery after small system power failure is realized. During normal operation, validity flag, version number and other metadata are added in the physical page OOB area to record the mapping state; when the mapping is updated, logical address, physical address and version are written into the log page in variable length log to form a chain structure, and the log position is recorded in the header of the dirty mapping page; a mapping table snapshot (reestablishment point) is generated periodically and stored in the recovery point block, and the Boot page records the address and log chain start point thereof. During restart recovery, the Boot page is read first to obtain the free block, recovery point and log chain information; the recovery point snapshot is loaded as the baseline mapping; the log chain is sequentially replayed, the validity flag and version number in the OOB are combined, and only the valid and new version mapping is updated; finally, the block state is synchronized and the invalid block is recycled, and the mapping table can be quickly reestablished without full disk scanning. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0028] Figure 1 A schematic flowchart of a small system fast file recovery method based on OOB enhanced mapping table according to the present application;

[0029] Figure 2 An operation flowchart of a small system fast file recovery method based on OOB enhanced mapping table according to the present application;

[0030] Figure 3 A power failure restart flowchart of a small system fast file recovery method based on OOB enhanced mapping table according to the present application. DETAILED DESCRIPTION

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] like Figure 1 As shown, a fast file recovery method for small systems based on an OOB enhanced mapping table is presented. This method, based on the page OOB structure extension mapping enhancement field, combined with the log chain structure and system rebuild point mechanism, enables fast file recovery without a full disk scan after a power outage in a small system.

[0033] Step S1: When data is written to an idle physical page, the mapping enhancement field of the OOB region of that physical page is expanded;

[0034] Step S2: When the mapping relationship is updated, the log record is written to the log page;

[0035] Step S3: When the mapped page becomes dirty, record the latest position information of the mapped page number and its associated log page;

[0036] Step S4: Periodically generate system rebuild points, generate a mapping table snapshot based on the mapping page status at that moment, and write the mapping table snapshot into a dedicated recovery point block;

[0037] Step S5: Based on the mapping enhancement field, log page, and recovery point block, realize fast file recovery after system restart.

[0038] Furthermore, in step S1, the mapping enhancement fields in the physical page OOB region include a validity flag, access count, version number, and hot / cold page flag. These mapping enhancement fields are used to record mapping metadata. In practical applications, the 1-bit validity flag field is used to mark whether the mapping is valid, the access count field records the access frequency for hot page identification, the 16-bit version number field records the mapping version number of the current logical page number, and the 1-bit hot / cold page flag field assists in garbage collection.

[0039] Furthermore, you can choose to add a logical page address checksum field or a complete logical page address field as needed. The logical page address checksum field is used to verify the integrity of the logical page address data, while the complete logical page address field is used to establish the mapping from logical pages to physical pages in a DRAM-free environment.

[0040] Furthermore, the small system fast file recovery method also includes: synchronously initializing the mapping enhancement field when writing a new data page, specifically:

[0041] First, the user data is written into the user data area of the idle physical page. At the same time, the OOB area corresponding to the physical page is initialized, including: setting the validity flag to binary value 1, explicitly identifying that the current mapping entry is in the valid state, for subsequent file recovery to quickly filter invalid entries with binary value 0 of the validity flag; the initial value of the access count is set to 1, and the access count value is incremented when the host accesses the logical page subsequently, providing a basis for access frequency analysis when the mapping table is rebuilt; the hot and cold flag is marked according to the access count, such as defaulting to "cold page" for the first time, and dynamically updating according to the access count subsequently, for cache strategy optimization when the mapping table is rebuilt; the version number is generated by a global incremental counter, ensuring that each logical page update corresponds to a unique version number, supporting data version rollback and consistency check.

[0042] Further, in step S2, when the mapping relationship is updated, the log record is written into the log page. The log record includes the logical page number, the physical page number, the current version number, and the management flag field; the log record is stored in the log page in a variable-length compressed format, which can save storage space. In actual use, each page of the log can record 256-512 records.

[0043] Further, when writing the log record, if the current log page space is exhausted, the full state is marked at the page tail or the OOB area of the log page; at the same time, a new log page is allocated, and the physical address of the new log page is written into the page tail pointer field of the current log page, so that the current log page and the new log page form a chain structure, thereby forming a log chain.

[0044] In the system running process, when the mapping record changes, the system records the mapping change by using the log page to avoid frequent updating of the main mapping table. Specifically, when the mapping table needs to be modified, the system does not directly rewrite the mapping page in the storage medium, but appends the update operation to the current log page in a variable-length record format. Each log record contains a logical page number, a physical page number, a current version number, and a management flag, supporting subsequent data integrity verification. The system calculates the remaining space in real time when writing the log record. When the current log page has insufficient space to write the next log record, the log switching mechanism is triggered, and the state flag bit is set in the page tail or the OOB area of the current page to indicate that it is full. Then, an empty physical page is applied from the free block manager, and the physical address of the new log page is written into the page tail pointer field of the current log page to form a log chain, which supports sequential playback during recovery and facilitates reverse scanning during garbage collection.

[0045] Further, in step S3, the mapping page is dirty, that is, the system records the mapping page number and the latest location information of the associated log page when the mapping relationship is recorded to the log page, specifically:

[0046] Each mapping page header reserves an area for storing a dirty page flag, which indicates whether the mapping page is updated. When the records in the mapping page are updated, the dirty page flag of the corresponding mapping page is set to 1, i.e., the mapping page is marked as a dirty page. At the same time, the page number of the mapping page and the latest position information of the associated log page are written into the mapping page table header index area. The index information is stored in the header of each mapping page, rather than being written into the Boot page. Only the pointer of the global checkpoint is reserved in the Boot page. This design significantly reduces the wear and tear of the Boot page and prolongs the service life of the storage medium.

[0047] Further, in step S4, a system checkpoint is generated periodically, a mapping table snapshot is generated based on the state of the mapping page at this time, and the mapping table snapshot is written into a dedicated recovery point block. Specifically:

[0048] During system operation, the generation process of the checkpoint is automatically performed based on a preset time interval, such as every 5 minutes, or a trigger condition, such as the cumulative log block reaching a threshold, or after every N write. A mapping table snapshot is generated based on the state of the mapping page at this time. After the mapping table snapshot is generated, the free physical block group in the recovery point block area is located. The mapping table is divided by page and sequentially written into the data area of the recovery point block. Each checkpoint contains a complete mapping table snapshot. The checkpoint version number is set in the recovery point block, which is used to identify the time point of the mapping table snapshot during file recovery. At the same time, the starting address of the recovery point block, the version number, and the starting address of the associated log chain are recorded in the Boot page containing the free block list.

[0049] Further, in step S5, based on the mapping enhancement field, the log page, and the recovery point block, fast file recovery after system restart is realized. The fast file recovery process after system restart is as follows:

[0050] Read the Boot page: first read the Boot page, parse the free block list, the latest generated recovery point block physical address, and the log chain starting physical address stored therein, and provide a basic index for subsequent file recovery;

[0051] Restore the mapping table of the checkpoint: read the mapping table snapshot according to the recovery point block address obtained by parsing. The mapping table snapshot contains the mapping relationship between the current system effective physical address and logical address and the corresponding version number, and is loaded into the internal DRAM cache. The checkpoint version number of the recovery point block is used as the initial value of the current mapping version;

[0052] Sequentially play back the log chain: start from the log chain starting address, sequentially read each log page, check each log record, and perform an update operation. Through the page tail pointer field of the log page, sequentially traverse to the last page of the log page, i.e., the chain tail page;

[0053] State synchronization: after the mapping table is reconstructed, the internal block management table is updated according to the free block list parsed from the Boot page, and valid blocks and invalid blocks are marked; at the same time, a background garbage collection thread is started to collect invalid blocks such as old log pages that have expired after the snapshot is generated, old mapping pages that are replaced, and the like, and release the storage space.

[0054] Further, checking each log record and performing an update operation includes:

[0055] Parsing a log record: extracting the logical page number, physical page number and version number in the current log record;

[0056] Verifying mapping validity: based on the physical page number of the log record, reading the mapping enhancement field of the OOB area of the corresponding physical page, checking the validity flag therein: if the validity flag is valid, it is determined that the physical page mapping is valid; if the validity flag is invalid, it is determined as invalid mapping, and the current log record is directly discarded;

[0057] Version number comparison and update: if the mapping is valid, the version number of the log record is compared with the current mapping version: if the log version number is greater than the current mapping version number, the physical page address mapped by the logical page address in the mapping table is updated to the physical page address of the log record, and the current mapping version number is updated to the version number of the log record; if the log version number is less than or equal to the current mapping version number, it is determined as an expired record and is directly discarded.

[0058] In actual application, the small system fast file recovery method realizes the following processes:

[0059] As Figure 2 The operation flowchart of the small system fast file recovery method based on OOB enhanced mapping table according to the application, specifically includes:

[0060] Process P201, page writing and OOB updating, selecting an idle physical page, writing user data, extending the mapping enhancement field of the OOB area, and initializing the validity flag (VF), access count (AC) and version number (Version) fields of the mapping enhancement field;

[0061] Process P202, record mapping updates to the log page, generate a log record, and write it to the log page; wherein the log record includes the logical page number (LPN), the physical page number (PPN), and the current version number (Version), etc.; when writing the log record, first determine whether the current log page is full; if the current log page is full, a new log page needs to be allocated, and the full state is marked at the end of the log page or in the OOB area, and the physical address of the new log page is written in the end pointer field of the current log page to form a chain structure, which is used for sequentially traversing the log page when quickly recovering the file, and each log record of the log page is checked in turn. Otherwise, the log record is directly written in the log page;

[0062] Process P203, update the mapping page table header index, mark the dirty mapping page, and write the page number of the mapping page and the latest location information of the associated log page into the mapping page table header index area;

[0063] Process P204, generate a system rebuild point, read the mapping table snapshot, write it into a dedicated recovery point block, and record the recovery point version number in the recovery point block. At the same time, record the start address, version number, and associated log chain start address of the recovery point block in the Boot page containing the free block list.

[0064] As Figure 3 The power-off restart flowchart of the small system fast file recovery method based on the OOB enhanced mapping table according to the present application specifically includes:

[0065] Process P301, read the Boot page, and parse the stored free block list, the recently generated recovery point block address, and the log chain start address;

[0066] Process P302, rebuild the mapping table, and load the mapping table snapshot from the recovery point block; wherein the recovery point version number of the recovery point block is used as the initial value of the current mapping version;

[0067] Flow P303, traverse the log chain and update the mapping, read the starting pointer of each log page from the log starting physical page number, refer to the valid flag and version number of the mapping enhancement field of the OOB area, only apply the valid and latest mapping entry, update the DRAM mapping table; from the log chain starting address, read each log page in turn, check each log record and perform the update operation, and through the page tail pointer field of the log page, traverse to the last page of the log page, that is, the chain tail page. The update operation is specifically: read the mapping enhancement field of the corresponding physical page OOB area, if the valid flag value is 1, it is determined that the physical page mapping is valid; otherwise, it is determined as invalid mapping, and the current log record is discarded directly. If the mapping is valid, compare the version number of the log record with the current mapping version, if the log version number is greater than the current mapping version number, update the physical page address mapped by the logical page address in the mapping table to the physical page address of the log record, and update the current mapping version number to the version number of the log record; if the log version number is less than or equal to the current mapping version number, it is determined as an expired record, and is discarded directly;

[0068] Flow P304, state synchronization, after completing the mapping table reconstruction, update the internal block management table according to the free block list parsed by the Boot page, and mark the valid blocks and invalid blocks; at the same time, start the background garbage collection thread to collect the invalid blocks such as the old log pages expired after the snapshot generation and the old mapping pages replaced, and release the storage space.

[0069] Advantages of the embodiment of the application: through the enhanced OOB field, log chain and periodic reconstruction point, fast recovery after power failure of the small system is realized. In normal operation, the validity flag, version number and other metadata are added in the OOB area of the physical page, and the mapping state is recorded; when the mapping is updated, the logical address, physical address and version are written into the log page in the form of variable length log, a chain structure is formed, and the log position is recorded in the head of the dirty mapping page; a mapping table snapshot (reconstruction point) is generated periodically and stored in the recovery point block, and the Boot page records the address and log chain starting point thereof. When restarting and recovering, the Boot page is read first to obtain the free block, recovery point and log chain information; the recovery point snapshot is loaded as the baseline mapping; the log chain is played back in sequence, the validity flag and version number in the OOB are combined, and only the valid and new version mapping is updated; finally, the block state is synchronized and the invalid blocks are collected, and the mapping table can be reconstructed quickly without full disk scanning.

[0070] The above merely describes the specific implementation of the application, but the protection scope of the application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the application, which should be covered within the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A small system fast file recovery method based on OOB enhanced mapping table, characterized in that, The method comprises: When data is written into an idle physical page, the mapping enhancement field of the OOB area of the physical page is extended; When the mapping relationship is updated, a log record is written into a log page; When a mapping page is marked as a dirty page, the page number of the mapping page and the latest location information of the associated log page are recorded; A system rebuild point is generated periodically, a mapping table snapshot is generated based on the state of the mapping page at the moment when the system rebuild point is generated, and the mapping table snapshot is written into a dedicated recovery point block; Based on the mapping enhancement field, the log page and the recovery point block, fast file recovery after system restart is realized.

2. The small system quick file recovery method of claim 1, wherein, The mapping enhancement field comprises a validity flag, an access count, a version number and a hot / cold page flag field.

3. The small system quick file recovery method of claim 2, wherein, The mapping enhancement field further comprises a logical page address checksum field or a complete logical page address field.

4. The small system quick file recovery method of claim 3, wherein, The fast file recovery method of the small system further comprises initializing the mapping enhancement field: when user data is written, the mapping enhancement field is initialized synchronously.

5. The method of claim 1, wherein, The log record comprises a logical page number, a physical page number, a current version number and a management flag field; wherein the log record is stored in the log page in a variable-length compressed format.

6. The small system quick file recovery method of claim 5, wherein, When the log record is written, if the space of the current log page is exhausted, the full state is marked at the page tail or the OOB area of the log page; at the same time, a new log page is allocated, and the physical address of the new log page is written into the page tail pointer field of the current log page, so that the current log page and the new log page form a chain structure, thereby forming a log chain.

7. The small system quick file recovery method of claim 6, wherein, When the system rebuild point is generated periodically, a recovery point version number is set in the recovery point block, which is used to identify the logical version reference corresponding to the mapping table snapshot.

8. The small system quick file recovery method of claim 7, wherein, When the system rebuild point is generated periodically, the starting address, version number and associated log chain of the recovery point block are recorded in the Boot page containing the free block list.

9. The method of claim 1 to 8, wherein, The fast file recovery after system restart comprises: Reading the Boot page: analyzing the free block list, the recovery point block address and the starting address of the log chain; Recovering the rebuild point mapping table: reading the mapping table snapshot according to the recovery point address, and loading it into the internal DRAM cache, the recovery point version number being used as the initial value of the current mapping version; Sequentially playing back the log chain: sequentially traversing the log page from the starting address of the log chain, checking each log record and performing update operation, and traversing to the tail of the chain through the page tail pointer field of the log page; State synchronization: updating the block management state based on the free block list, and starting garbage collection.

10. The method of claim 9, wherein, The checking each log record and performing update operation comprises: Based on the log record, the validity flag field of the mapping enhancement field is read, if the validity flag is in the valid state and the version number of the log record is greater than the current mapping version, the mapping relationship and the current mapping version are updated, otherwise the log record is discarded.

Images