Read-write method and device for high-speed embedded file system

Abstract

The invention discloses a read-write method and device for a high-speed embedded file system, and the method comprises the steps: a construction step: constructing a file system FTL table and a maximum counter table in an RAM according to file system management information stored in an NVM, and carrying out the atomicity protection of the file system according to the validity of the FTL table and the maximum counter table; in the writing step, when the file is written, the newest file pointer and the newest page table are obtained from the FTL table, file data are cached in the page cache, and after the page cache is full of the page data, the file pointer and the newest page table are written into the NVM physical page of the user data area together; and a reading step: when the file is read, obtaining the latest file pointer from the FTL table, and obtaining the file data from the corresponding record according to the pointer. According to the invention, the writing performance of the existing embedded file system can be improved by 10 times, and the reading performance can be improved by 5 times; according to the method, the service life of the NVM can be prolonged by 10 times, and atomic protection of the file system can be realized through exception recovery.

Description

Technical Field

[0001] The present invention relates to the field of navigation technology, and in particular to a high-speed embedded file system reading and writing method and device. Background Art

[0002] In existing embedded file systems (such as SPIFFS), the minimum storage unit is mostly blocks. The file data and management information are indexed by blocks through index tables. Each time a file is stored, the blocks occupied by the file data need to be written to NVM (Non-Volatile Memory) multiple times, and the index information needs to be written to NVM multiple times. When writing a file, the NVM needs to be erased many times, and the atomicity of the file writing cannot be guaranteed.

[0003] The existing reading and writing methods have the following disadvantages: (1) Slow file erase and write speed will suppress system scheduling and algorithm calculation, resulting in system data loss or failure to ensure real-time performance. Especially in the 10hz scenario in navigation applications, the system response speed and algorithm calculation speed are extremely high, which cannot be met by existing file systems.

[0004] (2) In embedded systems, files are frequently updated. The number of times NVM is erased and written each time a file is updated directly affects the overall life of the NVM. Extending the life of the NVM has always been a difficult problem in the industry. Usually, the life of the NVM is 10,000 or 100,000 times. In navigation systems, the current file system erases the NVM too many times and cannot guarantee a 10-year life of the product.

[0005] (3) File updates cannot guarantee atomicity. When an exception occurs during a file update, such as a power outage, only part of the file data or index information can be updated. Index errors or file data errors may occur when reading the file, resulting in system abnormalities or navigation positioning deviations. Summary of the invention

[0006] The technical problem to be solved by the embodiments of the present invention is to provide a high-speed embedded file system reading and writing method and device to improve the reading and writing performance of the embedded file system and the NVM lifespan, and to ensure the atomicity of the file system.

[0007] In order to solve the above technical problems, an embodiment of the present invention proposes a high-speed embedded file system reading and writing method, wherein the data of the embedded file system is managed by an NVM memory in units of physical pages, wherein files are stored in the NVM in the form of records, and the page header and the record header respectively store NVM page management information and file management information; the reading and writing method comprises: Construction step: constructing a file system FTL table and a maximum counter table in the RAM according to the file system management information stored in the NVM; the FTL table includes the latest file pointer, the latest file list, and the page cache, and the maximum counter page table includes the maximum counter write page number and the maximum counter page table; Abnormal recovery steps: During power-on initialization, after building the FTL table and the maximum counter table, check the integrity and validity of the maximum counter table to determine whether to perform abnormal recovery to ensure the atomicity of the file system; Writing steps: When writing a file, get the latest file list and the latest file pointer from the FTL table, cache the file data in the page cache, and when the page cache is full, write it to the NVM physical page in the user data area together; Reading step: When reading a file, obtain the latest file pointer from the FTL table, and obtain file data from the corresponding record according to the pointer.

[0008] Accordingly, an embodiment of the present invention further provides a high-speed embedded file system reading and writing device, wherein the data of the embedded file system is managed by an NVM memory in units of physical pages, wherein files are stored in the NVM in the form of records, and the page header and the record header respectively store NVM page management information and file management information; the reading and writing device comprises: Construction module: construct a file system FTL table and a maximum counter table in RAM according to the file system management information stored in NVM; the FTL table includes the latest file pointer, the latest file list, and the page cache, and the maximum counter page table includes the maximum counter write page number and the maximum counter page table; Abnormal recovery module: During power-on initialization, after building the FTL table and the maximum counter table, it checks the integrity and validity of the maximum counter table to determine whether to perform abnormal recovery to ensure the atomicity of the file system; Write module: When writing a file, it obtains the latest file list and the latest file pointer from the FTL table, caches the file data in the page cache, and when the page cache is full, writes it to the NVM physical page in the user data area. Reading module: When reading a file, it obtains the latest file pointer from the FTL table, and obtains file data from the corresponding record according to the pointer.

[0009] The beneficial effects of the present invention are as follows: the present invention can improve the writing performance of the current embedded file system by 10 times and the reading performance by 5 times; the present invention can increase the life of the NVM by 10 times, thereby increasing the product life of the navigation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 Schematic diagram of the structure of the NVM according to an embodiment of the present invention.

[0011] Figure 2 It is a schematic diagram of the structure of the FTL table of an embodiment of the present invention.

[0012] Figure 3 It is a schematic diagram of the process of creating an FTL table according to an embodiment of the present invention.

[0013] Figure 4 It is a flowchart of the abnormal recovery step of an embodiment of the present invention.

[0014] Figure 5 It is a flowchart of file writing according to an embodiment of the present invention.

[0015] Figure 6 It is a flowchart of file reading according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present invention is further described in detail below in conjunction with the drawings and specific embodiments.

[0017] In the embodiments of the present invention, if there are directional indications (such as up, down, left, right, front, back, etc.), they are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0018] In addition, in the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0019] The file data of the embedded file system of the embodiment of the present invention is managed by NVM memory in units of physical pages, wherein the file is stored in the NVM in the form of records, and the page header and the record header respectively store NVM page management information and file management information. The structure of the NVM physical page is as follows: Figure 1 As shown in the figure. NVM with word write feature uses record + mark to manage the validity of files, which is convenient for garbage collection and FTL table management. NVM with page erase feature uses write counter + record to manage valid files, which is convenient for better garbage collection and atomicity.

[0020] The high-speed embedded file system reading and writing method of the embodiment of the present invention comprises a construction step, an abnormal recovery step, a writing step, and a reading step.

[0021] Construction step: Construct the file system FTL table and the maximum counter table in the RAM according to the file system management information stored in the NVM. The FTL table contains the latest file pointer, the latest file list, and the page cache, and the maximum counter page table contains the maximum counter write page number and the maximum counter page table; The structure of the FTL table is as follows Figure 2 Please refer to Figure 3 , power on initialization, create FTL table. When NVM storage space is less than the set recycling threshold, garbage collection will be automatically performed.

[0022] The navigation chip of the present invention constructs the file system FTL table through the file system management information stored in NVM when it is powered on and initialized. During system operation, the file positioning uses the FTL table, and the file search positioning does not need to access NVM. Compared with the existing file search by accessing NVM, the search performance is qualitatively improved.

[0023] Abnormal recovery steps: Please refer to Figure 4 During power-on initialization, after constructing the FTL table and the maximum counter table, check the integrity and validity of the maximum counter table to determine whether to perform abnormal recovery to ensure the atomicity of the file system. The present invention protects the atomicity of the file system based on the validity of the FTL table and the maximum counter table. The present invention can achieve atomicity protection of the file system through abnormal recovery (abnormal recovery is also often called power-off recovery. In embedded systems, NVM erasing abnormalities generally occur when power is off, but there are also other abnormal situations).

[0024] As an implementation method, in the abnormal recovery step, the integrity and validity of the maximum counter table are checked. If the number of pages written to the maximum counter is different from the number of pages in the maximum counter page table, or the NVM page management information in the maximum counter page table is abnormal, or the file management information is abnormal, then an abnormality occurred when the file system was last updated to NVM, and abnormal recovery is performed to clear all NVM pages included in the maximum counter table, and delete all file information included in the maximum counter page table in the FTL, thereby ensuring the atomicity of the entire file system.

[0025] Writing steps: When writing a file, get the latest file list and the latest file pointer from the FTL table, cache the file data in the page cache, and when the page cache is full, write it together to the NVM physical page in the user data area.

[0026] As an implementation method, please refer to Figure 5In the writing step, if there is not enough space in the page cache to store the new file, it is determined whether there are enough empty NVM physical pages in the user data area to store the new file. If so, an empty NVM physical page is applied for and the file is written into the applied empty NVM physical page; if not, the old NVM physical page or the invalid NVM physical page is erased and the file is written into the erased NVM physical page.

[0027] The present invention only needs to access the file FTL table during the allocation of old pages and new pages, and does not need to access the slower NVM for search. The present invention uses page cache, and file data and file system management information are cached in the page cache. When the latest NVM physical page data cache is full, they are written to the NVM physical page together, and the file system management information can be saved without consuming additional NVM write times.

[0028] Reading steps: Please refer to Figure 6 When reading a file, the latest file pointer is obtained from the FTL table, and the file data is obtained from the corresponding record according to the pointer. The access to the file of the present invention directly uses the FTL table in the RAM, and does not need to be searched in the file system, so the access performance is greatly improved.

[0029] As an implementation mode, in the writing step, if the space in the user data area is insufficient, garbage collection is performed on the NVM physical page.

[0030] By storing NVM page management information, file management information and file data in one NVM physical page, the file erasing performance can reach the physical limit of NVM, that is, the file system management information can be saved without consuming additional NVM erasing.

[0031] The present invention can be adapted to ETOX, Snos and MRM types of NVM.

[0032] The high-speed embedded file system reading and writing device of the present invention comprises: Construction module: construct a file system FTL table and a maximum counter table in RAM according to the file system management information stored in NVM; the FTL table includes the latest file pointer, the latest file list, and the page cache, and the maximum counter page table includes the maximum counter write page number and the maximum counter page table; Abnormal recovery module: During power-on initialization, after building the FTL table and the maximum counter table, it checks the integrity and validity of the maximum counter table to determine whether to perform abnormal recovery to ensure the atomicity of the file system; Write module: When writing a file, it obtains the latest file list and the latest file pointer from the FTL table, caches the file data in the page cache, and when the page cache is full, writes it to the NVM physical page in the user data area. Reading module: When reading a file, it obtains the latest file pointer from the FTL table, and obtains file data from the corresponding record according to the pointer.

[0033] As an implementation method, the exception recovery module checks the integrity and validity of the maximum counter table. If the number of pages written into the maximum counter is different from the number of pages in the maximum counter page table, or the NVM page management information in the maximum counter page table is abnormal, or the file management information is abnormal, then an exception occurred when the file system was last updated to NVM, and exception recovery is performed to clear all NVM pages included in the maximum counter table, and delete all file information included in the maximum counter page table in the FTL, thereby ensuring the atomicity of the entire file system.

[0034] As an implementation mode, if there is not enough space in the page cache to store a new file, the write module determines whether there are enough empty NVM physical pages in the user data area to store the new file. If so, an empty NVM physical page is applied for and the file is written into the applied empty NVM physical page; if not, the old NVM physical page or the invalid NVM physical page is erased and the file is written into the erased NVM physical page.

[0035] As an implementation mode, the write module performs garbage collection on the NVM physical page if the space in the user data area is insufficient.

[0036] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A high-speed embedded file system reading and writing method, characterized in that: The data of the embedded file system is managed by NVM memory in units of physical pages, wherein files are stored in NVM in the form of records, and NVM page management information and file management information are stored in page headers and record headers respectively; the reading and writing method comprises: Construction step: constructing a file system FTL table and a maximum counter table in the RAM according to the file system management information stored in the NVM; the FTL table includes the latest file pointer, the latest file list, and the page cache, and the maximum counter table includes the maximum counter write page number and the maximum counter page table; Abnormal recovery steps: During power-on initialization, after building the FTL table and the maximum counter table, check the integrity and validity of the maximum counter table to determine whether to perform abnormal recovery to ensure the atomicity of the file system; Writing steps: When writing a file, get the latest file list and the latest file pointer from the FTL table, cache the file data in the page cache, and when the page cache is full, write it to the NVM physical page in the user data area together; Reading step: When reading a file, obtain the latest file pointer from the FTL table, and obtain file data from the corresponding record according to the pointer.

2. The high-speed embedded file system reading and writing method as claimed in claim 1, characterized in that: In the abnormal recovery step, the integrity and validity of the maximum counter table are checked. If the number of pages written to the maximum counter is different from the number of pages in the maximum counter page table, or the NVM page management information in the maximum counter page table is abnormal, or the file management information is abnormal, then an abnormality occurred when the file system was last updated to NVM. Exception recovery is performed, all NVM pages contained in the maximum counter table are cleared, and all file information contained in the maximum counter page table is deleted in the FTL, thereby ensuring the atomicity of the entire file system.

3. The high-speed embedded file system reading and writing method as claimed in claim 1, characterized in that: In the writing step, if there is not enough space in the page cache to store the new file, it is determined whether there are enough empty NVM physical pages in the user data area to store the new file. If so, an empty NVM physical page is applied for and the file is written into the applied empty NVM physical page; if not, the old NVM physical page or the invalid NVM physical page is erased and the file is written into the erased NVM physical page.

4. The high-speed embedded file system reading and writing method as claimed in claim 1, characterized in that: In the writing step, if the space in the user data area is insufficient, garbage collection is performed on the NVM physical page.

5. A high-speed embedded file system reading and writing device, characterized in that: The data of the embedded file system is managed by NVM memory in units of physical pages, wherein files are stored in NVM in the form of records, and NVM page management information and file management information are stored in page headers and record headers respectively; the reading and writing device comprises: Construction module: construct a file system FTL table and a maximum counter table in the RAM according to the file system management information stored in the NVM; the FTL table includes the latest file pointer, the latest file list, and the page cache, and the maximum counter table includes the maximum counter write page number and the maximum counter page table; Abnormal recovery module: During power-on initialization, after building the FTL table and the maximum counter table, it checks the integrity and validity of the maximum counter table to determine whether to perform abnormal recovery to ensure the atomicity of the file system; Write module: When writing a file, it obtains the latest file list and the latest file pointer from the FTL table, caches the file data in the page cache, and when the page cache is full, writes it to the NVM physical page in the user data area. Reading module: When reading a file, it obtains the latest file pointer from the FTL table, and obtains file data from the corresponding record according to the pointer.

6. The high-speed embedded file system reading and writing device as claimed in claim 5, characterized in that: The exception recovery module checks the integrity and validity of the maximum counter table. If the number of pages written to the maximum counter is different from the number of pages in the maximum counter page table, or the NVM page management information in the maximum counter page table is abnormal, or the file management information is abnormal, an exception occurred when the file system was last updated to NVM. Exception recovery is performed, all NVM pages contained in the maximum counter table are cleared, and all file information contained in the maximum counter page table is deleted in the FTL, thereby ensuring the atomicity of the entire file system.

7. The high-speed embedded file system reading and writing device as claimed in claim 5, characterized in that: If there is not enough space in the page cache to store the new file, the write module determines whether there are enough empty NVM physical pages in the user data area to store the new file. If so, an empty NVM physical page is requested and the file is written into the requested empty NVM physical page; if not, the old NVM physical page or the invalid NVM physical page is erased and the file is written into the erased NVM physical page.

8. The high-speed embedded file system reading and writing device as claimed in claim 5, characterized in that: If the space in the user data area is insufficient, the write module performs garbage collection on the NVM physical page.

Images