An embedded file system implementation method based on SPI Flash

Abstract

The application discloses a kind of based on SPI Flash's embedded file system implementation method, comprising: initialization reads superblock, establishes the state mapping table of file record item and data sector;When opening file, record item is quickly matched by file name metadata, and new item is allocated if it fails;When reading and writing, aggregate data through write cache, asynchronously update to flash memory and refresh mapping table;When closing file, if change occurs, then allocate new space and mark original space as invalid, avoid real-time erasing;Finally, garbage collection and fragmentation are arranged in system idle period.This method reduces direct access to SPI Flash through memory mapping table, reduces the frequency of erasing and writing by using write cache and asynchronous mechanism, moves time-consuming erasing operation to the background, significantly improves the processing efficiency of file operation and system real-time, while prolonging the service life of storage medium.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of embedded system, and particularly relates to an embedded file system implementation method based on SPI Flash. BACKGROUND

[0002] In embedded devices, data storage is often performed by externally connecting SPI Flash and transplanting open-source file systems (such as FATFS, LITFS, etc.). However, due to the slow erasing speed of SPI Flash, the operation must be performed in units of sectors, and the traditional file system does not fully consider the physical characteristics (such as bit flipping only supporting 1→0, requiring aligned writing) of SPI Flash, resulting in long time consumption of file creation, reading and writing, deletion and other operations, which cannot meet the real-time requirements of efficient and high-performance embedded devices. SUMMARY

[0003] (I) Objectives of the Invention

[0004] In order to overcome the above shortcomings, the purpose of the present application is to provide an embedded file system implementation method based on SPI Flash, so as to solve the problem that the existing embedded file system does not fully consider the physical characteristics of SPI Flash, resulting in low efficiency of file operation and inability to meet the needs of high-performance embedded devices.

[0005] (II) Technical Solutions

[0006] In order to achieve the above-mentioned purpose, the technical solutions provided by the present application are as follows:

[0007] An embedded file system implementation method based on SPI Flash, comprising the following steps:

[0008] S1. File system initialization: reading superblock, traversing all data sectors in file record area and data area, establishing file record item state mapping table and data sector state mapping table;

[0009] S2. File opening or creation: matching according to file name length and check value in file record item state mapping table, if matching is successful, opening the file, otherwise allocating a new file record item;

[0010] S3. File reading and writing operation: temporarily storing file data through write cache, and asynchronously writing data sectors when the cache is full or the file is closed, and updating the data sector state mapping table;

[0011] S4. File closing: judging whether the file record item or file data has been changed, if changed, allocating a new record item or data sector and marking the original record item or original data sector as invalid state to avoid erasing operation;

[0012] S5. Garbage collection and defragmentation: traverse all sector states when system is idle, reclaim invalid sectors and defragment file record area.

[0013] By establishing a mapping table in memory to manage file record and data sector states, the number of direct accesses to low-speed SPI Flash is greatly reduced; by using a fast matching strategy based on file name metadata, the opening and creation of files are efficiently implemented; by using write cache to aggregate write operations and using an asynchronous update mechanism, the frequency of frequent flash erase operations is greatly reduced; by using the strategy of allocating new space and marking the original space as invalid state when the file is closed, the time-consuming erase operation on the critical path of file operation is completely avoided; finally, garbage collection and defragmentation and other maintenance operations are moved to the idle period of the system to effectively prevent the interference of these background tasks on the performance of real-time tasks. In summary, the scheme fully adapts to the physical characteristics of SPI Flash, and solves the key problem that the traditional file system operation is inefficient and cannot meet the real-time requirements of high-performance embedded devices, while helping to balance wear and tear and prolong the life of storage media.

[0014] In some embodiments, the file record item state mapping table in step S1 includes the attribute state, file name length, and file name check value of each record item.

[0015] This embodiment precisely defines the core metadata contained in the mapping table, so that only a small amount of key information is needed to quickly filter and locate files in memory, greatly reducing the dependence on slow storage media and improving the efficiency of file management operations.

[0016] In some embodiments, the check value in step S2 is a CRC16 check value.

[0017] CRC16 has the advantages of fast calculation speed and low conflict probability, and is very suitable for embedded systems that are sensitive to computing resources, providing a reliable and efficient hash value for fast file name comparison.

[0018] In some embodiments, the size of the write cache in step S3 is consistent with the size of the data sector.

[0019] This alignment design allows the contents of the cache to be written directly to an empty physical sector as a complete unit after each full cache, without the need for additional data splitting, merging, or partial programming operations, achieving maximum efficiency in write operations.

[0020] In some embodiments, marking the original record item or the original data sector as invalid in step S4 is achieved by flipping the state bit of the corresponding sector in SPI Flash from 1 to 0.

[0021] The state persistence mark can be realized by simple bit flipping operation, completely avoiding slow erasing on the critical path.

[0022] In some embodiments, the fragment consolidation in step S5 includes:

[0023] reading data of a sector in the file record area into a cache;

[0024] judging whether the proportion of invalid record items exceeds a threshold value;

[0025] if yes, clearing the invalid items and writing back to the Flash.

[0026] When it is detected that the proportion of invalid records in a certain sector reaches a certain threshold value, it indicates that the sector space utilization is low. At this time, the sector data of the entire data sector is read into the memory, the invalid records are removed, and the consolidated valid data is written back to the Flash. This process recycles the fragmented space. Only when the fragmentation degree affects the space utilization, the consolidation is triggered, avoiding unnecessary overhead, effectively merging the storage space, and maintaining the long-term efficient operation of the file system.

[0027] In some embodiments, the allocation of file record items and data sectors adopts a random strategy.

[0028] This embodiment provides that when a new file record item or data sector needs to be allocated, it is randomly selected from the free resource pool, instead of simple sequential allocation. This strategy aims to uniformly distribute the write operation to different physical areas of the entire Flash storage medium. It realizes the balanced wear of the SPI Flash storage space, avoids the premature damage of some specific sectors due to frequent writing, thereby prolonging the service life of the entire Flash chip and improving the reliability of the system.

[0029] In some embodiments, step S2 further includes:

[0030] S2.1 calculating the file name length and the check value of the file to be opened;

[0031] S2.2 searching the file record item state mapping table to find whether there is a record item with the same file name length and check value;

[0032] S2.3 if yes, reading the corresponding file record item in the SPI Flash, and performing file name comparison;

[0033] S2.4 if consistent, opening the file and returning the file handle;

[0034] S2.5 if not or inconsistent, allocating an unused record item from the file record item state mapping table, and updating its state to valid.

[0035] In some embodiments, step S3 further comprises:

[0036] S3.1 judging whether the last data sector of the file is full;

[0037] S3.2 if not, loading sector data in the data sector to the write cache;

[0038] S3.3 appending data to be written to the write cache;

[0039] S3.4 if the write cache is full or the file is closed, allocating a new sector from the data sector state mapping table and writing the cache data to the new sector;

[0040] S3.5 updating the last sector pointer and file size in the file record item.

[0041] In some embodiments, step S4 further comprises:

[0042] S4.1 judging whether the file data or record item is changed;

[0043] S4.2 if changed, allocating a new record item or data sector;

[0044] S4.3 marking the original record item or data sector as invalid state without erasing operation;

[0045] S4.4 if it is the first write, directly synchronizing to the SPI Flash. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a flowchart of opening a file in the SPI Flash-based embedded file system implementation method of the present application;

[0047] Figure 2 is a flowchart of writing a file in the SPI Flash-based embedded file system implementation method of the present application;

[0048] Figure 3 is a flowchart of closing a file in the SPI Flash-based embedded file system implementation method of the present application. DETAILED DESCRIPTION

[0049] To make the objects, technical solutions and advantages of the present application clearer, further detailed description will be given below in conjunction with specific embodiments and reference to the drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present application. In addition, in the following description, the description of well-known structures and techniques is omitted to avoid unnecessary confusion of the concept of the present application.

[0050] In the method, the SPI Flash is managed in a sector of 4KB size, and the entire storage space is divided into regions of fixed size, and the physical layout is as follows:

[0051] Superblock: occupies the first sector (4KB), used to store key metadata of the file system, such as version number, region division information, etc.

[0052] File record area: immediately following the superblock, occupying 8 consecutive sectors (32KB in total), used to store record items of all files. Each file record item is assigned a fixed number (0 to 1024), and this area is used to centrally manage the metadata of files.

[0053] Data area: occupies the entire remaining sector of the Flash, used to actually store file data, and each sector used to actually store file data is also called a "data sector", and multiple "data sectors" form a "data area".

[0054] The method for implementing an embedded file system based on SPI Flash provided by the application comprises the following steps: first, file system initialization, which obtains file system metadata by reading the superblock stored in the first sector of the SPI Flash, and on this basis, traverses the file record area and the data area, and then establishes a file record item state mapping table and a data sector state mapping table in the memory of the microcontroller (MCU), which can significantly improve the access efficiency of subsequent file operations. In particular, the file record item state mapping table contains key information such as the attribute state (such as read-only, hidden) of each record item, the file name length, and the file name check value, and the data sector state mapping table records the use state of each data sector. It is worth noting that these mapping tables are resident in memory during system operation to achieve fast query.

[0055] Further, the file opening or creation operation is quickly matched in the file record item state mapping table according to the file name length and the pre-calculated check value, and if the matching is successful, the file is directly opened, otherwise a new file record item is assigned. Preferably, the check value is calculated by using the CRC16 algorithm, which has small calculation amount and low conflict probability, and is very suitable for resource-limited embedded environment. On this basis, the step further comprises the following sub-steps: calculating the file name length and the CRC16 check value of the file to be opened; then searching in the mapping table whether there is a record item with the same length and check value; if there is, reading the corresponding complete record item in the SPI Flash for accurate file name comparison; if the comparison is consistent, opening the file and returning the handle; if there is no or the comparison is inconsistent, assigning a new record item and marking it as valid.

[0056] The file read-write operation temporarily stores data through a write cache with a size aligned with the physical sector of the SPI Flash, and then asynchronously writes the data sector and synchronously updates the data sector state mapping table when the cache is full or the file is closed. In particular, the size of the write cache is preferably set to 4KB to match the most common SPI Flash sector size, thereby realizing efficient batch writing of data. The process further includes: judging whether the last data sector of the file is full, and if not, loading the data thereof to the write cache; appending new data to the cache; allocating a new data sector and writing data when the cache is full; and finally updating the related pointer and size information in the file record item.

[0057] When the file is closed, it is necessary to judge whether the file record item or file data has been changed, and if changed, a new record item or data sector is allocated, and the original record item or original data sector is marked as invalid. The marking operation is realized by flipping the corresponding state bit in the SPI Flash from 1 to 0, thereby completely avoiding time-consuming erase operations. It is worth noting that for the first time writing data, a direct synchronization method is used to write the SPI Flash to ensure data reliability. The step further includes: judging whether the data or metadata has been changed; if changed, allocating a new resource; marking the original resource as invalid; and handling special cases of first-time writing.

[0058] Garbage collection and defragmentation are automatically triggered when the system detects an idle period (triggered when the proportion of idle data sectors is below a certain threshold). The process traverses all sector states, recycles sectors marked as invalid, and defragments the file record area. In particular, the defragmentation operation includes: reading the entire 4KB sector data in the file record area into the cache; judging whether the proportion of invalid record items exceeds a preset threshold; if so, clearing all invalid items and rewriting the defragmented data to the SPI Flash. On this basis, the allocation of file record items and data sectors adopts a random strategy, which can ensure that the write operation is evenly distributed across the entire Flash medium, thereby effectively achieving balanced wear and tear and prolonging the service life of the device.

[0059] Preferably, the above-mentioned file record item state mapping table can be stored in the internal RAM in an array structure, each item of which corresponds to a file record item in the SPI Flash. The data sector state mapping table can be managed in bitmap form to save storage space. In addition, the CRC16 check algorithm can be realized by table lookup method to balance the calculation speed and resource overhead. It is worth noting that the method described in the present application can also be applied to other types of serial Flash memory, as long as the write cache size and management parameters are adjusted according to the specific sector size.

[0060] To make the step clearer, the following three typical examples are given to illustrate the file data change:

[0061] Example 1: Only file data content is changed

[0062] Suppose a user opens an existing log file log.txt and appends new log content. When the file is closed, the system detects that the file data has changed, but the file name and other metadata have not changed. The system will perform the following operations: first, allocate a new, erased blank data sector from the data sector state mapping table; then, write the contents of the write cache containing the new data to the new sector; then, since the file size and end sector pointer and other meta-information have changed, the system also needs to allocate a new file record item for this file and write the updated metadata (such as the new file size and the pointer to the new data sector) into it; finally, mark the data sector storing the old data and the old file record item as invalid. At this point, the file closing operation is completed, and the entire process does not need to perform any erase operation.

[0063] Example 2: Only file metadata is changed

[0064] Suppose a user only renames the file old.txt to new.txt without modifying its content. When the file is closed, the system detects that the file name has changed, but the file data content has not changed. The system will perform the following operations: first, allocate a new file record item from the file record item state mapping table; then, write the original metadata (such as file size, data sector pointer) along with the new file name new.txt into the new record item; finally, mark the old file record item containing the original file name old.txt as invalid. Since the data content has not changed, there is no need to allocate a new data sector, and the new record item still points to the original data sector. This process also completely avoids the erase operation.

[0065] Example 3: Both file data and metadata are changed

[0066] Suppose a user opens a configuration file config.ini, modifies some parameter content, and then changes its file attributes (such as read-only flag). When the file is closed, the system detects that both the file data and the metadata have changed. The system will perform the following operations: first, allocate a new data sector for the new data content and write it; then, allocate a new file record item for the new metadata (updated file attributes, new data sector pointer, and file size) and write it; finally, mark the original data sector and file record item as invalid. This example combines the operations of the previous two examples and is the most complex case in the file closing process, but its core mechanism is consistent, that is, to avoid real-time erasing by allocating new resources and marking old resources as invalid.

[0067] The following describes the opening file process, the writing file process, and the closing file process:

[0068] Figure 1 The specific implementation process of the opening file process is as follows: the system first calculates the file name length and the CRC16 check value of the file to be operated, and performs a quick comparison in the file record item state mapping table (File record manage), to find out whether there is a record item with the same file name length and check value. If there is a matching item, the complete file record item in the SPI Flash file information area is read through the index corresponding to the record item, and an accurate file name string comparison is performed. If the accurate comparison is consistent, the file is opened and the file handle structure is returned. If no matching item is found in the mapping table or the accurate comparison fails, the system randomly allocates a record item with an unused state from the file record item state mapping table (File record manage), updates the state of the record item in the mapping table to valid, and saves the relevant information of the new file to the returned file handle buffer, thereby completing the file creation or opening operation.

[0069] Figure 2 The specific implementation process of the writing file process is as follows: when performing file writing operation, the system first judges whether the last data sector of the file has been written full. If it is judged that the last data sector has not been written full, the existing data of the sector is completely loaded into a write cache with a size of 4KB to ensure the continuity of the data. If the last data sector has been written full, the system immediately finds an idle sector with an unused state marker from the data sector state mapping table, selects one of the idle sectors using a random allocation strategy, updates the state of the sector in the mapping table to used, then takes the newly allocated sector as the current operation sector, and resets the write cache pointer to the starting position to prepare for receiving new data. Subsequently, the new data to be written is sequentially appended to the write cache, and the state change of the write cache is monitored in real time. In this process, if it is detected that the write cache is full, the entire cache data is synchronously written into the SPI Flash in batches. After completing the data synchronization, the system clears the write cache and resets the cache pointer, and then judges whether all the data to be written has been completely processed. If the data has not been written completely, the above-mentioned process of allocating a new sector is repeated, and then the new data to be written is sequentially appended to the write cache. If the data has been written completely, the writing file operation is completed. Through this asynchronous batch writing mechanism, multiple scattered small data writing is aggregated into one complete sector writing, and finally all the data is completely written, which significantly reduces the actual erase and write operation times, improves the writing efficiency, and at the same time realizes the balanced wear of the Flash memory through the random allocation strategy, prolonging the service life of the device.

[0070] Figure 3 The specific implementation process of the file closing procedure is as follows: in the file closing phase, the system first judges whether there is file data to be synchronized to the SPI Flash. If there is data to be synchronized, the data synchronization operation is first performed. Then it is checked whether the file information record needs to be synchronized. If it needs to be synchronized, it is judged whether it is the first record. If it is not the first record, the strategy of allocating a new record item and marking the original record item as invalid is adopted. If it is the first record, it is directly synchronized to the SPI Flash. For the deletion operation, the state bit of the corresponding file record item and its data sector is marked as invalid, and the actual physical erasing operation is delayed to be performed by the garbage collection mechanism when the system is idle.

[0071] It should be understood that the above specific embodiments of the present application are only used for illustrative or explanatory purposes of the principles of the present application, and do not constitute a limitation on the present application. Therefore, any modification, equivalent replacement, improvement, etc. made without departing from the spirit and scope of the present application shall be included in the protection scope of the present application. In addition, the appended claims of the present application are intended to cover all variations and modifications falling within the scope and boundary of the appended claims, or the equivalent forms of such scope and boundary.

Claims

1. A method for implementing an embedded file system based on SPI Flash, characterized in that, The method comprises the following steps: S1. File system initialization: read the superblock, traverse all data sectors in the file record area and data area, and establish a file record item state mapping table and a data sector state mapping table; S2. File opening or creation: match the file name length and the check value in the file record item state mapping table, open the file if the match is successful, or allocate a new file record item if the match is not successful; S3. File read-write operation: temporarily store file data in the write cache, asynchronously write the data sectors when the cache is full or the file is closed, and update the data sector state mapping table; S4. File closing: determine whether the file record item or the file data has been changed, allocate a new record item or data sector and mark the original record item or the original data sector as invalid if the change has occurred, and avoid the erasing operation; S5. Garbage collection and fragmentation consolidation: traverse all sector states when the system is idle, recover the invalid sectors and consolidate the file record area fragmentation; wherein, The file record item state mapping table in step S1 comprises the attribute state, the file name length and the file name check value of each record item; The marking of the original record item or the original data sector as invalid in step S4 is achieved by flipping the state bit of the corresponding sector in the SPI Flash from 1 to 0.

2. The method of claim 1, wherein, The check value in step S2 is a CRC16 check value.

3. The method of claim 1, wherein, The size of the write cache in step S3 is consistent with the size of the data sector.

4. The method of claim 1, wherein, The fragmentation consolidation in step S5 comprises: reading the data of a sector in the file record area to the cache; determining whether the proportion of invalid record items exceeds the threshold value; if yes, clearing the invalid items and rewriting them to the Flash.

5. The method of claim 1, wherein, The allocation of the file record item and the data sector both adopts a random strategy.

6. The method of claim 1, wherein, Step S2 further comprises: S2.1 calculating the file name length and the check value of the file to be opened; S2.2 searching for a record item with the same file name length and check value in the file record item state mapping table; S2.3 reading the corresponding file record item in the SPI Flash if the record item exists, and performing file name comparison; S2.4 opening the file and returning the file handle if the comparison is consistent; S2.5 allocating an unused record item from the file record item state mapping table and updating its state to valid if the record item does not exist or the comparison is inconsistent.

7. The method of claim 1, wherein, Step S3 further comprises: S3.1 determining whether the last data sector of the file is full; S3.2 loading the sector data in the data sector to the write cache if the data sector is not full; S3.3 appending the data to be written to the write cache; S3.4 allocating a new sector from the data sector state mapping table and writing the cache data to the new sector if the write cache is full or the file is closed; S3.5 updating the last sector pointer and the file size in the file record item.

8. The method of claim 1, wherein, Step S4 further comprises: S4.1 determining whether the file data or the record item has been changed; S4.2 allocating a new record item or data sector if the change has occurred; S4.3 marking the original record item or the original data sector as invalid, without the need for erasing operation; S4.4 directly synchronizing to the SPI Flash if it is the first writing.

Images