Implementation method of embedded file system based on SPI Flash

Abstract

The invention discloses an embedded file system implementation method based on SPI Flash, which comprises the following steps: reading a super block during initialization, and establishing a state mapping table of file record items and data sectors; when the file is opened, quickly matching record items through file name metadata, and if the matching fails, allocating new items; during reading and writing, aggregating data through the write cache, asynchronously updating the data to the flash memory and refreshing the mapping table; if a change occurs when the file is closed, distributing a new space and marking the original space as failure to avoid real-time erasing; and finally, garbage collection and defragmentation are executed in the idle period of the system. According to the method, the direct access to the SPI Flash is reduced through the memory mapping table, the erasing frequency is reduced by utilizing the write cache and an asynchronous mechanism, and the time-consuming erasing operation is transferred to the background, so that the processing efficiency of file operation and the real-time performance of the system are remarkably improved, and meanwhile, the service life of a storage medium is prolonged.

Description

Technical Field

[0001] This invention belongs to the technical field of embedded systems, and particularly relates to an implementation method of an embedded file system based on SPI Flash. Background Technology

[0002] In embedded devices, data storage is often performed by using an external SPI Flash and porting open-source file systems (such as FATFS, LITFS, etc.). However, due to the slow write speed of SPI Flash, the need to operate on a sector-by-sector basis, and the fact that traditional file systems do not fully consider its physical characteristics (such as bit flipping only supporting 1→0 and requiring aligned writing), file creation, reading, writing, and deletion operations take a long time and cannot meet the real-time requirements of high-efficiency and high-performance embedded devices. Summary of the Invention

[0003] (a) Purpose of the invention To overcome the above shortcomings, the present invention aims to provide an embedded file system implementation method based on SPI Flash, so as to solve the problem that the existing embedded file system does not fully consider the physical characteristics of SPI Flash, resulting in low file operation efficiency and failing to meet the needs of high-performance embedded devices.

[0004] (2) Technical solution To achieve the above objectives, the technical solution provided in this application is as follows: An embedded file system implementation method based on SPI Flash includes the following steps: S1. File system initialization: Read the superblock, traverse all data sectors in the file record area and data area, and establish a file record entry status mapping table and a data sector status mapping table; S2. File Opening or Creation: Match the filename length with the checksum in the file record entry status mapping table. If the match is successful, open the file; otherwise, allocate a new file record entry. S3. File read and write operations: File data is temporarily stored in the write cache. When the cache is full or the file is closed, the data sector is asynchronously written and the data sector status mapping table is updated. S4. File Closure: Determine if file records or file data have been changed. If changed, allocate new records or data sectors and mark the original records or data sectors as invalid to avoid erasure operations. S5. Garbage Collection and Defragmentation: When the system is idle, iterate through the status of all sectors, reclaim failed sectors, and defragment the file record area.

[0005] By establishing a mapping table in memory to manage file records and data sector states, the number of direct accesses to the low-speed SPI Flash is greatly reduced. A fast matching strategy based on filename metadata is employed to efficiently open and create files. Write operations are aggregated using a write cache and an asynchronous update mechanism, significantly reducing the frequency of frequent Flash erases and writes. By allocating new space and marking the original space as invalid when a file is closed, time-consuming erase operations on critical file operation paths are completely avoided. Finally, maintenance operations such as garbage collection and defragmentation are moved to system idle periods, effectively preventing these background tasks from interfering with the performance of real-time tasks. In summary, this solution fully adapts to the physical characteristics of SPI Flash, comprehensively solving the key problems of low efficiency and inability to meet the real-time requirements of high-performance embedded devices in traditional file systems, while also helping to even out wear and extend the lifespan of the storage medium.

[0006] In some embodiments, the file record item status mapping table in step S1 includes the attribute status, file name length, and file name check value for each record item.

[0007] This embodiment, by precisely defining the core metadata contained in the mapping table, enables rapid file filtering and location in memory using only a small amount of key information, greatly reducing reliance on slow storage media and improving the efficiency of file management operations.

[0008] In some embodiments, the verification value in step S2 is the CRC16 verification value.

[0009] CRC16 has the advantages of fast calculation speed and low collision probability, making it very suitable for embedded systems, which are sensitive to computing resources, and providing a reliable and efficient hash value for fast filename comparison.

[0010] In some embodiments, the size of the write cache in step S3 is the same as the data sector size.

[0011] This alignment design allows the contents of the cache to be written directly to a free physical sector as a complete unit each time the cache is full, without the need for additional data splitting, merging, or partial programming operations, thus maximizing the efficiency of write operations.

[0012] In some embodiments, marking the original record item or original data sector as invalid in step S4 is achieved by flipping the status bit of the corresponding sector in the SPIFlash from 1 to 0.

[0013] Persistent state marking can be achieved through simple bit-flipping operations, completely avoiding slow erasure on the critical path.

[0014] In some embodiments, defragmentation in step S5 includes: Read data from one sector of the file record area into the buffer; Determine whether the proportion of invalid records exceeds a threshold; If so, clear the invalid items and write them back to Flash.

[0015] When the proportion of invalid records in a sector reaches a certain threshold, it indicates that the sector's space utilization is low. At this point, the entire data sector's data is read into memory, invalid records are removed, and the cleaned-up valid data is written back to Flash. This process reclaims fragmented space. Cleaning is only triggered when fragmentation affects space utilization, avoiding unnecessary overhead, effectively merging storage space, and maintaining the long-term efficient operation of the file system.

[0016] In some embodiments, the allocation of file record items and data sectors adopts a random strategy.

[0017] This embodiment specifies that when new file records or data sectors need to be allocated, they are randomly selected from the free resource pool, rather than using a simple sequential allocation. This strategy aims to distribute write operations evenly across different physical areas of the entire Flash storage medium. This achieves balanced wear on the SPI Flash storage space, preventing certain sectors from prematurely failing due to frequent writes, thereby extending the overall lifespan of the Flash chip and improving system reliability.

[0018] In some embodiments, step S2 further includes: S2.1 Calculate the filename length and checksum of the file to be opened; S2.2 Search the file record entry status mapping table for a record entry with the same filename length and checksum. S2.3 If it exists, read the corresponding file record in the SPI Flash and compare the file names; S2.4 If the comparison matches, open the file and return the file handle; S2.5 If the record does not exist or the comparison is inconsistent, allocate an unused record from the file record status mapping table and update its status to valid.

[0019] In some embodiments, step S3 further includes: S3.1 Determine if the last data sector of the file is full; S3.2 If the data sector is not full, load the sector data in that data sector into the write cache; S3.3 Append the data to be written to the write cache; S3.4 If the write cache is full or the file is closed, allocate a new sector from the data sector status map table and write the cached data to the new sector; S3.5 Updates the last sector pointer and file size in the file record entry.

[0020] In some embodiments, step S4 further includes: S4.1 Determine whether file data or record items have been changed; S4.2 If a change occurs, allocate a new record or data sector; S4.3 Mark the original record or data sector as invalid, no erasure operation is required; If this is the first write operation, S4.4 will be directly synchronized to the SPI Flash. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the process of opening a file in the embedded file system implementation method based on SPI Flash of the present invention; Figure 2 This is a flowchart illustrating the file writing process in the embedded file system implementation method based on SPI Flash of the present invention; Figure 3 This is a flowchart illustrating the process of closing a file in the embedded file system implementation method based on SPI Flash of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0023] In this method, the SPI Flash is managed in 4KB sectors, and the entire storage space is divided into fixed-size regions, with the following physical layout: Superblock: Occupies sector 1 (4KB) and is used to store critical metadata of the file system, such as version number and region partitioning information.

[0024] File Record Area: Following the superblock, this area occupies eight consecutive sectors (32KB in total) and stores all file record entries. Each file record entry is assigned a fixed number (0 to 1024). This area is used for centralized management of file metadata.

[0025] Data area: Occupies all remaining sectors of Flash memory and is used to actually store file data. Each sector used to actually store file data is also called a "data sector", and multiple "data sectors" make up the "data area".

[0026] This invention provides an embedded file system implementation method based on SPI Flash, comprising the following steps: First, file system initialization is performed. This step obtains file system metadata by reading the superblock stored in the first sector of the SPI Flash. Based on this, the file record area and data area are traversed, and then a file record item status mapping table and a data sector status mapping table are established in the microcontroller (MCU) memory. This significantly improves the access efficiency of subsequent file operations. Specifically, the file record item status mapping table contains key information such as the attribute status of each record item (e.g., read-only, hidden), filename length, and filename checksum, while the data sector status mapping table records the usage status of each data sector. It is worth noting that these mapping tables reside in memory during system operation to enable fast lookup.

[0027] Furthermore, the file opening or creation operation performs a fast match between the filename length and a pre-calculated checksum in the file record entry status mapping table. If a match is found, the file is opened directly; otherwise, an unused file record entry is allocated for the new file. Preferably, the checksum is calculated using the CRC16 algorithm, which has low computational complexity and a low probability of collision, making it very suitable for resource-constrained embedded environments. Based on this, this step further includes the following sub-steps: calculating the filename length and CRC16 checksum of the file to be opened; then searching the mapping table for a record entry with the same length and checksum; if it exists, reading the corresponding complete record entry from the SPI Flash for precise filename comparison; if the comparison matches, the file is opened and a handle is returned; if it does not exist or the comparison does not match, a new record entry is allocated and marked as valid.

[0028] File read / write operations temporarily store data using a write buffer whose size is aligned with the physical sectors of the SPI Flash. Data is then asynchronously written to the data sectors when the buffer is full or the file is closed, and the data sector status map is updated synchronously. Specifically, the write buffer size is preferably set to 4KB to match the most common SPI Flash sector size, thereby achieving efficient batch data writing. The process further includes: determining if the last data sector of the file is full; if not, loading its data into the write buffer; appending new data to the buffer; allocating new data sectors and writing data when the buffer is full; and finally updating the relevant pointers and size information in the file record.

[0029] When a file is closed, it's necessary to determine if any file entries or data have been modified. If modified, a new entry or data sector is allocated, and the original entry or data sector is marked as invalid. This marking is achieved by toggling the corresponding status bit in the SPI Flash from 1 to 0, thus completely avoiding time-consuming erase operations. It's worth noting that for the first write, a direct synchronous method is used to write to the SPI Flash to ensure data reliability. This step further includes: determining if data or metadata has changed; if changed, allocating new resources; marking the original resource as invalid; and handling special cases of the first write.

[0030] Garbage collection and defragmentation are automatically triggered during idle periods detected by the system (triggered when the proportion of idle data sectors falls below a certain threshold). This process iterates through all sector states, reclaims sectors marked as invalid, and defragments the file record area. Specifically, the defragmentation operation includes: reading the entire 4KB sector data from the file record area into the buffer; determining whether the proportion of invalid records exceeds a preset threshold; if it does, clearing all invalid records and writing the defragmented data back to the SPI Flash. Furthermore, the allocation of file records and data sectors employs a random strategy, which ensures that write operations are evenly distributed across the entire Flash medium, effectively achieving wear leveling and extending device lifespan.

[0031] Preferably, the aforementioned file record status mapping table can be stored in internal RAM using an array structure, with each entry corresponding to a file record in the SPI Flash. The data sector status mapping table can be managed using a bitmap to save storage space. Furthermore, the CRC16 checksum algorithm can be implemented using a lookup table method to balance computational speed and resource overhead. It is worth noting that the method described in this invention can also be applied to other types of serial Flash memories; simply adjust the write cache size and management parameters according to the specific sector size.

[0032] To illustrate this step more clearly, the following three typical examples demonstrate how file data changes occur: Example 1: Only the file data content has changed. Suppose a user opens an existing log file named log.txt and appends new log content. When the file is closed, the system detects that the file data has changed, but metadata such as the filename remains unchanged. The system will perform the following operations: First, it allocates a new, erased, blank data sector from the data sector status map table; then, it writes the write cache content containing the new data to this new sector; next, since metadata such as the file size and the last sector pointer have changed, the system also needs to allocate a new file record for this file and write the updated metadata (such as the new file size and the pointer to the new data sector) into it; finally, it marks the data sector that originally stored the old data and the old file record as invalid. The file closing operation is now complete, and no erasure operation is required throughout the process.

[0033] Example 2: Only file metadata changes Suppose a user simply renames the file old.txt to new.txt without modifying its content. When the file is closed, the system detects the filename change, but the file data remains unchanged. The system will perform the following operations: First, it allocates a new file entry from the file entry status map; then, it writes the existing metadata (such as file size and data sector pointers) along with the new filename new.txt to the new entry; finally, it marks the old file entry containing the original filename old.txt as invalid. Since the data content has not changed, there is no need to allocate new data sectors, and the new entry still points to the original data sectors. This process also completely avoids the erasure operation.

[0034] Example 3: Both file data and metadata have changed. Suppose a user opens a configuration file named config.ini, modifies some parameters, and then changes its file attributes (such as the read-only flag). When the file is closed, the system detects that both the file data and metadata have changed. The system will perform the following operations: First, it allocates new data sectors for the new data content and writes them; then, it allocates new file records for the new metadata (updated file attributes, new data sector pointers, and file size) and writes them; finally, it marks the existing data sectors and file records as invalid. This example combines the operations of the previous two examples and represents the most complex case in the file closing process, but its core mechanism remains the same: avoiding real-time erasure by allocating new resources and marking old resources as invalid.

[0035] The following describes the processes for opening, writing, and closing a file: Figure 1The specific implementation process of the file opening procedure is as follows: The system first calculates the filename length and CRC16 checksum of the file to be operated on, and performs a quick comparison in the file record status mapping table to check if there is a record with the same filename length and checksum. If a match is found, the complete file record in the SPI Flash file information area is read through the index corresponding to the record, and a precise filename string comparison is performed. If the precise comparison matches, the file is opened and a file handle structure is returned. If no match is found in the mapping table or the precise comparison fails, the system randomly allocates an unused record from the file record status mapping table, updates the status of the record in the mapping table to valid, and saves the relevant information of the new file to the returned file handle cache, thereby completing the file creation or opening operation.

[0036] Figure 2 The specific implementation process of the file writing procedure is as follows: When performing a file writing operation, the system first determines whether the last data sector of the file has been filled. If it is determined that the last data sector is not yet full, the existing data of that sector is fully loaded into a 4KB write buffer to ensure data continuity. If the last data sector is full, the system immediately searches for a free sector marked as unused in the data sector status mapping table, selects one free sector using a random allocation strategy, updates the status of that sector in the mapping table to used, and then uses this newly allocated sector as the current operating sector, resetting the write buffer pointer to the starting position to prepare for receiving new data. Subsequently, the new data to be written is appended to the write buffer in sequence, and the status changes of the write buffer are monitored in real time. During this process, if the write buffer is detected to be full, the entire buffer data is batch synchronously written to the SPI Flash. After data synchronization is completed, the system clears the write buffer and resets the buffer pointer, then determines whether all data to be written has been completely processed. If the data is not yet fully written, the process of allocating new sectors is repeated, and the new data to be written is appended to the write buffer in sequence. If all the data has been written, the file write operation is complete. This asynchronous batch write mechanism aggregates multiple small, scattered data writes into a single complete sector write, ensuring that all data is ultimately written completely. This significantly reduces the number of actual erase / write operations, improves write efficiency, and, through a random allocation strategy, achieves balanced wear of the Flash memory, extending the device's lifespan.

[0037] Figure 3The specific implementation process of the file closing procedure is as follows: During the file closing phase, the system first determines whether there is file data that needs to be synchronized to the SPI Flash. If there is data that needs to be synchronized, the data synchronization operation is performed first. Then, it checks whether the file information record needs to be synchronized. If synchronization is required, it determines whether it is the first record. If it is not the first record, a new record entry is allocated and the original record entry is marked as invalid. If it is the first record, it is directly synchronized to the SPI Flash. For deletion operations, the system marks the status bits of the corresponding file record entry and its data sector as invalid. The actual physical erasure operation is delayed until the system is idle and is performed by the garbage collection mechanism.

[0038] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A method for implementing an embedded file system based on SPI Flash, characterized in that, Includes the following steps: S1. File system initialization: Read the superblock, traverse all data sectors in the file record area and data area, and establish a file record entry status mapping table and a data sector status mapping table; S2. File Opening or Creation: Match the filename length with the checksum in the file record entry status mapping table. If the match is successful, open the file; otherwise, allocate a new file record entry. S3. File read and write operations: File data is temporarily stored in the write cache. When the cache is full or the file is closed, the data is asynchronously written to the data sector and the data sector status mapping table is updated. S4. File Closure: Determine if file records or file data have been changed. If changed, allocate new records or data sectors and mark the original records or data sectors as invalid to avoid erasure operations. S5. Garbage Collection and Defragmentation: When the system is idle, iterate through the status of all sectors, reclaim failed sectors, and defragment the file record area.

2. The method according to claim 1, characterized in that, The file record item status mapping table mentioned in step S1 includes the attribute status, file name length, and file name verification value of each record item.

3. The method according to claim 1, characterized in that, The verification value mentioned in step S2 is the CRC16 verification value.

4. The method according to claim 1, characterized in that, The size of the write cache mentioned in step S3 is the same as the size of the data sector.

5. The method according to claim 1, characterized in that, The step S4 of marking the original record item or original data sector as invalid is achieved by flipping the status bit of the corresponding sector in the SPI Flash from 1 to 0.

6. The method according to claim 1, characterized in that, The fragmentation process described in step S5 includes: Read data from one sector of the file record area into the buffer; Determine whether the proportion of invalid records exceeds a threshold; If so, clear the invalid items and write them back to Flash.

7. The method according to claim 1, characterized in that, The allocation of file record items and data sectors adopts a random strategy.

8. The method according to claim 1, characterized in that, Step S2 further includes: S2.1 Calculate the filename length and checksum of the file to be opened; S2.2 Search the file record entry status mapping table for a record entry with the same filename length and checksum. S2.3 If it exists, read the corresponding file record in the SPI Flash and compare the file names; S2.4 If the comparison matches, open the file and return the file handle; S2.5 If the record does not exist or the comparison is inconsistent, allocate an unused record from the file record status mapping table and update its status to valid.

9. The method according to claim 1, characterized in that, Step S3 further includes: S3.1 Determine if the last data sector of the file is full; S3.2 If the data sector is not full, load the sector data in that data sector into the write cache; S3.3 Append the data to be written to the write cache; S3.4 If the write cache is full or the file is closed, allocate a new sector from the data sector status map table and write the cached data to the new sector; S3.5 Updates the last sector pointer and file size in the file record entry.

10. The method according to claim 1, characterized in that, Step S4 further includes: S4.1 Determine whether file data or record items have been changed; S4.2 If a change occurs, allocate a new record or data sector; S4.3 Mark the original record or data sector as invalid, no erasure operation is required; If this is the first write operation, S4.4 will be directly synchronized to the SPI Flash.

Images