A High-Speed Access Method for Embedded File Systems Based on Asynchronous Buffering
By introducing an asynchronous RAM buffer management layer into the embedded file system, the system lag problem caused by Nor Flash operation latency is solved, and file system operations with fast response and good compatibility are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN JULINGSHEN TECH CO LTD
- Filing Date
- 2026-05-06
- Publication Date
- 2026-07-31
AI Technical Summary
In embedded MCU platforms, page programming and sector erasure operations of Nor Flash are time-consuming, causing system lag when the file system is frequently erased and written. Existing technologies cannot effectively isolate application layer operations from underlying physical delays, affecting system real-time performance.
An asynchronous buffer management layer based on RAM is introduced between the file system driver layer and the physical Flash driver layer. This layer intercepts Flash operation requests and performs data writing and status marking in RAM. Data is then asynchronously synchronized to the physical Flash via a background thread, transforming time-consuming operations into fast RAM operations.
It greatly improves application responsiveness, eliminates lag, has good compatibility, is easy to integrate, and does not require modification of existing file system code.
Smart Images
Figure CN122489005A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the technical field of embedded systems and storage, and specifically relates to a method and apparatus for improving the file system performance of Nor Flash memory in a microcontroller unit platform. Background Technology
[0002] In embedded MCU platforms, file systems such as LittleFS and SPIFFS are commonly used to manage Nor Flash memory. These file systems perform read, write, and erase operations on the Flash memory through interfaces provided by the Flash driver. For example, patent application 202110736016.9 discloses an SPI NOR FLASH test platform, test method, test apparatus, and electronic device. However, the physical characteristics of Nor Flash result in long page programming and sector erase operations, typically measured in milliseconds. When the file system performs frequent erase and write operations, these delays directly cause application stuttering and reduce system real-time performance.
[0003] Existing technical solutions such as Figure 1 As shown, the `prog` and `erase` interfaces of the file system directly trigger corresponding operations on the physical Flash memory, becoming a system performance bottleneck. While performance can be partially improved by optimizing the file system logic itself or increasing the communication rate between the MCU and the Flash chip, the blocking problem caused by the latency of Flash physical operations cannot be fundamentally eliminated. Therefore, a technical solution is needed that can effectively isolate application-layer operations from underlying physical latency and significantly improve the response speed of file system operations to address the above problems. Summary of the Invention
[0004] To address the aforementioned problems, the present invention aims to provide a high-speed access method for embedded file systems based on asynchronous buffering. This method introduces a RAM-based asynchronous buffering layer between the file system driver layer and the physical Flash driver layer, thereby resolving the issues of large response delays and system lag caused by direct physical programming and erasure when operating Nor Flash in existing embedded file systems.
[0005] Another objective of this invention is to provide a high-speed access method for embedded file systems based on asynchronous buffering. This method transforms time-consuming Flash physical operations into fast RAM operations, making file system interface calls almost non-blocking, greatly improving the response speed of applications and eliminating lag.
[0006] Another objective of this invention is to provide a high-speed access method for an embedded file system based on asynchronous buffering. This method is transparent to the upper-layer file system and application layer, requires no modification to the existing file system code, has good compatibility, and is easy to integrate.
[0007] To achieve the above objectives, the technical solution of the present invention is as follows: A high-speed access method for an embedded file system based on asynchronous buffering includes the following steps: Step S1: Allocate a region in the system RAM as a virtual image buffer for Flash, whose logical address space corresponds to the physical Flash region that needs to be managed; Step S2: Intercept all read, write, and erase operation requests from the file system to the Flash; when the file system calls the prog or erase interface, it does not directly operate the physical Flash, but only writes data and marks the status in the corresponding area in the RAM buffer, and immediately returns success, thus enabling the file system operation to be completed at high speed; Step S3: Set up an independent background synchronization thread; this thread monitors the status of the RAM buffer and asynchronously and in batches synchronizes the buffer data that has been written to the physical Flash in the background. Step S4: When the file system or application needs to read data, prioritize reading the latest data from the RAM buffer to ensure data consistency.
[0008] Furthermore, the paths for read and write requests are as follows: application layer, file system, asynchronous buffer management layer, physical Flash driver layer, and Nor Flash hardware; read and write requests are intercepted by the asynchronous buffer management layer and routed to the RAM buffer, while synchronous operations are directed to the physical Flash by a background thread.
[0009] Furthermore, when writing data, if the target physical Flash area is already cached in RAM, the data is directly written to the RAM buffer without triggering the physical Flash driver prog interface call; if the target physical Flash area is not yet cached in RAM, the mapping of the Flash area will be established in RAM first, and then the data will be written to the corresponding RAM buffer.
[0010] Furthermore, when reading data, if the target physical Flash area is already cached in RAM, the data is read directly from the RAM buffer; if the target physical Flash area is not yet cached in RAM, the data is read through the physical Flash driver read interface.
[0011] Furthermore, based on the capacity of the Flash to be managed, a contiguous buffer is allocated in RAM, and a certain number of buffer blocks are allocated based on the RAM space currently available to the MCU; each buffer block contains a data area, an address tag, and a status flag.
[0012] Furthermore, a buffer management driver layer is implemented, which provides the exact same interface as the original Flash driver layer. The .prog and .erase callback functions in the LittleFS configuration are pointed to the corresponding functions in the buffer management driver layer. When LittleFS needs to be programmed, the prog function of the buffer management driver writes the target address and data to the corresponding RAM buffer block, marks the block as "dirty", and then returns successfully. When LittleFS needs to be erased, the erase function of the buffer management driver returns directly without triggering the corresponding operation of the underlying physical Flash driver.
[0013] Furthermore, a low-priority background synchronization thread is created. This thread runs continuously, and its task is to traverse all buffer blocks and find blocks with a "dirty" status. When a dirty block is found, the thread calls the underlying physical Flash driver to perform the actual erase and programming operations, writing the data from the RAM buffer block to the corresponding address in the physical Flash. At this point, the status can be marked as "synchronizing". After completion, the status of the buffer block is set to "clean".
[0014] Furthermore, for read operations, the read function provided by the buffer management driver layer first checks whether the requested address exists in a certain buffer block; if it does, it directly reads the latest data from that buffer block; otherwise, it directly calls the read function of the underlying physical Flash driver.
[0015] Furthermore, to prevent data loss due to power failure before synchronization is complete, critical metadata can be synchronized back to Flash when the system is idle or periodically.
[0016] Furthermore, further optimizations will be made, including: a. Configure the triggering strategy for the synchronization thread; While the buffer management layer is processing read / write requests from application layer threads for a certain buffer, the background synchronization thread synchronizes data from another dirty buffer back to Flash. b. After merging frequently written buffer blocks, write them to the physical Flash all at once.
[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. By converting time-consuming Flash physical operations into fast RAM operations, file system interface calls become virtually unblocked, greatly improving application response speed and eliminating lag.
[0018] 2. By using a background thread for asynchronous synchronization, fragmented write operations can be merged, optimizing the write mode of Flash and potentially reducing the number of erase / write cycles, thereby extending the lifespan of Flash.
[0019] 3. It is transparent to the upper-level file system and application layer, requires no modification to the existing file system code, has good compatibility, and is easy to integrate. Attached Figure Description
[0020] Figure 1 This is a flowchart of the file system interface call process in existing technology.
[0021] Figure 2 This is a block diagram of the system structure implemented by the present invention.
[0022] Figure 3 This is a flowchart of the write request processing of the step buffer management layer implemented in this invention.
[0023] Figure 4 This is a flowchart of the asynchronous buffer management layer read request processing implemented in this invention.
[0024] Figure 5 This is the buffer state transition diagram implemented in this invention. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0026] Reference Figure 2 The diagram shows the system architecture implemented by this invention. From top to bottom, it consists of: application layer, file system, asynchronous buffer management layer provided by this invention, physical Flash driver layer, and Nor Flash hardware. Arrows indicate the paths for read and write requests. Read and write requests are intercepted by the asynchronous buffer management layer and routed to the RAM buffer, while synchronous operations are handled by a background thread pointing to the physical Flash.
[0027] Figure 3 The asynchronous buffer management layer of this invention handles write requests as follows: If the target physical Flash region is already cached in RAM, the data is directly written to the RAM buffer without triggering the physical Flash driver prog interface call; if the target physical Flash region is not yet cached in RAM, a mapping of the Flash region is first established in RAM, and then the data is written to the corresponding RAM buffer.
[0028] Figure 4 The asynchronous buffer management layer of this invention handles read requests as follows: if the target physical Flash area is already cached in RAM, data is read directly from the RAM buffer; if the target physical Flash area is not yet cached in RAM, data is read through the physical Flash driver read interface.
[0029] The high-speed access method for embedded file systems provided by this invention specifically includes the following steps: Step S1: Allocate a region in the system RAM as a virtual image buffer for Flash, whose logical address space corresponds to the physical Flash region that needs to be managed.
[0030] Step S2: Intercept all read, write, and erase operation requests from the file system to the Flash. When the file system calls the prog or erase interface, it does not directly operate on the physical Flash, but only writes data and marks the status in the corresponding area of the RAM buffer, and immediately returns success, thus enabling the file system operation to be completed at high speed.
[0031] Step S3: Set up an independent background synchronization thread. This thread monitors the status of the RAM buffer and asynchronously, in batches, synchronizes buffer data that has been written to the physical Flash memory in the background.
[0032] Step S4: When the file system or application needs to read data, prioritize reading the latest data from the RAM buffer to ensure data consistency.
[0033] Therefore, the present invention also provides an apparatus for implementing the above method, comprising: The buffer management module is used to manage the RAM buffer, handle read, write and erase commands from the file system, and maintain data buffer block markers.
[0034] The synchronous scheduling module runs in the background thread and is used to schedule dirty buffer data to be written to the physical Flash.
[0035] The read / write routing module is used to intercept file system read / write requests and correctly route them to the RAM buffer or physical Flash.
[0036] Figure 5 This is the state transition diagram of the buffer of the present invention. This embodiment takes running the LittleFS file system on the MCU to manage an SPI Nor Flash as an example.
[0037] (1) During system initialization, a contiguous buffer is allocated in RAM based on the capacity of the Flash memory to be managed. For example, if the Flash capacity is 8MB, with 4KB as the smallest management unit, a certain number of buffer blocks are allocated based on the available RAM space of the MCU. Each buffer block contains a data area, an address tag, and a status flag. The status flag is used to identify whether the buffer block is "clean," "dirty," or "synchronizing." The initial status is "clean," meaning there is no data.
[0038] (2) Implement a buffer management driver layer that provides the same interface as the original Flash driver layer. In this invention, the .prog and .erase callback functions in the LittleFS configuration are set to the corresponding functions in the buffer management driver layer. When LittleFS needs programming, the prog function of the buffer management driver writes the target address and data to the corresponding RAM buffer block, marks the block as "dirty," and then returns successfully. When LittleFS needs erasing, the erase function of the buffer management driver returns directly without triggering the corresponding operation of the underlying physical Flash driver.
[0039] (3) Create a low-priority background synchronization thread. This thread runs continuously, and its task is to traverse all buffer blocks and find blocks with a "dirty" status. When a dirty block is found, the thread calls the underlying physical Flash driver to perform the actual erase and programming operations, writing the data of the RAM buffer block to the corresponding address of the physical Flash. At this time, the status can be marked as "synchronizing". After completion, the status of the buffer block is set to "clean".
[0040] (4) For read operations, the read function provided by this driver layer will first check if the requested address exists in a certain buffer block. If it does, the latest data is read directly from the buffer block; otherwise, the read function of the underlying physical Flash driver is called directly.
[0041] (5) To prevent data loss due to power failure before synchronization is complete, key metadata can be synchronized back to Flash when the system is idle or periodically.
[0042] (6) Then further optimization is performed, including: a. Configure the triggering strategy for the synchronization thread.
[0043] When the buffer management layer is processing read / write requests from application layer threads for a certain buffer, the background synchronization thread can synchronize data from another dirty buffer back to Flash. Thus, the system operates in parallel on a macroscopic level during the physical Flash operation, improving operational efficiency.
[0044] b. After merging frequently written buffer blocks, write them to the physical Flash all at once.
[0045] The mainstream Flash memory that supports erasing in 64KB blocks takes less time than the sum of 16 4KB erases. Therefore, when conditions permit, the background synchronization thread can synchronize the buffer data back to the physical Flash in 64KB blocks.
[0046] In summary, the present invention has the following advantages: 1. It can convert time-consuming Flash physical operations into fast RAM operations, making file system interface calls almost unblocked, greatly improving the response speed of applications and eliminating lag.
[0047] 2. By using a background thread for asynchronous synchronization, fragmented write operations can be merged, optimizing the write mode of Flash and potentially reducing the number of erase / write cycles, thereby extending the lifespan of Flash.
[0048] 3. It is transparent to the upper-level file system and application layer, requires no modification to the existing file system code, has good compatibility, and is easy to integrate.
[0049] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An asynchronous buffer-based embedded file system high-speed access method, characterized in that, Includes the following steps: Step S1: Allocate a region in the system RAM as a virtual image buffer for Flash, whose logical address space corresponds to the physical Flash region that needs to be managed; Step S2: Intercept all read, write, and erase operation requests from the file system to the Flash; when the file system calls the prog or erase interface, it does not directly operate the physical Flash, but only writes data and marks the status in the corresponding area in the RAM buffer, and immediately returns success, thus enabling the file system operation to be completed at high speed; Step S3: Set up a separate background synchronization thread; This thread monitors the status of the RAM buffer and asynchronously, in batches, synchronizes buffer data that has been written to the physical Flash in the background. Step S4: When the file system or application needs to read data, prioritize reading the latest data from the RAM buffer to ensure data consistency.
2. The asynchronous buffer-based embedded file system high access method according to claim 1, wherein, The path for read and write requests is as follows: application layer, file system, asynchronous buffer management layer, physical Flash driver layer, and Nor Flash hardware. Read and write requests are intercepted by the asynchronous buffer management layer and routed to the RAM buffer, while synchronous operations are directed to the physical Flash by a background thread.
3. The asynchronous buffer-based embedded file system high access method according to claim 2, wherein, When writing data, if the target physical Flash area is already cached in RAM, the data is written directly to the RAM buffer without triggering the physical Flash driver prog interface call. If the target physical Flash region has not yet been cached in RAM, the mapping of the Flash region will be established in RAM first, and then the data will be written to the corresponding RAM buffer.
4. The asynchronous buffer-based embedded file system high access method according to claim 2, wherein, When reading data, if the target physical Flash area is already cached in RAM, the data is read directly from the RAM buffer; if the target physical Flash area is not yet cached in RAM, the data is read through the physical Flash driver read interface.
5. The asynchronous buffer-based embedded file system high access method according to claim 1, wherein, Based on the capacity of the Flash to be managed, a contiguous buffer is allocated in RAM. Based on the available RAM space of the MCU, a certain number of buffer blocks are allocated. Each buffer block contains a data area, an address tag, and a status flag.
6. The asynchronous buffer-based embedded file system high access method according to claim 5, wherein, Implement a buffer management driver layer that provides the same interface as the original Flash driver layer; point the .prog and .erase callback functions in the LittleFS configuration to the corresponding functions in the buffer management driver layer; when LittleFS needs programming, the prog function of the buffer management driver writes the target address and data to the corresponding RAM buffer block, marks the block as "dirty", and then returns successfully; when LittleFS needs erasing, the erase function of the buffer management driver returns directly without triggering the corresponding operation of the underlying physical Flash driver.
7. The high-speed access method for an embedded file system based on asynchronous buffering as described in claim 8, characterized in that, Create a low-priority background synchronization thread. This thread runs continuously, and its task is to traverse all buffer blocks and find blocks with a "dirty" status. When a dirty block is found, the thread calls the underlying physical Flash driver to perform the actual erase and programming operations, writing the data from the RAM buffer block to the corresponding address in the physical Flash. At this point, the status can be marked as "synchronizing". After completion, the status of the buffer block is set to "clean".
8. The high-speed access method for an embedded file system based on asynchronous buffering as described in claim 7, characterized in that, For read operations, the read function provided by the buffer management driver layer first checks whether the requested address exists in a certain buffer block; if it does, it directly reads the latest data from that buffer block; otherwise, it directly calls the read function of the underlying physical Flash driver.
9. The high-speed access method for an embedded file system based on asynchronous buffering as described in claim 8, characterized in that, To prevent data loss due to power failure before synchronization is complete, critical metadata can be synchronized back to Flash when the system is idle or periodically.
10. The high-speed access method for an embedded file system based on asynchronous buffering as described in claim 9, characterized in that, Further optimizations will be made, including: a. Configure the triggering strategy for the synchronization thread; While the buffer management layer is processing read / write requests from application layer threads for a certain buffer, the background synchronization thread synchronizes data from another dirty buffer back to Flash. b. After merging frequently written buffer blocks, write them to the physical Flash all at once.