Compatible design method for flash file system of embedded real-time operating system

Abstract

The invention relates to a compatible design method for a flash file system of an embedded real-time operating system, and belongs to the field of embedded real-time operating systems. Fusion and management of functions of the flash memory file system are achieved through the management core layer of the flash memory log file system, a garbage collection mechanism and a power failure protection mechanism are optimized, and the lightweight flash memory log file system suitable for an embedded real-time system is formed. The requirements of a lightweight, high-reliability, high-performance and low-storage-overhead flash log file system can be quickly met in a resource-limited embedded real-time system; and the flash memory equipment driving interface layer can be compatible with Nor type and Nand type flash memory equipment, so that the portability and expandability effectiveness of hardware platform equipment is ensured.

Description

Technical Field

[0001] The invention belongs to the field of embedded real-time operating systems, and in particular relates to a flash memory file system compatible design method for an embedded real-time operating system. Background Art

[0002] As embedded systems develop in the direction of informatization, intelligence and integration, large-scale model applications driven by massive data and data-intensive computing tasks have emerged in application scenarios, which puts forward new requirements for embedded system storage, such as miniaturization, high performance, large capacity, power-off protection and long-term data preservation. It has become a trend to upgrade storage media from traditional hard disks HDD or solid-state drives SSD to more space-saving and higher-performance on-chip Flash memory. The traditional non-logging file system (the most commonly used fat32) of domestic embedded real-time operating systems can no longer meet the recording and storage requirements based on flash media. It is necessary to implement a lightweight, highly reliable, high-performance and low-storage-overhead flash log file system to manage storage.

[0003] At present, domestic embedded real-time operating systems have a relatively complete file system layer, support some file systems, and provide a standard file system POSIX interface and file system function operation table.

[0004] The current implementations of flash log file systems are generally divided into two categories: one is a file system based on the translation layer, and the other is a dedicated file system for flash memory devices.

[0005] The file system implementation method based on the translation layer is to add a translation layer between the file system and the flash memory device. This translation layer simulates the flash memory device as a block device for the file system to control and use. This method maps the virtual block number to the physical address of the actual flash memory device, but this linear mapping has certain problems. First, this mapping method has the problem of wear imbalance. The number of writes to some flash memory physical blocks (such as physical blocks storing attribute information) is much greater than that of other flash memory physical blocks (such as physical blocks storing data). The traditional file system does not consider the problem of wear balance; secondly, there is a durability problem. The flash memory needs to erase the blocks to be written before writing, which causes the writing work of the traditional file system to become a two-step work of writing and erasing. The number of times the flash memory can be erased is limited. Repeated erasing of some physical blocks will greatly shorten the life of the flash memory; finally, there is a problem of power-off data protection. When modifying the data of a block, the data of the entire erase block needs to be added to the memory for modification, and then the entire Flash memory erase block is erased, and finally the modified data is written back to the erase block. If a power outage occurs during the modification process, the data in the entire erase block will be lost.

[0006] Dedicated file system implementation for flash memory devices. This type of file system mainly includes YAFFS, JFFS2, UBIFS, LogFS, etc. These file systems designed for flash memory devices in embedded systems have their own advantages and disadvantages in design and characteristics. For example, JFFS2 provides good reliability and consistency, can be used for Nor flash and Nand flash, supports dynamic node allocation, but does not support direct block erase and is not suitable for large-capacity flash; YAFFS provides high reliability and availability, supports Nor flash and Nand flash, has power-off data protection function, but does not support dynamic node allocation; LogFS has good write performance and is suitable for large-capacity NAND flash, but the technology is not mature yet and the stability needs to be improved; UBIFS has good performance and space utilization, is used for NAND flash, supports dynamic node allocation, but is not the best choice for Nor flash. In addition, the above flash file systems are widely used in embedded LINUX systems, rather than embedded real-time operating systems.

[0007] Based on this, the present invention proposes a method for designing a log file system compatibility layer to mount a flash file system without changing the original file system subsystem. This method allows the file system to be expanded to adapt to various existing and future data storage requirements, which is more convenient and more scalable than directly modifying the file system subsystem to implement a flash log file system. Summary of the invention

[0008] 1. Technical issues to be resolved

[0009] The technical problem to be solved by the present invention is how to provide a flash file system compatible design method for an embedded real-time operating system to meet the needs of resource-constrained embedded real-time systems for the rapid implementation of a lightweight, highly reliable, high-performance and low-storage-overhead flash log file system.

[0010] (II) Technical solution

[0011] In order to solve the above technical problems, the present invention proposes a flash file system compatible design method for an embedded real-time operating system, wherein the flash file system includes the following three modules:

[0012] Flash log file system compatibility layer module: includes a set of file system interfaces for embedded real-time operating system interfaces and a set of POSIX file operations; the file system interface for embedded real-time operating system interfaces implements two data structures OS_filesystem_file_handlers_r and OS_filesystem_operations_table; the data structure OS_filesystem_file_handlers_r is used to complete the flash file system directory operation interface registration, file operation interface registration, and file link interface registration; the data structure OS_filesystem_operations_table is used to register the file system interface; the POSIX file operation interface implements a set of operating system general interfaces;

[0013] Flash log file system management core layer module: used to implement the file read and write functions, address mapping, bad block detection and management, wear leveling, garbage collection and power-off protection functions of the flash log file system;

[0014] Flash device driver interface layer module: extracts the common features of the flash file system, optimizes and simplifies the Flash driver interface, and can adapt to most flash devices. The user application only needs to adapt the interface, which is defined by the structure struct flash_chip_config; after the user adapts the interface structure, call the function Fs_install_drv_flash to mount the driver to the flash file system.

[0015] (III) Beneficial effects

[0016] The present invention proposes a flash memory file system compatibility design method for an embedded real-time operating system. By adopting the flash memory file system compatibility design method based on the embedded real-time operating system, a flash memory file system can be quickly deployed without changing the original file system subsystem through a flash memory log file system compatibility layer, which is more convenient and more scalable than directly modifying the file system subsystem to realize the flash memory log file system; the flash memory log file system management core layer is used to realize the integration and management of the flash memory file system functions, and the garbage collection mechanism and the power-off protection mechanism are optimized to form a lightweight flash memory log file system suitable for the embedded real-time system, which can quickly realize the requirements of the flash memory log file system with lightweight, high reliability, high performance and low storage overhead in the resource-constrained embedded real-time system; the flash memory device driver interface layer can be compatible with Nor-type and Nand-type flash memory devices, thereby ensuring the portability and scalability of the hardware platform device. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1This is a functional block diagram of the flash file system compatibility technology of the present invention;

[0018] Figure 2 This is a flowchart of the garbage collection mechanism;

[0019] Figure 3 This is the schematic diagram of the power-off protection mechanism. DETAILED DESCRIPTION

[0020] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0021] The purpose of the present invention is to propose a design method for flash memory file system compatibility based on an embedded real-time operating system. The method designs a set of log file system compatibility layers to mount the flash memory file system without changing the original file system subsystem, realizes the integration and management of the flash memory file system functions, and optimizes the flash memory file system core to form a lightweight flash memory log file system suitable for embedded real-time systems, meeting the needs of resource-constrained embedded real-time systems for the rapid implementation of a lightweight, highly reliable, high-performance and low-storage-overhead flash memory log file system.

[0022] Figure 1 It is a functional block diagram of the technical solution of the present invention. Figure 1 As shown, the present invention provides a flash memory file system compatibility design method for an embedded real-time operating system. The flash memory file system of the present invention includes the following three modules: a flash memory log file system compatibility layer module, a flash memory log file system management core layer module and a flash memory device driver interface layer module. The flash memory log file system compatibility layer module, the flash memory log file system management core layer module and the flash memory device driver interface layer module are implemented in the embedded operating system and are arranged from top to bottom. The flash memory log file system compatibility layer module is connected to the original file system subsystem and the file system interface, and the flash memory device driver interface layer module is connected to the hardware device layer, namely, Nor type and Nand type flash memory devices.

[0023] 1. Flash log file system compatibility layer module

[0024] This module is a bridge to integrate the flash log file system into the existing file subsystem of the embedded real-time operating system and is implemented in the embedded operating system. This module includes a set of file system interfaces for the embedded real-time operating system interface and a set of POSIX file operations.

[0025] Among them, the file system interface facing the embedded real-time operating system interface is the key to the compatibility layer being attached to the original file subsystem in the present invention, and needs to realize two data structures OS_filesystem_file_handlers_r and OS_filesystem_operations_table. The data structure OS_filesystem_file_handlers_r is used to complete the flash file system directory operation interface registration, file operation interface registration and file link interface registration; the data structure OS_filesystem_operations_table is used to register the file system interface.

[0026] The design of the file system interface of the flash log file system compatibility layer module is the key to mounting the flash file system to the original file subsystem. By replacing the relevant function registration in the original system data structure, the flash file system directory operation interface, file operation interface and file link operating system interface can be quickly registered.

[0027] The specific definitions are as follows:

[0028] Among them, the functions starting with OriginalOS_ are interfaces of the original file system, and the functions starting with flashfs_ are added interfaces related to the flash memory.

[0029] 1) Directory operation interface registration

[0030]

[0031] 2) File operation interface registration

[0032]

[0033]

[0034] 3) File link interface registration

[0035] 4) File system interface registration

[0036]

[0037] The POSIX file operation interface is to implement a set of common operating system interfaces for user development or migration, including file opening and closing, file reading and writing, file position control, file descriptor operation, dup, file and directory management, stat / fstat / mkdir / rmdir, file management, link / unlink / rename, file stream operations, fopen / fclose / fread / fwrite, etc.

[0038] 2. Flash log file system management core layer module

[0039] This module implements the file read and write functions, address mapping, bad block detection and management, wear leveling, garbage collection and power-off protection functions of the flash log file system. The module can be independently linked to the system in the form of a static library. This module is implemented on an embedded operating system. In addition, in order to meet the needs of lightweight, low energy consumption and high reliability of embedded real-time operating systems, it is necessary to further optimize the garbage collection mechanism and power-off protection mechanism. Specific implementation:

[0040] Garbage collection mechanism:

[0041] The garbage collection mechanism of the existing flash log file system usually has a control threshold. When the number of free blocks in a page is less than the threshold, the garbage collection process is started. The file system will find the dirty block to be recycled according to the mapping table or bitmap for managing valid nodes (in the file system, the management method for valid nodes is usually in the form of a mapping table or a bitmap, which is used to mark the position of the "valid node" in the Flash storage page. When the state of the valid node changes (for example, from an empty state to a full state, or from a full state to a dirty state, etc.), the value of the mapping table or bitmap will change accordingly. This process is called "managing a mapping table or bitmap for valid nodes"), read the data content of the valid node into the cache and write it into a new free block, then modify the index pointer corresponding to the file, replace the old page with the new page, and invalidate the original page of the old data. Finally, delete the invalid data, recycle the invalid page, and release the space. However, file systems rarely consider the balance between garbage collection delays, wear leveling, and resource usage. When garbage collection performs data migration, it will take up a lot of CPU time, causing delays in other operations. When garbage collection performs data migration, it will occupy the chips and data channels of the original and target pages of the data, affecting other data read and write operations. Appropriate adjustments need to be made based on specific application scenarios.

[0042] Based on the above problems, the improved garbage collection mechanism is as follows Figure 2As shown, based on the characteristics of the embedded real-time operating system, the garbage collection operation is controlled by starting the foreground execution task and the background daemon task at the same time. Specifically:

[0043] a) The background daemon task is active recycling. When the processor is idle or the daemon task is on other cores of the processor, garbage collection is performed regularly. This task does not affect the execution process of other tasks and does not preempt the processor to cause delays in other file system operations, thereby improving the execution efficiency of the file system. The setting of the execution frequency should take into account wear leveling and write amplification issues.

[0044] b) The foreground execution task is passive recycling. When a write operation occurs, a signal is released to the task. The task starts the recycling operation when it determines that the number of free blocks is less than the set threshold. Use SIT to find invalid data blocks, and use the CAT algorithm to select a block from the selected recyclable blocks for recycling; use the delayed migration strategy to migrate valid data pages in the block. This delayed migration of valid data can reduce the number of writes and improve work efficiency. Then erase the recycled data block, and after writing the checkpoint, the block can be rewritten with data.

[0045] Power-off protection mechanism:

[0046] The power-off protection mechanism is a mechanism to improve reliability by ensuring that the integrity of the metadata is maintained and useful data is restored (protected) to the maximum extent possible when the system suddenly loses power. When the system power is restored, the on-site data can be restored in a timely manner to avoid confusion in the file system and cause boot failures.

[0047] The improved power-off protection mechanism is as follows Figure 3 As shown, it is implemented by the log management module, power-off protection module and file access module in the flash log file system.

[0048] a) Log management module, which divides an independent space in the disk as log space, saves log super blocks and log description information, and is used to record all basic information in the file system log area. This module determines whether the file operation transaction is to be submitted to the disk, whether the transaction data has been refreshed to the disk, and realizes disk synchronization;

[0049] b) Power-off protection module: The embedded real-time operating system runs the power-off protection module function by starting a special background task, reading the super block of the log partition to determine whether the file needs to be restored. If the file system is normally unloaded, no recovery is required; otherwise, the start and end points of the log need to be found in the log space, the description information is obtained to match the file processing method, and the corresponding sectors and blocks of the log area file are found. Finally, the file is restored to the Flash memory, and the information in the log super block is updated to complete the data recovery.

[0050] 3. Flash device driver interface layer module

[0051] The driver interface layer module in the present invention supports both norflash and nandflash flash memory types, and supports 64-bit / 32-bit compatible access. In order to meet the requirements of high performance and low storage overhead of embedded real-time operating systems, the driver interface needs to be optimized. The present invention proposes to extract the common characteristics of the flash file system, optimize and simplify the Flash flash driver interface, and adapt to most flash memory devices. The user application only needs to adapt to the interface. The interface is defined by the structure struct flash_chip_config as follows:

[0052] typedefstruct flash_chip_config

[0053] {

[0054] const char*name; / / name

[0055] / / Chip parameters

[0056] intn_blocks; / / Total number of flash chip blocks

[0057] intbytes_per_block; / / Flash chip block size, that is, the actual erase area size. nandflash can be configured without (and does not need to include spare area), norflash must be configured (including spare area)

[0058] / / Configure the parameters for the log file system

[0059] intpages_per_block; / / Number of pages configured for each block

[0060] intdata_bytes_per_page; / / Configure the size of the data area for each page

[0061] int spare_bytes_per_page; / / Configure spare area size for each page

[0062] / / Configure the system to be installed between [start_block, end_block], 0 <= start_block <end_block<=(n_blocks-1)

[0063] int start_block;

[0064] int end_block;

[0065] / / intbadmarker_offset that the flash driver needs to configure; / / Offset of badmark in the spare area, norflash is -1

[0066] intdataecc_offset; / / Offset of data area ecc in spare area, if not needed = -1

[0067] inttags_offset; / / tag offset in spare area

[0068] ecc_modes_t ecc_mode; / / Driver calculation ecc mode,

[0069] intbus_width_shift; / / Bus width. If it is composed of multiple flash chips, the bus width may require buffer alignment. =0 no alignment requirement, =1 alignment to 2 bytes, =2 alignment to 4 bytes

[0070] / / Initialize flash, called when mounting, returns -1 if failed, 0 if successful, can be empty

[0071] int(*_init)(void);

[0072] / / Reset flash, called when unmounting, returns -1 if failed, 0 if successful, can be empty

[0073] int(*_deinit)(void);

[0074] / / Erase block, return -1 if failed, return 0 if successful, cannot be empty

[0075] int(*_erase)(uint32_tblock_num);

[0076] int(*_read)(uint32_t page_num,uint32_t column_addr,uint8_t*buf,intbytes);

[0077] int(*_write)(uint32_t page_num,uint32_t column_addr,const uint8_t*buf,intbytes);

[0078] / / Print possible statistics, can be empty

[0079] void(*_dump)(void);

[0080] } flash_chip_config_t;

[0081] After the user has adapted the interface structure, the function Fs_install_drv_flash is called to mount the driver to the flash file system.

[0082] struct Fs_dev*Fs_install_drv_flash(const char*mount_path,flash_chip_config_t*chip);

[0083] The key of the present invention:

[0084] The present invention proposes a flash memory file system compatible design method based on an embedded real-time operating system, which allows the data storage requirements of Nor and Nand flash memory devices to be met through the expansion of the file system. The design of the file system interface of the compatibility layer is the key to the flash memory file system being attached to the original file subsystem, and the flash memory file system directory operation interface, file operation interface and file link operating system interface are quickly registered by replacing the relevant function registration in the original system data structure.

[0085] The garbage collection operation is controlled by the background daemon task and the front-end execution task at the same time, the idle time of the processor is fully utilized to recycle the data blocks in the background, and the delayed migration strategy is used to migrate the valid data pages in the blocks, thereby improving the working efficiency of the file system.

[0086] Effects of the present invention:

[0087] The design method for flash memory file system compatibility based on embedded real-time operating system of the present invention can quickly deploy a set of flash memory file system without changing the original file system subsystem through the flash memory log file system compatibility layer, which is more convenient and more scalable than directly modifying the file system subsystem to realize the flash memory log file system; the flash memory log file system management core layer is used to realize the integration and management of the flash memory file system functions, and the garbage collection mechanism and the power-off protection mechanism are optimized to form a lightweight flash memory log file system suitable for embedded real-time systems, which can quickly realize the requirements of lightweight, highly reliable, high-performance and low storage overhead flash memory log file system in resource-constrained embedded real-time systems; the flash memory device driver interface layer can be compatible with Nor-type and Nand-type flash memory devices, ensuring the portability and scalability of hardware platform devices.

[0088] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A flash file system compatible design method for an embedded real-time operating system, characterized in that: The flash file system includes the following three modules: Flash log file system compatibility layer module: includes a set of file system interfaces for embedded real-time operating system interfaces and a set of POSIX file operations; the file system interface for embedded real-time operating system interfaces implements two data structures OS_filesystem_file_handlers_r and OS_filesystem_operations_table; the data structure OS_filesystem_file_handlers_r is used to complete the flash file system directory operation interface registration, file operation interface registration, and file link interface registration; the data structure OS_filesystem_operations_table is used to register the file system interface; the POSIX file operation interface implements a set of operating system general interfaces; Flash log file system management core layer module: used to implement the file read and write functions, address mapping, bad block detection and management, wear leveling, garbage collection and power-off protection functions of the flash log file system; Flash device driver interface layer module: extracts the common features of the flash file system, optimizes and simplifies the Flash driver interface, and can adapt to most flash devices. The user application only needs to adapt the interface, which is defined by the structure struct flash_chip_config; after the user adapts the interface structure, call the function Fs_install_drv_flash to mount the driver to the flash file system.

2. The flash file system compatible design method of the embedded real-time operating system as claimed in claim 1, characterized in that: The POSIX file operation interface includes file opening and closing, file reading and writing, file position control, file descriptor operation, dup, file and directory management, stat / fstat / mkdir / rmdir, file management, link / unlink / rename, and file stream operations, fopen / fclose / fread / fwrite.

3. The flash file system compatible design method of the embedded real-time operating system as claimed in claim 1, characterized in that: The flash log file system compatibility layer module realizes the fast registration of the flash file system directory operation interface, file operation interface and file link operating system interface by replacing the relevant function registration in the original system data structure.

4. The flash file system compatible design method of the embedded real-time operating system as claimed in claim 1, characterized in that: The flash log file system management core layer module is independently linked to the system in the form of a static library.

5. The flash file system compatible design method of the embedded real-time operating system as claimed in claim 1, characterized in that: Garbage collection mechanism: Control garbage collection operations by starting foreground execution tasks and background daemon tasks at the same time; The background daemon task is active recycling. When the processor is idle or the daemon task is on other cores of the processor, garbage collection is performed regularly. This task does not affect the execution process of other tasks and does not preempt the processor to cause delays in other file system operations. The foreground execution task is passive recycling. When a write operation occurs, a signal is released to the task. When the task determines that the number of free blocks is less than the set threshold, the recycling operation is started.

6. The method for designing a flash memory file system compatible with an embedded real-time operating system as claimed in claim 5, characterized in that: When actively recycling, the execution frequency should be set to take into account wear leveling and write amplification issues.

7. The method for designing a flash file system compatible with an embedded real-time operating system as claimed in claim 5, characterized in that: During passive recycling, invalid data blocks are found through SIT, and a block is selected from the selected recyclable blocks using the CAT algorithm for recycling; the valid data pages in the block are migrated using the delayed migration strategy, and then the recycled data block is erased. After the checkpoint is written, the block can be rewritten with data.

8. The method for designing a flash memory file system compatible with an embedded real-time operating system as claimed in claim 1, characterized in that: The power-off protection mechanism is implemented by the log management module, power-off protection module and file access module in the flash log file system; Log management module: divides an independent space in the disk as log space, saves log super blocks and log description information, and is used to record all basic information of the file system log area; This module determines whether the file operation transaction is to be submitted to the disk and whether the transaction data has been refreshed to the disk, thus realizing disk synchronization. Power-off protection module: The embedded real-time operating system runs the power-off protection module function by starting a special background task, reading the super block of the log partition to determine whether the file needs to be restored. If the file system is unmounted normally, no recovery is required; otherwise, it is necessary to find the start and end points of the log in the log space, obtain the description information to match the file processing method, and find the corresponding sectors and blocks of the log area file; finally, restore the file to the Flash memory, and update the information in the log super block to complete the data recovery.

9. The method for designing a flash file system compatible with an embedded real-time operating system as claimed in claim 1, characterized in that: The flash device driver interface layer module supports both norflash and nandflash flash memory types, and supports 64-bit / 32-bit compatible access.

10. The flash file system compatible design method of the embedded real-time operating system according to claim 1, characterized in that: The flash log file system compatibility layer module, the flash log file system management core layer module and the flash device driver interface layer module are implemented in the embedded operating system and are arranged from top to bottom. The flash log file system compatibility layer module is connected to the original file system subsystem and file system interface, and the flash device driver interface layer module is connected to the hardware device layer, namely, Nor and Nand flash memory devices.

Images