A partition management method and system for an embedded storage system

Abstract

The application discloses a partition management method of an embedded storage system, comprising the following steps: partition division: dividing the total storage space of a nonvolatile memory into at least two independent functional partitions, wherein the independent functional partitions comprise a file system partition and an original storage partition; partition adaptation management: setting an embedded file system and a corresponding file operation interface for the file system partition, and setting an original storage access interface for the original storage partition; address mapping: realizing the mapping of a logical storage unit to a physical storage address through an address conversion mechanism; boundary protection: limiting the operation range of each independent functional partition through an effectiveness check mechanism to avoid cross-partition out-of-bound access. The application solves the technical problems of the prior art, such as the single storage mode in the embedded storage system being unable to consider two types of data storage requirements, the address mapping mechanism being nonstandard, the partition boundary protection being absent, and the file system adaptation being poor.

Description

Technical Field

[0001] This invention relates to the field of embedded storage technology, and in particular to a partition management method and system for embedded storage systems. Background Technology

[0002] Embedded devices are widely used in industrial control, IoT sensing, smart home and other fields. Their storage systems need to meet the storage requirements of two types of data at the same time: one is structured data such as configuration files and operation logs, which needs to be frequently queried, modified and classified; the other is raw binary data collected by sensors, which needs to be stored at high speed and continuously, and has extremely high requirements for storage latency.

[0003] Existing embedded storage management solutions have significant drawbacks: First, using a single storage model to manage the entire storage space, while using a file system (such as FATFS or SPIFFS) to manage all data can meet the flexible management needs of structured data, the metadata maintenance of the file system consumes additional hardware resources, leading to increased latency in raw data storage and making it unsuitable for high-speed acquisition scenarios. If a direct access mode to raw sectors is used, although high-speed storage of raw data can be achieved, the lack of file management capabilities makes the classification, retrieval, and modification of structured data extremely difficult, increasing the complexity of application development. Second, existing partition management solutions do not clearly define the functional boundaries of partitions, easily leading to cross-partition access problems, resulting in corruption of system kernel data or user data. Although some solutions divide into multiple partitions, the data partitions are highly coupled with the system boot partitions (such as the U-boot partition and kernel partition), and data partition management operations can easily affect the normal operation of the system. Third, traditional embedded storage systems often use general-purpose file systems with high resource consumption, unsuitable for embedded devices with weak computing power and small memory, such as microcontrollers, and lack robust power-loss protection and wear-leveling mechanisms, resulting in poor storage system stability and short lifespan.

[0004] Therefore, there is an urgent need for an embedded storage partition management solution that balances flexible management of structured data with high-speed storage of raw data, has boundary protection capabilities, and is adaptable to resource-constrained scenarios, in order to address the shortcomings of existing technologies. Summary of the Invention

[0005] The purpose of this invention is to provide a partition management method and system for embedded storage systems, in order to solve the technical problems in existing embedded storage systems where a single storage mode cannot meet the needs of two types of data storage, the address mapping mechanism is not standardized, partition boundary protection is lacking, and file system adaptability is poor.

[0006] To achieve the above objectives, the present invention provides a partition management method for an embedded storage system, comprising the following steps: Partitioning: Divide the total storage space of the non-volatile memory into at least two independent functional partitions, including a file system partition and a raw storage partition; Partition adaptation management: Set up embedded file systems and corresponding file operation interfaces for file system partitions, and set up raw storage access interfaces for raw storage partitions; Address mapping: The mapping from logical storage units to physical storage addresses is achieved through an address translation mechanism; Boundary protection: Restrict the scope of operations on each independent functional partition through a validity verification mechanism to prevent cross-partition access.

[0007] Preferably, the file system partition occupies n% of the total storage space, and the original storage partition occupies m% of the total storage space, where 0 < n + m ≦ 100.

[0008] Preferably, the file system partition is used to store structured data, and the raw storage partition is used to store raw acquired data; the structured data includes configuration files and log files, and the raw acquired data includes sensor acquired data.

[0009] Preferably, the file operation interface is used for at least one of file creation, deletion, reading and writing, directory management, and file attribute management.

[0010] Preferably, the raw storage access interface is used to include at least one of reading data by storage cell number, writing data by storage cell number, and erasing one or more storage cells, wherein the storage cell is a sector.

[0011] Preferably, the total storage space of the non-volatile memory also includes a Uboot partition and a kernel partition. The Uboot partition is used for hardware initialization and system kernel booting, and the kernel partition is the system kernel that runs on the embedded device. The file system partition and the raw storage partition are dedicated partitions for data storage, and the partitioning and management of the file system partition and the raw storage partition are independent of the operation of the Uboot partition and the kernel partition.

[0012] Preferably, the non-volatile memory is Flash memory, and the embedded file system is LittleFS.

[0013] In another aspect, the present invention provides a partition management system for an embedded storage system, comprising: The partitioning module is used to divide the total storage space of the non-volatile memory into at least two independent functional partitions, including a file system partition and a raw storage partition. The adaptation management module is used to configure the embedded file system and corresponding file operation interface for the file system partition, and to configure the raw storage access interface for the raw storage partition. The address mapping module is used to map logical storage units to physical storage addresses through an address translation mechanism; The boundary protection module is used to restrict the scope of operations on each independent functional partition through a validity verification mechanism, so as to avoid cross-partition out-of-bounds access.

[0014] Preferably, the non-volatile memory is Flash memory; the embedded file system is LittleFS; the file operation interface is used for file creation, deletion, reading and writing, directory management and file attribute management, and the raw storage access interface is used for reading data by sector number, writing data by sector number and erasing one or more sectors.

[0015] The beneficial effects of this invention are: First, by dividing the same physical storage device into file system partitions and raw storage partitions, and providing corresponding access interfaces for each, the embedded system can select the most appropriate access method based on data type and application scenario. Structured data is managed using the file system, which is simple and convenient; unstructured or high-speed data is accessed using raw data, which is efficient and accurate.

[0016] Second, file system partitions meet the needs of routine data management, while raw storage partitions eliminate redundant overhead in the file system, providing a fast channel for data access that requires high throughput or low latency.

[0017] Third, by using address mapping and boundary protection mechanisms, access isolation between different partitions is ensured, effectively preventing cross-partition data corruption caused by program bugs or abnormal operations, and improving the overall stability of the system.

[0018] Fourth, by rationally partitioning the storage space, the limited embedded storage resources can be utilized most effectively, avoiding the performance or functional compromises caused by a single storage management mode. Attached Figure Description

[0019] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0020] Figure 1 This is a partition layout diagram of the embedded storage system in an embodiment of the present invention.

[0021] Figure 2 This is an architectural block diagram of the embedded storage system in an embodiment of the present invention.

[0022] Figure 3 This is a flowchart of the file system initialization process in an embodiment of the present invention.

[0023] Figure 4 This is a flowchart of the original storage area access operation in an embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram of the address mapping principle in an embodiment of the present invention. Detailed Implementation

[0025] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Example

[0026] This embodiment takes an IoT temperature and humidity sensor device as an example. The device uses an STM32L476RG microcontroller as the main control chip, a W25Q64 SPI Flash (W25Q64 serial peripheral interface flash memory) with a capacity of 8MB, and FreeRTOS (real-time operating system) as the embedded operating system. The specific implementation steps are as follows: First, perform partitioning step S1: Use a partitioning tool to divide the 8MB total storage space of the W25Q64 SPI Flash into four independent functional partitions, refer to... Figure 1 The specific division scheme is as follows: 1. Uboot partition: Occupies 5% of the total storage space (409.6KB), with an address range of 0x000000-0x00064000. It is used to store the bootloader and its main function is to complete the initialization of the microcontroller hardware (such as CPU, memory, and SPI interface initialization) and load the system kernel of the boot kernel partition.

[0027] 2. Kernel partition: Occupies 15% of the total storage space (1228.8KB), with an address range of 0x00064000-0x001C8000. It is used to store the FreeRTOS system kernel, providing core support for the overall operation of the device.

[0028] 3. File system partition: Occupies 30% of the total storage space (2457.6KB), with addresses 0x001C8000-0x00432000, used to store structured data, such as device WiFi configuration files (key-value pair format) and temperature and humidity acquisition logs (timestamp + temperature value + humidity value format).

[0029] 4. Raw storage partition: occupies 50% of the total storage space (4096KB), with an address range of 0x00432000-0x00832000. It is used to store the raw binary data collected by the temperature and humidity sensor (each data is 2 bytes, corresponding to the raw sampled values ​​of temperature and humidity respectively).

[0030] The file system partition and the raw storage partition are dedicated partitions for data storage. Their partitioning and subsequent management operations (such as file reading and writing, sector erasure) are independent of the Uboot partition and the kernel partition. This ensures that system startup and kernel operation are not interfered with by data management operations.

[0031] Then, perform partition adaptation management step S2, refer to... Figure 2 : 1. File system partition adaptation: corresponding Figure 2 The interaction logic between the "storage media layer - file system partition" and the "hardware abstraction layer" is described. The file system partition in the "storage media layer" is the physical carrier for structured data storage. In this embodiment, the LittleFS embedded file system is configured and mounted for this file system partition, following the specified procedures. Figure 3 The logical execution of the file system initialization flowchart is as follows: First, the system initialization phase is entered, sequentially completing "initializing memory connection" (establishing a hardware communication link between the STM32L476RG microcontroller and the W25Q64 SPI Flash to ensure normal interaction of the underlying hardware), "registering hardware operation functions" (registering the underlying driver functions such as Flash read, write, and erase to the file system adaptation layer to provide hardware operation support for the file system), and "initializing file system configuration" (loading the core parameters of LittleFS such as sector size, mount point, and cache size to adapt to the W25Q64 SPI in this embodiment). (Flash characteristics); After system initialization, the system enters the file system initialization phase. First, "File System Initialization" is executed, initializing the LittleFS control structure. Then, "File System Mounting" is attempted. The adaptation management module calls the LittleFS file system mount verification interface to verify the mount result. If the mount fails (e.g., due to file system corruption, unformatted initial use, etc., the verification result is "No"), the adaptation management module triggers a "Formatting" operation to rebuild the file system. After completion, it returns to the "File System Mounting" step. If the mount succeeds (verification result is "Yes"), the system enters the "File System Management" phase, and the file system partition officially acquires structured data storage capabilities. Based on this, standardized file operation interfaces are encapsulated using LittleFS, specifically including file creation, file writing, file reading, directory management, and file attribute management interfaces. These interfaces can respectively implement functions such as structured file creation, data writing, data reading, log directory classification management, and file creation / modification time query. Figure 2 The functional modules of the "hardware abstraction layer" are exposed to the "application layer" and can be called by applications to perform structured data management operations such as WiFi configuration file creation and temperature and humidity collection and logging.

[0032] 2. Original storage partition adaptation: corresponding Figure 2 The interaction logic between the "Storage Media Layer - Raw Storage Partition" and the "Hardware Abstraction Layer" is described. The raw storage partition in the "Storage Media Layer" is the physical carrier for raw data storage. In this embodiment, the storage unit of this raw storage partition is a sector (the size of a single sector in a W25Q64 SPI Flash is 4096 bytes). The core of the adaptation management is to encapsulate three types of raw storage access interfaces for this raw storage partition: a read interface by sector number, a write interface by sector number, and a sector erase interface. These interfaces also belong to... Figure 2 The unique feature of the "Hardware Abstraction Layer" module is that it directly calls the Flash underlying driver function of the "Storage Media Layer", skipping the file system metadata maintenance process, thereby enabling the "Application Layer" to read and write data to the "raw storage partition" at high speed.

[0033] The process by which applications access sectors through these interfaces must follow... Figure 4 The logical execution process is as follows: First, the application (such as a temperature and humidity acquisition program) calls the raw storage access interface (such as the sector_write interface) encapsulated by the adaptation management module according to the raw data storage requirements, and passes in three core parameters: the target partition (raw storage partition), the logical sector number (such as logical sector 10), and the raw binary data to be stored (such as 2 bytes of temperature and humidity data); then, after receiving the request, the raw storage access interface of the hardware abstraction layer first performs parameter format verification (such as verifying whether the logical sector number is a valid integer and whether the data length meets the interface requirements). After the verification is successful, the address mapping module is automatically triggered; subsequently, refer to Figure 5 The address mapping module calculates the target physical address (physical address = partition start address + logical sector number × single sector size) according to a preset formula (e.g., 0x00432000 + 10 × 4096 = 0x0043C000 in this embodiment) and calls the system kernel interface (e.g., the ioremap function) to map the physical address to a virtual address operable by the system kernel. Then, the adaptation management module calls the W25Q64 SPI Flash underlying driver function (e.g., the SPI write operation function) through the mapped virtual address to perform the actual sector write operation. After the operation is completed, the address mapping module calls the kernel interface (e.g., the iounmap function) to release the mapping relationship between the virtual address and the physical address and release kernel resources. Finally, the adaptation management module feeds back the "operation success / failure" result (including error code and reason, such as "illegal sector number") to the application through the raw storage access interface.

[0034] Next, step S3, address mapping, is executed.

[0035] In this embodiment, the address mapping module implements the conversion between the W25Q64 SPI Flash physical address and the FreeRTOS system kernel virtual address through a standardized process. The specific process and implementation are as follows: 1) When the temperature and humidity acquisition program needs to write raw data to the raw storage partition, it sends a request to the driver layer by calling the ioctl function. The parameters passed in include the target physical address (calculated by the address formula, specifically 0x0043C000) and the data length (2 bytes). 2) After receiving a data storage request, the driver's ioctl processing function first performs a preliminary validity check on the passed parameters. After confirming that the passed parameter format is correct, it calls the ioremap function to map the passed physical address 0x0043C000 to a virtual address available to the system kernel (such as 0xFFFF0000, the specific address is allocated by the system kernel). 3) The driver layer uses the allocated virtual address 0xFFFF0000 to call the writel function to write 2 bytes of raw temperature and humidity data to the physical address corresponding to the W25Q64 SPI Flash; 4) After the original temperature and humidity data is written, the driver layer calls the iounmap function to unmap the virtual address 0xFFFF0000 and the physical address 0x0043C000, release the system kernel resources, and then return the "write successful" result to the application.

[0036] Through this process, the application never directly manipulates physical addresses, strictly adhering to FreeRTOS's virtual memory management rules. Furthermore, if the physical address changes due to a replacement of the Flash memory, only the parameters of the address mapping module need to be modified, without altering the application code, effectively improving system portability.

[0037] For example, if an application needs to write raw temperature and humidity data to logical sector 10 of the raw storage partition, and the starting physical address of the raw storage partition is known to be 0x00432000, then the target physical address can be calculated using the formula (physical address = partition start address + logical sector number × single sector size): 0x00432000 + 10 × 4096 = 0x0043C000. This calculation process is automatically completed by the address mapping module. The application only needs to pass in two parameters: "raw storage partition" and "logical sector 10" to achieve accurate data storage.

[0038] Finally, step S4 is executed: boundary protection. In this embodiment, the boundary protection module specifically implements boundary protection through the following methods: 1) The driver layer predefines the legal access parameters for each partition. The starting address of the raw storage partition is SA=0x00432000, the total length is FLA_LEN=4096KB, the corresponding legal address range is [0x00432000, 0x00832000), and the logical sector number range is 0-1023; the starting address of the file system partition is SA=0x001C8000, the total length is FLA_LEN=2457.6KB, and the legal address range is [0x001C8000, 0x00432000). 2) The boundary protection module receives the storage operation request from the application, parses it to obtain the request parameters, including the target partition type (such as the original storage partition), the requested access address (or logical sector number), and the data length; 3) If a request is made to access logical sector 1024 of the original storage partition, it is first converted to a physical address using the address formula: 0x00432000 + 1024 × 4096 = 0x00832000. It is then determined whether this address is within the valid range [0x00432000, 0x00832000] of the original storage partition. Since 0x00832000 is not less than the maximum value of the range, the address verification fails. If the application requests to write 5000 bytes of data to logical sector 10 of the original storage partition, the calculated address + data length = 0x0043C000 + 5000 = 0x0043D388, which exceeds the address limit of 0x0043D000 corresponding to the capacity of a single sector (4096 bytes). Therefore, the length verification fails. 4. If any verification fails, the boundary protection module immediately terminates the current operation, returns the corresponding error message to the application (such as "illegal logical sector number" or "data length exceeds the limit"), and does not perform any physical layer read or write operations; if all verifications pass, the address mapping module and partition adaptation management module are allowed to execute subsequent operations. Example

[0039] This embodiment provides a partition management system for implementing the partition management method of the above-described embedded storage system. This partition management system is configured in an STM32L476RG microcontroller and works in conjunction with a W25Q64 SPI Flash. The functions and specific implementation methods of each core module are as follows: The partitioning module is primarily responsible for the functional partitioning planning of the total storage space of the W25Q64 SPI Flash. Specifically, a partitioning configuration function is written in C language, which permanently defines the starting address, capacity, and function of the U-boot partition, kernel partition, file system partition, and raw storage partition. Simultaneously, partition isolation logic is integrated within the function to ensure that the partitioning and management of the file system partition and raw storage partition do not affect the normal operation of the U-boot and kernel partitions. This configuration function is automatically executed when the device powers on and boots, completing the partition initialization.

[0040] The partition adaptation management module is primarily responsible for providing adapted storage access capabilities for different partitions. In its implementation, it integrates the LittleFS open-source library to provide lightweight file system support for file system partitions and encapsulates file operation interfaces based on LittleFS, including lfs_file_create (create), lfs_file_write (write), and lfs_file_read (read). Simultaneously, it encapsulates direct sector access interfaces for the original storage partitions, covering core operations such as sector_read (read), sector_write (write), and sector_erase (erase). Furthermore, the module integrates file system mount / unmount management logic to ensure stable operation of both data partitions and ultimately provides a unified function call interface for easy development of upper-layer applications.

[0041] The address mapping module addresses the issue of user-space applications being unable to directly access hardware physical addresses. Specifically, it uses C language to write address mapping processing functions, integrating ioctl request parsing, ioremap mapping, and iounmap demapping to achieve a standardized conversion from logical storage units to physical addresses. The workflow is as follows: first, it parses the physical address and data length parameters passed from the application via ioctl; then, it calls ioremap to convert the physical address to a system kernel virtual address and passes it to the adaptation management module for hardware operations; after the operation is complete, it calls iounmap to demapping and returns the result to the application. Simultaneously, the address mapping module includes an address calculation formula function that can calculate the target physical address based on the logical sector number and partition start address, which can be called by the application when making a request. This design abstracts away the details of the underlying physical storage, decoupling upper-layer applications from physical addresses and improving system portability.

[0042] The boundary protection module is the core of ensuring storage system security. It uses a verification function written in C to rigorously control application storage access requests. This verification function is automatically triggered when an application calls the storage access interface. First, it reads the pre-defined partition start address (SA) and total length (FLA_LEN) parameters within the function. Then, it parses the target partition, access address / logical sector number, data length, and other parameters passed by the application. Next, it performs dual logic for address and length verification (implemented through conditional statements). If the verification fails, it immediately returns the corresponding error code (e.g., 0x01 for invalid address, 0x02 for invalid length) and terminates subsequent operations. If the verification succeeds, it returns a success code, triggering the address mapping module and the adaptation management module to execute subsequent processes. This design restricts application access to a preset "secure access domain," preventing cross-partition access from damaging the file system or critical system data.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the embodiments of the present invention have been described in detail, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of protection of the claims of the present invention.

Claims

1. A partition management method for an embedded storage system, characterized in that, Includes the following steps: S1 Partitioning: Divide the total storage space of the non-volatile memory into at least two independent functional partitions, wherein the independent functional partitions include a file system partition and a raw storage partition; S2 Partition Adaptation Management: Set up an embedded file system and corresponding file operation interface for the file system partition, and set up a raw storage access interface for the raw storage partition; S3 address mapping: It realizes the mapping from logical storage units to physical storage addresses through an address translation mechanism; S4 boundary protection: Restricts the scope of operations on each independent functional partition through a validity verification mechanism to prevent cross-partition access.

2. The partition management method for an embedded storage system according to claim 1, characterized in that, The file system partition occupies n% of the total storage space, and the original storage partition occupies m% of the total storage space, where 0 < n + m ≦ 100.

3. The partition management method for an embedded storage system according to claim 2, characterized in that, The file system partition is used to store structured data, and the raw storage partition is used to store raw collected data; the structured data includes configuration files and log files, and the raw collected data includes sensor collected data.

4. The partition management method for an embedded storage system according to claim 1, characterized in that, The file operation interface is used for at least one of the following: file creation, deletion, reading and writing, directory management, and file attribute management.

5. The partition management method for an embedded storage system according to claim 1, characterized in that, The original storage access interface includes reading data by storage cell number, writing data by storage cell number, and erasing at least one of one or more storage cells, wherein the storage cell is a sector.

6. The partition management method for an embedded storage system according to claim 1, characterized in that, The total storage space of the non-volatile memory also includes a Uboot partition and a kernel partition. The Uboot partition is used for hardware initialization and system kernel booting, and the kernel partition is the system kernel that runs on the embedded device. The file system partition and the raw storage partition are dedicated data storage partitions, and the division and management of the file system partition and the raw storage partition are independent of the operation of the Uboot partition and the kernel partition.

7. The partition management method for an embedded storage system according to claim 1, characterized in that, The non-volatile memory is Flash memory, and the embedded file system is LittleFS.

8. A partition management system for an embedded storage system, characterized in that, include: The partitioning module is used to divide the total storage space of the non-volatile memory into at least two independent functional partitions, wherein the independent functional partitions include a file system partition and a raw storage partition; The partition adaptation management module is used to configure an embedded file system and corresponding file operation interface for the file system partition, and to configure a raw storage access interface for the raw storage partition. The address mapping module is used to map logical storage units to physical storage addresses through an address translation mechanism. The boundary protection module is used to restrict the scope of operations on each independent functional partition through a validity verification mechanism, so as to avoid cross-partition out-of-bounds access.

9. The partition management system for an embedded storage system according to claim 8, characterized in that, The non-volatile memory is a Flash memory; the embedded file system is LittleFS; the file operation interface is used for file creation, deletion, reading and writing, directory management and file attribute management; the raw storage access interface is used for reading data by sector number, writing data by sector number and erasing one or more sectors.

Images