File batch management method and device based on Fatfs and embedded system

Abstract

The invention provides a file batch management method and device based on Fatfs and an embedded system. The method comprises the steps that a Fatfs file system is configured in storage equipment; to-be-stored data are received through the Fatfs file system, a plurality of storage partitions are divided and created in the storage device according to the data types of the to-be-stored data, the storage partitions correspond to the data types one to one, and the data types at least comprise log data and sensor data; and respectively storing the to-be-stored data of each data type into the corresponding storage partitions through the Fatfs file system. According to the method, different types of data are stored in a partitioned manner, so that the influence of fragmentation generated by frequent operation of small files such as logs on the read-write speed of large files is avoided, and the problem that in the prior art, the system performance is reduced due to frequent file read-write generated by storage of a large amount of data in a small and medium-sized embedded system is solved.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and more specifically, to a method, apparatus, computer-readable storage medium, and embedded system for batch file management based on Fatfs. Background Technology

[0002] Edge computing for the Internet of Things (IoT) requires reducing latency and improving data response speed, and rapid data processing and decision-making are crucial to the efficiency of edge computing.

[0003] The Fatfs file system is lightweight and highly portable, making it commonly used in embedded systems. However, due to the large data throughput in edge computing used by embedded systems in the Internet of Things (IoT), storage devices relying on the Fatfs file system may experience performance degradation. Specifically, in small embedded systems, the frequent file creation and deletion generated by edge computing can lead to fragmentation of the Fatfs file system, affecting its read and write speeds. At the same time, a large number of file operations will consume a lot of memory, further impacting the read and write speeds of the Fatfs file system.

[0004] Therefore, when the existing Fatfs file system is applied to small embedded systems, its performance usually gradually decreases during use due to the large data throughput. Summary of the Invention

[0005] The main objective of this application is to provide a Fatfs-based method, apparatus, computer-readable storage medium, and embedded system for batch file management, addressing the problem of performance degradation caused by frequent file read / write operations resulting from large amounts of data storage in small embedded systems, at least in the prior art.

[0006] To achieve the above objectives, according to one aspect of this application, a Fatfs-based batch file management method is provided, comprising: configuring a Fatfs file system in a storage device; receiving data to be stored through the Fatfs file system, and dividing and creating multiple storage partitions in the storage device according to the data type of the data to be stored, wherein the storage partitions correspond one-to-one with the data types, and the data types include at least log data and sensor data; and storing the data to be stored of each data type into the corresponding storage partitions through the Fatfs file system.

[0007] Optionally, after creating multiple storage partitions in the storage device according to the data type of the data to be stored, the method further includes: traversing the storage data in each storage partition through the Fatfs file system to determine whether each storage data is expired or abnormal; and formatting the corresponding storage data through the Fatfs file system if the storage data is expired or abnormal.

[0008] Optionally, if the stored data is expired or is abnormal, the corresponding stored data can be formatted using the Fatfs file system. This can be achieved by: creating multiple empty partitions on the storage device using the Fatfs file system; moving the expired stored data to any empty partition using the Fatfs file system if some stored data in the storage partition is expired; moving the abnormal stored data to any empty partition using the Fatfs file system if some stored data in the storage partition is abnormal; and formatting the empty partition using the Fatfs file system if the usage rate of the empty partition is greater than or equal to a first threshold, where the usage rate is the ratio of the occupied space of the storage partition to the total space.

[0009] Optionally, if the stored data has expired or is abnormal, the corresponding stored data may be formatted. The method may also include: if all stored data in the storage partition has expired, formatting the storage partition using the Fatfs file system; if all stored data in the storage partition is abnormal, formatting the storage partition using the Fatfs file system.

[0010] Optionally, after storing the data to be stored for each data type into the corresponding storage partition, the method further includes: traversing each storage partition through the Fatfs file system to obtain the usage rate of each storage partition; if the usage rate of two or more storage partitions is greater than a first threshold, merging all corresponding storage partitions with usage rates greater than the first threshold through the Fatfs file system; determining the number of storage partitions to be merged, obtaining the target number, and creating the target number of storage partitions in the storage device through the Fatfs file system, wherein the storage partitions to be merged correspond one-to-one with the newly created storage partitions.

[0011] Optionally, after storing the data to be stored for each data type into the corresponding storage partition, the method further includes: receiving a deletion command input by the user and determining at least one object to be deleted based on the deletion command; receiving the data to be stored, and if the data type of the object to be deleted is the same as that of the data to be stored, overwriting the object to be deleted with the data to be stored through the Fatfs file system.

[0012] Optionally, after receiving the data to be stored, the method further includes: if the data type of the data to be stored is different from that of the object to be deleted, determining the storage partition to which the object to be deleted belongs as the target area; traversing the storage data in the target area through the Fatfs file system, and if all the storage data has expired, overwriting the storage data in the target area with the data to be stored; if there is unexpired storage data in the target area, moving the unexpired storage data to an empty partition, and overwriting the storage data in the target area with the data to be stored.

[0013] According to another aspect of this application, a Fatfs-based batch file management device is provided. The device includes: a first processing unit for configuring a Fatfs file system in a storage device; a first acquisition unit for receiving data to be stored through the Fatfs file system and creating multiple storage partitions in the storage device according to the data type of the data to be stored, wherein the storage partitions correspond one-to-one with the data types, and the data types include at least log data and sensor data; and a first storage unit for storing the data to be stored of each data type into the corresponding storage partition through the Fatfs file system.

[0014] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform any one of the methods.

[0015] According to another aspect of this application, an embedded system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for performing any one of them.

[0016] Applying the technical solution of this application, in the aforementioned Fatfs-based batch file management method, firstly, a Fatfs file system is configured in the storage device; then, the data to be stored is received through the Fatfs file system, and multiple storage partitions are created in the storage device according to the data type of the data to be stored. Each storage partition corresponds one-to-one with a data type, which includes at least log data and sensor data; finally, the data to be stored for each data type is stored separately in its corresponding storage partition through the Fatfs file system. This application solves the problem of system performance degradation caused by frequent file read / write operations resulting from large amounts of data storage in small embedded systems in the prior art by partitioning and storing different types of data, thus isolating log data operations from sensor data and avoiding interference from file system fragmentation caused by frequent log operations on the same partition. Attached Figure Description

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for a Fatfs-based batch file management method provided in an embodiment of this application is shown.

[0018] Figure 2 A flowchart illustrating a Fatfs-based batch file management method according to an embodiment of this application is shown.

[0019] Figure 3 A flowchart illustrating a specific Fatfs-based batch file management method according to another embodiment of this application is shown.

[0020] Figure 4 A structural block diagram of a Fatfs-based batch file management device provided according to an embodiment of this application is shown.

[0021] The above figures include the following reference numerals:

[0022] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] As described in the background section, the performance of the existing Fatfs file system, when applied to small embedded systems, usually gradually declines due to the large data throughput. To address the problem of performance degradation caused by frequent file read / write operations resulting from the large amount of data storage in small embedded systems, embodiments of this application provide a Fatfs-based file batch management method, apparatus, computer-readable storage medium, and embedded system.

[0027] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0028] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a Fatfs-based batch file management method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0029] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the Fatfs-based batch file management method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] This embodiment provides a Fatfs-based batch file management method that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] Figure 2 This is a flowchart of a Fatfs-based batch file management method according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0032] Step S201: Configure the Fatfs file system on the storage device;

[0033] Specifically, the Fatfs file system, which is compatible with small embedded systems, is used to ensure that the file system can run stably in a specific hardware environment. The storage device is initialized and necessary parameters, such as cluster size and number of root directory nodes, are configured to adapt to different types of data storage needs.

[0034] Step S202: Receive the data to be stored through the Fatfs file system, and create multiple storage partitions in the storage device according to the data type of the data to be stored. The storage partitions correspond one-to-one with the data types, and the data types include at least log data and sensor data.

[0035] Specifically, the system automatically identifies data types based on data characteristics, which can include video, audio, logs, and sensor data. Storage partitions are dynamically allocated based on data type, with dedicated partitions created for each type of data to be stored.

[0036] Understandably, video and audio data, due to their large file size, are allocated to separate, large-capacity partitions. Log and sensor data, on the other hand, are smaller files and are frequently manipulated, so they are allocated to smaller partitions. This isolates data of different types and prevents fragmentation of frequently manipulated small files from affecting the reading and writing of large files.

[0037] Step S203: Store the data to be stored for each data type into the corresponding storage partition using the Fatfs file system.

[0038] Specifically, the identified data to be stored is stored in the corresponding storage partitions according to its data type.

[0039] Understandably, by creating dedicated storage partitions for different data types, data can be categorized and stored, different data types can be isolated, file system fragmentation can be reduced, thereby improving file read / write speed and storage efficiency, and solving the performance bottleneck problem caused by storing a large number of different types of data in the same partition.

[0040] In this embodiment, firstly, a Fatfs file system is configured in the storage device; then, the Fatfs file system receives the data to be stored, and according to the data type, multiple storage partitions are created in the storage device, with each storage partition corresponding to a data type, including at least log data and sensor data; finally, the Fatfs file system stores the data of each data type into its corresponding storage partition. This application solves the problem of performance degradation caused by frequent file read / write operations on large amounts of data stored in small embedded systems in existing technologies by partitioning and storing different types of data, thus isolating log data operations from sensor data and preventing file system fragmentation caused by frequent log operations from interfering with sensor data.

[0041] In another embodiment, when creating a storage partition, the partition size can be set according to the historical usage of the corresponding data type and the predicted data generation pattern to meet the usage requirements of the data planned to be stored in the storage partition.

[0042] Furthermore, the size of the storage partition can be dynamically adjusted. For example, when the utilization rate of a certain storage partition is greater than 80%, other storage partitions can be traversed, and a portion of space can be temporarily used from the storage partition with the lowest utilization rate.

[0043] In practice, the dynamic adjustment of partition size can be achieved by using a data prediction model to predict future data volume changes based on the historical trends of the data type, thereby adjusting the partition size in advance and optimizing data read and write speeds.

[0044] To improve the processing speed of the data to be stored, in one optional implementation, after creating multiple storage partitions in the storage device according to the data type of the data to be stored, the method further includes:

[0045] Step S301: Traverse the storage data in each storage partition through the Fatfs file system to determine whether the storage data is expired or abnormal.

[0046] Specifically, by periodically starting a program through the file system to check the stored data in the storage partition, it can be understood that the above detection is carried out in two aspects. On the one hand, expiration detection is performed, that is, using the timestamp information in the file metadata to determine whether the data has exceeded the preset retention period; on the other hand, anomaly identification is performed, that is, using data analysis algorithms to identify abnormal patterns or errors in the stored data.

[0047] Step S302: If the stored data is expired or is abnormal, the corresponding stored data is formatted using the Fatfs file system.

[0048] Specifically, when expired or abnormal stored data is detected, a corresponding data clearing strategy is employed to format the stored data.

[0049] The above embodiments demonstrate that data erasure through formatting, rather than deleting files one by one, significantly reduces data cleanup time. This is particularly beneficial in small embedded systems where data has a short lifespan and requires frequent cleanup operations to achieve high data throughput within limited performance constraints. Furthermore, formatting, rather than deletion, reduces unnecessary I / O operations, lowers CPU load, and extends the lifespan of the storage device.

[0050] In an optional implementation, to clean up unwanted stored data, step S302 includes:

[0051] Step S3021: Create multiple empty partitions on the storage device using the Fatfs file system;

[0052] Specifically, multiple empty partitions are created as buffers for temporary data migration or data storage, enabling data interaction between partitions, thereby reducing direct operations on the main storage area and reducing performance overhead.

[0053] Step S3022: If some of the stored data in the storage partition has expired, move the corresponding stored data to any empty partition using the Fatfs file system.

[0054] Specifically, during file system operation, expired data is periodically filtered out. If only a portion of the data in a storage partition has expired, the expired data is moved from the storage partition to an empty partition.

[0055] Step S3023: If some of the stored data in the storage partition is abnormal, move the corresponding stored data to any empty partition using the Fatfs file system.

[0056] Similarly, during file system operation, anomalies are periodically filtered out. If only part of the data in a storage partition is abnormal, the abnormal data is moved from the storage partition to an empty partition.

[0057] Understandably, processing stored data through move operations, compared to delete operations, does not update the FAT table, reduces file system write operations, and helps maintain storage performance.

[0058] Step S3024: If the usage rate of the empty partition is greater than or equal to the first threshold, the empty partition is formatted using the Fatfs file system. The usage rate is the ratio of the space occupied by the storage partition to the total space.

[0059] Specifically, the usage of empty partitions is monitored. Once the usage rate reaches a preset first threshold (e.g., 80%), it is considered that the partition has accumulated enough discarded data and can be formatted to quickly clear all data in the empty partition and reclaim storage space.

[0060] Understandably, formatting completely erases all data within a partition, avoiding file fragmentation caused by deletion and improving storage space utilization. Furthermore, batch formatting expired data in an empty partition avoids individual deletion, improving data processing efficiency. Additionally, moving unnecessary data from the storage partition reduces node usage and improves read / write performance. Moreover, the move operation reduces rewriting and deletion operations, extending the partition's lifespan and mitigating system instability caused by frequent file operations.

[0061] In an optional implementation, to clean up unwanted stored data, step S302 further includes:

[0062] Step S3025: If all stored data in the storage partition has expired, format the storage partition using the Fatfs file system.

[0063] Specifically, after determining through file metadata that all stored data in the storage partition has expired, the Fatfs file system's formatting command is invoked to clear the data in the partition, reorganize the file system structure, and prepare for the storage of new data.

[0064] Step S3026: If all stored data in the storage partition is abnormal, format the storage partition using the Fatfs file system.

[0065] Similarly, if a custom anomaly detection algorithm determines that all stored data in a storage partition is abnormal, the Fatfs file system's formatting command is invoked to clear the data in the partition, reorganize the file system structure, and prepare for the storage of new data.

[0066] Through the above embodiments, when all data in a certain storage partition is no longer useful, this application sets up a method to format the entire partition. This extreme case handling strategy can not only free up a large amount of storage space, but also simplify the complexity of data management and shorten the data processing time. Furthermore, the formatting operation can also clear fragmented files in the entire storage partition and restore the performance degradation that is inevitable due to the operation of the file system.

[0067] In another embodiment, to prevent critical data from being deleted, priorities can be set for stored data, for example, giving higher priority to media data and lower priority to log data.

[0068] To further improve the performance of the storage device, in one optional implementation, after storing the data to be stored for each data type into the corresponding storage partition, the method further includes:

[0069] Step S401: Traverse each storage partition through the Fatfs file system to obtain the usage rate of each storage partition;

[0070] Specifically, the system periodically traverses each storage partition through the Fatfs file system API to obtain the current usage rate of the partition.

[0071] Step S402: If the usage rate of two or more storage partitions is greater than the first threshold, merge all the corresponding storage partitions with usage rates greater than the first threshold through the Fatfs file system.

[0072] Specifically, a first threshold is pre-set, such as 95% (to indicate that a storage partition is about to be full). When the utilization rate of any partition exceeds this threshold, the system begins to consider merging partitions to optimize storage resources. Specifically, high-utilization storage partitions are selected. If there are two or more such storage partitions, the file system performs a partition merging operation. Subsequently, this partition will no longer be written to, only read-enabled. Then, after the data's lifecycle is reached, it will be formatted to free up more nodes and optimize the partition layout.

[0073] Step S403: Determine the number of storage partitions to be merged, obtain the target number, and create the target number of storage partitions in the storage device using the Fatfs file system, wherein the storage partitions to be merged correspond one-to-one with the newly created storage partitions.

[0074] Specifically, in order to store subsequently acquired new data, in one optional implementation, the same number of new partitions are created, each new partition corresponding to one of the original high-usage partitions. New data is written to the corresponding new partition.

[0075] To optimize storage space utilization, this solution introduces a dynamic merging and reallocation mechanism for storage partitions, as described in the above embodiments. By periodically checking the utilization rate of storage partitions, if multiple partitions are found to have utilization rates exceeding a preset first threshold, it indicates that these partitions have high space utilization and may be under storage pressure. In this case, merging these high-utilization partitions not only frees up contiguous storage space but also allows for a more rational allocation of storage resources through the creation of new partitions.

[0076] To reduce the number of storage device deletion operations and thus minimize system performance loss, in one optional implementation, after storing the data to be stored for each data type into its corresponding storage partition, the method further includes:

[0077] Step S501: Receive a deletion instruction input by the user, and determine at least one object to be deleted based on the deletion instruction;

[0078] Specifically, after receiving the user's deletion command, the system parses the keywords in the command to determine the data type of the object to be deleted, and uses the file system API to search for matching files in the specified data type partition as the deletion object.

[0079] Step S502: Receive the data to be stored. If the data type of the object to be deleted is the same as that of the data to be stored, overwrite the object to be deleted with the data to be stored through the Fatfs file system.

[0080] Specifically, upon receiving new data, the system checks if its data type matches that of the object to be deleted. If the data types match, the system directly writes the new video data to the sector containing the video file to be deleted, overwriting the existing file content.

[0081] After the file system table maintenance overwrite deletion is completed, the system updates the information of the relevant entries in the file system table, such as file size, creation date, and last modification date, to reflect the latest file status. It is important to note that although overwrite deletion avoids the physical deletion of the file system, the file system table still needs to be properly maintained; otherwise, data loss or file system corruption may occur.

[0082] Understandably, traditional file deletion operations update the file system table, leading to additional storage overhead and potential file fragmentation. The overwrite deletion strategy, on the other hand, is based on the principle of "writing new data to overwrite old data." By directly writing new data to the location of the file to be deleted, rather than performing a physical deletion, it not only avoids updating the file system table but also effectively utilizes storage space, reduces fragmentation, and thus optimizes system performance.

[0083] To reduce deletion operations on storage devices and thus minimize system performance degradation, in one optional implementation, after receiving the data to be stored, the method further includes:

[0084] Step S601: If the data types of the data to be stored and the data to be deleted are different, the storage partition to which the data to be deleted belongs is determined as the target area;

[0085] Specifically, when the received data to be stored does not match the type of the object to be deleted specified by the user, the system will mark the storage partition where the object to be deleted originally belonged as the "target area".

[0086] Step S602: Traverse the stored data in the target area through the Fatfs file system. If all the stored data has expired, overwrite the stored data in the target area with the data to be stored.

[0087] Specifically, the system iterates through and evaluates all stored data in the target region using the Fatfs file system, checking the data's expiration date. If all data in the target region is determined to be expired, the system can directly overwrite the data in that region with the data to be stored.

[0088] Step S603: If there is unexpired stored data in the target area, move the unexpired stored data to the empty partition and overwrite the stored data in the target area with the data to be stored.

[0089] Specifically, if there is unexpired data in the "target area," the system migrates this data to an existing empty partition, and then overwrites the remaining expired data area with the data to be stored. This protects important data while effectively utilizing storage space.

[0090] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the Fatfs-based batch file management method of this application will be described in detail below with reference to specific embodiments.

[0091] This embodiment relates to a specific method for batch file management based on Fatfs, such as... Figure 3 As shown, it includes the following steps:

[0092] Step S1: Initialize the disk, that is, initialize the SD card, flash and some storage devices, and format the storage devices to the Fatfs file system;

[0093] Step S2: Create partitions to store video, audio, logs, sensor data, and two or more empty partitions;

[0094] Step S3: Collect data and place the collected data into the corresponding partitions;

[0095] Step S4: Check if there is any data in all partitions that does not need to be stored, such as expired video or log data that needs to be deleted. Move this unnecessary data to an empty partition. When this empty partition has stored a certain amount of data, perform a unified format of the partition.

[0096] Step S5: Check if all data in all partitions needs to be deleted. If all data in a partition is expired, or the data is abnormal and needs to be deleted, or there is not enough storage space, then format the partition.

[0097] Step S6: If the data in the partition does not need to be deleted, check if all partitions are full. If two or more partitions are full, merge these full partitions (including partitions belonging to different attributes) into one partition, reducing the number of partitions.

[0098] Step S7: Upon receiving a deletion command, check whether the data in the storage partition of the object to be deleted is needed. If not, use new data to overwrite the original data. If it is needed, migrate the required data to the new partition and overwrite the original data.

[0099] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0100] This application also provides a Fatfs-based file batch management device. It should be noted that this Fatfs-based file batch management device can be used to execute the Fatfs-based file batch management method provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0101] The following describes the Fatfs-based batch file management device provided in the embodiments of this application.

[0102] Figure 4 This is a structural block diagram of a Fatfs-based batch file management device according to an embodiment of this application. Figure 4 As shown, the device includes:

[0103] The first processing unit 10 is used to configure the Fatfs file system in the storage device;

[0104] Specifically, the Fatfs file system, which is compatible with small embedded systems, is used to ensure that the file system can run stably in a specific hardware environment. The storage device is initialized and necessary parameters, such as cluster size and number of root directory nodes, are configured to adapt to different types of data storage needs.

[0105] The first acquisition unit 20 is used to receive data to be stored through the Fatfs file system, and to divide and create multiple storage partitions in the storage device according to the data type of the data to be stored. The storage partitions correspond one-to-one with the data types, and the data types include at least log data and sensor data.

[0106] Specifically, the system automatically identifies data types based on data characteristics, which can include video, audio, logs, and sensor data. Storage partitions are dynamically allocated based on data type, with dedicated partitions created for each type of data to be stored.

[0107] Understandably, video and audio data, due to their large file size, are allocated to separate, large-capacity partitions. Log and sensor data, on the other hand, are smaller files and are frequently manipulated, so they are allocated to smaller partitions. This isolates data of different types and prevents fragmentation of frequently manipulated small files from affecting the reading and writing of large files.

[0108] The first storage unit 30 is used to store data of various data types into the corresponding storage partitions through the Fatfs file system.

[0109] Specifically, the identified data to be stored is stored in the corresponding storage partitions according to its data type.

[0110] Understandably, by creating dedicated storage partitions for different data types, data can be categorized and stored, different data types can be isolated, file system fragmentation can be reduced, thereby improving file read / write speed and storage efficiency, and solving the performance bottleneck problem caused by storing a large number of different types of data in the same partition.

[0111] In this embodiment, the first processing unit configures a Fatfs file system in the storage device; the first acquisition unit receives data to be stored through the Fatfs file system and, according to the data type of the data to be stored, creates multiple storage partitions in the storage device, with each storage partition corresponding to a data type, including at least log data and sensor data; the first storage unit stores the data to be stored for each data type into its corresponding storage partition through the Fatfs file system. This application solves the problem of system performance degradation caused by frequent file read / write operations resulting from large amounts of data storage in small embedded systems in the prior art by partitioning and storing different types of data, thus isolating log data operations from sensor data and avoiding interference from file system fragmentation caused by frequent log operations on the same partition.

[0112] In another embodiment, when creating a storage partition, the partition size can be set according to the historical usage of the corresponding data type and the predicted data generation pattern to meet the usage requirements of the data planned to be stored in the storage partition.

[0113] Furthermore, the size of the storage partition can be dynamically adjusted. For example, when the utilization rate of a certain storage partition is greater than 80%, other storage partitions can be traversed, and a portion of space can be temporarily used from the storage partition with the lowest utilization rate.

[0114] In practice, the dynamic adjustment of partition size can be achieved by using a data prediction model to predict future data volume changes based on the historical trends of the data type, thereby adjusting the partition size in advance and optimizing data read and write speeds.

[0115] To improve the processing speed of the data to be stored, in one optional embodiment, the above-mentioned apparatus further includes:

[0116] The first checking unit is used to determine whether the stored data is expired or abnormal after multiple storage partitions are created in the storage device according to the data type of the data to be stored, by traversing the stored data in each storage partition through the Fatfs file system.

[0117] Specifically, by periodically starting a program through the file system to check the stored data in the storage partition, it can be understood that the above detection is carried out in two aspects. On the one hand, expiration detection is performed, that is, using the timestamp information in the file metadata to determine whether the data has exceeded the preset retention period; on the other hand, anomaly identification is performed, that is, using data analysis algorithms to identify abnormal patterns or errors in the stored data.

[0118] The second processing unit is used to format the corresponding stored data through the Fatfs file system when the stored data has expired or is abnormal.

[0119] Specifically, when expired or abnormal stored data is detected, a corresponding data clearing strategy is employed to format the stored data.

[0120] The above embodiments demonstrate that data erasure through formatting, rather than deleting files one by one, significantly reduces data cleanup time. This is particularly beneficial in small embedded systems where data has a short lifespan and requires frequent cleanup operations to achieve high data throughput within limited performance constraints. Furthermore, formatting, rather than deletion, reduces unnecessary I / O operations, lowers CPU load, and extends the lifespan of the storage device.

[0121] In an optional implementation, to clean up unwanted stored data, the second processing unit includes:

[0122] Create a module for creating multiple empty partitions on a storage device using the Fatfs file system;

[0123] Specifically, multiple empty partitions are created as buffers for temporary data migration or data storage, enabling data interaction between partitions, thereby reducing direct operations on the main storage area and reducing performance overhead.

[0124] The first storage module is used to move the corresponding storage data to any empty partition via the Fatfs file system when some of the stored data in the storage partition has expired.

[0125] Specifically, during file system operation, expired data is periodically filtered out. If only a portion of the data in a storage partition has expired, the expired data is moved from the storage partition to an empty partition.

[0126] The second storage module is used to move the corresponding stored data to any empty partition via the Fatfs file system when some of the stored data in the storage partition is abnormal.

[0127] Similarly, during file system operation, anomalies are periodically filtered out. If only part of the data in a storage partition is abnormal, the abnormal data is moved from the storage partition to an empty partition.

[0128] Understandably, processing stored data through move operations, compared to delete operations, does not update the FAT table, reduces file system write operations, and helps maintain storage performance.

[0129] The first processing module is used to format the empty partition using the Fatfs file system when the usage rate of the empty partition is greater than or equal to a first threshold. The usage rate is the ratio of the space occupied by the storage partition to the total space.

[0130] Specifically, the usage of empty partitions is monitored. Once the usage rate reaches a preset first threshold (e.g., 80%), it is considered that the partition has accumulated enough discarded data and can be formatted to quickly clear all data in the empty partition and reclaim storage space.

[0131] Understandably, formatting completely erases all data within a partition, avoiding file fragmentation caused by deletion and improving storage space utilization. Furthermore, batch formatting expired data in an empty partition avoids individual deletion, improving data processing efficiency. Additionally, moving unnecessary data from the storage partition reduces node usage and improves read / write performance. Moreover, the move operation reduces rewriting and deletion operations, extending the partition's lifespan and mitigating system instability caused by frequent file operations.

[0132] In an optional implementation, to clean up unwanted stored data, the second processing unit further includes:

[0133] The second processing module is used to format the storage partition using the Fatfs file system when all stored data in the storage partition has expired.

[0134] Specifically, after determining through file metadata that all stored data in the storage partition has expired, the Fatfs file system's formatting command is invoked to clear the data in the partition, reorganize the file system structure, and prepare for the storage of new data.

[0135] The third processing module is used to format the storage partition using the Fatfs file system when all stored data in the storage partition is abnormal.

[0136] Similarly, if a custom anomaly detection algorithm determines that all stored data in a storage partition is abnormal, the Fatfs file system's formatting command is invoked to clear the data in the partition, reorganize the file system structure, and prepare for the storage of new data.

[0137] Through the above embodiments, when all data in a certain storage partition is no longer useful, this application sets up a method to format the entire partition. This extreme case handling strategy can not only free up a large amount of storage space, but also simplify the complexity of data management and shorten the data processing time. Furthermore, the formatting operation can also clear fragmented files in the entire storage partition and restore the performance degradation that is inevitable due to the operation of the file system.

[0138] In another embodiment, to prevent critical data from being deleted, priorities can be set for stored data, for example, giving higher priority to media data and lower priority to log data.

[0139] To further improve the performance of the storage device, in one optional embodiment, the above-mentioned apparatus further includes:

[0140] The second checking unit is used to traverse each storage partition through the Fatfs file system after storing the data to be stored for each data type into the corresponding storage partition to obtain the usage rate of each storage partition.

[0141] Specifically, the system periodically traverses each storage partition through the Fatfs file system API to obtain the current usage rate of the partition.

[0142] The third processing unit is used to merge all the corresponding storage partitions with usage rates greater than the first threshold through the Fatfs file system when the usage rate of two or more storage partitions is greater than the first threshold.

[0143] Specifically, a first threshold is pre-set, such as 95% (to indicate that a storage partition is about to be full). When the utilization rate of any partition exceeds this threshold, the system begins to consider merging partitions to optimize storage resources. Specifically, high-utilization storage partitions are selected. If there are two or more such storage partitions, the file system performs a partition merging operation. Subsequently, this partition will no longer be written to, only read-enabled. Then, after the data's lifecycle is reached, it will be formatted to free up more nodes and optimize the partition layout.

[0144] A unit is created to determine the number of storage partitions to be merged, obtain the target number, and create the target number of storage partitions in the storage device using the Fatfs file system. Each storage partition involved in the merge corresponds one-to-one with the newly created storage partitions.

[0145] Specifically, in order to store subsequently acquired new data, in one optional implementation, the same number of new partitions are created, each new partition corresponding to one of the original high-usage partitions. New data is written to the corresponding new partition.

[0146] To optimize storage space utilization, this solution introduces a dynamic merging and reallocation mechanism for storage partitions, as described in the above embodiments. By periodically checking the utilization rate of storage partitions, if multiple partitions are found to have utilization rates exceeding a preset first threshold, it indicates that these partitions have high space utilization and may be under storage pressure. In this case, merging these high-utilization partitions not only frees up contiguous storage space but also allows for a more rational allocation of storage resources through the creation of new partitions.

[0147] To reduce the number of deletion operations on storage devices and thus reduce system performance loss, in one optional embodiment, the above-mentioned apparatus further includes:

[0148] The second acquisition unit is used to receive a deletion instruction input by the user after storing the data to be stored of each data type into the corresponding storage partition, and to determine at least one object to be deleted based on the deletion instruction.

[0149] Specifically, after receiving the user's deletion command, the system parses the keywords in the command to determine the data type of the object to be deleted, and uses the file system API to search for matching files in the specified data type partition as the deletion object.

[0150] The third acquisition unit is used to receive the data to be stored. If the data type of the object to be deleted is the same as that of the data to be stored, the data to be stored will be used to overwrite the object to be deleted through the Fatfs file system.

[0151] Specifically, upon receiving new data, the system checks if its data type matches that of the object to be deleted. If the data types match, the system directly writes the new video data to the sector containing the video file to be deleted, overwriting the existing file content.

[0152] After the file system table maintenance overwrite deletion is completed, the system updates the information of the relevant entries in the file system table, such as file size, creation date, and last modification date, to reflect the latest file status. It is important to note that although overwrite deletion avoids the physical deletion of the file system, the file system table still needs to be properly maintained; otherwise, data loss or file system corruption may occur.

[0153] Understandably, traditional file deletion operations update the file system table, leading to additional storage overhead and potential file fragmentation. The overwrite deletion strategy, on the other hand, is based on the principle of "writing new data to overwrite old data." By directly writing new data to the location of the file to be deleted, rather than performing a physical deletion, it not only avoids updating the file system table but also effectively utilizes storage space, reduces fragmentation, and thus optimizes system performance.

[0154] To reduce the number of deletion operations on storage devices and thus reduce system performance loss, in one optional embodiment, the above-mentioned apparatus further includes:

[0155] The fourth processing unit is used to determine the storage partition to which the object to be deleted belongs as the target area after receiving the data to be stored, in the case that the data to be stored and the data type of the object to be deleted are different.

[0156] Specifically, when the received data to be stored does not match the type of the object to be deleted specified by the user, the system will mark the storage partition where the object to be deleted originally belonged as the "target area".

[0157] The third checking unit is used to traverse the stored data in the target area through the Fatfs file system, and if all the stored data has expired, the data to be stored will overwrite the stored data in the target area.

[0158] Specifically, the system iterates through and evaluates all stored data in the target region using the Fatfs file system, checking the data's expiration date. If all data in the target region is determined to be expired, the system can directly overwrite the data in that region with the data to be stored.

[0159] The fifth processing unit is used to move the unexpired stored data to an empty partition when there is unexpired stored data in the target area, and to overwrite the stored data in the target area with the data to be stored.

[0160] Specifically, if there is unexpired data in the "target area," the system migrates this data to an existing empty partition, and then overwrites the remaining expired data area with the data to be stored. This protects important data while effectively utilizing storage space.

[0161] The aforementioned Fatfs-based batch file management device includes a processor and a memory. The first processing unit, first acquisition unit, and first storage unit are all stored as program units in the memory, and the processor executes these program units to implement their respective functions. All of the above modules reside in the same processor; alternatively, the modules may be located in different processors in any combination.

[0162] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and the read / write speed of the embedded system can be improved by adjusting kernel parameters.

[0163] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0164] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the Fatfs-based batch file management method.

[0165] This invention provides a processor for running a program, wherein the program executes the Fatfs-based batch file management method during runtime.

[0166] This invention provides an embedded system, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements at least the steps of the Fatfs-based batch file management method described above.

[0167] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform an initialization program having at least the steps of the Fatfs-based file batch management method described above.

[0168] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0169] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0173] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0174] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0175] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0176] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0177] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0178] 1) The Fatfs-based batch file management method of this application first configures a Fatfs file system in the storage device; then, it receives the data to be stored through the Fatfs file system and creates multiple storage partitions in the storage device according to the data type of the data to be stored, with each storage partition corresponding to a data type, including at least log data and sensor data; finally, it stores the data to be stored for each data type into its corresponding storage partition through the Fatfs file system. This application solves the problem of system performance degradation caused by frequent file read / write operations due to large amounts of data storage in small embedded systems in the prior art by partitioning and storing different types of data, isolating log data operations from sensor data, and avoiding interference from file system fragmentation caused by frequent log operations on different data in the same partition.

[0179] 2) The Fatfs-based batch file management device of this application includes a first processing unit that configures a Fatfs file system in a storage device; a first acquisition unit that receives data to be stored through the Fatfs file system and, according to the data type of the data to be stored, creates multiple storage partitions in the storage device, with each storage partition corresponding to a data type, including at least log data and sensor data; and a first storage unit that stores the data to be stored of each data type into its corresponding storage partition through the Fatfs file system. This application solves the problem of system performance degradation caused by frequent file read / write operations resulting from large amounts of data storage in small embedded systems in the prior art by partitioning and storing different types of data, thus isolating log data operations from sensor data and avoiding interference from file system fragmentation caused by frequent log operations on the same partition.

[0180] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A batch file management method based on Fatfs, characterized in that, include: Configure the Fatfs file system on the storage device; The Fatfs file system receives data to be stored and, based on the data type of the data to be stored, creates multiple storage partitions in the storage device. Each storage partition corresponds one-to-one with a data type, which includes at least log data and sensor data. The Fatfs file system is used to store the data to be stored for each of the data types into the corresponding storage partitions.

2. The method according to claim 1, characterized in that, After creating multiple storage partitions in the storage device according to the data type of the data to be stored, the method further includes: The storage data in each storage partition is traversed through the Fatfs file system to determine whether the storage data has expired or whether the storage data is abnormal. If the stored data is expired or is abnormal, the corresponding stored data will be formatted using the Fatfs file system.

3. The method according to claim 2, characterized in that, In cases where the stored data is expired or corrupted, the corresponding stored data is formatted using the Fatfs file system. The method includes: Multiple empty partitions are created on the storage device using the Fatfs file system; If some of the stored data in the storage partition has expired, the corresponding stored data can be moved to any of the empty partitions using the Fatfs file system. If some of the stored data in the storage partition is abnormal data, the corresponding stored data shall be moved to any of the empty partitions using the Fatfs file system. If the usage rate of the empty partition is greater than or equal to a first threshold, the empty partition is formatted using the Fatfs file system, where the usage rate is the ratio of the space occupied by the storage partition to the total space.

4. The method according to claim 3, characterized in that, In cases where the stored data has expired or is abnormal, the corresponding stored data is formatted. The method further includes: If all the stored data in the storage partition has expired, the storage partition is formatted using the Fatfs file system. If all the stored data in the storage partition is the abnormal data, the storage partition is formatted using the Fatfs file system.

5. The method according to claim 1, characterized in that, After storing the data to be stored for each of the aforementioned data types into the corresponding storage partitions, the method further includes: The usage rate of each storage partition is obtained by traversing the Fatfs file system. If the usage rate of two or more of the aforementioned storage partitions exceeds a first threshold, all the corresponding storage partitions with usage rates exceeding the first threshold will be merged through the Fatfs file system. The number of storage partitions to be merged is determined to obtain the target number, and the target number of storage partitions are created in the storage device through the Fatfs file system, wherein the storage partitions to be merged correspond one-to-one with the newly created storage partitions.

6. The method according to claim 3, characterized in that, After storing the data to be stored for each of the aforementioned data types into the corresponding storage partitions, the method further includes: Receive a deletion command input by the user, and determine at least one object to be deleted based on the deletion command; Receive the data to be stored, and if the data type of the object to be deleted is the same as that of the data to be stored, overwrite the object to be deleted with the data to be stored through the Fatfs file system.

7. The method according to claim 6, characterized in that, After receiving the data to be stored, the method further includes: If the data to be stored is different from the data type of the object to be deleted, the storage partition to which the object to be deleted belongs is determined as the target area; The storage data in the target area is traversed through the Fatfs file system. If all the storage data has expired, the storage data in the target area is overwritten with the data to be stored. If there is unexpired stored data in the target area, the unexpired stored data is moved to the empty partition, and the stored data in the target area is overwritten by the data to be stored.

8. A batch file management device based on Fatfs, characterized in that, The device includes: The first processing unit is used to configure the Fatfs file system in the storage device; The first acquisition unit is used to receive data to be stored through the Fatfs file system, and to divide and create multiple storage partitions in the storage device according to the data type of the data to be stored. The storage partitions correspond one-to-one with the data types, and the data types include at least log data and sensor data. The first storage unit is used to store the data to be stored of each data type into the corresponding storage partition through the Fatfs file system.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 7.

10. An embedded system, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising methods for performing any one of claims 1 to 7.

Images