Data circulation storage method based on FAT file system optimization
By employing a circular data storage method in the FAT file system and optimizing file naming and deletion algorithms, the problem of low file storage and deletion efficiency in embedded devices is solved, enabling fast file creation and deletion, reducing cluster fragmentation, and supporting the storage of Chinese names.
Patent Information
- Application Number
- CN202411153378.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-29
AI Technical Summary
The existing FAT32 file system has low efficiency in file storage and deletion in small embedded devices, especially when storing data in multi-channel data, and its efficiency is significantly reduced. In addition, it suffers from file name length limitations and severe cluster fragmentation.
It adopts a data circular storage method based on the FAT file system. In offline mode, it simulates the formatting of the file system using a USB flash drive. In online mode, it reads configuration file information and creates storage files. The file name is ID number + configuration information + channel number + file name + .dat. When storing, it searches for consecutive directory entries from the last index position. When deleting, it starts deleting from the file with the smallest number in a circular manner. It simplifies the GBK and Unicode conversion table and supports Chinese names.
It improves file creation and deletion speed, reduces cluster fragmentation, enhances file system real-time performance and storage efficiency, supports the storage of Chinese names, and improves storage speed through EMMC multi-sector writing.
Smart Images

Figure HDA0005004227240000011 
Figure HDA0005004227240000012 
Figure HDA0005004227240000021
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of embedded devices, and in particular relates to a data circulation storage method based on FAT file system optimization. Background Art
[0002] With the development of embedded technology, more and more scenarios need to record data quickly and in real time and store them in mainstream file system formats, such as the FAT32 file system format. The FATFS file system is a completely free and open source file system module that supports FAT32. It is specially designed to adapt to the small storage characteristics of small embedded systems. It is completely written in standard C language and has good hardware platform independence. It can be ported to various DSP, ARM and other microprocessors. The basic structure of the FAT32 file system is mainly composed of the following three parts: the boot area (BOOT area), the file allocation table area (FAT area), and the data area (DATA area). The FAT32 file system also includes directory entries. Except for all files in the root directory, a set of data is described as a file, which is called a file directory entry. The file directory entries of all files and subdirectories in each file directory entry are stored in the storage space corresponding to this directory, mainly including the file name, file attributes, file size, file storage location, and a series of event descriptions. By combining the file path, file directory entry and FAT table, the storage status of a file in the hard disk can be determined. When creating a file, first check whether there is a file with the same name in the current directory, then query and allocate continuous available directory entries to establish the directory, and finally allocate storage data area to update the FAT table; when searching for a file, find the cluster of the directory where the file is located through the path, compare the file name in the space corresponding to the directory, find the file directory entry corresponding to the file, determine the starting cluster and file size of the file through the file directory entry, and determine the storage space of the file according to the cluster chain in the FAT table. When deleting a file, find the cluster of the directory where the file is located through the path, compare the file name in the space corresponding to the directory, find the file directory entry and file information corresponding to the file, delete the file directory entry, and delete the linked list in the FAT table.
[0003] Embedded devices are characterized by small size and low power consumption, but their processing speed is much slower than that of PCs, which magnifies some of the shortcomings of the FAT32 file system. For example, when creating a file, it will search for files with the same name and find the location of the last directory entry. If there are many created files, the search time will be longer; when storing files, if there are many non-free clusters between two free clusters, or if the clusters are severely fragmented, the efficiency of searching for free clusters will be reduced, thereby reducing the efficiency of file creation and storage; in addition, when searching for deleted files, the directory will also be traversed. If there are many directory entries, the search efficiency will also be reduced. The above factors will cause the real-time performance of the FAT32 file system to deteriorate, and this situation will be exacerbated when multi-channel data and multiple files are stored at the same time.
[0004] In order to solve the above problems, the efficiency of file storage can be improved from aspects such as directory search efficiency or cluster allocation efficiency. For example, the patent with application number [CN201510355756] mentions that the root directory and subdirectory are read into the memory for search to increase the speed, but for small embedded devices, the memory is usually small and not applicable; the patent with application number [CN201910574107] proposes a data block storage method, which is equivalent to pre-allocating multiple continuous clusters for file storage. If the file storage length is uncertain, the remaining clusters will be discarded after the storage is completed, resulting in a waste of space resources; the patent with application number [CN202110868087] shortens the time by pre-allocating the file name position through the "first file name", but it still stores the file directory traversal time and limits the length of the file name; the patent with application number [CN201110327108] proposes a method of simultaneously alternating the search for free clusters and deleted directories to speed up the creation of file directory entries, but when the deleted file directory entry is located relatively late, the search efficiency will also be significantly reduced. Summary of the invention
[0005] The object of the present invention is to provide a data circulation storage method based on FAT file system optimization to solve the technical problem of low data storage and deletion efficiency of small and medium-sized embedded devices in the prior art.
[0006] In order to achieve the above object, the present invention adopts the following technical scheme:
[0007] A data cyclic storage method based on FAT file system optimization comprises the following steps:
[0008] The recorder enters offline mode to simulate USB flash drive formatting file system and configuration file update;
[0009] After the recorder enters the online mode and completes the device initialization and file system mounting, it reads the configuration file information in the root directory, creates a file for storing data, and starts receiving and storing data;
[0010] When storing data, different files are named, and the directory entry index information of the file is stored after each file is created. When the next file is created, the continuous directory entry that can be allocated is directly searched from the index position stored last time to create a new file, and the size of the accepted data is determined and saved in the file;
[0011] When data is deleted, when the storage area is less than the limited remaining storage capacity or the number of files is greater than the limited number of files, the file with the smallest ID number in each data channel is queried and deleted, thereby cyclically storing data, and recording the directory entry index of the deleted file after each file is deleted. When the file is deleted next time, the next file to be deleted is directly searched from the directory entry index position; each time a file is deleted, the directory entry index of the currently deleted file is recorded in the storage device, and when the file is deleted next time, the file with the smallest file ID to be deleted is directly searched from the recorded deleted index position, and the file information is obtained and then the file is deleted;
[0012] The file naming method is ID number+configuration file information+channel number and file name of multi-channel data+.dat, where dat means that each channel creates its own dat file to store data after starting data storage.
[0013] Furthermore, after the recorder is powered on, the working mode is identified by the mode recognition GPIO input pin level, a low level is identified as an offline mode, and a high level is identified as an online mode.
[0014] Furthermore, the recorder enters the offline mode to simulate a USB flash drive, and interacts with the PC through the USB interface to copy data, format the file system, and update the configuration file.
[0015] Furthermore, the formatted file system is a FAT32 file system.
[0016] Furthermore, the recorder enters the online mode to complete the initialization of MCU and EMMC.
[0017] Furthermore, before the data is stored, a corresponding number of subfolders are created in the root directory according to the number of data channels to be recorded, and the folder names are recorded to identify the channel numbers.
[0018] Furthermore, the start and end file number IDs of each data channel are recorded in the memory.
[0019] Furthermore, each time a new file is created, the file ID number is incremented by 1.
[0020] Furthermore, the steps of determining the data size and saving it to a file are:
[0021] Check whether the data is full of 4k, 2k or 512 bytes in turn. If the data is full of 4k, 2k or 512 bytes, save the data and write it to the file;
[0022] If the data is less than 512 bytes, the circular buffer continues to receive data to detect the data size.
[0023] Furthermore, the conversion table between GBK and Unicode is simplified to common Chinese characters, the character code conversion table is shortened, and support for Chinese names of files is achieved through the simplified character code conversion table.
[0024] Compared with the prior art, the present invention has the following beneficial technical effects:
[0025] The present invention provides a data circular storage method based on FAT file system optimization. The method completes the updating of the formatted file and the configuration file in the offline mode of the recorder, so that the recorder enters the online mode to read the configuration file information in the root directory, creates a file for storing data, starts to receive and store data, and when storing and deleting data, the directory item index information of the file is stored each time the file is created or deleted. When the next file is created or deleted, it can be directly searched from the index position. The file creation speed is improved by respectively establishing a folder and storing data and a file ID number through multiple data channels, and the directory item index position of the newly created file is recorded. The file search and deletion speed is improved by recording the directory item index position of the deleted file. In addition, when the file is deleted, the file is deleted from the position with the smallest number to store the data circularly, thereby reducing the fragmentation of the cluster.
[0026] The file naming method is ID number + configuration file information + channel number and file name of multi-channel data + .dat. Each time a new file is created, the file ID number is incremented by 1, which avoids file renaming and increases the speed of file creation, search and deletion.
[0027] Furthermore, when the file stores data, when the cached data is large, EMMC multi-sector writing is used to increase the file storage speed.
[0028] Furthermore, by simplifying the conversion table of GBK and Unicode to commonly used Chinese characters, the character code conversion table is reduced to adapt to embedded devices with smaller flash storage, and at the same time support for Chinese file names is achieved, that is, support for Chinese file names is achieved in limited storage space.
[0029] Furthermore, when judging the data size and saving it in a file, it is detected whether the data band meets the requirements and then determines whether to write it into the file, thereby improving the efficiency of data reading and storage. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 A schematic diagram of a data circulation storage method based on FAT file system optimization in an embodiment of the present invention;
[0031] Figure 2 It is a device structure block diagram of a data circulation storage method based on FAT file system optimization in an embodiment of the present invention;
[0032] Figure 3A detailed flow chart of a data circulation storage method based on FAT file system optimization of the present invention;
[0033] Figure 4 A schematic diagram of a method for creating a file directory entry search method based on a data circular storage method optimized for a FAT file system according to the present invention;
[0034] Figure 5 The present invention is a schematic diagram of a method for searching for deleted file directory entries in a data circular storage method optimized based on a FAT file system. DETAILED DESCRIPTION
[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0036] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.
[0037] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0038] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.
[0039] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.
[0040] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.
[0041] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.
[0042] The present invention is further described in detail below in conjunction with the accompanying drawings:
[0043] like Figure 1 As shown, a data circulation storage method based on FAT file system optimization includes the following steps:
[0044] Step 1: The recorder enters offline mode to simulate USB formatting of the file system and configuration file update:
[0045] Specifically, after the recorder is powered on, the working mode is identified by the GPIO input pin level. A low level is identified as an offline mode, and a high level is identified as an online mode. The overall workflow is as follows: Figure 2 As shown;
[0046] In offline mode, the recorder simulates a USB flash drive and interacts with the PC through the USB interface to quickly copy data, format, update configuration files, etc. The configuration file contains the configuration file name information.
[0047] Step 2: After the recorder enters the online mode and completes the device initialization and file system mounting, it reads the configuration file information in the root directory, creates a file to store data, and starts receiving and storing data:
[0048] Specifically, in online mode, after completing device initialization and file system mounting, read the configuration file information in the root directory, obtain the configuration file name "NAMEINFO", initialize the communication interface, create a file to store data, and start receiving and storing data, such as Figure 3 shown.
[0049] Step 3: When storing data, different files are named, and the directory entry index information of the file is stored after each file is created. When the next file is created, the continuous directory entry that can be allocated is directly searched from the index position to create a new file, and the size of the accepted data is determined and saved in the file;
[0050] Specifically, before storing data, a corresponding number of subfolders are created in the root directory according to the number of channels of data to be recorded, and the channel number and file name are recorded as COMN.
[0051] After each channel starts storing data, it creates its own dat file to store data. The files are numbered from 1, and the file name is in the form of "ID number_NAMEINFO_COMN.dat" ("NAMEINFO" comes from the configuration file). Each time a new file is created, the ID is incremented by 1. The uniqueness of the file number ensures that the file will not have the same name. This eliminates the step of searching for duplicate files in the f_open function in the FATFS file system, increasing the speed of file creation. The starting and ending file number IDs of each data channel are recorded in the memory. In addition, the search algorithm for allocating free directory entries when creating files is optimized. When a new file is created, the directory entry index of the file with the largest number is recorded. When a new file is created next time, it will directly search backwards from the directory entry index position recorded last time for continuous directory entries that can be allocated. It will no longer search for free directory entries from the beginning of the current subdirectory. This greatly reduces the traversal time, thereby speeding up file creation when there are many files, such as Figure 4 shown.
[0052] Step 4, when data is deleted, when the storage area is less than the limited remaining storage capacity or the number of files is greater than the limited number of files, the file with the smallest ID number in each data channel is queried and deleted, thereby storing data in a loop, and the directory entry index of the deleted file is recorded after each file is deleted. When the file is deleted next time, the next file to be deleted is directly searched from the directory entry index position; each time a file is deleted, the directory entry index of the currently deleted file is recorded in the memory, and when the file is deleted next time, the file with the smallest file ID to be deleted is directly searched from the recorded deleted index position, the file information is obtained, and then the file is deleted.
[0053] Specifically, when the total number of files exceeds the set limit or the remaining capacity exceeds the limit, the file with the smallest file number in each data channel is deleted. This ensures that continuous free clusters are released starting from the low address direction of the memory, avoiding a large number of storage cluster fragments, thereby improving the efficiency of obtaining free clusters and thus improving the data storage speed. In addition, the search algorithm for querying files to be deleted is optimized. Each time a file is deleted, the directory entry index of the currently deleted file is recorded in the memory. When the file is deleted next time, the file with the smallest file ID to be deleted is directly searched from the recorded deleted index position, the file information is obtained, and then the file is deleted, avoiding traversing the search from the beginning of the directory, greatly shortening the traversal search time, and speeding up the search and deletion of files. In addition, due to the uniqueness of the known file name, the file duplicate name search is removed in the FATFS file system file deletion function f_unlink to improve the file deletion speed. For example, Figure 5 shown.
[0054] When storing data in files, if there is a lot of cached data, use EMMC multi-sector writing to increase the file storage speed.
[0055] In addition, by simplifying the conversion table of GBK and Unicode to commonly used Chinese characters, the character code conversion table is reduced to adapt to embedded devices with smaller flash storage, and support for Chinese file names is achieved.
[0056] The file mentioned above is named as ID number + configuration file information + channel number and file name of multi-channel data + .dat, where dat means that each channel creates its own dat file to store data after starting data storage.
[0057] Example
[0058] A data circulation storage method based on FAT file system optimization, comprising:
[0059] The recorder operation modes include online mode and offline mode. The online mode creates files to store data, and the offline mode simulates a USB flash drive.
[0060] Before data storage, set the recorder to offline mode to identify it as a USB flash drive, format it to FAT32 file system, set the cluster size to 64KB, and store the configuration file containing the file name information in the root directory. After the above preparations are completed, enter the online mode to store data.
[0061] In online mode, after the recorder is powered on, the MCU and EMMC are initialized, the file system is mounted, the configuration file is read, the file name and other information are obtained, the three serial ports are initialized, the mode is recognized as online mode to store data, and the serial ports are reset. Figure 2 shown.
[0062] Three folders are created in the root directory for the three serial ports. The data of each serial port is stored in its own folder. The ID of the data file name is increased by 1 each time the power is turned on. The naming method is "ID number_NAMEINFO_COMN.dat". The Chinese name of the file is supported through a simplified character code conversion table.
[0063] Each time a file is created, the directory entry index information of the file is stored. When the next file is created, the continuous directory entries available for allocation are directly searched from the index position to create a new file.
[0064] The PC sends data to the MCU through three serial ports, detects the data in turn and writes it into the file when it is full of 4K, 2K or 512 bytes, and calls the f_sync function for synchronization to prevent the entire file data from being lost due to power failure.
[0065] When the storage area is smaller than the limited remaining storage capacity or the number of files is larger than the limited number of files, the file with the smallest ID in each data channel is queried and deleted, thereby cyclically storing data.
[0066] Each time a file is deleted, the directory entry index of the deleted file is recorded. When a file is deleted next time, the next file to be deleted is directly searched from the directory entry index position to obtain the next file to be deleted.
[0067] When the recorder is in offline mode, it is identified as a simulated USB flash drive and the data is copied to the PC via the USB high-speed interface.
[0068] The specific effects that the present invention can achieve are as follows:
[0069] (1) By storing data files in folders, numbering file names with IDs, and recording directory entry index information of files, the speed of file creation is effectively improved. At the same time, by simplifying the transcoding table to common Chinese characters, Chinese names can be supported when the storage space is relatively small.
[0070] (2) Reduce file fragmentation by deleting files with low ID numbers and use EMMC multi-sector operations to increase file storage speed.
[0071] (3) The next file to be deleted is located by recording the index information of the directory entry of the previously deleted file. The file duplicate name check is skipped by ensuring that the file ID does not have a duplicate name, thereby increasing the file deletion speed.
[0072] (4) In offline mode, high-speed data copying is performed via USB mode.
[0073] Take the CH32V307 platform as an example. Figure 2 As shown, the EMMC is connected to the host computer PC through the serial port RS422 and USB2.0 to form a hardware platform for data sending and receiving records.
[0074] The data recording software is burned into the MCU and tested by the matching host computer software, and all functions are realized. The test results show that the present invention has good feasibility and the performance meets expectations.
[0075] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims
1. A data circulation storage method based on FAT file system optimization, characterized in that: The following steps are involved: The recorder enters offline mode to simulate USB flash drive formatting file system and configuration file update; After the recorder enters the online mode and completes the device initialization and file system mounting, it reads the configuration file information in the root directory, creates a file for storing data, and starts receiving and storing data; When storing data, different files are named, and the directory entry index information of the file is stored after each file is created. When the next file is created, the continuous directory entry that can be allocated is directly searched from the index position stored last time to create a new file, and the size of the accepted data is determined and saved in the file; When data is deleted, when the storage area is less than the limited remaining storage capacity or the number of files is greater than the limited number of files, the file with the smallest ID number in each data channel is queried and deleted, thereby cyclically storing data, and recording the directory entry index of the deleted file after each file is deleted. When the file is deleted next time, the next file to be deleted is directly searched from the directory entry index position; each time a file is deleted, the directory entry index of the currently deleted file is recorded in the storage device, and when the file is deleted next time, the file with the smallest file ID to be deleted is directly searched from the recorded deleted index position, and the file information is obtained and then the file is deleted; The file naming method is ID number+configuration file information+channel number and file name of multi-channel data+.dat, where dat means that each channel creates its own dat file to store data after starting data storage.
2. The data circulation storage method based on FAT file system optimization according to claim 1, characterized in that: After the recorder is powered on, the working mode is identified by the mode recognition GPIO input pin level, a low level is identified as an offline mode, and a high level is identified as an online mode.
3. The data circulation storage method based on FAT file system optimization according to claim 1, characterized in that: The recorder enters the offline mode to simulate a USB flash drive, and interacts with the PC through the USB interface to copy data, format the file system, and update the configuration file.
4. A data circulation storage method based on FAT file system optimization according to claim 1 to 3, characterized in that: The formatted file system is the FAT32 file system.
5. The data circulation storage method based on FAT file system optimization according to claim 1, characterized in that: The recorder enters the online mode to complete the initialization of MCU and EMMC.
6. The data circulation storage method based on FAT file system optimization according to claim 1, characterized in that: Before the data is stored, a corresponding number of subfolders are created in the root directory according to the number of data channels to be recorded, and the folder name is recorded to identify the channel number.
7. The data circulation storage method based on FAT file system optimization according to claim 1, characterized in that: The starting and ending file IDs of each data channel are recorded in the memory.
8. The data circulation storage method based on FAT file system optimization according to claim 1, characterized in that: Each time a new file is created, the file ID number is incremented by 1.
9. The data circulation storage method based on FAT file system optimization according to claim 1, characterized in that: The steps of determining the data size and saving it to a file are: Check whether the data is full of 4k, 2k or 512 bytes in turn. If the data is full of 4k, 2k or 512 bytes, save the data and write it to the file; If the data is less than 512 bytes, the circular buffer continues to receive data to detect the data size.
10. The data circulation storage method based on FAT file system optimization according to claim 1, characterized in that: Simplify the conversion table of GBK and Unicode for commonly used Chinese characters, reduce the character code conversion table, and support the Chinese name of the file through the simplified character code conversion table.
