A method for improving the reliability of a file system based on a bitmap mapping table mechanism

Abstract

The present invention provides a method for improving the reliability of a file system based on a bitmap mapping table mechanism, which is used to solve the problems of inconsistent read and write data, data bit flipping, etc. in unreliable storage media under abnormal power-off, long-term static state, etc., resulting in a reduction in the reliability of the file system. A method for improving the reliability of a file system based on a bitmap mapping table mechanism includes three key aspects: 1. Create a logical block checksum bitmap mapping table when formatting a volume of the file system and dynamically maintain it during operation; 2. Implement the physical sector dual-copy read and write function in the storage medium adaptation management layer; 3. When reading a logical block in the file system, query the checksum bitmap mapping table. If the judgment result is that the data is incorrect, continue to read the copy and synchronously update the correct data to the position where the error data is located. After implementing the method combining the checksum bitmap mapping table and dual-copy read and write designed by the present invention, the reliability of the file system can be effectively improved on unreliable storage media.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer system software, and specifically provides a method for improving the reliability of a file system based on a bitmap mapping table mechanism. Background Art

[0002] When a file system runs normally on an unreliable storage medium, due to problems such as inconsistent reading and writing of the storage medium itself and data jumping, the reliability of the file system will be reduced, and even the entire file system will crash and become unavailable. For an unreliable storage medium, if effective fault tolerance and error correction methods cannot be adopted at the software level, the files stored by users are at risk of being lost at any time, and the entire file system cannot provide stable and reliable file services to users. How to effectively solve the problem of the decline in the reliability of the file system caused by the defects of the unreliable storage medium itself at the software level is a problem that needs to be studied key. Therefore, a method for improving the reliability of a file system based on a bitmap mapping table mechanism is proposed. Summary of the Invention

[0003] Objectives of the present invention:

[0004] In order to solve the problems that when an unreliable storage medium has inconsistent reading and writing data, data bit jumping, etc. in the case of abnormal power-off, long-term static, etc., resulting in a decrease in the reliability of the file system, the present invention is hereby proposed by adopting a combination of a checksum bitmap mapping table and dual-copy reading and writing to effectively improve the reliability of the file system on an unreliable storage medium.

[0005] Technical solution of the present invention: A method for improving the reliability of a file system based on a bitmap mapping table mechanism, the method comprising the following steps:

[0006] Format the physical structure of the checksum bitmap mapping table;

[0007] Mount the management structure of the checksum bitmap mapping table;

[0008] Hook the read hook function and write hook function of the dual-copy (i.e., the original physical sector range and the copy physical sector range) of the corresponding storage medium physical sector, and create a copy physical sector mapping information table;

[0009] Calculate the first checksum corresponding to the data of the logical block to be written;

[0010] Execute the dual-copy write operation of the corresponding storage medium physical sector;

[0011] Update the first checksum at the table entry offset position in the checksum bitmap mapping table;

[0012] Obtain the first checksum of the corresponding logical block from the checksum bitmap mapping table;

[0013] Read data from the original physical sector range, calculate the second check code, and compare the first check code with the second check code; if the first check code is the same as the second check code, return the data read from the original physical sector range;

[0014] If the first check code is different from the second check code, read data from the copy physical sector range, calculate the third check code, and compare the first check code with the third check code; if the first check code is the same as the third check code, synchronously update the correct data read from the copy physical sector range to the original physical sector range, and then return the data read from the copy physical sector range; if the first check code is different from the third check code, return an error.

[0015] Specifically, format the physical structure of the check code bitmap mapping table, which specifically includes:

[0016] When the file system formats a volume, according to the total number of logical blocks it contains, create and initialize a check code bitmap mapping table at a specified logical offset of the volume, map the check code of each logical block in bitmap mode and store it at the corresponding position in the table, and determine the corresponding check code generation algorithm.

[0017] Specifically, mount the management structure of the check code bitmap mapping table, which specifically includes:

[0018] When the file system mounts a volume, create a management structure for the logical block check code bitmap mapping table. The logical block check code bitmap mapping table includes the starting logical offset, total size, and single logical block check code size on the volume.

[0019] Specifically, hook the read hook function and write hook function for the double copy of the physical sector of the corresponding storage medium, and create a copy physical sector mapping information table, which specifically includes:

[0020] When the storage medium adaptation management layer is initialized, hook the read hook function and write hook function for the double copy of the physical sector of the corresponding storage medium according to the configuration, and create a copy physical sector mapping information table; the copy physical sector mapping information table includes the original physical sector range and the copy physical sector range.

[0021] Specifically, perform a double copy write operation on the physical sector of the corresponding storage medium, which specifically includes:

[0022] The file system obtains the physical sector range of the corresponding storage medium for the logical block according to the address mapping relationship, executes the write hook function for the double copy of the physical sector of the corresponding storage medium, writes the data to the original physical sector range, then calculates the corresponding copy physical sector range, writes the data and updates the copy physical sector mapping table.

[0023] Specifically, read data from the original physical sector range, and calculate the second check code, which specifically includes:

[0024] After the file system successfully obtains the first checksum of the logical block, obtain the physical sector range of the storage medium corresponding to the logical block according to the address mapping relationship, execute the double-copy read hook function for the corresponding physical sectors of the storage medium, read data from the original physical sector range, and select the corresponding checksum generation algorithm according to the configuration, and then calculate the second checksum of the read logical block data.

[0025] Specifically, the size of the file system logical block is 2 n times that of the physical sector size of the storage medium, and n is an integer greater than or equal to 0.

[0026] Specifically, the size of the checksum for a single logical block of the file system can be configured to 4, 8, or 16 bytes.

[0027] Advantages and effects of the present invention: After implementing the method of the present invention, problems such as inconsistent read and write data and data bit flips in unreliable storage media under conditions such as abnormal power-off and long-term static placement can be solved, resulting in a reduction in the reliability of the file system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of a bitmap mapping checksum method provided by an embodiment of the present application;

[0029] Figure 2 Flowchart of the execution of a bitmap mapping checksum method provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] A file system volume consists of multiple logical blocks, which are the smallest data management units; a storage medium consists of multiple physical sectors, which are the smallest read / write units; a logical block of a file system consists of multiple physical sectors of the storage medium. The stable and reliable operation of the file system depends on the consistency of reading and writing data from logical block data to physical sector data, and unreliable storage media will have such problems. To improve the reliability of the file system, a method combining a checksum bitmap mapping table and double-copy read / write is adopted, as Figure 1 shown.

[0031] Embodiment 1

[0032] A method for improving the reliability of a file system based on a bitmap mapping table mechanism, comprising the following steps:

[0033] Step 1: Format the physical structure of the checksum bitmap mapping table;

[0034] Specifically, when the file system formats a volume, according to the total number of logical blocks it contains, a checksum bitmap mapping table is created and initialized at a specified logical offset of the volume. Each logical block's checksum is mapped and stored in the corresponding position of the table in bitmap form, and the corresponding checksum generation algorithm is determined.

[0035] Step 2: Mount the management structure of the checksum bitmap mapping table;

[0036] Specifically, when the file system mounts a volume, a management structure for the logical block checksum bitmap mapping table is created. The logical block checksum bitmap mapping table includes the starting logical offset, total size, and single logical block checksum size on the volume.

[0037] Step 3: Hang the read hook function and write hook function for the double copies of the physical sectors of the corresponding storage medium, and create a copy physical sector mapping information table;

[0038] Specifically, during the initialization of the storage medium adaptation management layer, according to the configuration, the read hook function and write hook function for the double copies of the physical sectors of the corresponding storage medium are hung, and a copy physical sector mapping information table is created; the copy physical sector mapping information table includes the original physical sector range and the copy physical sector range.

[0039] Step 4: Calculate the first checksum of the data of the logical block to be written;

[0040] It should be noted that when writing data to a logical block during the normal operation of the file system, according to the configuration, the corresponding checksum generation algorithm is selected, and then the first checksum of the data of the logical block to be written is calculated.

[0041] Step 5: Perform a double-copy write operation on the physical sectors of the corresponding storage medium;

[0042] Specifically, the file system obtains the physical sector range of the storage medium corresponding to the logical block according to the address mapping relationship, executes the write hook function for the double copies of the physical sectors of the corresponding storage medium, writes the data to the original physical sector range, then calculates the corresponding copy physical sector range, writes the data and updates the copy physical sector mapping table.

[0043] Step 6: Update the first checksum at the table entry offset position in the checksum bitmap mapping table;

[0044] Specifically, after the file system successfully writes the data of the logical block, according to the logical block number and the checksum size, the table entry offset position in the checksum bitmap mapping table of the logical block is obtained, and then the first checksum at this position is updated.

[0045] Step 7: Obtain the first checksum of the corresponding logical block from the checksum bitmap mapping table;

[0046] Specifically, when reading data from a logical block during the normal operation of the file system, the offset position of the entry in the logical block checksum bitmap mapping table is obtained according to the logical block number and the size of the checksum, and then the first checksum at that position is obtained.

[0047] Step 8: Read data from the original physical sector range, calculate the second checksum, and compare the first checksum and the second checksum; if the first checksum and the second checksum are the same, return the data read from the original physical sector range.

[0048] Specifically, after the file system successfully obtains the first checksum of the logical block, the physical sector range of the storage medium corresponding to the logical block is obtained according to the address mapping relationship, the double-copy read hook function of the corresponding storage medium physical sector is executed, data is read from the original physical sector range, and the corresponding checksum generation algorithm is selected according to the configuration, and then the second checksum of the read logical block data is calculated.

[0049] Compare the first checksum with the second checksum: If the two are the same, it indicates that the read data content is correct, return the data, and the execution is completed; otherwise, it indicates that the read data content is incorrect, and proceed to Step 9.

[0050] Step 9: If the first checksum and the second checksum are different, read data from the copy physical sector range, calculate the third checksum, and compare the first checksum and the third checksum; if the first checksum and the third checksum are the same, synchronously update the correct data read from the copy physical sector range to the original physical sector range, and then return the data read from the copy physical sector range; if the first checksum and the third checksum are different, return an error.

[0051] Specifically, according to the copy physical sector mapping information table maintained by the storage medium adaptation management layer, obtain the copy physical sector range corresponding to the original physical sector range, read the data, and select the corresponding checksum generation algorithm according to the configuration, and then calculate the third checksum of the read logical block data.

[0052] Compare the first checksum with the third checksum: If the two are different, it indicates that the read data content is still incorrect, return an error, and the execution is completed; otherwise, it indicates that the data content of the copy physical sector is correct, synchronously update the correct data to the original physical sector, return the data, and the execution is completed.

[0053] The described method for improving the reliability of the file system is applicable to embedded devices.

[0054] The logical block size of the described file system is 2 n times the physical sector size of the storage medium, where n is an integer greater than or equal to 0.

[0055] The logical offset of the file system is represented by the logical block number.

[0056] The size of the checksum for a single logical block in the file system can be configured to 4, 8, or 16 bytes.

[0057] The physical sector range of the storage medium is represented by the starting physical sector number and the number of consecutive physical sectors.

[0058] Embodiment 2

[0059] Reference Figures 1 to 2 , the specific implementation steps of using the method for improving the reliability of a file system based on a bitmap mapping table mechanism described in the present invention are as follows:

[0060] 1) Partition and format a file system volume on the storage medium, where the physical sector size of the storage medium is 512B, the logical block size of the file system is 4KB (including 8 physical sectors), the size of the file system volume is 2GB (including 524,288 logical blocks), and the size of the checksum for a single logical block is configured to 8B. Then, create and initialize a logical block checksum bitmap mapping table at the offset of logical block number 1 in the file system volume, with a total size of 4MB (including 1024 logical blocks).

[0061] 2) When mounting the file system volume, create a management structure for the logical block checksum bitmap mapping table, set the starting logical offset on the volume to 1, the total size to 4MB, and the size of the checksum for a single logical block to 8B.

[0062] 3) When initializing the storage medium adaptation management layer, according to the configuration, hook up the double-copy read and write functions dupSecReadHook and dupSecWriteHook for the corresponding physical sectors of the storage medium, and create a copy physical sector mapping information table, with the table entry format being (<original starting physical sector number, original number of consecutive physical sectors>, <copy starting physical sector number, copy number of consecutive physical sectors>).

[0063] 4) When writing data to logical block number 10000 during the normal operation of the file system, calculate an 8B logical block first checksum. Then, according to the address mapping relationship, obtain the physical sector range of the storage medium corresponding to logical block number 10000, which is 8 consecutive sectors starting from physical sector number 82000. Execute the double-copy write hook function dupSecWriteHook for the corresponding physical sectors of the storage medium, write the data to the 8 consecutive sectors in sequence, and then calculate the corresponding copy physical sector range, which is 8 consecutive sectors starting from physical sector number 4276304. Write the data to the 8 consecutive sectors in sequence and update the copy physical sector mapping table.

[0064] 5) Obtain the table entry offset position in the logical block checksum bitmap mapping table according to the logical block number 10000 and the checksum size of 8B, which is at the 2176-byte offset of the logical block number 20 of the file system volume, and then update the first 8-byte checksum starting from this position.

[0065] 6) When reading data from the logical block number 10000 during the normal operation of the file system, obtain the table entry offset position in the logical block checksum bitmap mapping table according to the logical block number 10000 and the checksum size of 8B, which is at the 2176-byte offset of the logical block number 20 of the file system volume, and then obtain the first 8-byte checksum starting from this position.

[0066] 7) According to the address mapping relationship, the physical sector range of the storage medium corresponding to the logical block number 10000 is 8 consecutive sectors starting from the physical sector number 82000. Execute the double-copy read hook function dupSecReadHook for the corresponding physical sectors of the storage medium. After sequentially reading data from 8 consecutive sectors, then calculate the 8B second checksum of the read logical block data, and compare the first checksum with the second checksum: If the two are the same, it indicates that the read data content is correct, return the data, and the execution is completed; otherwise, it indicates that the read data content is incorrect, and go to 8).

[0067] 8) Calculate the corresponding copy physical sector range for 8 consecutive sectors starting from the physical sector number 82000, which is 8 consecutive sectors starting from the physical sector number 4276304. Read the copy physical sector data, and then calculate the 8B third checksum of the read logical block data, and compare the first checksum with the third checksum: If the two are different, it indicates that the read data content is still incorrect, return an error, and the execution is completed; otherwise, it indicates that the copy physical sector data content is correct. Synchronously update the correct data read from the copy physical sector range to the original physical sector range, and then return the data, and the execution is completed.

[0068] Implementing according to the above working steps can achieve the effects of the invention.

Claims

1. A method for improving the reliability of a file system based on a bitmap mapping table mechanism, characterized in that, The method includes the following steps: Format the physical structure of the checksum bitmap mapping table; Mount the management structure of the checksum bitmap mapping table; Hook the read hook function and write hook function of the double copy of the physical sector of the corresponding storage medium. The double copy refers to the original physical sector range and the copy physical sector range, and create a copy physical sector mapping information table; Calculate the first checksum corresponding to the data of the logical block to be written; Perform a double-copy write operation on the physical sector of the corresponding storage medium; Update the first checksum at the table entry offset position in the checksum bitmap mapping table; Obtain the first checksum corresponding to the logical block from the checksum bitmap mapping table; Read data from the original physical sector range, calculate the second checksum, and compare the first checksum and the second checksum; if the first checksum and the second checksum are the same, return the data read from the original physical sector range; If the first checksum and the second checksum are different, read data from the copy physical sector range, calculate the third checksum, and compare the first checksum and the third checksum; if the first checksum and the third checksum are the same, synchronously update the correct data read from the copy physical sector range to the original physical sector range, and then return the data read from the copy physical sector range; if the first checksum and the third checksum are different, return an error.

2. The method for improving the reliability of a file system based on a bitmap mapping table mechanism according to claim 1, characterized in that, Format the physical structure of the checksum bitmap mapping table, specifically including: When formatting a volume in the file system, according to the total number of logical blocks it contains, create and initialize a checksum bitmap mapping table at the specified logical offset of the volume, map and store the checksum of each logical block in a bitmap manner to the corresponding position in the table, and determine the corresponding checksum generation algorithm.

3. The method for improving the reliability of a file system based on a bitmap mapping table mechanism according to claim 1, characterized in that, Mount the management structure of the checksum bitmap mapping table, specifically including: When mounting a volume in the file system, create a management structure for the logical block checksum bitmap mapping table. The logical block checksum bitmap mapping table includes the starting logical offset, total size, and single logical block checksum size on the volume.

4. The method for improving the reliability of a file system based on a bitmap mapping table mechanism according to claim 1, characterized in that, Hook the read hook function and write hook function of the double copy of the physical sector of the corresponding storage medium, and create a copy physical sector mapping information table, specifically including: When initializing the storage medium adaptation management layer, hook the read hook function and write hook function of the double copy of the physical sector of the corresponding storage medium according to the configuration, and create a copy physical sector mapping information table; the copy physical sector mapping information table includes the original physical sector range and the copy physical sector range.

5. The method for improving the reliability of a file system based on a bitmap mapping table mechanism according to claim 1, characterized in that, Perform a double-copy write operation on the physical sector of the corresponding storage medium, specifically including: The file system obtains the physical sector range of the storage medium corresponding to the logical block according to the address mapping relationship, executes the write hook function of the double copy of the physical sector of the corresponding storage medium, writes the data into the original physical sector range, then calculates the corresponding copy physical sector range, writes the data and updates the copy physical sector mapping table.

6. The method for improving the reliability of a file system based on a bitmap mapping table mechanism according to claim 1, characterized in that, Read data from the original physical sector range and calculate the second checksum, specifically including: After the file system successfully obtains the first checksum of the logical block, obtain the physical sector range of the storage medium corresponding to the logical block according to the address mapping relationship, execute the double-copy read hook function of the corresponding physical sector of the storage medium, read data from the original physical sector range, and select the corresponding checksum generation algorithm according to the configuration, and then calculate the second checksum of the read logical block data.

7. The method for improving the reliability of a file system based on a bitmap mapping table mechanism according to claim 1, characterized in that, The logical block size of the file system is 2 times the physical sector size of the storage medium. n n is an integer greater than or equal to 0.

8. The method for improving the reliability of a file system based on a bitmap mapping table mechanism according to claim 1, characterized in that, The size of the checksum of a single logical block in the file system can be configured to 4, 8, or 16 bytes.

Images