An online file system error detection method and device

Abstract

The application relates to an online file system error detection method and device, and belongs to the technical field of file system detection, which solves the problem that the file system error detection and repair in the prior art are time-consuming and need to be stopped, thereby affecting business continuity. The method comprises the following steps: obtaining the path of a file system to be detected, and then obtaining the type of the file system to be detected; based on the path and the type of the file system to be detected, reading superblock metadata of the file system to be detected, and then obtaining global information of the file system to be detected; based on the global information of the file system to be detected, obtaining the reading rate of each metadata type; based on the type of the file system to be detected and the reading rate of each metadata type, performing metadata detection on the file system to be detected, and obtaining a metadata detection result. The file system error detection is realized online and efficiently.

Description

Technical Field

[0001] This invention relates to the field of file system detection technology, and in particular to an online file system error detection method and apparatus. Background Technology

[0002] A file system partition is the space where the operating system stores file data. The storage medium provides a large, contiguous block of physical or virtual space, and the specific file system type is determined by dividing this large block of space according to the format specifications defined by the file system. This step is called file system partition formatting. After formatting and loading by the operating system, files can be read and written to the corresponding partition. File system partitions involve two types of data: metadata and data. Metadata is divided into two types: one type maintains the file system partition and includes information about file system usage; the other type records information related to the actual data. During long-term use of a file system partition, some abnormal operations may cause metadata corruption, resulting in the file system partition being unable to perform normal read and write operations.

[0003] When a file system partition error occurs, the current approach is to stop file read / write operations on the partition and unmount it. Then, a file system checking tool is used to scan, analyze, and process the error. The scan time is directly proportional to the partition size. Nowadays, file system partitions are usually quite large, reaching 10TB or more. This will lead to a prolonged business interruption and affect business continuity. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide an online file system error detection method and apparatus to solve the problem that existing file system error detection and repair takes too long and requires downtime, affecting business continuity.

[0005] On one hand, embodiments of the present invention provide an online file system error detection method, comprising the following steps: Obtain the path of the file system to be tested, and then obtain the type of the file system to be tested; Based on the path and type of the file system to be tested, the superblock metadata of the file system to be tested is read, and then the global information of the file system to be tested is obtained. Based on the global information of the file system to be tested, the read rate of each metadata type is obtained; Based on the type of the file system to be tested and the read rate of each metadata type, metadata testing is performed on the file system to be tested to obtain metadata testing results.

[0006] Furthermore, based on the set metadata detection rate and the total data size of each type of metadata, the read rate of each metadata type is determined; wherein, the data size of each type of metadata is obtained through the global information of the file system to be detected.

[0007] Furthermore, the read rate for each metadata type is expressed as follows: ; In the formula, Indicates the first i The read rate of this type of metadata Indicates the first i The total data size of each type of metadata This indicates the set metadata detection rate. This indicates rounding down to the nearest integer.

[0008] Furthermore, the metadata detection rate is dynamically adjusted based on the disk utilization and read / write queue length of the file system to be detected.

[0009] Furthermore, the metadata detection rate is dynamically adjusted based on the disk utilization and read / write queue length of the file system to be detected in the following ways: Based on the set time intervals, monitor disk utilization and read / write queue length, and determine: If the current disk utilization is greater than the set third threshold or the current read / write queue length is greater than the set fourth threshold, then the current metadata detection rate is set to the minimum detection rate or 0. Otherwise, based on the current disk utilization and the current read / write queue length, the current load score is obtained, and then the current metadata detection rate is obtained.

[0010] Furthermore, the current load score Represented as: ; In the formula, Indicates the current length of the read / write queue. This represents the maximum value of the read / write queue length. , These represent the first weight and the second weight, respectively. This represents the current disk utilization rate, and min() indicates taking the minimum value.

[0011] Furthermore, the current metadata detection rate is obtained based on the current load score, and is expressed as: ; In the formula, Indicates the current load score. This indicates the current metadata detection rate. This indicates the maximum detection rate.

[0012] On the other hand, embodiments of the present invention provide an online file system error detection device, comprising: The file system information acquisition module is used to obtain the path of the file system to be detected, and then to obtain the type of the file system to be detected; The global information acquisition module of the file system is used to read the superblock metadata of the file system to be detected based on the path and type of the file system to be detected, and then obtain the global information of the file system to be detected. The read rate determination module is used to obtain the read rate of each metadata type based on the global information of the file system to be tested; The file system detection module is used to perform metadata detection on the file system to be detected based on the type of the file system and the read rate of each metadata type, and obtain the metadata detection results.

[0013] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects: This invention provides an online file system error detection method and apparatus. By obtaining the path and type of the file system to be detected, the superblock metadata of the file system to be detected is read to obtain the global information of the file system to be detected, thereby obtaining the read rate of each metadata type. Then, based on the type of the file system to be detected and the read rate of each metadata type, the metadata of the file system to be detected is detected to obtain the metadata detection result. It can perform file system error detection online without unloading or interrupting the IO operation of business applications. Furthermore, it can control the amount of metadata detected per unit time during the file system error detection process, so as not to affect the normal IO operation of business applications.

[0014] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0015] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0016] Figure 1 This is a flowchart illustrating the online file system error detection method provided in Embodiment 1 of the present invention. Detailed Implementation

[0017] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0018] Example 1 One specific embodiment of the present invention discloses an online file system error detection method, such as... Figure 1 As shown, it includes the following steps: S1. Obtain the path of the file system to be tested, and then obtain the type of the file system to be tested.

[0019] Specifically, the file system path and type to be detected are obtained through tool calls or system interface calls.

[0020] More specifically, for the file system to be tested, the user can manually specify the corresponding block device path or UUID information, and obtain the path information of the file system corresponding to the specified device through methods including but not limited to the blkid tool or system call interface; then, based on the obtained file system path, the file system type to be tested, such as ext4 or xfs, can be obtained through methods including but not limited to the blkid tool or system call interface.

[0021] It should be noted that the metadata specification of a file system includes metadata types, the storage format of metadata on disk, its location and indexing method, verification logic, and consistency guarantee mechanisms. By obtaining the file system type, the actual organizational layout of its metadata on disk can be determined according to the metadata specification corresponding to that file system type, facilitating the reading and detection of specific metadata using a matching method.

[0022] S2. Based on the path and type of the file system to be detected, read the superblock metadata of the file system to be detected, and then obtain the global information of the file system to be detected.

[0023] Specifically, based on the path of the file system to be detected, the corresponding file system is opened online in read-only mode through the system call interface, and the corresponding superblock metadata information is read using the system call interface according to the metadata organization form corresponding to the type of the file system to be detected.

[0024] For example, if the file system to be tested is an ext4 file system, the primary superblock on the first block can be read; if the file system to be tested is an xfs file system, the superblock in the first block of each partition (AG) can be obtained.

[0025] Specifically, the global information of the file system is parsed from the superblock metadata of the file system to be detected. The global information includes, but is not limited to, the total number of blocks, block size, inode size, and metadata type of the file system. For example, the metadata type of the ext4 file system includes block group descriptors, inode table, data block bitmap, and inode bitmap.

[0026] S3. Based on the global information of the file system to be tested, obtain the read rate of each metadata type.

[0027] Specifically, the read rate of each metadata type is determined based on the set metadata detection rate and the total data size of each type of metadata; wherein, the data size of each type of metadata is obtained through global information of the file system to be detected.

[0028] Specifically, the read rate for each metadata type is expressed as follows: ; In the formula, Indicates the first i The read rate of this type of metadata Indicates the first i The total data size of each type of metadata This indicates the set metadata detection rate. This indicates rounding down; where, the metadata detection rate is the maximum amount of metadata that can be detected per unit time, and the read rate is the amount of metadata that can be detected per unit time.

[0029] Optionally, the metadata detection rate is a fixed value, which can be set according to the user's specific needs.

[0030] Optionally, the metadata detection rate can be dynamically adjusted based on the disk utilization and read / write queue length of the file system to be detected.

[0031] Specifically, the metadata detection rate is dynamically adjusted based on the disk utilization and read / write queue length of the file system to be detected in the following ways: Based on the set time intervals, monitor disk utilization and read / write queue length, and determine: If the current disk utilization is greater than the set third threshold or the current read / write queue length is greater than the set fourth threshold, then the current metadata detection rate is set to the minimum detection rate or 0. Otherwise, based on the current disk utilization and the current read / write queue length, the current load score is obtained, and then the current metadata detection rate is obtained.

[0032] More specifically, current load score Represented as: ; In the formula, Indicates the current length of the read / write queue. This represents the maximum value of the read / write queue length. , Let represent the first weight and the second weight respectively, and min() means taking the minimum value.

[0033] Furthermore, the first and second weights are set according to the business I / O characteristics (such as CPU-intensive or I / O-intensive); for example, they are set to 0.6 and 0.4 respectively.

[0034] More specifically, the current metadata detection rate is obtained based on the current load score and is expressed as: ; In the formula, This indicates the maximum detection rate.

[0035] Specifically, the maximum detection rate can be set according to the user's specific needs.

[0036] Understandably, the above method of dynamically adjusting the metadata detection rate based on the disk utilization and read / write queue size of the file system to be detected is suitable for systems that require fine-grained control.

[0037] Specifically, the metadata detection rate is dynamically adjusted based on the disk utilization and read / write queue length of the file system to be detected in the following ways: Historical load data is acquired based on the set predictive acquisition interval, and then the following is executed: Based on the historical load data at the current acquisition time, the current feature vector is obtained; where the historical load data is the time series data of disk utilization and read / write queue length at each acquisition time before the current acquisition time; The current feature vector is input into the trained disk load prediction model to obtain the load score prediction value at the current collection time. Based on the current load score prediction value, the metadata detection rate at the current collection time is obtained.

[0038] It is understandable that the metadata detection rate predicted by the disk load prediction model is a prediction interval. In other words, the metadata detection rate at the current collection time is the metadata detection rate for the time interval between the current collection time and the next collection time.

[0039] More specifically, based on historical load data at the current acquisition time, time-series context features and historical load features are extracted to construct the current feature vector. Among them, historical load features include instantaneous features, concurrent features, and trend features. Furthermore, time-series context features include the time period of the day and the day of the week in which the current acquisition time occurs; instantaneous features include disk utilization and read / write queue length at the most recent acquisition time; concurrent features include disk utilization and read / write queue length at the current acquisition time on the previous day and at the current acquisition time in the previous week; and trend features are the moving average of disk utilization and queue length within a defined time window and the slope calculated after linear fitting of the average.

[0040] More specifically, the disk load prediction model selects the gradient boosting decision tree model.

[0041] More specifically, the training of the disk load prediction model includes pre-training and online training. Pre-training is understood to be performed when the system is first deployed or the model needs to be completely reset, such as when the system is first installed, the model performance is severely degraded, or the system version is upgraded. Online training is performed periodically and automatically when the system is running normally.

[0042] Furthermore, the disk load prediction model is pre-trained in the following manner: Historical load data at each historical acquisition time is obtained, feature vectors at each historical acquisition time are extracted, and labels are added to each feature vector to construct a historical data sample set; where the label is the load score at the acquisition time corresponding to the feature vector. The historical data sample set is divided into a validation set and a training set to pre-train the disk load prediction model, resulting in a pre-trained disk load prediction model.

[0043] Furthermore, the disk load prediction model is trained online in the following way: During system operation, load data is continuously collected to build or update the sample set. When the number of built sample sets exceeds the set number, the previously trained disk load prediction model is fine-tuned based on the current sample set, and the current trained disk load prediction model is then used. The pre-trained disk load prediction model is the initial trained disk load prediction model.

[0044] More specifically, based on the current load score prediction value, the metadata detection rate at the current collection moment is obtained, including: If the current load score prediction value is less than or equal to the set low load threshold, the metadata detection rate at the current collection time is set to the maximum detection rate. If the current load score prediction value is greater than the set high load threshold, the metadata detection rate at the current collection time is set to a detection rate of 0. Otherwise, based on the current load score prediction value, the metadata detection rate at the current collection time is obtained.

[0045] Understandably, the above-mentioned method of dynamically adjusting the metadata detection rate based on the disk utilization and read / write queue length of the file system to be detected is suitable for systems with periodic but complex business workloads.

[0046] S4. Based on the type of the file system to be tested and the read rate of each metadata type, perform metadata detection on the file system to be tested and obtain the metadata detection results.

[0047] Specifically, based on calling the system interface to open the corresponding block device online, the metadata in the superblock is first detected according to the metadata specification of the file system type to be detected. Then, based on the read rate of each metadata type, the metadata of each corresponding type of the file system to be detected, excluding the superblock, is read, and the metadata is detected according to the metadata specification of the file system type to be detected. Among them, the detection includes, but is not limited to, verifying the consistency of metadata. It should be noted that the metadata detection in the superblock is not limited by the read rate. The metadata of the superblock is detected first, and then other types of metadata are detected based on the global information recorded in the superblock.

[0048] It should be noted that the file system metadata specification defines the file system verification logic and consistency determination method. The detection is to check whether the metadata meets the requirements of the file system metadata specification.

[0049] More specifically, the detection rules need to match the metadata specification corresponding to the type of the file system being detected.

[0050] For example, taking a portion of the detection content of the ext4 file system as an example, the detection rules include: (1) Check whether the metadata in the superblock conforms to the metadata format specification of the ext4 file system, such as whether the magic number recorded in the superblock is 0xEF53, and whether the data block size and other data are within a reasonable range.

[0051] (2) Check whether the number of free blocks recorded in each block group descriptor is consistent with the number of free blocks recorded in the corresponding block bitmap; (3) Check whether the number of free inodes recorded in each block group descriptor is consistent with the number of free inodes recorded in the corresponding inode bitmap; (4) Recursively check the inode information in each directory entry, obtain the logical block number and physical block number corresponding to the directory inode, and if the mapping between the logical block number and the physical block number is abnormal, there is a directory hole.

[0052] Specifically, the metadata detection results include whether file system errors exist or not; if file system errors exist, error information is also included.

[0053] More specifically, error messages can be identified using predefined error codes, and the recorded specific error content matches the error type. The error message content includes the error code, the location where the error occurred, the detection time, and other information, so as to facilitate subsequent analysis, alarms, and repair operations. The specific error data can be recorded using formats including but not limited to JSON.

[0054] Preferably, error messages can be printed directly or stored as needed, such as using a database, log file, or uploading to cloud storage.

[0055] It should be noted that the file system error detection method in this embodiment can be executed periodically as routine maintenance, or error detection can be performed when needed; the execution cycle can be set according to specific requirements.

[0056] It should be noted that traditional file system checks involve a large number of I / O operations. In a dual-machine hot standby software environment, some types of file system errors (such as directory hole errors) do not affect the current I / O, but after a failover, the file system will be unable to be mounted, causing business applications to fail to start and resulting in business interruption. The detection method in this embodiment can be used for routine online file system error detection. It can be combined with dual-machine hot standby software as a file system check item before a failover. When a file system error is detected that will prevent the file system from being mounted, it can provide alarm information to the dual-machine hot standby software, which can prevent situations where the file system partition cannot be mounted normally after a failover, thus affecting business operations.

[0057] Therefore, this implementation also includes: In dual-machine hot standby software, periodic checks are performed based on the aforementioned error detection methods, or checks are performed before switching machines. The results of these checks determine whether an alert needs to be issued to the file system.

[0058] Specifically, the dual-machine software can be configured to directly call the specific implementation interface of the aforementioned error detection method, or the specific implementation interface of the aforementioned error detection method can be directly embedded into the dual-machine software module to integrate the aforementioned error detection method with the dual-machine hot standby software.

[0059] More specifically, the judgment criteria are set by comprehensively considering the metadata specifications of the corresponding file system and the impact of errors on actual business.

[0060] For example, taking the ext4 file system as an example, when the number of free blocks recorded in the block bitmap is inconsistent with the number of free blocks recorded in the block group descriptor, the number of free blocks recorded in the inode bitmap is inconsistent with the number of free inodes recorded in the block group descriptor, or when there are directory holes, it indicates that there are serious errors in the file system, and there is a risk of loss of business data. After a dual-machine switchover, the business may not be able to start normally. At this time, it is necessary to immediately issue an alarm and notify the administrator to stop the business and repair the file system in a timely manner.

[0061] More specifically, if the detection results show serious errors in the file system, the dual-machine software will alert the operator and provide alarm information to the administrator to prevent the operator from performing the machine switch without knowing about the file system error, thus preventing the business from being affected by the inability to mount the file system partition properly after the machine switch.

[0062] Compared with existing technologies, the online file system error detection method provided in this embodiment reads the superblock metadata of the file system to be detected by obtaining the path and type of the file system to be detected, thereby obtaining the global information of the file system to be detected, and thus obtaining the read rate of each metadata type. Then, based on the type of the file system to be detected and the read rate of each metadata type, the file system to be detected is subjected to metadata detection to obtain the metadata detection result. It can perform file system error detection online without unloading or interrupting the IO operation of business applications. Furthermore, it can control the amount of metadata detected per unit time during the file system error detection process, so as not to affect the normal IO operation of business applications.

[0063] Example 2 A specific embodiment of the present invention discloses an online file system error detection device, comprising: The file system information acquisition module is used to obtain the path of the file system to be detected, and then to obtain the type of the file system to be detected; The global information acquisition module of the file system is used to read the superblock metadata of the file system to be detected based on the path and type of the file system to be detected, and then obtain the global information of the file system to be detected. The read rate determination module is used to obtain the read rate of each metadata type based on the global information of the file system to be tested; The file system detection module is used to perform metadata detection on the file system to be detected based on the type of the file system and the read rate of each metadata type, and obtain the metadata detection results.

[0064] The specific implementation process of this invention can be found in the above method embodiments, and will not be repeated here.

[0065] Since this embodiment is based on the same principle as the above method embodiment, this device also has the corresponding technical effects of the above method embodiment.

[0066] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0067] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An online file system error detection method, characterized in that, Includes the following steps: Obtain the path of the file system to be tested, and then obtain the type of the file system to be tested; Based on the path and type of the file system to be tested, the superblock metadata of the file system to be tested is read, and then the global information of the file system to be tested is obtained. Based on the global information of the file system to be tested, the read rate of each metadata type is obtained; Based on the type of the file system to be tested and the read rate of each metadata type, metadata detection is performed on the file system to be tested to obtain metadata detection results. Among them, the metadata detection in the superblock is not limited by the read rate. The metadata of the superblock is detected first, and then other types of metadata are detected based on the global information recorded in the superblock. Among these, the metadata detection rate is dynamically adjusted based on the disk utilization and read / write queue length of the file system to be detected, including: Based on the set time intervals, monitor disk utilization and read / write queue length, and determine: If the current disk utilization is greater than the set third threshold or the current read / write queue length is greater than the set fourth threshold, then the current metadata detection rate is set to the minimum detection rate or 0. Otherwise, based on the current disk utilization and the current read / write queue length, the current load score is obtained, and then the current metadata detection rate is obtained; The current load score Represented as: ； In the formula, Indicates the current length of the read / write queue. This represents the maximum value of the read / write queue length. , These represent the first weight and the second weight, respectively. This indicates the current disk utilization rate, and min() represents taking the minimum value; The current metadata detection rate is obtained based on the current load score and is expressed as follows: ； In the formula, Indicates the current load score. This indicates the current metadata detection rate. This indicates the maximum detection rate.

2. The online file system error detection method according to claim 1, characterized in that, Based on the set metadata detection rate and the total data size of each type of metadata, the read rate of each metadata type is determined; whereby the data size of each type of metadata is obtained through global information of the file system to be detected.

3. The online file system error detection method according to claim 2, characterized in that, The read rates for each metadata type are expressed as follows: ； In the formula, Indicates the first i The read rate of this type of metadata Indicates the first i The total data size of each type of metadata This indicates the set metadata detection rate. This indicates rounding down to the nearest integer.

4. An online file system error detection device, characterized in that, include: The file system information acquisition module is used to obtain the path of the file system to be detected, and then to obtain the type of the file system to be detected; The global information acquisition module of the file system is used to read the superblock metadata of the file system to be detected based on the path and type of the file system to be detected, and then obtain the global information of the file system to be detected. The read rate determination module is used to obtain the read rate of each metadata type based on the global information of the file system to be tested; The file system detection module is used to perform metadata detection on the file system to be detected based on the type of the file system to be detected and the read rate of each metadata type, and obtain the metadata detection results. Among them, the metadata detection in the superblock is not limited by the read rate. The metadata of the superblock is detected first, and then other types of metadata are detected based on the global information recorded in the superblock. Among these, the metadata detection rate is dynamically adjusted based on the disk utilization and read / write queue length of the file system to be detected, including: Based on the set time intervals, monitor disk utilization and read / write queue length, and determine: If the current disk utilization is greater than the set third threshold or the current read / write queue length is greater than the set fourth threshold, then the current metadata detection rate is set to the minimum detection rate or 0. Otherwise, based on the current disk utilization and the current read / write queue length, the current load score is obtained, and then the current metadata detection rate is obtained; The current load score Represented as: ； In the formula, Indicates the current length of the read / write queue. This represents the maximum value of the read / write queue length. , These represent the first weight and the second weight, respectively. This indicates the current disk utilization rate, and min() represents taking the minimum value; The current metadata detection rate is obtained based on the current load score and is expressed as follows: ； In the formula, Indicates the current load score. This indicates the current metadata detection rate. This indicates the maximum detection rate.

Images