A general hard disk state monitoring method based on multi-dimensional data fusion

Abstract

The application relates to a hard disk state general monitoring method based on multi-dimensional data fusion, and belongs to the technical field of data storage. In the scheme, a set of hard disk space state lightweight mapping table DMAP and a hard disk fine-grained IO statistical method are designed; multi-dimensional characteristic data obtained by processing and screening SMART data, Kernel logs, DMAP and IO fine-grained information are horizontally compared, and an abnormal hard disk is positioned in real time and quickly; multi-dimensional data of the abnormal hard disk is fused and decided, state misjudgment is excluded, and a slow disk and a fault disk are accurately found. The application is a lightweight application technology, can provide the service of quickly positioning the slow disk and the bad disk without affecting the storage business, and effectively supports the high performance and high reliability of a storage system.

Description

Technical Field

[0001] This invention belongs to the field of data storage technology, specifically relating to a general method for monitoring hard disk status based on multi-dimensional data fusion. Background Technology

[0002] In production environments, hard drive problems occur frequently, threatening data security and storage system performance. Under high concurrency and high bandwidth performance requirements, hard drive failure is no longer the primary factor limiting storage system performance; slow drives have become the main cause of performance limitations in small and medium-sized storage systems. Existing hard drive status monitoring mostly uses SMART information and machine learning methods based on SMART information. However, SMART information cannot report errors before they occur and can only report a limited number of faults; different manufacturers and models of hard drives report different amounts of information—for example, SAS drives and SSDs only report a few features. Therefore, machine learning methods are only applicable to a specific model of hard drive with a large amount of data and cannot detect slow drives in the system, making it not a universal hard drive status monitoring technology. For small and medium-sized storage systems and hard drives that report limited SMART information, a practical and universal hard drive status monitoring technology is urgently needed. Summary of the Invention

[0003] (a) Technical problems to be solved

[0004] The technical problem to be solved by this invention is to provide a practical and usable abnormal state monitoring method for various mechanical disks and solid-state disks in small and medium-sized storage systems in a production environment.

[0005] (II) Technical Solution

[0006] To address the aforementioned technical problems, this invention provides a general hard disk status monitoring method based on multi-dimensional data fusion, comprising the following steps:

[0007] Collect multidimensional data from the hard drives in the storage system, including hard drive SMART data, kernel log data, lightweight mapping table DMAP data, fine-grained IO statistics, and storage system operation log data;

[0008] Multidimensional feature data is obtained by performing model matching, key information extraction, or statistical analysis on the multidimensional data, and the multidimensional feature data is filtered using operation log data.

[0009] The filtered multidimensional feature data is compared, and the obtained abnormal data is fused and processed for decision-making to locate hard drives in two states: slow and faulty.

[0010] Preferably, after collecting the multidimensional data, the DMAP data therein is also managed as follows:

[0011] The first step is the initialization and management of DMAP data. The physical hard disk address space is sliced ​​into blocks of 1024M each. Each block of address space corresponds to one byte of DMAP data, which records the number of times the corresponding address space data is inconsistent. Multiple bytes are used to represent the state of a hard disk.

[0012] After completing the initialization of DMAP data, the storage system verifies the data, calls the API to submit the verification result, and parses the mapping relationship between the data address and the physical address based on the verification result.

[0013] Finally, maintain the dynamic mapping between DMAP data and hard disk physical addresses to identify added or removed physical disks.

[0014] Preferably, of the collected hard disk SMART data, kernel log data, storage system operation log data, DMAP data, and IO fine-grained statistics, the first three are data that the storage system can directly provide, while the latter two are two types of data generated to characterize the hard disk status. Among them, the storage system operation log data is not included in the horizontal comparison.

[0015] Preferably, the multidimensional feature data is obtained by performing model matching, key information extraction, or statistical analysis on the multidimensional data as follows:

[0016] First, a series of SMART features are obtained by model matching of hard disk SMART data. Second, Kernel log data is analyzed to extract warning and error messages related to the hard disk, resulting in Kernel features. Then, DMAP data obtained from storage system data verification is analyzed and statistically analyzed to obtain DMAP features. Finally, fine-grained IO features are obtained based on IO fine-grained statistical data, namely, the latency, bandwidth, proportion, and retry count of IO at different levels within a statistical period. The SMART features, Kernel features, DMAP features, and fine-grained IO features together constitute multi-dimensional feature data.

[0017] Preferably, the fine-grained statistical information of IO in the collected multidimensional data is obtained in the following way:

[0018] Based on I / O size, I / O is divided into four levels: 0-4K, 4-16K, 16-512K, and >512K. The statistics service runs in kernel mode, collects information on each I / O request, performs statistical analysis once per unit time, stores the proportion of different I / O levels, latency, bandwidth, and retry count information in a circular queue, and sends the data to the application via socket.

[0019] Preferably, the specific process of using operation log data to filter multidimensional feature data is as follows:

[0020] Parse storage system operation log data to extract illegal operation information;

[0021] List the multidimensional feature data and the time-related data of the illegal operation;

[0022] The correlation between the analysis operation and the feature data is analyzed, and strongly correlated data in the multidimensional feature data is filtered out.

[0023] Preferably, when comparing the filtered multidimensional feature data, the comparison objects are hard drives of the same type, that is, hard drives with the same model and the same role and status in the storage system; in the process of horizontal comparison of multidimensional data, the features are compared one by one to find the features and hard drives with abnormal deviations, and the judgment criteria for whether they are abnormal are preset; and the comparison of each dimension is based on the consistency of the elements.

[0024] Preferably, when performing a horizontal comparison of multidimensional feature data of similar hard drives, the horizontal comparison process includes three parts: system analysis, hard drive classification, and data comparison.

[0025] First, analyze the storage system and, based on its characteristics and data storage rules, label the hard drives according to their roles, locations, and models.

[0026] Secondly, based on the different labels of the hard drives, the hard drives are divided into different groups. Hard drives in the same group have the same role and model, and the storage system distributes the data evenly on the hard drives.

[0027] Finally, the multidimensional feature data of these hard drives are compared one by one in groups to identify the hard drives with anomalies, thus preparing data for the fusion processing and decision-making of abnormal data.

[0028] Preferably, the obtained abnormal data is fused and processed for decision-making. When locating hard drives in two states—slow speed and fault—related features from these multi-dimensional feature data are fused to obtain a series of unrelated features. Finally, a weighted sum is performed to determine the hard drive's state, including normal, slow, and fault states. During the weighted summation, different weighting coefficients correspond to different decision objectives. For slow drive decisions, the weights of fine-grained IO features and SMART features are greater than the weights of the other two features. For faulty drive decisions, the weights of DMAP features, Kernel features, and SMART features are greater than the weights of fine-grained IO features. The decision result is two binary (0, 1) values, representing the detection results for the slow and faulty hard drive states, respectively.

[0029] The present invention also provides a monitoring system for implementing the method.

[0030] (III) Beneficial Effects

[0031] To accurately and promptly locate slow and faulty disks in storage systems, this invention presents a general hard disk status monitoring method based on multi-dimensional data fusion. This scheme employs a lightweight hard disk space status mapping table (DMAP) and a fine-grained hard disk I / O statistics method. By comparing multi-dimensional feature data obtained through processing and filtering SMART data, kernel logs, DMAP, and fine-grained I / O information, abnormal hard disks can be located quickly and in real time. Furthermore, by fusing multi-dimensional data from abnormal hard disks, false status judgments are eliminated, accurately identifying slow and faulty disks. This invention is a lightweight application technology that provides rapid location of slow and faulty disks without impacting storage operations, effectively supporting the high performance and high reliability of storage systems. Attached Figure Description

[0032] Figure 1 This is a diagram of the multidimensional data fusion decision architecture of the present invention;

[0033] Figure 2 This is a diagram of the DMAP management architecture of the present invention;

[0034] Figure 3 This is a schematic diagram of the horizontal comparison service design of the present invention;

[0035] Figure 4 This is a multidimensional data fusion diagram of the present invention. Detailed Implementation

[0036] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0037] This invention proposes a general method for monitoring hard disk status based on multi-dimensional data fusion. For example... Figure 1 As shown, the method of the present invention is implemented using five modules. The data acquisition module is used to collect multidimensional data from the hard disk in the storage system, including hard disk SMART data, kernel log data, DMAP data, fine-grained IO statistics data, and storage system operation log data. The data preprocessing module is used to process the multidimensional data by means of model matching, key information extraction or statistical analysis to obtain multidimensional feature data, and to filter the data using operation log data. The horizontal comparison module is used to compare the filtered multidimensional feature data and submit abnormal data to the data fusion decision module for fusion processing and decision-making.

[0038] The lightweight mapping table (DMAP) is managed in the multidimensional data collected by the data acquisition module as follows:

[0039] This invention utilizes the data verification function of the storage system to provide a universal API for operating on the Disk Status Map (DMAP). Figure 2 The management architecture of DMAP is described, with the DMAP Manager module being the core of the entire architecture. The management of DMAP is mainly divided into three processes: initialization, update, and remapping.

[0040] First, there's the initialization management of DMAP. The DMAP Manager slices the physical hard drive address space into 1024MB blocks, with each block corresponding to one byte of DMAP. This byte records the number of times data inconsistencies have occurred in the corresponding address space, using multiple bytes to represent the state of a hard drive. Normally, the probability of a healthy hard drive experiencing data flips and write errors is close to zero.

[0041] After the DMAP initialization is completed, the data verification module of the storage system can submit the verification result to the DMAP manager by calling the API after verifying the data. The DMAP manager parses the mapping relationship between the data address and the physical address and updates the result to the DMAP.

[0042] Finally, the DMAP manager maintains a dynamic mapping between DMAPs and hard drive physical addresses, identifying added or removed physical disks.

[0043] The data acquisition module is the foundation of this invention. Of the hard drive SMART data, Kernel log data, storage system operation log data, DMAP data, and fine-grained IO statistics it collects, the first three are data directly provided by the storage system, while the latter two are data generated by this invention to accurately characterize the hard drive status. Storage system operation log data is not included in the horizontal comparison. To obtain multi-dimensional feature data, the data preprocessing module first obtains a series of SMART features by model matching the hard drive SMART data; secondly, it extracts warning and error messages related to the hard drive by analyzing the Kernel log, obtaining Kernel features; then, it analyzes and statistically analyzes the DMAP data obtained from storage system data verification to obtain DMAP features; finally, it obtains fine-grained IO features based on information such as latency, bandwidth, proportion, and retry count of IO at different levels over a statistical period. These SMART features, Kernel features, DMAP features, and fine-grained IO features constitute the multi-dimensional feature data.

[0044] The method for obtaining fine-grained IO statistics from the multidimensional data collected by the data acquisition module is as follows:

[0045] This invention provides a finer-grained breakdown of hard disk I / O statistics, facilitating component analysis of each I / O operation. Based on I / O size, it categorizes I / O into four levels: 0-4K, 4-16K, 16-512K, and >512K. The statistics service runs in kernel mode, collecting information from each I / O request and performing statistical analysis every unit of time. It stores the proportion of different I / O levels, latency, bandwidth, and retry information (number of retries) in a circular queue and sends the data to the application via a socket.

[0046] Multidimensional feature data needs to undergo a crucial data filtering process before it can be used for horizontal comparison. Data filtering is a vital step in reducing misjudgments, eliminating feature data that is prone to misjudgment due to non-compliant operation and maintenance. The specific implementation process is as follows:

[0047] Parse storage system operation log data to extract illegal operation information;

[0048] List the multidimensional feature data and the time-related data of the illegal operation;

[0049] The correlation between the analysis operation and the feature data is analyzed, and strongly correlated data in the multidimensional feature data is filtered out.

[0050] The core of this invention is the horizontal comparison of multidimensional feature data. Its foundation (the comparison object) is similar hard drives, meaning hard drives of the same model that play the same role and have the same status in the storage system. Modern storage systems are designed with the goal of uniform data distribution and a symmetrical architecture. In production environments, storage systems of the same batch mostly use the same type of hard drives, which provides the material basis for horizontal comparison. In the horizontal comparison of multidimensional data, features are compared one by one to identify abnormal features and hard drives.

[0051] The essence of horizontal comparison algorithms lies in the fact that the comparison of each dimension is based on the consistency of elements. For example, fine-grained IO feature comparison requires that the IO components are the same, and kernel feature comparison requires that the information is of the same type.

[0052] The horizontal comparison module performs horizontal comparisons of multi-dimensional feature data of similar hard drives as follows:

[0053] This invention identifies hard drives with abnormal conditions through horizontal comparison of similar hard drives, without relying on big data analysis, and can effectively operate on small to medium-sized file systems. Figure 3 As shown, the horizontal comparison process is mainly divided into three parts: system analysis, hard drive classification, and data comparison.

[0054] First, analyze the storage system. Currently, the mainstream storage systems supported are RAID, CEPH, and Gluster. Based on the characteristics of the storage system and the data storage rules, label the hard drives according to their roles, locations, models, etc.

[0055] Secondly, based on the different labels of the hard drives, the hard drives are divided into different groups. Hard drives in the same group have the same role and model. The storage system distributes the data evenly on the hard drives, and these hard drives should have similar external characteristics.

[0056] Finally, the multidimensional feature data of these hard drives are compared one by one in groups to identify the hard drives with anomalies, thus preparing data for multidimensional data fusion decision-making.

[0057] Design of a multi-dimensional data fusion decision model for the data fusion decision module:

[0058] This invention enables the fusion of multi-dimensional data, avoiding performance and cost losses in storage systems caused by misjudgments in single-dimensional data decision-making. For example... Figure 4 As shown, the multidimensional data mainly includes hard drive SMART data, DMAP data, kernel log data, and fine-grained IO statistics. After data preprocessing, these data form multiple feature data. Multidimensional data fusion merges related features from these feature data to obtain a series of unrelated features. Finally, these features are further weighted for decision-making to obtain the hard drive status (normal, slow, faulty). Multidimensional data fusion decision-making is the ultimate goal of this invention. The data fusion process involves the fusion of related features, feature quantification, and weighted summation. Different decision objectives correspond to different weighting coefficients. For slow drive decisions, IO and SMART features have a large weighting; for faulty drive decisions, the focus is on DMAP, kernel, and SMART features. The decision result is two binary (0, 1) values, representing the detection results for the slow and faulty states of the hard drive, respectively, for use by storage system and maintenance personnel.

[0059] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A general hard disk status monitoring method based on multi-dimensional data fusion, characterized in that, Includes the following steps: Collect multidimensional data from the hard drives in the storage system, including hard drive SMART data, kernel log data, lightweight mapping table DMAP data, fine-grained IO statistics, and storage system operation log data; Multidimensional feature data is obtained by performing model matching, key information extraction, or statistical analysis on the multidimensional data, and the multidimensional feature data is filtered using operation log data. The filtered multidimensional feature data is compared, and the obtained abnormal data is fused and processed for decision-making to locate hard drives in two states: slow and faulty. After collecting the multidimensional data, the DMAP data is also managed as follows: First, there is the initialization management of DMAP data. The physical hard disk address space is sliced ​​into blocks of 1024M each. Each block of address space corresponds to one byte of DMAP data. It records the number of times the data in the corresponding address space is inconsistent. Multiple bytes are used to represent the state of a hard disk. After completing the initialization of DMAP data, the storage system verifies the data, calls the API to submit the verification result, and parses the mapping relationship between the data address and the physical address based on the verification result. Finally, maintain the dynamic mapping between DMAP data and hard disk physical addresses, and identify the added or removed physical disks. Of the collected hard disk SMART data, kernel log data, storage system operation log data, DMAP data, and fine-grained IO statistics, the first three are data that the storage system can directly provide, while the latter two are two types of data generated to characterize the hard disk status. Among them, the storage system operation log data is not included in the horizontal comparison. The multidimensional feature data obtained by performing model matching, key information extraction, or statistical analysis on the multidimensional data are as follows: First, a series of SMART features are obtained by model matching of hard drive SMART data; second, by analyzing kernel log data, warning and error messages related to the hard drive are extracted to obtain kernel features. Then, the DMAP data obtained by verifying the storage system data is analyzed and statistically analyzed to obtain DMAP features; finally, based on the fine-grained IO statistics, namely the latency, bandwidth, proportion, and retry count of IO of different sizes within a certain period of time, fine-grained IO features are obtained. The SMART features, Kernel features, DMAP features, and fine-grained IO features constitute multi-dimensional feature data. The fine-grained statistics on IO in the collected multidimensional data are obtained in the following ways: Based on IO size, IO is divided into four levels: 0-4K, 4-16K, 16-512K, and >512K. The statistics service runs in kernel mode, collects information on each IO request, performs statistical analysis once per unit time, stores the proportion of different IO levels, latency, bandwidth, and retry count information in a circular queue, and sends the data to the application via socket. The specific process of using operation log data to filter multidimensional feature data is as follows: Parse storage system operation log data to extract illegal operation information; List the multidimensional feature data and the time-related data of the illegal operation; Analyze the correlation between the operation and the feature data, and filter out strongly correlated data in the multidimensional feature data; When comparing the filtered multidimensional feature data, the comparison objects are hard drives of the same type, that is, hard drives with the same model and the same role and status in the storage system; in the process of horizontal comparison of multidimensional data, the features are compared one by one to find the features and hard drives with abnormal deviations. The judgment criteria for whether they are abnormal are preset; and the comparison of each dimension is based on the consistency of the elements. When performing a horizontal comparison of multidimensional feature data of similar hard drives, the horizontal comparison process includes three parts: system analysis, hard drive classification, and data comparison. First, analyze the storage system and, based on its characteristics and data storage rules, label the hard drives according to their roles, locations, and models. Secondly, based on the different labels of the hard drives, the hard drives are divided into different groups. Hard drives in the same group have the same role and model, and the storage system distributes the data evenly on the hard drives. Finally, the multidimensional feature data of these hard drives are compared one by one in groups to identify the hard drives with abnormalities, thus preparing data for abnormal data fusion processing and decision-making. The obtained abnormal data is fused and processed for decision-making. When locating hard drives in two states—slow and faulty—related features from these multi-dimensional feature data are fused to obtain a series of unrelated features. Finally, a weighted sum is calculated to determine the hard drive's state, including normal, slow, and faulty states. During the weighted summation, different weighting coefficients correspond to different decision objectives. For slow drive decisions, the weights of fine-grained IO features and SMART features are larger than the weights of the other two features. For faulty drive decisions, the weights of DMAP features, Kernel features, and SMART features are larger than the weights of fine-grained IO features. The decision result is two binary (0, 1) values, representing the detection results for the slow and faulty states of the hard drive, respectively.

Images