Hard disk performance analysis method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of distributed storage systems, and discloses a hard disk performance analysis method and device, computer equipment and a storage medium, and the hard disk performance analysis method comprises the steps: collecting the time delay of each hard disk in a storage node of a distributed storage system, and obtaining monitoring data; grouping the monitoring data based on a preset acquisition interval to obtain at least one monitoring data group; determining a to-be-analyzed hard disk of which the time delay meets an abnormal condition in each monitoring data group, and determining an abnormal frequency of the to-be-analyzed hard disk; and performing performance analysis on the to-be-analyzed hard disk based on the abnormal frequency to obtain a target hard disk meeting a fault condition. According to the method, the monitoring data in the distributed system can be longitudinally compared and summarized and analyzed, false alarms caused by load fluctuation and the like are reduced, and the accuracy of fault hard disk recognition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed storage systems, and in particular to a hard disk performance analysis method, device, computer equipment and storage medium. Background Art

[0002] In a distributed storage system, the storage pool is a logical partition of the storage object. Its performance directly affects the data read and write latency and speed when the client accesses the storage cluster. The storage pool data read and write generated by the client access eventually falls on the hard disk under the OSD (Object-based Storage Device). Because of the crush algorithm, the data read and write to each hard disk in the storage pool is relatively uniform. Under normal circumstances, the performance and specifications of each hard disk in the storage pool are consistent, and at the same time, the read and write latency of the hard disk will not differ too much.

[0003] However, if some hard disks in the storage pool fail, the latency will increase when the storage pool read and write operations are sent to these failed hard disks, but data can still be read and written. This will increase the data read and write latency of the storage pool and reduce the throughput, resulting in client data backlogs.

[0004] Therefore, it is necessary to identify the faulty hard disk through performance analysis. However, in the related hard disk performance analysis method, the hard disk latency of each storage node in the distributed storage system is usually monitored in real time, and the hard disk with a latency of one is determined as a faulty hard disk, which leads to false alarms due to fluctuations in the workload on the hard disk, affecting the accuracy of faulty hard disk identification. Summary of the invention

[0005] In view of this, the present invention provides a hard disk performance analysis method, apparatus, computer equipment and storage medium to solve the problem of poor accuracy in fault hard disk identification caused by real-time fault identification in a distributed storage system.

[0006] In a first aspect, the present invention provides a hard disk performance analysis method, which is applied to a distributed storage system, and the method comprises:

[0007] Collecting the latency of each hard disk in the storage node of the distributed storage system to obtain monitoring data;

[0008] Grouping the monitoring data based on a preset collection interval to obtain at least one monitoring data group;

[0009] Determine in each of the monitoring data groups the hard disks to be analyzed whose delays meet the abnormal condition, and determine the abnormal frequency of the hard disks to be analyzed;

[0010] Based on the abnormal frequency, the performance of the hard disk to be analyzed is analyzed to obtain a target hard disk that meets the failure condition.

[0011] In an embodiment of the present invention, the time delay of each hard disk in the storage node of the distributed storage system can be first collected to obtain monitoring data, and the monitoring data can be grouped based on the preset collection interval to obtain at least one monitoring data group. Then, the hard disk to be analyzed whose delay meets the abnormal condition can be determined in each monitoring data group to obtain the abnormal frequency of the hard disk to be analyzed, and the performance analysis of the hard disk to be analyzed is performed based on the abnormal frequency to obtain the target hard disk, thereby performing a longitudinal comparison and summary analysis of the monitoring data in the distributed system, reducing false alarms caused by load fluctuations and other reasons, and improving the accuracy of faulty hard disk identification.

[0012] In an optional implementation, determining a hard disk to be analyzed whose latency satisfies an abnormal condition in each monitoring data group, and determining the abnormal frequency of the hard disk to be analyzed includes:

[0013] Determining the amount of time delay data in the monitoring data group;

[0014] Based on the amount of data, determine the average delay value corresponding to the monitoring data group;

[0015] Determine in each of the monitoring data groups the hard disks to be analyzed whose delay exceeds the corresponding average delay value, and obtain a hard disk collection;

[0016] Based on the hard disk collection, the number of abnormalities of each hard disk to be analyzed is determined, and the abnormality frequency of the hard disk to be analyzed is determined according to the number of abnormalities.

[0017] In an embodiment of the present invention, the hard disks to be analyzed whose delays exceed the corresponding delay average value can be determined in each monitoring data group to obtain a hard disk collection, and the number of abnormalities of each hard disk to be analyzed can be determined based on the hard disk collection, so as to determine the abnormal frequency of the hard disk to be analyzed according to the abnormal number of times, thereby providing a technical basis for performing performance analysis on the hard disk to be analyzed based on the abnormal frequency and obtaining an implementation method of a target hard disk that meets the failure condition.

[0018] In an optional implementation, the performance analysis of the hard disk to be analyzed is performed based on the abnormal frequency to obtain a target hard disk that meets the fault condition, including:

[0019] Counting the abnormal frequency of the hard disk to be analyzed to obtain the total number of abnormalities;

[0020] When the proportion of the abnormal frequency of the hard disk to be analyzed in the total abnormal quantity exceeds a first proportion threshold, the hard disk to be analyzed is determined as a target hard disk.

[0021] In an embodiment of the present invention, the abnormal data can be summarized and analyzed to obtain total abnormal data, and the hard disk to be analyzed whose abnormal frequency accounts for a proportion of the total abnormal number that exceeds a first ratio threshold is determined as a target hard disk, thereby reducing server resource consumption while improving the reliability of the located faulty hard disk to better suit large-scale distributed storage systems.

[0022] In an optional implementation, the performance analysis of the hard disk to be analyzed is performed based on the abnormal frequency to obtain a target hard disk that meets the fault condition, including:

[0023] Based on the abnormal frequency, determining the abnormal duration corresponding to the hard disk to be analyzed;

[0024] When the proportion of the abnormal duration in the preset collection interval exceeds a second proportion threshold, the hard disk to be analyzed is determined as a target hard disk.

[0025] In an embodiment of the present invention, while determining that a faulty target hard disk is used through a first ratio threshold, a second ratio threshold can also be used to identify faults on the hard disk to be analyzed based on the abnormal duration, thereby providing another fault identification strategy, which expands the usage scenarios of the present invention while ensuring the reliability of the identification results.

[0026] In an optional embodiment, the method further includes:

[0027] After obtaining the abnormal frequency of the hard disk to be analyzed, establishing a mapping relationship between the monitoring data group and the corresponding preset collection interval;

[0028] Based on the mapping relationships corresponding to all the monitoring data groups, mapping information is determined.

[0029] In the embodiment of the present invention, based on the mapping relationship between the monitoring data group and the preset collection interval, the mapping relationship between the abnormal frequencies corresponding to each hard disk pair to be analyzed in the monitoring data group can be determined at the same time to obtain mapping information. This is to facilitate the subsequent process of grouping the monitoring data based on the preset collection interval based on the mapping information, thereby determining the target hard disk that may have a fault relative to the current moment, and realizing real-time fault alarm operation, so as to avoid affecting the operating efficiency of the distributed storage system.

[0030] In an optional embodiment, the method further includes:

[0031] After the monitoring data is updated, determining an updated monitoring data group based on the updated data;

[0032] The mapping information is updated based on the mapping relationship between the hard disks to be analyzed and the corresponding abnormal frequencies in the update monitoring data group.

[0033] In the embodiment of the present invention, since the monitoring data is updated in real time, the mapping information is also updated accordingly. Therefore, the distance of the storage queue L can be limited, thereby avoiding the unlimited expansion of the content occupied by the mapping information, resulting in excessive memory usage and reduced fault location efficiency.

[0034] In an optional embodiment, the method further includes:

[0035] In response to the analysis instruction, based on the updated mapping information, the hard disk to be analyzed and the abnormal frequency corresponding to the hard disk to be analyzed are determined to perform performance analysis on the hard disk to be analyzed based on the abnormal frequency to obtain the target hard disk.

[0036] In an embodiment of the present invention, considering that after detecting that the above-mentioned mapping information is updated, it indicates that a new hard disk to be analyzed is generated, therefore, fault identification can be performed based on the abnormal frequency corresponding to the hard disk to be analyzed to achieve real-time positioning of the faulty hard disk, thereby ensuring the operating efficiency of the distributed storage system.

[0037] In a second aspect, the present invention provides a hard disk performance analysis device, comprising:

[0038] A collection module, used to collect the delay of each hard disk in the storage node of the distributed storage system to obtain monitoring data;

[0039] A grouping module, used to group the monitoring data based on a preset collection interval to obtain at least one monitoring data group;

[0040] A determination module, configured to determine, in each monitoring data group, a hard disk to be analyzed whose delay meets an abnormal condition, and obtain an abnormal frequency of the hard disk to be analyzed;

[0041] The analysis module is used to perform performance analysis on the hard disk to be analyzed based on the abnormal frequency to obtain a target hard disk that meets the fault condition.

[0042] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the hard disk performance analysis method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the hard disk performance analysis method of the first aspect or any corresponding embodiment thereof.

[0044] In a fifth aspect, the present invention provides a computer program product, including computer instructions, and the computer instructions are used to enable a computer to execute the hard disk performance analysis method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 is a flowchart of a hard disk performance analysis method according to an embodiment of the present invention;

[0047] Figure 2 is a schematic diagram of a process of collecting and storing monitoring data of a hard disk according to an embodiment of the present invention;

[0048] Figure 3 is a schematic diagram of storing monitoring data according to an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of determining a hard disk to be analyzed according to an embodiment of the present invention;

[0050] Figure 5 is a structural block diagram of a hard disk performance analysis device according to an embodiment of the present invention;

[0051] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0053] In combination with the application scenarios on which the execution of the hard disk performance analysis method depends, the application scenarios are described here.

[0054] In a distributed storage system, the storage pool is a logical partition of the storage object. Its performance directly affects the data read and write latency and speed when the client accesses the storage cluster. The storage pool data read and write generated by the client access eventually falls on the hard disk under the OSD (Object-based Storage Device). Because of the crush algorithm, the data read and write to each hard disk in the storage pool is relatively uniform. Under normal circumstances, the performance and specifications of each hard disk in the storage pool are consistent, and at the same time, the read and write latency of the hard disk will not differ too much.

[0055] However, if some hard disks in the storage pool fail, the latency will increase when the storage pool read and write operations are sent to these failed hard disks, but data can still be read and written. This will increase the data read and write latency of the storage pool and reduce the throughput, resulting in client data backlogs.

[0056] Therefore, it is necessary to identify the faulty hard disk through performance analysis. However, in the related hard disk performance analysis method, the hard disk latency of each storage node in the distributed storage system is usually monitored in real time, and the hard disk with a latency of one is determined as a faulty hard disk, which leads to false alarms due to fluctuations in the workload on the hard disk, affecting the accuracy of faulty hard disk identification.

[0057] Specifically, when real-time faulty hard disk identification is performed, the hard disk can be considered abnormal when its latency exceeds the threshold. However, if the threshold is set improperly or too low, it is very easy to cause false alarms during business peak hours or when clients frequently write small objects. If the threshold is set too high, the monitoring method will not be effective.

[0058] In addition, the average latency of the hard disk can be calculated in real time. If the latency of a hard disk is significantly higher than the average, the hard disk is considered abnormal. However, if the latency of most hard disks in the host is relatively high, it is impossible to accurately determine whether it is a fault, and this scenario is not uncommon in distributed storage systems.

[0059] Based on this, an embodiment of the present invention provides a hard disk performance analysis method, which first collects the latency of each hard disk in the storage node of the distributed storage system to obtain monitoring data, and groups the monitoring data based on a preset collection interval to obtain at least one monitoring data group. Then, the hard disk to be analyzed whose latency meets the abnormal condition can be determined in each monitoring data group to obtain the abnormal frequency of the hard disk to be analyzed, and the performance of the hard disk to be analyzed is analyzed based on the abnormal frequency to obtain the target hard disk, thereby performing a longitudinal comparison and summary analysis of the monitoring data in the distributed system, reducing false alarms caused by load fluctuations and other reasons, and improving the accuracy of faulty hard disk identification.

[0060] According to an embodiment of the present invention, an embodiment of a hard disk performance analysis method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0061] In this embodiment, a hard disk performance analysis method is provided, which can be used in the above-mentioned distributed storage system. Figure 1 is a flow chart of a hard disk performance analysis method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0062] Step S101, collecting the latency of each hard disk in the storage node of the distributed storage system to obtain monitoring data.

[0063] In an embodiment of the present invention, a distributed storage system is a computer data storage architecture that stores data in a dispersed manner on multiple independent devices. In this distributed storage system, data can be stored in fragments, and replica replication technology can be used to ensure the reliability and availability of data in each storage node, thereby realizing distributed storage and management of data.

[0064] When collecting monitoring data, a monitoring service can be set in each storage node of the distributed storage system. The monitoring service is used to use the iostat command to continuously monitor and collect performance data such as hard disk latency according to a preset collection interval to obtain monitoring data.

[0065] Specifically, since a storage node may include different types of hard disks, such as a system disk, an acceleration disk, a data disk, etc., and different types of hard disks have different storage functions, the read and write frequencies and latency conditions between different types of hard disks are usually different.

[0066] On this basis, when collecting monitoring data, the monitoring service can collect data separately according to the type of hard disk to obtain the monitoring data corresponding to each type of hard disk in the storage node, and store the monitoring data corresponding to each type of hard disk separately according to different storage locations, so as to facilitate fault identification of each hard disk separately, thereby improving the accuracy of fault identification results.

[0067] Here we take the data disk and acceleration disk as an example. Figure 2 The figure shows a flow chart of collecting and storing monitoring data of a hard disk, wherein firstly, the data disk and the acceleration disk in the storage node can be identified, and the monitoring data in the data disk and the acceleration disk can be collected through the iostat command. Then, the collected monitoring data can be parsed to mark the type of the monitoring data, and the marked monitoring data can be stored in the storage location of the corresponding type.

[0068] In addition, if Figure 3 The diagram shows a schematic diagram of storing monitoring data, wherein the monitoring service corresponding to each storage node in the distributed storage system can push the collected monitoring data to the monitoring center (i.e. Figure 3 Monitorcenter in the system) to manage the monitoring data through the monitoring center.

[0069] Step S102, grouping the monitoring data based on the preset collection interval to obtain at least one monitoring data group. In the embodiment of the present invention, the preset collection interval can be adjusted according to the data level of the hard disk. Specifically, if the data level is high and the data volume is large, the preset collection interval can be shorter, otherwise it can be longer. For example, the default preset collection interval T can be 20s.

[0070] Here, when the monitoring data is grouped based on the preset collection interval, a monitoring data group may be generated for the monitoring data corresponding to each preset collection interval, or the monitoring data under multiple preset collection intervals may be processed to jointly determine a monitoring data group for the multiple preset collection intervals.

[0071] Here, we take the generation of monitoring data groups based on monitoring data at multiple preset collection intervals as an example. Specifically, if the monitoring data includes m time delay data obtained by monitoring: v1-v m , then the monitoring data set can be expressed as {v1, v2, …v m}.

[0072] Step S103, determining the hard disk to be analyzed whose delay meets the abnormal condition in each monitoring data group, and determining the abnormal frequency of the hard disk to be analyzed.

[0073] In the embodiment of the present invention, the hard disk to be analyzed may be a hard disk with abnormal latency. When determining the hard disk to be analyzed, the latency average value V may be calculated based on the monitoring data group, where V=(v1+v2+…+v m ) / m. Then, the half-time acquisition time t' can be calculated, t'=t0+T / 2, where T is the above-mentioned preset acquisition interval. Next, in v1-v m The hard disks to be analyzed whose delay exceeds V are determined, and a timeout list is obtained. For example, when the hard disks to be analyzed include dx, ..., the timeout list can be expressed as [dx, ...]. In addition, if v1-v m If there is no hard disk to be analyzed whose delay exceeds V, the timeout list is empty, represented by [].

[0074] Specifically, Figure 4The figure shows a schematic diagram for determining the hard disk to be analyzed, wherein the time period corresponding to the monitoring data group is [t1, t1+30s], wherein the delay exceeding the average value V can be considered as an abnormal value, and the hard disk corresponding to the abnormal value is determined as the hard disk to be analyzed.

[0075] Here, the analysis data set (t', V, [dx, ...]) corresponding to the monitoring data group can be determined based on the above-mentioned delay average value V, the half-time collection time t' and the timeout list. Here, if the statistical period of the above-mentioned monitoring data group is 5 minutes, the total collection time is t1+N hours, which contains a total of n statistical periods, the monitoring data group can be expressed as: (t1', V1, [dx, ...]), (t2', V2, []), ... (t n ',V n , [dy, …]).

[0076] Next, the occurrence count of the hard disk to be analyzed can be counted based on the determined analysis data collection to obtain the abnormal frequency of each hard disk to be analyzed. For example, the occurrence count of the hard disk to be analyzed can be counted based on the timeout list in the analysis data collection to obtain the abnormal frequency of the hard disk to be analyzed.

[0077] Step S104: Perform performance analysis on the hard disk to be analyzed based on the abnormal frequency, and obtain a target hard disk that meets the fault condition.

[0078] In the embodiment of the present invention, the hard disk to be analyzed is a hard disk that may have a fault. Therefore, the performance analysis of the hard disk to be analyzed can be performed based on the abnormal frequency to obtain a target hard disk that meets the fault condition.

[0079] Specifically, the current operating parameters of the hard disk to be analyzed can be obtained, and based on the operating parameters, it can be analyzed whether the fault conditions are met, thereby improving the accuracy of fault judgment. For example, it can be determined whether the hard disk cannot be recognized in the BIOS. If so, the fault conditions are met, and there may be a problem with the hard disk connection or a fault in the hard disk itself.

[0080] In addition, it is also determined whether the abnormal frequency exceeds the frequency threshold in the fault condition. If so, it is determined that the hard disk to be analyzed meets the fault condition. If not, the hard disk to be analyzed is marked to continue the performance analysis of the hard disk to be analyzed, so that there is no need to obtain other data for fault judgment, so as to improve the efficiency of fault judgment.

[0081] It can be seen from the above description that in the embodiment of the present invention, first, the time delay of each hard disk in the storage node of the distributed storage system can be collected to obtain monitoring data, and the monitoring data can be grouped based on the preset collection interval to obtain at least one monitoring data group. Then, the hard disk to be analyzed whose delay meets the abnormal condition can be determined in each monitoring data group, the abnormal frequency of the hard disk to be analyzed can be obtained, and the performance analysis of the hard disk to be analyzed can be performed based on the abnormal frequency to obtain the target hard disk, thereby performing a longitudinal comparison and summary analysis of the monitoring data in the distributed system, reducing false alarms caused by load fluctuations and other reasons, and improving the accuracy of faulty hard disk identification.

[0082] In some optional implementations, the above step S103, determining the hard disk to be analyzed whose latency meets the abnormal condition in each monitoring data group, and determining the abnormal frequency of the hard disk to be analyzed, includes:

[0083] Step S1031, determining the amount of delay data in the monitoring data group.

[0084] Step S1032: Determine the average delay value corresponding to the monitoring data group based on the data quantity.

[0085] Step S1033, determining in each monitoring data group the hard disks to be analyzed whose delay exceeds the corresponding average delay value, and obtaining a hard disk collection.

[0086] Step S1034, based on the hard disk collection, determine the number of abnormalities of each hard disk to be analyzed, and determine the abnormal frequency of the hard disk to be analyzed according to the number of abnormalities.

[0087] In an embodiment of the present invention, the average latency value is the above-mentioned V, and the hard disk collection is a set of timeout lists corresponding to each monitoring data group. For example, the number of monitoring data groups is n, including [dx, ...]-[dy, ...], then the hard disk collection can be expressed as [dx, ..., dy, ...].

[0088] After determining the hard disk collection, the number of times the hard disk to be analyzed appears in the hard disk collection can be counted to obtain the abnormal frequency of the hard disk to be analyzed. For example, if the number of times the hard disk to be analyzed dx appears in the hard disk collection [dx, ..., dy, ...] is a, the abnormal frequency of the hard disk to be analyzed dx can be determined based on the number a.

[0089] In an embodiment of the present invention, the hard disks to be analyzed whose delays exceed the corresponding delay average value can be determined in each monitoring data group to obtain a hard disk collection, and the number of abnormalities of each hard disk to be analyzed can be determined based on the hard disk collection, so as to determine the abnormal frequency of the hard disk to be analyzed according to the abnormal number of times, thereby providing a technical basis for performing performance analysis on the hard disk to be analyzed based on the abnormal frequency and obtaining an implementation method of a target hard disk that meets the failure condition.

[0090] In some optional implementations, the above step S104, performing performance analysis on the hard disk to be analyzed based on the abnormal frequency to obtain a target hard disk that meets the fault condition, includes:

[0091] Step S11, counting the abnormal frequencies of each hard disk to be analyzed to obtain a total number of abnormalities.

[0092] Step S12: when the proportion of the abnormal frequency of the hard disk to be analyzed in the total abnormal quantity exceeds a first proportion threshold, the hard disk to be analyzed is determined as a target hard disk.

[0093] In the embodiment of the present invention, the total number of abnormalities may be the sum of the abnormal frequencies corresponding to each hard disk to be analyzed. For example, the abnormal frequency corresponding to the hard disk dx to be analyzed is a1, ..., and the abnormal frequency corresponding to the hard disk dy to be analyzed is a1. n , then the total number of anomalies x = a1 + ... + a n Then, the proportion of the abnormal frequency of the hard disk to be analyzed in the total number of abnormalities can be calculated. For example, the proportion corresponding to the hard disk dx to be analyzed can be expressed as a1 / x.

[0094] When determining the first ratio threshold, it can be determined based on empirical values, for example, the first ratio threshold is set to half of the total number of abnormalities. Alternatively, it can be determined based on the total collection time t of the monitoring data. n -t1 is longer than the above-mentioned preset acquisition interval T to calculate the first ratio threshold. For example, the first threshold can be expressed as: (t n -t1) / T / 2. For another example, the first ratio threshold may be determined according to the number n of monitoring data groups, and the first ratio threshold may be expressed as n / 2.

[0095] In an embodiment of the present invention, the abnormal data can be summarized and analyzed to obtain total abnormal data, and the hard disk to be analyzed whose abnormal frequency accounts for a proportion of the total abnormal number that exceeds a first ratio threshold is determined as a target hard disk, thereby reducing server resource consumption while improving the reliability of the located faulty hard disk to better suit large-scale distributed storage systems.

[0096] In some optional implementations, the above step S104, performing performance analysis on the hard disk to be analyzed based on the abnormal frequency to obtain a target hard disk that meets the fault condition, further includes:

[0097] Step S21, based on the abnormal frequency, determine the abnormal duration corresponding to the hard disk to be analyzed.

[0098] Step S22: when the proportion of the abnormal duration in the preset collection interval exceeds a second proportion threshold, the hard disk to be analyzed is determined as a target hard disk.

[0099] In an embodiment of the present invention, when determining the abnormal time corresponding to the hard disk to be analyzed based on the abnormal frequency, the delay duration corresponding to each abnormal frequency in the preset collection interval T can be summed to obtain the total abnormal time b corresponding to the hard disk to be analyzed.

[0100] Then, it is possible to calculate whether the proportion b / T of the abnormal duration in the preset collection interval exceeds the second proportion threshold c. If b / T>c, the hard disk to be analyzed is determined as the target hard disk; if not b / T<c, the hard disk to be analyzed is marked to continue performance analysis of the hard disk to be analyzed.

[0101] In an embodiment of the present invention, while determining that a faulty target hard disk is used through a first ratio threshold, a second ratio threshold can also be used to identify faults on the hard disk to be analyzed based on the abnormal duration, thereby providing another fault identification strategy, which expands the usage scenarios of the present invention while ensuring the reliability of the identification results.

[0102] In some optional embodiments, the above Figure 1 The corresponding embodiments also include:

[0103] Step S31, after obtaining the abnormal frequency of the hard disk to be analyzed, establishing a mapping relationship between the monitoring data group and the corresponding preset collection interval;

[0104] Step S32: Determine mapping information based on the mapping relationships corresponding to all monitoring data groups.

[0105] In the embodiment of the present invention, considering the need to locate the faulty hard disk in real time in the above-mentioned distributed storage system, a sliding window can be pre-set, and the monitoring data collected within the sliding window can be stored. Specifically, the length of the sliding window can be determined according to the actual usage scenario, for example, 2 hours, and the collection end node of the sliding window is usually the current time.

[0106] When the monitoring data in the sliding window is stored, a storage list L may be established to store the mapping information corresponding to the monitoring data in the list L, so that the mapping relationship in the storage list L is updated in real time according to the update of the monitoring data in the sliding window.

[0107] When determining the mapping information, the analysis data collection corresponding to the preset collection interval can be obtained, and a mapping between the analysis data collection and the hard disk to be analyzed can be established, so as to obtain a mapping relationship between the abnormal frequency and the hard disk to be analyzed, and the mapping information can be determined based on the mapping relationship, so as to execute a subsequent process of grouping the monitoring data based on the preset collection interval based on the mapping information, so as to determine the target hard disk that may have a fault relative to the current moment.

[0108] Specifically, we can first determine the abnormal frequency corresponding to each hard disk to be analyzed in a certain monitoring data group, and establish an association relationship until the mapping relationship corresponding to all monitoring data groups is determined. For example, if the abnormal frequency corresponding to the hard disk dx to be analyzed in data monitoring group 1 is 1, then the association relationship dx:1 can be established. If the abnormal frequency corresponding to the hard disk dy to be analyzed is m, then the association relationship dy:m can be established, and finally the sub-map corresponding to monitoring data group 1 is obtained, that is, map1: {dx:1, dy:m, ...}. Then, the sub-maps corresponding to all data monitoring groups can be combined to obtain mapping information.

[0109] It should be understood that the historical faulty hard disks can be checked by setting the starting position of the sliding window, or a collection of analysis data for a time period in a preset collection interval can be obtained based on the mapping information, so as to analyze the performance of the storage node in that time period based on the collection of analysis data to improve operation and maintenance efficiency.

[0110] In the embodiment of the present invention, based on the mapping relationship between the monitoring data group and the preset collection interval, the mapping relationship between the abnormal frequencies corresponding to each hard disk pair to be analyzed in the monitoring data group can be determined at the same time to obtain mapping information. This is to facilitate the subsequent process of grouping the monitoring data based on the preset collection interval based on the mapping information, thereby determining the target hard disk that may have a fault relative to the current moment, and realizing real-time fault alarm operation, so as to avoid affecting the operating efficiency of the distributed storage system.

[0111] In some optional implementations, after the monitoring data is updated, the above Figure 1 The corresponding embodiments also include:

[0112] Step S41: after the monitoring data is updated, determine to update the monitoring data group based on the updated data.

[0113] Step S42: updating mapping information based on the mapping relationship between the hard disks to be analyzed and the corresponding abnormal frequencies in the updated monitoring data group.

[0114] In the embodiment of the present invention, since the monitoring data is updated in real time, the mapping information is also updated accordingly. To avoid the unlimited expansion of the content occupied by the mapping information and resulting in excessive memory usage, the length of the storage queue L can be limited.

[0115] Specifically, the implementation method of determining the hard disk to be analyzed in the update data and the corresponding abnormal frequency is as described above. Figure 1 The corresponding embodiments are not described in detail here. After the monitoring data is updated, the updated monitoring data group corresponding to the updated data can be n+1and the corresponding mapping relationship are put into the storage queue L, and the first monitoring data group e1 and the corresponding mapping relationship in the storage queue are cleared.

[0116] Here, the monitoring data group e1 in the storage queue L can be cleared first. For example, if the monitoring data groups contained in the storage queue L before the update include: (t1', V1, []), (t2', V2, []), (t3', V3, [dx, ...]) ... (t n ',V n, [d y, …]). Then, the monitoring data groups included in the update include: (t2', V2, []), (t3', V3, [dx, …])… (t n ',V n, [d y, …]), (t n+1 ',V n+1 ,[d z ,…]).

[0117] Next, the mapping relationship in the mapping information map can be updated through the following logic:

[0118]

[0119] In the embodiment of the present invention, since the monitoring data is updated in real time, the mapping information is also updated accordingly. Therefore, the distance of the storage queue L can be limited, thereby avoiding the unlimited expansion of the content occupied by the mapping information, resulting in excessive memory usage and reduced fault location efficiency.

[0120] In some optional embodiments, the above Figure 1 The corresponding embodiments also include:

[0121] In response to the analysis instruction, based on the updated mapping information, the hard disk to be analyzed and the abnormal frequency corresponding to the hard disk to be analyzed are determined to perform performance analysis on the hard disk to be analyzed based on the abnormal frequency to obtain the target hard disk.

[0122] In the embodiment of the present invention, after detecting that the mapping information is updated, it indicates that a new hard disk to be analyzed is generated. At this time, the data analysis collection corresponding to the updated data can be obtained based on the storage list L. n+1 ',V n+1 ,[d z ,…]), to determine the hard disk to be analyzed and the corresponding abnormal frequency through the data analysis collection, and then execute the implementation method corresponding to the above step S104 based on the abnormal frequency to determine whether the hard disk to be analyzed is a faulty target hard disk.

[0123] In addition, the present invention can analyze the performance of storage nodes in a certain period of time based on the data analysis collection in the storage list L to intuitively understand the performance, load and other operating conditions of the storage nodes, thereby providing data support for operation and maintenance decisions.

[0124] In an embodiment of the present invention, considering that after detecting that the above-mentioned mapping information is updated, it indicates that a new hard disk to be analyzed is generated, therefore, fault identification can be performed based on the abnormal frequency corresponding to the hard disk to be analyzed to achieve real-time positioning of the faulty hard disk, thereby ensuring the operating efficiency of the distributed storage system.

[0125] In summary, in the embodiment of the present invention, the time delay of each hard disk in the storage node of the distributed storage system can be first collected to obtain monitoring data, and the monitoring data can be grouped based on the preset collection interval to obtain at least one monitoring data group. Then, the hard disk to be analyzed whose delay meets the abnormal condition can be determined in each monitoring data group, the abnormal frequency of the hard disk to be analyzed can be obtained, and the performance analysis of the hard disk to be analyzed can be performed based on the abnormal frequency to obtain the target hard disk, thereby performing a longitudinal comparison and summary analysis of the monitoring data in the distributed system, reducing false alarms caused by load fluctuations and other reasons, and improving the accuracy of faulty hard disk identification.

[0126] In this embodiment, a hard disk performance analysis device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0127] This embodiment provides a hard disk performance analysis device, such as Figure 5 As shown, including:

[0128] The collection module 501 is used to collect the latency of each hard disk in the storage node of the distributed storage system to obtain monitoring data;

[0129] A grouping module 502, configured to group the monitoring data based on a preset collection interval to obtain at least one monitoring data group;

[0130] A determination module 503 is used to determine the hard disk to be analyzed whose delay meets the abnormal condition in each monitoring data group, and obtain the abnormal frequency of the hard disk to be analyzed;

[0131] The analysis module 504 is used to perform performance analysis on the hard disk to be analyzed based on the abnormal frequency, and obtain a target hard disk that meets the fault condition.

[0132] In some optional implementations, the determining module 503 includes:

[0133] Determine the amount of delay data in the monitoring data group;

[0134] Based on the amount of data, determine the average latency corresponding to the monitoring data group;

[0135] Determine in each monitoring data group the hard disk to be analyzed whose latency exceeds the corresponding latency average value, and obtain a hard disk collection;

[0136] Based on the hard disk collection, the number of abnormalities of each hard disk to be analyzed is determined, and the abnormal frequency of the hard disk to be analyzed is determined according to the number of abnormalities.

[0137] In some optional implementations, the determining module 503 includes:

[0138] Count the abnormal frequency of the hard disk to be analyzed and get the total number of abnormalities;

[0139] When the proportion of the abnormal frequency of the hard disk to be analyzed in the total abnormal quantity exceeds a first proportion threshold, the hard disk to be analyzed is determined as a target hard disk.

[0140] In some optional implementations, the analysis module 504 includes:

[0141] Based on the abnormal frequency, determine the abnormal duration of the hard disk to be analyzed;

[0142] When the proportion of the abnormal duration in the preset collection interval exceeds a second proportion threshold, the hard disk to be analyzed is determined as a target hard disk.

[0143] In some optional embodiments, the device further comprises:

[0144] After obtaining the abnormal frequency of the hard disk to be analyzed, a mapping relationship between the monitoring data group and the corresponding preset collection interval is established;

[0145] Based on the mapping relationships corresponding to all monitoring data groups, the mapping information is determined.

[0146] In some optional embodiments, the device further comprises:

[0147] After the monitoring data is updated, determining an updated monitoring data group based on the updated data;

[0148] Based on the mapping relationship between the hard disk to be analyzed and the corresponding abnormal frequency in the update monitoring data group, the mapping information is updated.

[0149] In some optional embodiments, the device further comprises:

[0150] In response to the analysis instruction, based on the updated mapping information, the hard disk to be analyzed and the abnormal frequency corresponding to the hard disk to be analyzed are determined to perform performance analysis on the hard disk to be analyzed based on the abnormal frequency to obtain the target hard disk.

[0151] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0152] The hard disk performance analysis device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0153] The embodiment of the present invention also provides a computer device having the above Figure 5 The hard disk performance analysis device is shown.

[0154] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0155] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0156] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0157] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0158] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0159] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0160] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0161] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0162] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A hard disk performance analysis method, characterized in that: Applied to a distributed storage system, the method comprises: Collecting the latency of each hard disk in the storage node of the distributed storage system to obtain monitoring data; Grouping the monitoring data based on a preset collection interval to obtain at least one monitoring data group; Determine in each of the monitoring data groups the hard disks to be analyzed whose delays meet the abnormal condition, and determine the abnormal frequency of the hard disks to be analyzed; Based on the abnormal frequency, the performance of the hard disk to be analyzed is analyzed to obtain a target hard disk that meets the failure condition.

2. The method according to claim 1, characterized in that The step of determining the hard disk to be analyzed whose delay meets the abnormal condition in each monitoring data group, and determining the abnormal frequency of the hard disk to be analyzed, includes: Determining the amount of time delay data in the monitoring data group; Based on the amount of data, determine the average delay value corresponding to the monitoring data group; Determine in each of the monitoring data groups the hard disks to be analyzed whose delay exceeds the corresponding average delay value, and obtain a hard disk collection; Based on the hard disk collection, the number of abnormalities of each hard disk to be analyzed is determined, and the abnormality frequency of the hard disk to be analyzed is determined according to the number of abnormalities.

3. The method according to claim 2, characterized in that The performing performance analysis on the hard disk to be analyzed based on the abnormal frequency to obtain a target hard disk that meets the fault condition includes: Counting the abnormal frequency of the hard disk to be analyzed to obtain the total number of abnormalities; When the proportion of the abnormal frequency of the hard disk to be analyzed in the total abnormal quantity exceeds a first proportion threshold, the hard disk to be analyzed is determined as a target hard disk.

4. The method according to claim 1, characterized in that: The performing performance analysis on the hard disk to be analyzed based on the abnormal frequency to obtain a target hard disk that meets the fault condition includes: Based on the abnormal frequency, determining the abnormal duration corresponding to the hard disk to be analyzed; When the proportion of the abnormal duration in the preset collection interval exceeds a second proportion threshold, the hard disk to be analyzed is determined as a target hard disk.

5. The method according to claim 1, characterized in that: The method further comprises: After obtaining the abnormal frequency of the hard disk to be analyzed, establishing a mapping relationship between the monitoring data group and the corresponding preset collection interval; Based on the mapping relationships corresponding to all the monitoring data groups, mapping information is determined.

6. The method according to claim 5, characterized in that The method further comprises: After the monitoring data is updated, determining an updated monitoring data group based on the updated data; The mapping information is updated based on the mapping relationship between the hard disks to be analyzed and the corresponding abnormal frequencies in the update monitoring data group.

7. The method according to claim 6, characterized in that The method further comprises: In response to the analysis instruction, based on the updated mapping information, the hard disk to be analyzed and the abnormal frequency corresponding to the hard disk to be analyzed are determined to perform performance analysis on the hard disk to be analyzed based on the abnormal frequency to obtain the target hard disk.

8. A hard disk performance analysis device, characterized in that: Applied to a distributed storage system, the device comprises: A collection module, used to collect the delay of each hard disk in the storage node of the distributed storage system to obtain monitoring data; A grouping module, used to group the monitoring data based on a preset collection interval to obtain at least one monitoring data group; A determination module, configured to determine, in each monitoring data group, a hard disk to be analyzed whose delay meets an abnormal condition, and obtain an abnormal frequency of the hard disk to be analyzed; The analysis module is used to perform performance analysis on the hard disk to be analyzed based on the abnormal frequency, and obtain a target hard disk that meets the fault condition.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the hard disk performance analysis method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the hard disk performance analysis method according to any one of claims 1 to 7.

Images