Server fault detection method, device, equipment and storage medium
By comparing the actual operating data of the server with the standard data, predicting the probability of failure, the problem of difficult to predict server failure in the existing technology is solved, and the effect of taking precautions in advance and improving server stability is achieved.
Patent Information
- Application Number
- CN202411891532.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-20
- Publication Date
- 2025-04-04
AI Technical Summary
The prior art is difficult to predict server failures, resulting in business interruptions and data loss, and the losses are large.
By obtaining the actual operating data of the server and comparing it with the standard operating data, we can determine whether it is abnormal data, determine the abnormal operating indicators, predict the probability of failure, and generate the fault detection result.
It can predict server failures in advance, reduce losses, and improve server stability.
Smart Images

Figure CN119759629A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of server technology, and in particular to a server fault detection method, device, equipment and storage medium. Background Art
[0002] A server is a device that provides computing services. Its stability is related to business continuity, data integrity, and overall operational efficiency. In some current technologies, by installing detection software, it is possible to detect faults that have already occurred in the server. However, when a fault is detected, it means that problems such as business interruption and data loss have already occurred, and the loss is still large. Therefore, it is necessary to predict server failures so that corresponding preventive measures can be taken in advance for possible server failures, thereby improving the stability of the server. Summary of the invention
[0003] In view of this, the present disclosure provides a server fault detection method, a server fault detection device, an electronic device and a computer-readable storage medium, which can predict server faults and improve the stability of the server.
[0004] In a first aspect, the present disclosure provides a server failure detection method, the method comprising:
[0005] Get the actual operation data of the server;
[0006] Comparing the actual operation data with the standard operation data, and determining whether the actual operation data is abnormal data based on the comparison result;
[0007] If the actual operation data is abnormal data, determining an abnormal operation index of the actual operation data, wherein the abnormal operation index is used to characterize a cumulative deviation degree of the actual operation data relative to the standard operation data;
[0008] Based on the abnormal operation indicator, the failure probability of the server is predicted, and according to the failure probability, a first fault detection result of the server is generated.
[0009] In a second aspect, the present disclosure provides a server fault detection device, the device comprising:
[0010] A data acquisition module is used to obtain the actual operation data of the server;
[0011] A data comparison module, used to compare the actual operation data with the standard operation data, and based on the comparison result, determine whether the actual operation data is abnormal data;
[0012] an indicator determination module, configured to determine an abnormal operation indicator of the actual operation data if the actual operation data is abnormal data, wherein the abnormal operation indicator is used to characterize a cumulative deviation degree of the actual operation data from the standard operation data;
[0013] The fault detection module is used to predict the fault probability of the server based on the abnormal operation indicator, and generate a first fault detection result of the server according to the fault probability.
[0014] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the above method by executing the computer instructions.
[0015] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the above method.
[0016] In the technical solutions of some embodiments of the present disclosure, after comparing the actual operation data of the server with the standard operation data, it can be determined whether the actual operation data is abnormal data. In the case where the actual operation data is abnormal data, it can be determined that the server is in an abnormal operation state. In the case where the server is in an abnormal operation state, although the server has not yet failed, there is a risk of failure in the future. Therefore, the failure probability of the server can continue to be predicted based on the abnormal operation indicators of the actual operation data, and the first fault detection result of the server can be obtained, so that the maintenance personnel can take fault prevention measures in advance based on the first fault detection result, thereby achieving the purpose of improving the stability of the server. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0018] Figure 1 is a schematic diagram of a system architecture provided by some embodiments of the present disclosure;
[0019] Figure 2 is a flow chart of a server fault detection method provided by some embodiments of the present disclosure;
[0020] Figure 3is a schematic diagram of the relationship between the data interval range of standard operating data, the first data threshold and the second data threshold provided by some embodiments of the present disclosure;
[0021] Figure 4 is a module schematic diagram of a server fault detection device provided by some embodiments of the present disclosure;
[0022] Figure 5 It is a schematic diagram of the structure of an electronic device provided by some embodiments of the present disclosure. DETAILED DESCRIPTION
[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.
[0024] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0025] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "some embodiments" or "the embodiment" should be understood as "at least some embodiments". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.
[0026] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.
[0027] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.
[0028] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scopes of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and their authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.
[0029] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to software or hardware such as an electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.
[0030] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message is sent to the relevant user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.
[0031] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0032] In view of the problem that some technologies cannot predict server failures, the present disclosure provides a server failure detection method that can predict server failures and improve server stability. The server failure detection method can be applied to Figure 1 The host computer in the system architecture shown.
[0033] Combined with reference Figure 1. In the host computer, the server information of one or more servers that need to be predicted for faults can be pre-configured. The server information may include but is not limited to the operating system IP of the server, the baseboard management controller IP, etc. Based on the server information, the host computer can communicate with the server and collect the actual operation data of the server. The actual operation data refers to the current operation data of the server, which may include but is not limited to the actual operation temperature, actual operation power, actual memory utilization, actual I / O read speed, packet loss rate, bandwidth utilization, data transmission rate, etc. of the server. For example, based on the baseboard management controller IP, the host computer can establish a connection with the baseboard management controller in the server, and then obtain the actual operation temperature and actual operation power of the server. For another example, based on the operating system IP, the host computer can establish a connection with the operating system of the server, and then obtain the actual memory utilization, actual I / O read speed, actual packet loss rate, actual data transmission rate and actual bandwidth utilization of the server.
[0034] Based on the actual operation data collected, the operation status of the server can be judged. In some embodiments of the present application, the operation status of the server can be divided into a normal operation status, an abnormal operation status and a faulty operation status. Among them, when all the actual operation data are within the normal range allowed by the server, it means that the server is in a normal operation state; when at least part of the actual operation data occasionally exceeds the normal range allowed by the server (that is, when the actual operation data fluctuates abnormally), it means that the server is in an abnormal operation state; when at least part of the actual operation data exceeds the normal range allowed by the server for a long time, it means that the server is in a faulty operation state. When the server is in a normal operation state, it means that the server can provide services to the outside normally and there is no risk of failure; when the server is in an abnormal operation state, it means that the server can provide services to the outside normally, but there is a risk of failure; when the server is in a faulty operation state, it means that the server can no longer provide services to the outside normally.
[0035] based on Figure 1 For a description of Figure 2 , which is a flow chart of a server fault detection method provided in some embodiments of the present disclosure. Figure 2 In the server fault detection method, the server fault detection method includes the following steps:
[0036] Step S201, obtaining actual operation data of the server.
[0037] Specifically, Figure 1 As described above, based on the pre-configured server information, the host computer can determine the servers that need to be predicted for faults and collect the actual operation data of these servers.
[0038] In this embodiment, based on the server failure to be predicted, actual operation data related to the server failure can be collected. For example, when predicting whether a server will have a storage failure in the future, the actual operation data related to storage of the server can be collected, such as actual memory utilization, actual I / O read speed, etc. For another example, when predicting whether a server will have a performance failure in the future, the actual operation data related to performance of the server can be collected, such as actual CPU utilization, actual number of concurrent requests, etc. Only collecting actual operation data that matches the server failure type can reduce the amount of data collected.
[0039] In this embodiment, the actual operation data may be collected once every preset time period. The actual operation data collected at different time points may be different. The collected actual operation data may be stored in a database.
[0040] Step S202: compare the actual operation data with the standard operation data, and determine whether the actual operation data is abnormal data based on the comparison result.
[0041] Specifically, abnormal data refers to the operation data when the server is in an abnormal operation state. Standard operation data refers to the data range of the server's operation data when the server is in a normal operation state. For example, the standard CPU utilization rate is 20% to 70%, and the standard concurrent request number is 100 to 2000 QPS (Queries Per Second, query rate per second).
[0042] When comparing the actual operation data with the standard operation data, the actual operation data collected within the most recent specified time period may be compared with the standard operation data. For example, the actual CPU utilization collected within the most recent hour is compared with the standard CPU utilization. The process of comparison is essentially to determine whether the actual operation data is within the data interval of the standard operation data. If within the most recent specified time period, the actual operation data at one or more time points are not within the data interval of the standard operation data, then it is considered that the actual operation data within the most recent specified time period is not within the data interval of the standard operation data. On the contrary, if within the most recent specified time period, the actual operation data at all time points are within the data interval of the standard operation data, then it is considered that the actual operation data within the most recent specified time period is within the data interval of the standard operation data. For example, assuming that the standard CPU utilization is 20% to 70%, and the actual CPU utilizations collected at multiple time points within the most recent hour are {22%, 50%, 50%, 60%, 99%, 98%} respectively. Since 99% and 98% are not within the data range of the standard CPU utilization, it can be considered that the actual CPU utilization in the last hour is not within the data range of the standard CPU utilization. On the contrary, if the actual CPU utilization collected at multiple time points in the last hour are {22%, 50%, 50%, 60%, 55%, 57%}, since the actual operation data at all time points in the last hour are within the data range of the standard CPU utilization, it can be considered that the actual CPU utilization in the last hour is within the data range of the standard CPU utilization.
[0043] Based on the comparison results, if the actual operation data is within the data interval of the standard operation data, it can be determined that the actual operation data is non-abnormal data. For example, when the standard CPU utilization is 20% to 70%, the collected actual CPU utilization is {22%, 50%, 50%, 60%, 55%, 57%}, and the actual CPU utilization can be considered as non-abnormal data.
[0044] If the actual operation data is not within the data interval of the standard operation data, it can be further determined whether the actual operation data is greater than a first data threshold or less than a second data threshold, wherein the first data threshold is greater than the maximum value within the data interval of the standard operation data, and the second data threshold is less than the minimum value within the data interval of the standard operation data. Figure 3, which is a schematic diagram of the relationship between the data interval range of standard operation data, the first data threshold and the second data threshold provided in some embodiments of the present disclosure. When the actual operation data is not within the data interval range of the standard operation data, there are two possibilities: one possibility is that the server is in an abnormal operation state, and the other possibility is that the server is in a faulty operation state. Normally, when the server is in a faulty operation state, its actual operation data is far away from the data interval range of the standard operation data, and when the server is in an abnormal operation state, its actual operation data is close to the data interval range of the standard operation data. For example, assuming that the standard CPU utilization is 20% to 70%, then when the server is in a faulty operation state, its actual CPU utilization may be 99% or 0%, and when the server is in an abnormal operation state, its actual CPU utilization may be 15% or 80%.
[0045] Based on this, the first data threshold and the second data threshold can be set on both sides of the data interval range of the standard operation data. If the actual operation data is not within the data interval range of the standard operation data, and the actual operation data is greater than the first data threshold or less than the second data threshold, it means that the server has currently failed, and the actual operation data can be determined to be the operation data when the server is in a faulty operation state; if the actual operation data is not within the data interval range of the standard operation data, and the actual operation data is not greater than the first data threshold and not less than the second data threshold, it means that the server can still provide services normally but its operation state has become abnormal, and the actual operation data can be determined to be abnormal data. For example, assuming that the standard CPU utilization is 20% to 70%, the first data threshold is 10%, and the second data threshold is 95%. If the actual CPU utilization collected at each time point within the recently specified time period is {99%, 100%, 100%, 100%, 100%, 100%}, it can be considered that the server has a performance failure. On the contrary, if the actual CPU utilization collected at each time point within the recent specified period is {80%, 75%, 70%, 50%, 45%, 75%}, it can be considered that the server is in an abnormal operating state and the actual operating data collected is abnormal data.
[0046] By setting the first data threshold and the second data threshold, it is possible to troubleshoot existing server faults during the server fault prediction process, so that the existing server faults can be repaired in time, reducing the losses caused by the server faults.
[0047] It should be noted that the above is only an example of comparing actual operating data with standard operating data, and does not constitute a limitation of the present application. For example, in some other embodiments, step S202 may specifically include: determining whether the actual operating data is within the data interval of the standard operating data. If the actual operating data is within the data interval of the standard operating data, it can be determined that the actual operating data is non-abnormal data; if the actual operating data is not within the data interval of the standard operating data, it can be determined that the actual operating data is abnormal data. In these embodiments, the operating data that is not within the data interval of the standard operating data is uniformly defined as abnormal data. In this way, the difficulty of data comparison can be reduced.
[0048] After the data comparison in step S202, if all of the multiple actual operation data collected for the same fault are abnormal data, step S203 can be continued. For example, suppose that when predicting whether a server will have a performance failure in the future, the actual CPU utilization and the actual number of concurrent requests of the server are collected. If both the actual CPU utilization and the actual number of concurrent requests are abnormal data, step S203 can be executed.
[0049] Step S203: if the actual operation data is abnormal data, an abnormal operation index of the actual operation data is determined, where the abnormal operation index is used to characterize the cumulative deviation degree of the actual operation data relative to the standard operation data.
[0050] Specifically, for different types of actual operation data, the corresponding abnormal operation indicators can be determined respectively. For example, for the actual CPU utilization and the actual number of concurrent requests, the abnormal operation indicator M1 of the actual CPU utilization and the abnormal operation indicator M2 of the actual number of concurrent requests can be determined respectively.
[0051] In this embodiment, the abnormal operation index of the actual operation data can be determined according to expression (1):
[0052] M=∑△ 偏 ×t (1)
[0053] Among them, M represents the abnormal operation index, △ 偏 It indicates the offset of the actual running data relative to the standard running data; t indicates the duration of the actual running data offset from the standard running data.
[0054] For example, suppose the actual CPU utilization collected at various time points in the last hour is {80%, 75%, 70%, 50%, 45%, 30%}, which means that in the last hour, the actual CPU utilization in the first 10 minutes is 80%, the actual CPU utilization in the second 10 minutes is 75%, the actual CPU utilization in the third 10 minutes is 70%, and so on. Then, when the standard CPU utilization is 20% to 70%, it can be considered that in the first 10 minutes, the actual CPU utilization offset △ 偏 10%, and the offset of the actual CPU utilization in the second 10 minutes 偏 The abnormal operation index of the standard CPU utilization rate in the last hour is M1 =∑10*10%+10*5%.
[0055] Step S204: predicting the failure probability of the server based on the abnormal operation index, and generating a first fault detection result of the server according to the failure probability.
[0056] In this embodiment, the abnormal operation index can be input into the fault risk prediction model, and the fault risk prediction model outputs the failure probability of the server. Specifically, the fault risk prediction model can be a logistic regression model, which can be specifically shown as expression (2):
[0057]
[0058] Among them, y=1 indicates server failure, G(y=1|M1,M2,…,M n ) represents the predicted server failure probability (i.e., the probability that the server will fail in the future);
[0059] y=0 means the server is running normally, G(y=0|M1,M2,…,M n ) represents the predicted normal probability of the server (i.e., the probability that the server will not fail in the future);
[0060] M1,M2,…,M n Indicates abnormal operation indicators of various types of actual operation data determined in step S203, for example, M1 is an abnormal operation indicator of actual CPU utilization, and M2 is an abnormal operation indicator of actual concurrent number of requests;
[0061] α, β1, …, β nRepresents the model coefficient of the fault risk prediction model. Specifically, the model coefficient can be obtained based on the following method: obtain the first historical operation data of one or more servers when a fault occurs and the second historical operation data when no fault occurs, and fit the fault risk prediction model based on the first historical operation data and the second historical operation data to obtain the model coefficient. Specifically, the model coefficients α, β1, ..., β n , and then for α, β1, …, β n Perform significance tests to determine the model coefficients α, β1, …, β n Whether it has a significant impact on the fault risk prediction model. According to the results of the significance check, the fault risk prediction model can be optimized to improve the accuracy and reliability of fault prediction.
[0062] In this embodiment, after the failure probability of the server is predicted based on the failure risk prediction model shown in expression (2), if the failure probability is greater than a preset probability threshold (such as 0.8), a first fault detection result of the server is generated to prompt that the server will have an operation failure in the future. In this way, the maintenance personnel can take corresponding preventive measures in advance for possible operation failures of the server based on the first fault detection result, thereby improving the stability of the server.
[0063] In summary, in the technical solutions of some embodiments of the present disclosure, after comparing the actual operation data of the server with the standard operation data, it can be determined whether the actual operation data is abnormal data. In the case where the actual operation data is abnormal data, it can be determined that the server is in an abnormal operation state. In the case where the server is in an abnormal operation state, although the server has not yet failed, there is a risk of failure in the future. Therefore, the failure probability of the server can continue to be predicted based on the abnormal operation indicators of the actual operation data, and the first fault detection result of the server can be obtained, so that the maintenance personnel can take fault prevention measures in advance based on the first fault detection result, thereby achieving the purpose of improving the stability of the server.
[0064] In some embodiments, the actual operation data includes the actual operation temperature of the server, and the standard operation data includes the standard operation temperature. The step S204 of predicting the failure probability of the server based on the abnormal operation index includes:
[0065] If the actual operating temperature is determined to be abnormal data based on the comparison result between the actual operating temperature and the standard operating temperature, the failure probability of the server is predicted based on the abnormal operating index of the actual operating temperature.
[0066] Specifically, similar to the above expressions (1) and (2), the abnormal operation index of the actual operating temperature can be determined according to expression (3):
[0067] M T =∑△ T ×t T (3)
[0068] Among them, M T Indicates the abnormal operation index of the actual operating temperature, △ T Indicates the deviation of the actual operating temperature from the standard operating temperature; t T Indicates the time duration that the actual operating temperature deviates from the standard operating temperature.
[0069] By inputting the abnormal operation index of the actual operating temperature into the failure risk prediction model shown in expression (4), the failure probability of the server can be obtained:
[0070]
[0071] In some embodiments, the actual operation data includes the actual operation power of the server, and the standard operation data includes the standard operation power. The step S204 predicts the failure probability of the server based on the abnormal operation index, including:
[0072] If the actual operating power is determined to be abnormal data based on the comparison result between the actual operating power and the standard operating power, the failure probability of the server is predicted based on the abnormal operating index of the actual operating power.
[0073] Specifically, similar to the above expressions (1) and (2), the abnormal operation index of the actual operation power can be determined according to expression (5):
[0074] M P =∑△ P ×t P (5)
[0075] Among them, M P Indicates the abnormal operation index of the actual operating power, △ P Indicates the deviation of actual operating power relative to standard operating power; t P Indicates the duration that the actual operating power deviates from the standard operating power.
[0076] By inputting the abnormal operation index of the actual operating power into the failure risk prediction model shown in expression (6), the failure probability of the server can be obtained:
[0077]
[0078] In some embodiments, the actual operation data includes the actual operation temperature and the actual operation power of the server, and the standard operation data includes the standard operation power and the standard operation power. In step S204, predicting the failure probability of the server based on the abnormal operation index includes:
[0079] If the actual operating temperature is determined to be abnormal data based on the comparison result between the actual operating temperature and the standard operating temperature, and the actual operating power is determined to be abnormal data based on the comparison result between the actual operating power and the standard operating power, then the abnormal operation index M based on the actual operating temperature is T and abnormal operation index M of actual operating power P , predict the failure probability of the server.
[0080] Specifically, the abnormal operation index M of the actual operating temperature can be determined according to the above expressions (3) and (5), respectively. T and abnormal operation index M of actual operating power P .
[0081] By inputting the abnormal operating indicators of the actual operating temperature and the actual operating power into the failure risk prediction model shown in expression (7), the failure probability of the server can be obtained:
[0082]
[0083] Furthermore, in some embodiments, the actual operating temperature of the server is the average of the actual operating temperatures of the server components, and the actual operating power of the server is the average of the actual operating powers of the server components. The server components include but are not limited to a central processing unit (CPU), memory, hard disk, etc.
[0084] Predicting whether a server will fail in the future based on the actual operating temperature and actual operating power of the server can predict the overall failure of multiple components in the server and reduce the complexity of failure prediction. However, this method cannot predict the specific component that has failed. In view of this, in the method disclosed in the present invention, it is also possible to predict whether a single server component will fail in the future based on the actual operating temperature and actual operating power of the single server component.
[0085] Specifically, in some implementations, for any target component in the server components, the actual operating temperature of the target component may be compared with the standard operating temperature of the target component. If the actual operating temperature of the target component is determined to be abnormal data based on the comparison result, the failure probability of the target component is predicted based on the abnormal operating index of the actual operating temperature of the target component.
[0086] In some embodiments, for any target component in the server components, the actual operating power of the target component may be compared with the standard operating power of the target component. If the actual operating power of the target component is determined to be abnormal data according to the comparison result, the failure probability of the target component is predicted based on the abnormal operating index of the actual operating power of the target component.
[0087] In some embodiments, for any target component in the server components, the actual operating temperature of the target component may be compared with the standard operating temperature of the target component, and the actual operating power of the target component may be compared with the standard operating power of the target component. If, according to the comparison result, the actual operating temperature and the actual operating power of the target component are determined to be abnormal data, then based on the abnormal operating index of the actual operating temperature of the target component and the abnormal operating index of the actual operating power, the failure probability of the target component is predicted.
[0088] In this way, fault prediction can be performed for individual server components, thereby improving the accuracy of fault prediction.
[0089] In some embodiments, the standard operating data in the present application can be obtained based on the following method:
[0090] Acquire third historical operation data of one or more servers at multiple time points when the servers are in a normal operating state;
[0091] Determine the observed kurtosis value of each third historical operation data respectively;
[0092] Based on the kurtosis observation value and the predetermined kurtosis test critical value, non-outlier historical operation data are screened out from the third historical operation data;
[0093] Based on the non-outlier historical operating data, standard operating data is determined.
[0094] Specifically, the kurtosis observation value refers to the kurtosis value of the third historical operation data. When the kurtosis observation value is greater than 3, it means that the third historical operation data has fewer measurement points near the mean, the tail of the distribution is thinner, and a sharp peak is formed; when the kurtosis observation value is less than 3, it means that the third historical operation data has more measurement points near the mean, the tail of the distribution is thicker, and a flat peak is formed. Since the kurtosis value of the normal distribution is 3, the kurtosis observation value can be used to determine whether the data distribution of the third historical operation data is close to the normal distribution.
[0095] The kurtosis test critical value is the limit value used to determine whether the data distribution conforms to the normal distribution when performing the kurtosis test. If the kurtosis observed value is within the range of the kurtosis test critical value, the hypothesis that the data follows the normal distribution cannot be rejected; if the kurtosis observed value exceeds the range of the kurtosis test critical value, the normal distribution hypothesis is rejected, that is, the data is considered to have abnormal kurtosis.
[0096] Based on the above description, when determining the standard operation data, the third historical operation data can be sorted in ascending order. Based on the sorted third historical operation data, the detection level of the third historical operation data can be determined. The so-called detection level refers to the significance level designated for detecting outliers in the process of statistically verifying the third historical operation data. According to the detection level, the kurtosis test critical value can be obtained. Furthermore, based on expression (3), the kurtosis observation value of each third historical operation data can be calculated separately:
[0097]
[0098] Where bk(n) is the observed kurtosis value of the nth third historical running data after sorting in ascending order;
[0099] is the average of all third historical running data;
[0100] x i The other third historical operation data are arranged before the nth third historical operation data after being sorted in ascending order.
[0101] For any third historical operation data, if the kurtosis observation value of the third historical operation data is greater than the kurtosis test critical value, then the third historical operation data can be considered as outlier historical operation data; if the kurtosis observation value of the third historical operation data is not greater than the kurtosis test critical value, then the third historical operation data can be considered as non-outlier historical operation data.
[0102] In some embodiments, determining standard operating data based on non-outlier historical operating data includes:
[0103] Determine the mean and standard deviation of non-outlier historical run data;
[0104] The average value is increased by a preset number of standard deviations to obtain an upper limit value, and the average value is decreased by a preset number of standard deviations to obtain a lower limit value;
[0105] The data between the upper limit and the lower limit are used as standard operation data.
[0106] For example, suppose the average value of the non-outlier historical running data is The standard deviation is s ′ , then the standard operating data can be
[0107] In specific implementation, different types of third historical operation data can be analyzed to obtain corresponding standard operation data. For example, according to the above method, the historical memory utilization of the server can be analyzed to obtain the standard memory utilization, and the historical concurrent request number of the server can be analyzed to obtain the standard concurrent request number.
[0108] In the above embodiment, the third historical operation data is analyzed by a statistical method, and the obtained standard operation data is relatively accurate.
[0109] In some embodiments, when fault detection is performed on the server according to actual operation data of the server, the method of the present disclosure further includes:
[0110] Sending a communication detection message to the server at a first time point;
[0111] In response to receiving a feedback message from the server regarding the communication detection message at the second time point, determining a duration between the first time point and the second time point;
[0112] If the duration between the first time point and the second time point exceeds the duration threshold, a second fault detection result of the server is generated.
[0113] Specifically, Figure 1 As shown, the host computer can send communication detection messages to each server that needs to be predicted for faults, so as to determine whether a network communication fault occurs between the host computer and each server. For any server, if the duration between the first time point and the second time point corresponding to the server exceeds the duration threshold, it can be determined that a network communication fault occurs between the host computer and the server, and then a second fault detection result of the server can be generated. In this way, the maintenance personnel can promptly handle the network communication fault problem between the host computer and the server based on the second fault detection result. After the communication between the host computer and the server returns to normal, the host computer can collect the actual operation data of the server from the server. Based on the actual operation data, the operating status of the server can be monitored, and it is predicted whether the server will fail in the future.
[0114] Combined with reference Figure 4 , which is a module diagram of a server fault detection device provided in some embodiments of the present disclosure. Figure 4 In the embodiment, the server fault detection device comprises:
[0115] The data acquisition module 401 is used to acquire the actual operation data of the server;
[0116] The data comparison module 402 is used to compare the actual operation data with the standard operation data, and determine whether the actual operation data is abnormal data based on the comparison result;
[0117] The indicator determination module 403 is used to determine the abnormal operation indicator of the actual operation data if the actual operation data is abnormal data, and the abnormal operation indicator is used to represent the cumulative deviation degree of the actual operation data relative to the standard operation data;
[0118] The fault detection module 404 is used to predict the fault probability of the server based on the abnormal operation index, and generate a first fault detection result of the server according to the fault probability.
[0119] In some embodiments, the actual operation data includes the actual operation temperature of the server, and the standard operation data includes the standard operation temperature; the fault detection module 404 is specifically used to:
[0120] If the actual operating temperature is determined to be abnormal data based on the comparison result between the actual operating temperature and the standard operating temperature, the failure probability of the server is predicted based on the abnormal operating index of the actual operating temperature; and / or
[0121] The actual operation data includes the actual operation power of the server, and the standard operation data includes the standard operation power. The fault detection module 404 is specifically used for:
[0122] If the actual operating power is determined to be abnormal data based on the comparison result between the actual operating power and the standard operating power, the failure probability of the server is predicted based on the abnormal operating index of the actual operating power.
[0123] In some embodiments, the standard operation data represents the data interval range of the operation data of the server when the server is in a normal operation state; the data comparison module 402 is specifically used for:
[0124] Determine whether the actual operation data is within the data interval of the standard operation data;
[0125] If the actual operation data is within the data interval of the standard operation data, the actual operation data is determined to be non-abnormal data;
[0126] If the actual operation data is not within the data interval of the standard operation data, and the actual operation data is greater than the first data threshold or less than the second data threshold, then the actual operation data is determined to be operation data when the server is in a faulty operation state;
[0127] If the actual operation data is not within the data interval of the standard operation data, and the actual operation data is not greater than the first data threshold and not less than the second data threshold, then the actual operation data is determined to be abnormal data;
[0128] The first data threshold is greater than the maximum value in the data interval where the standard operating data is located, and the second data threshold is less than the minimum value in the data interval where the standard operating data is located.
[0129] In some embodiments, the fault detection module 404 is specifically used to:
[0130] The abnormal operation index is input into the fault risk prediction model, and the fault risk prediction model outputs the failure probability of the server, wherein the fault risk prediction model is a logistic regression model and includes a model coefficient, and the model coefficient is obtained based on the following method:
[0131] Acquire first historical operation data of one or more servers when a failure occurs and second historical operation data of one or more servers when no failure occurs;
[0132] A fault risk prediction model is fitted based on the first historical operation data and the second historical operation data to obtain model coefficients.
[0133] In some embodiments, the fault detection module 404 is specifically configured to obtain standard operation data based on the following method:
[0134] Acquire third historical operation data of one or more servers at multiple time points when the servers are in a normal operating state;
[0135] Determine the observed kurtosis value of each third historical operation data respectively;
[0136] Based on the kurtosis observation value and the predetermined kurtosis test critical value, non-outlier historical operation data are screened out from the third historical operation data;
[0137] Based on the non-outlier historical operating data, standard operating data is determined.
[0138] In some embodiments, the fault detection module 404 is specifically used to:
[0139] Determine the mean and standard deviation of non-outlier historical run data;
[0140] The average value is increased by a preset number of standard deviations to obtain an upper limit value, and the average value is decreased by a preset number of standard deviations to obtain a lower limit value;
[0141] The data between the upper limit and the lower limit are used as standard operation data.
[0142] In some embodiments, when performing fault detection on a server according to actual operation data of the server, the fault detection module 404 is further used to:
[0143] Sending a communication detection message to the server at a first time point;
[0144] In response to receiving a feedback message from the server regarding the communication detection message at the second time point, determining a duration between the first time point and the second time point;
[0145] If the duration between the first time point and the second time point exceeds the duration threshold, a second fault detection result of the server is generated.
[0146] The server fault detection device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0147] The present disclosure also provides an electronic device having the above Figure 4 The server fault detection device shown.
[0148] Combined with reference Figure 5 , is a schematic diagram of the structure of an electronic device provided by some embodiments of the present disclosure. Figure 5 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.
[0149] The processor 10 may be a first PCIe device, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0150] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.
[0151] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0152] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0153] The electronic device further comprises a communication interface 30 for the electronic device to communicate with other devices or a communication network.
[0154] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0155] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.
[0156] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A server failure detection method, characterized in that: The method comprises: Get the actual operation data of the server; Comparing the actual operation data with the standard operation data, and determining whether the actual operation data is abnormal data based on the comparison result; If the actual operation data is abnormal data, determining an abnormal operation index of the actual operation data, wherein the abnormal operation index is used to characterize a cumulative deviation degree of the actual operation data relative to the standard operation data; Based on the abnormal operation indicator, the failure probability of the server is predicted, and according to the failure probability, a first fault detection result of the server is generated.
2. The method according to claim 1, characterized in that The actual operation data includes the actual operation temperature of the server, and the standard operation data includes the standard operation temperature; The predicting the failure probability of the server based on the abnormal operation indicator includes: If the actual operating temperature is determined to be abnormal data based on the comparison result between the actual operating temperature and the standard operating temperature, then based on the abnormal operating index of the actual operating temperature, predicting the failure probability of the server; and / or The actual operation data includes the actual operation power of the server, and the standard operation data includes the standard operation power; The predicting the failure probability of the server based on the abnormal operation indicator includes: If the actual operating power is determined to be abnormal data based on the comparison result between the actual operating power and the standard operating power, the failure probability of the server is predicted based on the abnormal operation index of the actual operating power.
3. The method according to claim 1 or 2, characterized in that: The standard operation data represents the data interval range of the operation data of the server when the server is in a normal operation state; The comparing the actual operation data with the standard operation data, and judging whether the actual operation data is abnormal data based on the comparison result, includes: Determining whether the actual operation data is within the data interval of the standard operation data; If the actual operation data is within the data interval of the standard operation data, determining that the actual operation data is non-abnormal data; If the actual operation data is not within the data interval of the standard operation data, and the actual operation data is greater than a first data threshold or less than a second data threshold, determining that the actual operation data is operation data when the server is in a faulty operation state; If the actual operation data is not within the data interval of the standard operation data, and the actual operation data is not greater than the first data threshold and not less than the second data threshold, determining that the actual operation data is abnormal data; The first data threshold is greater than the maximum value within the data interval where the standard operating data is located, and the second data threshold is less than the minimum value within the data interval where the standard operating data is located.
4. The method according to claim 1, characterized in that: The predicting the failure probability of the server based on the abnormal operation indicator includes: The abnormal operation index is input into a fault risk prediction model, and the fault risk prediction model outputs the failure probability of the server, wherein the fault risk prediction model is a logistic regression model and includes model coefficients, and the model coefficients are obtained based on the following method: Acquire first historical operation data of one or more servers when a failure occurs and second historical operation data of one or more servers when no failure occurs; The fault risk prediction model is fitted based on the first historical operation data and the second historical operation data to obtain the model coefficients.
5. The method according to claim 1, characterized in that The standard operating data is obtained based on the following method: Acquire third historical operation data of one or more servers at multiple time points when the servers are in a normal operating state; respectively determining the kurtosis observation value of each of the third historical operation data; Based on the kurtosis observation value and a predetermined kurtosis test critical value, screening out non-outlier historical operation data from the third historical operation data; The standard operating data is determined based on the non-outlier historical operating data.
6. The method according to claim 5, characterized in that The determining the standard operating data based on the non-outlier historical operating data includes: Determining the mean and standard deviation of the non-outlier historical operating data; Increasing the average value by a preset number of the standard deviations to obtain an upper limit value, and decreasing the average value by a preset number of the standard deviations to obtain a lower limit value; The data between the upper limit value and the lower limit value is used as the standard operation data.
7. The method according to claim 1, characterized in that When performing fault detection on the server according to the actual operation data of the server, the method further includes: Sending a communication detection message to the server at a first time point; In response to receiving a feedback message from the server in response to the communication detection message at a second time point, determining a duration between the first time point and the second time point; If the duration between the first time point and the second time point exceeds a duration threshold, a second fault detection result of the server is generated.
8. A server fault detection device, characterized in that: The device comprises: A data acquisition module is used to obtain the actual operation data of the server; A data comparison module, used to compare the actual operation data with the standard operation data, and based on the comparison result, determine whether the actual operation data is abnormal data; an indicator determination module, configured to determine an abnormal operation indicator of the actual operation data if the actual operation data is abnormal data, wherein the abnormal operation indicator is used to characterize a cumulative deviation degree of the actual operation data from the standard operation data; The fault detection module is used to predict the fault probability of the server based on the abnormal operation indicator, and generate a first fault detection result of the server according to the fault probability.
9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the server fault detection method according to any one of claims 1 to 7 by executing the computer instructions.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the server fault detection method according to any one of claims 1 to 7.
Citation Information
Cited By
Beidou short message bypass monitoring method and device for field monitoring station
CN120415549A
A beidou short message bypass monitoring method and device for a field monitoring station
CN120415549B