Method and system for processing large-scale host faults of cloud platform

Abstract

The invention discloses a cloud platform large-scale host fault processing method and system, and the method comprises the steps: obtaining all historical host analysis original data, and carrying out the feature extraction of all historical host analysis original data; comprehensive total analysis is performed based on massive historical analysis data in a host and a final analysis result, and the final analysis is that analysis features appearing in all historical analysis feature data with host fault types serve as a plurality of analysis features related to the host fault types. Meanwhile, the related weights between the host fault types and the analysis features are calculated and sequentially serve as data bases for matching degree calculation during subsequent analysis, then effective reuse of mass analysis data is achieved, the accuracy during analysis means recommendation is improved, the accumulated requirement for analysis experience of workers can be effectively reduced, and the analysis efficiency is improved. Effective reuse of mass analysis data is realized, and it is ensured that an optimal maintenance means can be recommended during host fault analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of host failure analysis and processing, and in particular to a method and system for processing large-scale host failures on a cloud platform. Background Art

[0002] A cloud platform refers to a service platform that provides computing, networking, and storage based on hardware and software resources. It is generally deployed in a large-scale cluster environment. When a host node fails and can no longer provide services, the cloud host running on it can be evacuated to other physical hosts through the high availability function to ensure that the virtual machine can be used normally and user services are not affected. Currently, the cloud platform provides basic high availability capabilities in computing, networking, storage, and management. For large-scale cluster environments, computing high availability (High Available, HA) has gradually become a must-have function. When a host node fails and can no longer provide services, the multiple cloud hosts corresponding to the host node can be evacuated to other physical hosts through the high availability function to ensure that the cloud host can be used normally and user services are not affected.

[0003] However, in the prior art, when determining a host fault, the staff often needs to perform a large amount of maintenance analysis on the host without much maintenance experience, and only after obtaining sufficient analysis data can the host fault be accurately identified, which is difficult to meet the needs of the staff. Summary of the invention

[0004] In order to solve the above technical problems, a method and system for handling large-scale host failures on a cloud platform are provided. This technical solution solves the above-mentioned problem that when determining the host failure existing in the host, the staff often needs to perform a large amount of maintenance analysis on the host based on the fact that they do not have a lot of maintenance experience, and only after obtaining sufficient analysis data can the host failure existing in the host be accurately identified.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is: A method for handling large-scale host failures on a cloud platform, comprising: Obtain all historical host analysis raw data, perform feature extraction on all historical host analysis raw data, and obtain some historical host analysis feature data; Obtain the final analysis result of each host analysis raw data and record it as historical analysis result data; Based on a number of historical host analysis feature data, an analysis prediction model is established, wherein the input of the analysis prediction model is the analysis data, and the output of the analysis prediction model is the analysis prediction result; Used to input analysis data into the analysis and prediction model to obtain analysis and prediction results; Based on the analysis and prediction results, an analysis recommendation plan is generated through analysis recommendation logic.

[0006] This solution is based on statistical analysis of the host analysis data to obtain an analysis prediction model. When analyzing the host, based on the host's analysis data, the analysis prediction model is used to predict the types of host failures that may exist in the host, and based on the types of host failures that may exist in the host, a recommended repair solution is output. In this way, the accuracy of the maintenance means when analyzing the host can be effectively improved.

[0007] Preferably, the establishment of an analysis and prediction model based on a number of historical host analysis feature data specifically includes: Establish a one-to-one correspondence between each piece of historical analysis result data and the corresponding historical host analysis feature data; Based on the host failure type of the analysis result, the historical analysis result data is classified to obtain a number of historical analysis result data corresponding to each host failure type; Based on the one-to-one correspondence between the historical analysis result data and the corresponding historical host analysis feature data, a number of historical host analysis feature data corresponding to each host failure type are obtained; Performing statistical analysis on several historical host analysis feature data corresponding to each host failure type, respectively obtaining several host analysis features related to each host failure type and the correlation weight between each host failure type and the host analysis feature; Establishing a feature matching sub-model, wherein the feature matching sub-model is used to calculate the matching degree between the input analysis data and each host fault type; A screening sub-model is established, and the screening sub-model screens out one or more host fault types that best match the input analysis data based on the calculation results of the feature matching sub-model, and outputs them as analysis prediction results.

[0008] Based on a comprehensive and total analysis of the massive historical analysis data in the host and the final analysis results, the final analysis is to use the analysis features that appear in all historical analysis feature data of the host fault type as several analysis features related to the host fault type. At the same time, the relevant weights between the host fault type and the analysis features are calculated, which are used as the data basis for matching degree calculation in subsequent analysis, thereby realizing the effective reuse of massive analysis data and improving the accuracy of recommending analysis methods.

[0009] Preferably, the calculation formula for the correlation weight between the host fault type and the host analysis feature is:

[0010] In the formula, is the correlation weight between the host fault type and the i-th analysis feature, is the total number of historical analysis feature data corresponding to the host fault type, The total number of historical analysis feature data with the i-th analysis feature appearing in the historical analysis feature data corresponding to the host fault type.

[0011] Preferably, the specific calculation formula of the feature matching sub-model is:

[0012] in, is the matching degree between the input analysis data and the jth host failure type, is the total number of historical analysis feature data corresponding to the jth host failure type, is the total number of features of the input analysis data, is the total number of identical features between the feature data of the input analysis data and the historical analysis feature data corresponding to the j-th host failure type, is the correlation weight between the j-th host failure type and the feature data of the l-th input analysis data and the same feature in the historical analysis feature data corresponding to the j-th host failure type, To find the maximum value function.

[0013] In this solution, the matching degree is calculated based on two indicators: one is the overlap of feature types, and the other is the relevant weight of the overlapping features and the host fault types. The matching degree between the analysis data and the host fault types is calculated by comprehensively calculating the two indicators, and the host fault types that the analysis data may suffer from can be accurately identified.

[0014] Preferably, the step of inputting the analysis data into the analysis prediction model and obtaining the analysis prediction results specifically includes: Perform feature extraction on the input analysis data to obtain a number of analysis features corresponding to the analysis data; The feature matching sub-model is called to calculate the matching degree between the input analysis data and each host fault type based on several analysis features.

[0015] Preferably, the step of inputting the analysis data into the analysis prediction model and obtaining the analysis prediction results specifically includes: Calling the screening sub-model to screen out one or more host fault types whose matching degree with the input analysis data is greater than a preset value as the predicted host fault type; A one-to-one correspondence is established between the predicted host fault type and the matching degree corresponding to the predicted host fault type, and then the result of analysis and prediction is obtained.

[0016] Preferably, generating an analysis recommendation solution based on the analysis prediction result by analysis recommendation logic includes: Determine whether there is a high-risk host failure among the predicted host failure types. If not, output the recommended repair method according to the host failure type that matches the input analysis data the most. If yes, output the recommended method according to the high-risk host failure type.

[0017] Preferably, the outputting of recommended maintenance measures according to the high-risk host failure types specifically includes: Establish a high-risk main engine fault database, and based on historical maintenance experience, set preliminary maintenance measures and re-inspection maintenance measures for each high-risk main engine fault in the high-risk main engine fault database; Based on the screened high-risk host failures, it is determined whether the matching degree is greater than a first matching preset value.

[0018] Preferably, the determining whether the matching degree is greater than a first matching preset value specifically includes: If so, the preliminary maintenance measures corresponding to the high-risk host failure are output as recommended maintenance measures; If not, the initial maintenance means and re-inspection maintenance means corresponding to the high-risk host failure are output as recommended maintenance means; The preliminary detection means is a detection means with low detection cost and can preliminarily identify whether there is no high-risk host failure; The re-inspection and repair method has a high detection cost and can accurately identify whether a high-risk host failure occurs.

[0019] In this solution, by screening out several possible predicted host failure types in the analysis data, recommended maintenance measures are output. If all possible predicted host failure types do not have high risks, it means that the host failure existing in the host is not serious. Therefore, based on the most likely host failure type of the host, recommended repair measures are output to guide the host repair; If there is a high risk among all possible predicted host failure types, the high-risk host failures are checked. If the host's analysis data has a high degree of match with the high-risk host failures, the host needs to be accurately repaired and screened for the corresponding high-risk host failures. Therefore, it is necessary to output re-inspection and repair means as recommended repair means. If the host's analysis data has a low degree of match with the high-risk host failures, it is only necessary to exclude and screen the high-risk host failures of the host. Therefore, preliminary repair means are output, and the next step of repair recommendation is made based on the repair results of the preliminary repair means.

[0020] A system for processing large-scale host failures on a cloud platform, used to implement a method for processing large-scale host failures on a cloud platform as described above, comprising: A processor, the processor is used to obtain all historical host fault analysis raw data, and perform feature extraction on all historical host analysis raw data, obtain a number of historical host analysis feature data, and obtain the final analysis result of each host analysis raw data, which is recorded as historical analysis result data; A building module, wherein the building module is used to establish an analysis and prediction model based on a number of historical host analysis feature data; An analysis module, wherein the analysis module is used to input analysis data into an analysis prediction model to obtain analysis prediction results; The recommendation module generates an analysis recommendation scheme based on the analysis prediction results through analysis recommendation logic.

[0021] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a method and system for handling large-scale host failures on a cloud platform. The method is based on a comprehensive and aggregated analysis of massive historical analysis data in the host and the final analysis results. The final analysis uses the analysis features that appear in all historical analysis feature data containing the host failure types as several analysis features related to the host failure types. At the same time, the relevant weights between the host failure types and the analysis features are calculated, which are used as the data basis for matching degree calculation in subsequent analysis, thereby achieving effective reuse of massive analysis data and improving the accuracy of recommended analysis methods. This can effectively reduce the need for staff to accumulate analysis experience, achieve effective reuse of massive analysis data, and ensure that the best maintenance methods can be recommended during host analysis, thereby ensuring the repair level received by the host. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the method flow of the present invention; Figure 2 A schematic diagram of a method flow for establishing an analysis and prediction model based on a number of historical host analysis feature data in the present invention; Figure 3 A schematic diagram of the process flow of the method for obtaining analysis and prediction results in the present invention; Figure 4 The figure is a flow chart of a method for outputting recommended maintenance measures according to the type of high-risk host failure in the present invention. DETAILED DESCRIPTION

[0023] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0024] Reference Figure 1 As shown, a method for handling large-scale host failures on a cloud platform includes: Obtain all historical host analysis raw data, perform feature extraction on all historical host analysis raw data, and obtain some historical host analysis feature data; Obtain the final analysis result of each host analysis raw data and record it as historical analysis result data; Based on several historical host analysis feature data, an analysis prediction model is established, the input of the analysis prediction model is the analysis data, and the output of the analysis prediction model is the analysis prediction result; Used to input analysis data into the analysis and prediction model to obtain analysis and prediction results; Based on the analysis and prediction results, an analysis recommendation plan is generated through analysis recommendation logic.

[0025] This solution is based on statistical analysis of the host analysis data to obtain an analysis prediction model. When analyzing the host, based on the host's analysis data, the analysis prediction model is used to predict the types of host failures that may exist in the host, and based on the types of host failures that may exist in the host, a recommended repair solution is output. In this way, the accuracy of the maintenance means when analyzing the host can be effectively improved.

[0026] Based on several historical host analysis feature data, the analysis and prediction model is established, including: Establish a one-to-one correspondence between each piece of historical analysis result data and the corresponding historical host analysis feature data; Based on the host failure type of the analysis result, the historical analysis result data is classified to obtain a number of historical analysis result data corresponding to each host failure type; Based on the one-to-one correspondence between the historical analysis result data and the corresponding historical host analysis feature data, a number of historical host analysis feature data corresponding to each host failure type are obtained; Performing statistical analysis on several historical host analysis feature data corresponding to each host failure type, respectively obtaining several host analysis features related to each host failure type and the correlation weight between each host failure type and the host analysis feature; Establish a feature matching sub-model, which is used to calculate the matching degree between the input analysis data and each host fault type; A screening sub-model is established. The screening sub-model screens out one or more host fault types that best match the input analysis data based on the calculation results of the feature matching sub-model, and outputs them as analysis prediction results.

[0027] Based on a comprehensive and total analysis of the massive historical analysis data in the host and the final analysis results, the final analysis is to use the analysis features that appear in all historical analysis feature data of the host fault type as several analysis features related to the host fault type. At the same time, the relevant weights between the host fault type and the analysis features are calculated, which are used as the data basis for matching degree calculation in subsequent analysis, thereby realizing the effective reuse of massive analysis data and improving the accuracy of recommending analysis methods.

[0028] The calculation formula for the correlation weight between the host fault type and the analysis feature is:

[0029] In the formula, is the correlation weight between the host fault type and the i-th analysis feature, is the total number of historical analysis feature data corresponding to the host fault type, The total number of historical analysis feature data with the i-th analysis feature appearing in the historical analysis feature data corresponding to the host fault type.

[0030] The following is a calculation formula for the correlation weight between the above host fault types and analysis features in conjunction with a specific embodiment; In some embodiments, the total number of all historical analysis feature data corresponding to host fault type A is 5000, of which 3985 historical analysis feature data include analysis feature b, and the correlation weight between host fault A and analysis feature b is 3985 / 5000=0.797.

[0031] The specific calculation formula of the feature matching sub-model is:

[0032] in, is the matching degree between the input analysis data and the jth host failure type, is the total number of historical analysis feature data corresponding to the jth host failure type, is the total number of features of the input analysis data, is the total number of identical features between the feature data of the input analysis data and the historical analysis feature data corresponding to the j-th host failure type, is the correlation weight between the j-th host failure type and the feature data of the l-th input analysis data and the same feature in the historical analysis feature data corresponding to the j-th host failure type, To find the maximum value function.

[0033] In this solution, the matching degree is calculated based on two indicators: one is the overlap of feature types, and the other is the relevant weight of the overlapping features and the host fault types. The matching degree between the analysis data and the host fault types is calculated by comprehensively calculating the two indicators, and the host fault types that the analysis data may suffer from can be accurately identified.

[0034] Input analysis data into the analysis and prediction model to obtain analysis and prediction results, including: Perform feature extraction on the input analysis data to obtain a number of analysis features corresponding to the analysis data; Call the feature matching sub-model to calculate the matching degree between the input analysis data and each host fault type based on several analysis features; Calling the screening sub-model to screen out one or more host fault types whose matching degree with the input analysis data is greater than a preset value as the predicted host fault type; A one-to-one correspondence is established between the predicted host fault type and the matching degree corresponding to the predicted host fault type, and then the result of analysis and prediction is obtained.

[0035] Based on the analysis and prediction results, the analysis recommendation solutions generated by the analysis recommendation logic include: Determine whether there is a high-risk host failure among the predicted host failure types. If not, output the recommended repair method according to the host failure type that matches the input analysis data the most. If yes, output the recommended maintenance method according to the high-risk host failure type.

[0036] Recommended maintenance measures based on high-risk host failure types include: Establish a high-risk main engine fault database, and based on historical maintenance experience, set preliminary maintenance measures and re-inspection maintenance measures for each high-risk main engine fault in the high-risk main engine fault database; Based on the screened high-risk host failures, determining whether the matching degree thereof is greater than a first matching preset value; If so, the preliminary maintenance measures corresponding to the high-risk host failure are output as recommended maintenance measures; If not, the preliminary maintenance means and re-inspection maintenance means corresponding to the high-risk host failure are output at the same time as the recommended maintenance means.

[0037] The preliminary detection means is a detection means with low detection cost and can preliminarily identify whether there is no high-risk host failure; The re-inspection and repair method has a high detection cost and can accurately identify whether there is a high-risk host failure.

[0038] In this solution, by screening out several possible predicted host failure types in the analysis data, recommended maintenance measures are output. If all possible predicted host failure types do not have high risks, it means that the host failure existing in the host is not serious. Therefore, based on the most likely host failure type of the host, recommended repair measures are output to guide the host repair; If there is a high risk among all possible predicted host failure types, the high-risk host failures are checked. If the host's analysis data has a high degree of match with the high-risk host failures, the host needs to be accurately repaired and screened for the corresponding high-risk host failures. Therefore, it is necessary to output re-inspection and repair means as recommended repair means. If the host's analysis data has a low degree of match with the high-risk host failures, it is only necessary to exclude and screen the high-risk host failures of the host. Therefore, preliminary repair means are output, and the next step of repair recommendation is made based on the repair results of the preliminary repair means.

[0039] A system for processing large-scale host failures on a cloud platform, used to implement a method for processing large-scale host failures on a cloud platform as described above, comprising: The processor is used to obtain all historical host fault analysis raw data, perform feature extraction on all historical host analysis raw data, obtain several historical host analysis feature data, and obtain the final analysis result of each host analysis raw data, which is recorded as historical analysis result data; Building a module, the building module is used to establish an analysis and prediction model based on several historical host analysis feature data; The analysis module is used to input analysis data into the analysis prediction model to obtain analysis prediction results; Recommendation module,The recommendation module generates analysis recommendation solutions based on the analysis and prediction results through analysis and recommendation logic.

[0040] It can be understood by those skilled in the art that the processor uses TPU, which is specially designed for machine learning tasks and is particularly suitable for the training and reasoning of deep learning models. It can efficiently perform large-scale matrix operations and is the brain of the system. The processor automatically obtains all historical host fault analysis raw data, extracts features from all historical host analysis raw data, obtains several historical host analysis feature data, obtains the final analysis result of each host analysis raw data, and transmits the data to the building module through electrical signals. The building module establishes an analysis prediction model and a feature matching sub-model. The analysis module analyzes the data and transmits the data to the feature matching sub-model, and screens out one or more host fault types whose matching degree with the input analysis data is greater than a preset value as the predicted host fault type. A one-to-one correspondence is established between the predicted host fault type and the matching degree corresponding to the predicted host fault type, and then the result of the analysis prediction is obtained. The analysis module and the recommendation module are also connected by electrical signals, so the analysis module transmits the analysis prediction result to the recommendation module. The recommendation module determines whether there is a high-risk host fault in the predicted host fault type. If not, the recommended repair means are output according to the host fault type with the largest matching degree with the input analysis data. If so, the recommended means are output according to the high-risk host fault type.

[0041] In summary, the advantages of the present invention are: it can effectively improve the accuracy of the maintenance means when the host performs fault analysis, can effectively reduce the need for the accumulation of analysis experience of the staff, realize the effective reuse of massive analysis data, and ensure that the best maintenance means can be recommended during the host analysis, thereby ensuring the repair level received by the host.

[0042] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for handling large-scale host failures on a cloud platform, characterized in that: include: Obtain all historical host analysis raw data, perform feature extraction on all historical host analysis raw data, and obtain some historical host analysis feature data; Obtain the final analysis result of each host analysis raw data and record it as historical analysis result data; Based on a number of historical host analysis feature data, an analysis prediction model is established, wherein the input of the analysis prediction model is the analysis data, and the output of the analysis prediction model is the analysis prediction result; Used to input analysis data into the analysis and prediction model to obtain analysis and prediction results; Based on the analysis and prediction results, an analysis recommendation plan is generated through analysis recommendation logic.

2. A method for handling large-scale host failures on a cloud platform according to claim 1, characterized in that: The establishment of an analysis and prediction model based on a number of historical host analysis feature data specifically includes: Establish a one-to-one correspondence between each piece of historical analysis result data and the corresponding historical host analysis feature data; Based on the host failure type of the analysis result, the historical analysis result data is classified to obtain a number of historical analysis result data corresponding to each host failure type; Based on the one-to-one correspondence between the historical analysis result data and the corresponding historical host analysis feature data, a number of historical host analysis feature data corresponding to each host failure type are obtained; Performing statistical analysis on several historical host analysis feature data corresponding to each host failure type, respectively obtaining several host analysis features related to each host failure type and the correlation weight between each host failure type and the host analysis feature; Establishing a feature matching sub-model, wherein the feature matching sub-model is used to calculate the matching degree between the input analysis data and each host fault type; A screening sub-model is established, and the screening sub-model screens out one or more host fault types that best match the input analysis data based on the calculation results of the feature matching sub-model, and outputs them as analysis prediction results.

3. A method for handling large-scale host failures on a cloud platform according to claim 2, characterized in that: The calculation formula of the correlation weight between the host fault type and the host analysis feature is: In the formula, is the correlation weight between the host fault type and the i-th analysis feature, is the total number of historical analysis feature data corresponding to the host fault type, The total number of historical analysis feature data with the i-th analysis feature appearing in the historical analysis feature data corresponding to the host fault type.

4. A method for handling large-scale host failures on a cloud platform according to claim 3, characterized in that: The specific calculation formula of the feature matching sub-model is: in, is the matching degree between the input analysis data and the jth host failure type, is the total number of historical analysis feature data corresponding to the jth host failure type, is the total number of features of the input analysis data, is the total number of identical features between the feature data of the input analysis data and the historical analysis feature data corresponding to the j-th host failure type, is the correlation weight between the j-th host failure type and the feature data of the l-th input analysis data and the same feature in the historical analysis feature data corresponding to the j-th host failure type, To find the maximum value function.

5. A method for handling large-scale host failures on a cloud platform according to claim 4, characterized in that: The inputting of analysis data into the analysis prediction model to obtain the analysis prediction results specifically includes: Perform feature extraction on the input analysis data to obtain a number of analysis features corresponding to the analysis data; The feature matching sub-model is called to calculate the matching degree between the input analysis data and each host fault type based on several analysis features.

6. A method for handling large-scale host failures on a cloud platform according to claim 5, characterized in that: The inputting of analysis data into the analysis prediction model and obtaining the analysis prediction results specifically further comprises: Calling the screening sub-model to screen out one or more host fault types whose matching degree with the input analysis data is greater than a preset value as the predicted host fault type; A one-to-one correspondence is established between the predicted host fault type and the matching degree corresponding to the predicted host fault type, and then the result of analysis and prediction is obtained.

7. A method for handling large-scale host failures on a cloud platform according to claim 6, characterized in that: The generating of the analysis recommendation scheme based on the analysis prediction result by the analysis recommendation logic includes: Determine whether there is a high-risk host failure among the predicted host failure types. If not, output the recommended repair method according to the host failure type that matches the input analysis data the most. If yes, output the recommended method according to the high-risk host failure type.

8. A method for handling large-scale host failures on a cloud platform according to claim 7, characterized in that: The output of recommended maintenance measures according to the high-risk host failure types specifically includes: Establish a high-risk main engine fault database, and based on historical maintenance experience, set preliminary maintenance measures and re-inspection maintenance measures for each high-risk main engine fault in the high-risk main engine fault database; Based on the screened high-risk host failures, it is determined whether the matching degree is greater than a first matching preset value.

9. A method for handling large-scale host failures on a cloud platform according to claim 8, characterized in that: The step of determining whether the matching degree is greater than the first matching preset value specifically includes: If so, the preliminary maintenance measures corresponding to the high-risk host failure are output as recommended maintenance measures; If not, the initial maintenance means and re-inspection maintenance means corresponding to the high-risk host failure are output as recommended maintenance means; The preliminary detection means is a detection means with low detection cost and can preliminarily identify whether there is no high-risk host failure; The re-inspection and repair method has a high detection cost and can accurately identify whether a high-risk host failure occurs.

10. A system for processing large-scale host failures on a cloud platform, used to implement a method for processing large-scale host failures on a cloud platform as claimed in claims 1 to 9, characterized in that: include: A processor, the processor is used to obtain all historical host fault analysis raw data, and perform feature extraction on all historical host analysis raw data, obtain a number of historical host analysis feature data, and obtain the final analysis result of each host analysis raw data, which is recorded as historical analysis result data; A building module, wherein the building module is used to establish an analysis and prediction model based on a number of historical host analysis feature data; An analysis module, wherein the analysis module is used to input analysis data into an analysis prediction model to obtain analysis prediction results; The recommendation module generates an analysis recommendation scheme based on the analysis prediction results through analysis recommendation logic.

Images