Fault diagnosis processing method and device of server, electronic equipment and storage medium

Abstract

The invention discloses a server fault diagnosis processing method and device, electronic equipment and a storage medium. The method comprises the following steps: in response to a received state monitoring instruction, obtaining running state data of a server; using a pre-training fault diagnosis model to perform fault diagnosis on the server based on the operation state data to obtain a fault type of the server, the pre-training fault diagnosis model being used for representing a pre-trained neural network model for performing fault diagnosis on the server; fault processing information is determined based on the fault type so as to carry out fault processing on the server, and the fault processing information is used for representing fault processing strategy information for carrying out fault processing on the server. According to the method and the device, the technical problem of relatively low efficiency of fault diagnosis processing on the server in related technologies is solved.

Description

Technical Field

[0001] The present invention relates to the field of server fault diagnosis and processing, and in particular to a server fault diagnosis and processing method, device, electronic equipment and storage medium. Background Art

[0002] When a server fails or experiences an abnormality, the cause of the problem is analyzed, detected, and located, and then appropriate measures are taken to repair the problem and restore the normal operation of the server, which is also called server fault diagnosis.

[0003] At present, the relevant technologies usually monitor and analyze the server through remote monitoring tools, and detect whether there are any abnormal conditions. Fault diagnosis and processing of servers requires professional knowledge and technology. If the operator lacks relevant experience and skills, it may lead to diagnosis failure or delay in repair time. And because server failures may be caused by a variety of reasons, there may be missed detection or misjudgment during the diagnosis process, which affects the accurate grasp of the problem and the formulation of solutions, resulting in low efficiency of fault diagnosis and processing of servers in the relevant technologies.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method, device, electronic device and storage medium for diagnosing and processing a server fault, so as to at least solve the technical problem of low efficiency in diagnosing and processing a server fault in the related art.

[0006] According to one aspect of an embodiment of the present application, a method for fault diagnosis and processing of a server is provided, comprising: in response to receiving a status monitoring instruction, obtaining operating status data of the server; using a pre-trained fault diagnosis model to perform fault diagnosis on the server based on the operating status data to obtain a fault type of the server, wherein the pre-trained fault diagnosis model is used to represent a pre-trained neural network model for performing fault diagnosis on the server; determining fault handling information based on the fault type to perform fault handling on the server, wherein the fault handling information is used to represent fault handling strategy information for performing fault handling on the server.

[0007] Optionally, a pre-trained fault diagnosis model is used to perform fault diagnosis on the server based on the operating status data to obtain the fault type of the server, including: extracting features from the operating status data to obtain target meta-features; using the pre-trained fault diagnosis model, comparing the target meta-features with a preset fault type knowledge base to obtain the fault type, wherein the preset fault type knowledge base is used to represent a knowledge base constructed based on the correspondence between sample operating status data of the server and sample fault types.

[0008] Optionally, using a pre-trained fault diagnosis model, the target meta-feature is compared with a preset fault type knowledge base to obtain the fault type, including: using a pre-trained fault diagnosis model, the target meta-feature is compared with a preset fault type knowledge base to obtain the confidence between the target meta-feature and the sample fault type; determining the fault type based on the confidence.

[0009] Optionally, the method further includes: acquiring sample operating status data of the server; labeling the fault type of the sample operating status data to obtain a sample fault type; and training a pre-trained fault diagnosis model based on the sample operating status data and the sample fault type.

[0010] Optionally, obtaining sample operation status data of the server includes: obtaining historical operation status data of the server, and performing feature extraction on the historical operation status data to obtain historical meta-features; and concatenating the historical operation status data and the historical meta-features to obtain sample operation status data.

[0011] Optionally, based on the sample operating status data and the sample fault type, a pre-trained fault diagnosis model is trained, including: based on the sample operating status data and the sample fault type, a base learner in a preset fault diagnosis model is trained to obtain a pre-trained fault diagnosis model, wherein the sample operating status data serves as the input of the base learner, and the sample fault type serves as the output of the base learner.

[0012] According to another aspect of an embodiment of the present invention, a server fault diagnosis and processing device is also provided, including: an acquisition module, used to obtain the operating status data of the server in response to receiving a status monitoring instruction; a diagnosis module, used to use a pre-trained fault diagnosis model to perform fault diagnosis on the server based on the operating status data to obtain the fault type of the server, wherein the pre-trained fault diagnosis model is used to represent a pre-trained neural network model for performing fault diagnosis on the server; a processing module, used to determine fault handling information based on the fault type to perform fault handling on the server, wherein the fault handling information is used to represent fault handling strategy information for performing fault handling on the server.

[0013] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present invention is executed when the program is running.

[0014] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0015] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0016] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0017] According to another aspect of the embodiments of the present invention, a computer program is further provided. When the computer program is executed by a processor, the methods in the embodiments of the present invention are implemented.

[0018] In an embodiment of the present invention, a method for fault diagnosis and processing of a server is provided, comprising: in response to receiving a status monitoring instruction, obtaining the operating status data of the server; using a pre-trained fault diagnosis model, diagnosing the server based on the operating status data to obtain the fault type of the server, wherein the pre-trained fault diagnosis model is used to represent a pre-trained neural network model for performing fault diagnosis on the server; determining fault handling information based on the fault type to perform fault handling on the server, wherein the fault handling information is used to represent fault handling strategy information for performing fault handling on the server. It is easy to notice that the present application uses a pre-trained fault diagnosis model to analyze the operating status data of the server, which can more accurately diagnose the fault type of the server, avoid human misjudgment or missed diagnosis, and improve the accuracy of fault diagnosis. The pre-trained fault diagnosis model has been trained with a large amount of data, and can quickly diagnose the fault of the server, shorten the time of fault diagnosis, and improve the efficiency of fault diagnosis. By determining fault handling information based on the fault type, corresponding fault handling strategy information can be provided according to the specific fault type, helping operation and maintenance personnel to handle server faults more quickly and accurately, improving the efficiency of fault handling, and providing personalized fault handling information for the server, helping operation and maintenance personnel to better handle server faults and improve the stability and reliability of the system. This solves the technical problem of low efficiency in server fault diagnosis and processing in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 is a flow chart of a method for diagnosing and processing a server fault according to an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of an optional server fault diagnosis processing process according to an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of an optional cluster server fault diagnosis processing process according to an embodiment of the present invention;

[0023] Figure 4 It is a schematic diagram of a server fault diagnosis and processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] According to an embodiment of the present invention, an embodiment of a server fault diagnosis and processing method is provided. It can be explained that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] Figure 1 FIG. 1 is a flow chart of a method for diagnosing and processing a server fault according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0028] Step S102: in response to receiving the status monitoring instruction, obtaining the operation status data of the server.

[0029] In an optional embodiment, in response to receiving a status monitoring instruction, the running status data of the server can be obtained. Specifically, a monitoring agent can be deployed on the server side first, which is responsible for regularly collecting the running status data of the server and storing it in a local database. The monitoring agent can obtain key indicators such as the processor utilization, memory usage, network traffic, etc. of the server through the application program interface provided by the operating system or a third-party monitoring tool. At the same time, the monitoring agent can also monitor the status of various services and processes running on the server, such as database services, website servers, etc. When receiving a status monitoring instruction, the monitoring agent can obtain the corresponding running status data from the database according to the instruction content, and package it into a response message and return it to the requester. In this process, a reasonable data structure and communication protocol can be designed to ensure the accuracy and reliability of the data. The state data can also be obtained and responded to in an asynchronous processing manner to improve the performance and response speed of the system. By using multi-threading or event-driven technologies, the concurrent processing capability of the monitoring agent can be realized, so as to respond to the state monitoring instruction faster and obtain the running status data of the server. The present application can also add corresponding access control and encryption mechanisms to the monitoring agent to ensure that only authorized users can obtain the server's operating status data, and the data cannot be stolen or tampered with during transmission. In the above process, by regularly collecting and monitoring the server's operating status data, abnormal situations such as server performance problems and service failures can be discovered in a timely manner, which helps to prevent and solve failures in advance. By monitoring the server's operating status data, key indicators such as the server's load and resource utilization can be understood, which helps to optimize the system's configuration and performance tuning, and improve the system's stability and reliability.

[0030] Step S104, using the pre-trained fault diagnosis model, diagnose the server fault based on the operating status data to obtain the fault type of the server.

[0031] The pre-trained fault diagnosis model is used to represent a pre-trained neural network model for performing fault diagnosis on a server.

[0032] In an optional embodiment, the use of a pre-trained fault diagnosis model for fault diagnosis can improve the accuracy and efficiency of fault diagnosis. The pre-trained fault diagnosis model can be a neural network model obtained by training a large amount of operating status data, which can learn the characteristic representation of different fault types and accurately identify the fault type of the server in the new operating status data. Specifically, the operating status data of the server can be collected first, including indicators such as processor utilization, memory usage, network traffic, disk read and write speed, etc. These data can be collected in real time through monitoring tools, or can be obtained through log files and other methods. The collected data can be pre-processed, including data cleaning, missing value processing, data normalization and other operations to ensure the quality and consistency of the data. Then, the neural network model can be trained for fault diagnosis using a large amount of pre-collected operating status data. The training process includes steps such as feature extraction of input data, initialization of model parameters, definition of loss function, and back propagation. Then, the real-time collected operating status data can be input into the trained model, and the model can output the fault type of the server. According to the output results, the server can be troubleshooted and repaired in a timely manner. In the above process, the pre-trained fault diagnosis model can learn the characteristic representation of different fault types through learning and training of a large amount of data, thereby improving the accuracy of fault diagnosis and being able to locate and identify the fault type of the server more quickly. The pre-trained fault diagnosis model can realize real-time diagnosis in actual applications, without the need for model training, and can quickly diagnose server faults, saving diagnosis time and improving the efficiency of troubleshooting. Using the pre-trained fault diagnosis model for fault diagnosis can reduce the need for manual intervention, reduce the cost and time of manual diagnosis, and improve the efficiency of fault handling.

[0033] Step S106: determine fault handling information based on the fault type to perform fault handling on the server.

[0034] The fault handling information is used to indicate fault handling strategy information for handling faults on the server.

[0035] In an optional embodiment, a corresponding fault handling strategy can be formulated according to different types of faults to ensure that the server can resume normal operation as soon as possible, which can effectively improve the efficiency and accuracy of fault handling, and can also reduce maintenance costs and reduce system downtime. Specifically, the faults occurring in the server can be diagnosed and classified first to determine the specific type of fault. This can be done by monitoring system logs, performance indicators, and sensor data. According to different fault types, corresponding handling strategies can be formulated. Fault handling information related to a specific fault type can be collected, including processing steps, operating guidelines, troubleshooting methods, fault recovery strategies, etc., which can be obtained through professional knowledge bases, documentation, experience summaries, etc. The collected fault handling information can be matched with the identified fault type to determine the processing information applicable to the current fault, which can be done by establishing a fault handling information library, through data matching and rule matching, etc. Finally, according to the matched fault handling information, corresponding fault handling operations can be performed, which can include restarting the server, replacing hardware equipment, repairing software programs, adjusting configuration parameters, etc. During the execution process, operations can be performed according to the steps and instructions in the processing information to ensure the effectiveness and correctness of the fault handling. In the above process, according to different fault types, corresponding processing strategies are formulated, which can find the cause of the fault more quickly and take effective processing measures to shorten the fault repair time. By preparing processing information for different fault types in advance, the time and manpower cost of troubleshooting can be reduced, and system risks caused by incorrect operations can be avoided.

[0036] In an embodiment of the present invention, a method for fault diagnosis and processing of a server is provided, comprising: in response to receiving a status monitoring instruction, obtaining the operating status data of the server; using a pre-trained fault diagnosis model, diagnosing the server based on the operating status data to obtain the fault type of the server, wherein the pre-trained fault diagnosis model is used to represent a pre-trained neural network model for performing fault diagnosis on the server; determining fault handling information based on the fault type to perform fault handling on the server, wherein the fault handling information is used to represent fault handling strategy information for performing fault handling on the server. It is easy to notice that the present application uses a pre-trained fault diagnosis model to analyze the operating status data of the server, which can more accurately diagnose the fault type of the server, avoid human misjudgment or missed diagnosis, and improve the accuracy of fault diagnosis. The pre-trained fault diagnosis model has been trained with a large amount of data, and can quickly diagnose the fault of the server, shorten the time of fault diagnosis, and improve the efficiency of fault diagnosis. By determining fault handling information based on the fault type, corresponding fault handling strategy information can be provided according to the specific fault type, helping operation and maintenance personnel to handle server faults more quickly and accurately, improving the efficiency of fault handling, and providing personalized fault handling information for the server, helping operation and maintenance personnel to better handle server faults and improve the stability and reliability of the system. This solves the technical problem of low efficiency in server fault diagnosis and processing in related technologies.

[0037] Optionally, a pre-trained fault diagnosis model is used to perform fault diagnosis on the server based on the operating status data to obtain the fault type of the server, including: extracting features from the operating status data to obtain target meta-features; using the pre-trained fault diagnosis model, comparing the target meta-features with a preset fault type knowledge base to obtain the fault type, wherein the preset fault type knowledge base is used to represent a knowledge base constructed based on the correspondence between sample operating status data of the server and sample fault types.

[0038] In an optional embodiment, feature extraction can be performed on the operating status data. Feature extraction can be a process of converting raw data into a numerical value or vector that can describe the data characteristics. In the fault diagnosis of the server, features related to the fault, such as outliers, volatility, etc., can be extracted. Then, the pre-trained fault diagnosis model can be used for fault diagnosis, and the target meta-features can be compared with the preset fault type knowledge base to obtain the fault type of the server. The preset fault type knowledge base includes a knowledge base constructed based on the correspondence between the sample operating status data of the server and the sample fault type. By comparing the target meta-features with the data in the knowledge base, the fault type of the server can be accurately diagnosed. In the above process, the pre-trained fault diagnosis model and the knowledge base can be used to accurately diagnose the fault type of the server, avoiding the subjectivity and misjudgment of manual judgment. The automated fault diagnosis process can save manpower and time costs, quickly and accurately diagnose the server fault, and improve the efficiency of fault handling.

[0039] Optionally, using a pre-trained fault diagnosis model, the target meta-feature is compared with a preset fault type knowledge base to obtain the fault type, including: using a pre-trained fault diagnosis model, the target meta-feature is compared with a preset fault type knowledge base to obtain the confidence between the target meta-feature and the sample fault type; determining the fault type based on the confidence.

[0040] In an optional embodiment, the target meta-feature can be input into a pre-trained fault diagnosis model, and the confidence between the target meta-feature and the sample fault type can be obtained through the forward propagation process of the model, and then the final fault type can be determined according to the size of the confidence. A threshold can be set, and when the confidence exceeds the threshold, the fault type can be determined. In the above process, the fault can be quickly judged through the pre-trained fault diagnosis model, which improves the efficiency of fault diagnosis. The pre-trained fault diagnosis model can perform adaptive learning according to different data inputs and is suitable for different server environments and fault conditions. The fault diagnosis method based on the pre-trained model can be flexibly expanded and updated, and the model can be retrained and optimized with new data and fault types.

[0041] Optionally, the method further includes: acquiring sample operating status data of the server; labeling the fault type of the sample operating status data to obtain a sample fault type; and training a pre-trained fault diagnosis model based on the sample operating status data and the sample fault type.

[0042] In an optional embodiment, the acquired sample operation status data can be labeled with fault types, and the data can be analyzed by professionals and labeled according to specific fault types. Labeling can facilitate the training of machine learning models to identify different fault types. Based on the sample operation status data and the sample fault type, a pre-trained fault diagnosis model can be trained using a machine learning method. This model can predict the possible fault types in unknown data by learning the patterns and rules in the sample data. In the process of training the model, a variety of machine learning algorithms, such as neural networks, support vector machines, etc., can be used to improve the accuracy and generalization ability of the model. In the above process, the accuracy of fault diagnosis can be improved by training the pre-trained fault diagnosis model. The model can predict the possible fault types in unknown data by learning the patterns and rules in the sample data, thereby helping administrators to locate and solve faults more quickly and accurately. When the model detects the corresponding fault type, it can issue an alarm in time to help administrators deal with the fault in time and avoid serious consequences.

[0043] Optionally, obtaining sample operation status data of the server includes: obtaining historical operation status data of the server, and performing feature extraction on the historical operation status data to obtain historical meta-features; and concatenating the historical operation status data and the historical meta-features to obtain sample operation status data.

[0044] In an optional embodiment, by acquiring the historical running status data of the server and performing feature extraction on it, historical meta-features can be obtained. By analyzing these features, the regularities and abnormalities of the server during operation can be found, providing data basis for subsequent fault diagnosis. Specifically, the running status data of the server can be collected in real time through monitoring tools or software, including information such as processor utilization, memory usage, network traffic, etc., and these data can be recorded in the form of time series for subsequent analysis. Then, feature extraction can be performed on the collected data, and meaningful features can be extracted through statistical methods or machine learning algorithms, such as the mean and variance of processor utilization, the degree of fluctuation of memory usage, etc. can be calculated. The extracted features can be combined together to obtain historical meta-features, which can reflect the running status of the server in the past period of time and can help understand the normal running rules and abnormalities of the server. Finally, the historical running status data and the historical meta-features can be spliced ​​to obtain sample running status data, so as to combine the running status data and feature information of the server together to form a complete sample data set. In the above process, by combining the historical running status data and feature information together, more comprehensive data information can be provided, which helps to more accurately analyze the running status of the server. By extracting and combining historical meta-features, better feature expressions can be obtained, which helps to discover potential regularities and anomalies of the server. By analyzing sample operation status data, it is possible to more accurately determine whether the server is faulty, and to predict potential faults in advance, which helps to take timely measures to repair them.

[0045] Optionally, based on the sample operating status data and the sample fault type, a pre-trained fault diagnosis model is trained, including: based on the sample operating status data and the sample fault type, a base learner in a preset fault diagnosis model is trained to obtain a pre-trained fault diagnosis model, wherein the sample operating status data serves as the input of the base learner, and the sample fault type serves as the output of the base learner.

[0046] In an optional embodiment, a pre-trained fault diagnosis model can be trained based on sample operation status data and sample fault types. Specifically, sample data can be collected, including the operation status data of the server and the corresponding fault type, which can be obtained through a monitoring system, log files, etc. The collected data can be pre-processed, including data cleaning, removal of outliers, data normalization, etc., to ensure the accuracy and consistency of the data. And a suitable machine learning algorithm, such as a support vector machine, a decision tree, a neural network, etc., can be selected as a base learner. The sample operation status data is used as input and the sample fault type is used as output to train the base learner to obtain a pre-trained fault diagnosis model. Then, the model can be evaluated by methods such as cross-validation, and the model parameters can be adjusted to improve the accuracy and generalization ability of the model. When a server fails, the current operation status data of the server can be input into the pre-trained fault diagnosis model, and the pre-trained fault diagnosis model can predict the possible fault type based on the input data, helping engineers to quickly locate and solve the problem. In the above process, the pre-trained fault diagnosis model is trained based on a large amount of sample data, has a strong generalization ability, and can adapt to different server environments and fault types. At the same time, the pre-trained fault diagnosis model is automatically learned through machine learning algorithms, which can effectively reduce manual intervention, improve the degree of automation of fault diagnosis, and reduce maintenance costs and risks.

[0047] The technical solution proposed in this application is described below in conjunction with an optional embodiment. This application proposes a cluster server fault diagnosis and processing method, which relates to the field of computer room cluster server operation and maintenance. In the field of civil aviation, due to the large number of system services, the associated background systems are also diverse, and each system has a large number of background interfaces. In order to meet the higher system access volume, horizontal expansion can be achieved by building a cluster. Through horizontal expansion, the system load can be balanced to multiple servers, thereby avoiding single point failure. This is a system composed of multiple independent computer nodes, which are interconnected through a network to complete a task together, and the cluster service has a high degree of scalability, reliability and fault tolerance. However, cluster services are also faced with many challenges, such as data consistency, service performance degradation, some interfaces timeout or stuck, etc. Therefore, the fault diagnosis and processing of cluster servers are more important.

[0048] The present application provides a distributed server intelligent monitoring, analysis and fault handling system and method, the system includes: a server cluster information management module, which is used to maintain the information of each server, generate server sessions, and store them in a session resource pool; a server operation monitoring module, which is used to monitor the operation data of the corresponding server through sessions, and store them in an information resource pool and a database; a server operation status analysis module, which is used to extract operation data, analyze and compare it with preset thresholds, and send fault information and sessions to a fault handling resource pool when a fault occurs; a server operation fault handling module, which is used to extract fault information, find corresponding processing strategies, and handle and repair faults through corresponding sessions. Through distributed management, the present application modularizes operations, realizes large-scale server cluster management, monitors server operation status in real time, repairs server faults in a timely manner, ensures normal operation of servers, and reduces human monitoring costs.

[0049] The fault diagnosis of cluster servers in this application belongs to the fault diagnosis of distributed systems based on machine learning. The method includes: analyzing and sampling system sensitive logs, preprocessing historical system fault data; constructing a filling model for inferring missing values ​​of historical system fault data; using a pre-processing model based on a shallow neural network to fit the historical system fault data processed by the filling model, and obtaining a deeper feature embedding between features; combining feature embedding and original features to obtain an expanded data set; training the base learner of the fault diagnosis model, taking the probability representation of the base learner output as a meta-feature, used as the input of the meta-learner logistic regression, training the meta-learner to obtain a diagnosis model; using the same method for new data samples to obtain the final meta-feature, and inputting the diagnosis model for prediction. Fault diagnosis based on neural networks does not require the establishment of an accurate mathematical model, can handle nonlinear problems, has parallel computing capabilities, does not require diagnosis and reasoning rules, and can self-learn according to set criteria through the mapping relationship between the input and output of a set of samples.

[0050] When a fault occurs, the system will generate sensitive logs. The logs generated by the system are captured and collected to analyze and determine the cause of the fault. Corresponding solutions are taken to form decision tasks and added to the queue for execution. According to the execution feedback results, the next task is decided. If the decision is successfully executed, the fault is reviewed to see if it has been completely eliminated. If it has been eliminated, the whole process ends. If the fault still exists, the fault judgment and location are optimized, a new decision task is generated and added to the queue, and the process is re-executed. If the decision fails to execute, the decision task is optimized. After analyzing the cause of the failure, the optimized decision task is added to the queue for execution.

[0051] This application applies artificial neural networks to fault diagnosis, and the specific implementation steps are: collect system status sample data within the historical time interval and perform preprocessing. Classify the fault types of the processed sample data set, count the fault types in the sample data set, and build a system fault type knowledge base. Use a multi-layer perceptron pre-processing model to fit the data set, extract the deep feature embedding between the fault features, and splice the deep feature embedding and the preprocessed sample data set together to obtain an extended data set. The probability of occurrence of various types of faults is used as the meta-feature input of the meta-learner, and the label in the extended data set is used as the output of the meta-learner. The meta-learner is trained and its parameters are retained to obtain a diagnostic model with input as feature representation and output as the final fault type probability. The features in the extended data set are used as the input of the base learner, and the labels in the extended data set are used as the target of the base learner. The base learner is trained and its parameters are retained. The base learner is used to predict the features in the extended data set to obtain the probability of occurrence of various types of faults. Obtain sample data of the distributed system status information to be diagnosed, input the sample data information of each node in the data set into the pre-processing model based on the multi-layer perceptron to generate deep feature embedding, merge with the original features to obtain an expanded data set, and input it into the base learner to obtain the meta-feature input of the meta-learner. Input the meta-feature into the diagnosis model, compare it with the fault type in the system fault type knowledge base, obtain the probability set of which fault type each sample ultimately belongs to, and determine the fault type of the faulty node according to the confidence given by the diagnosis model.

[0052] In the process of fault judgment, the application can use the basic performance and status of the server as the main indicators, mainly including: processor (CPU) utilization, CPU load, memory utilization, disk space utilization, disk I / O, network traffic and system-specific processes, etc. At the same time, it also supports user-level customization of specific content, such as the setting of specific operation information parameters of the server, such as the monitoring of network traffic needs to obtain the transmission data packets within the specified time and store them, the detection of system-specific processes needs to obtain process status information regularly and be able to kill the process, and there is a specific threshold for detection of other aspects, and an alarm is issued when the threshold is exceeded. In the server status acquisition module, a Shell script is used to obtain the operating status information of the server. The Shell script calls a C language program with execution parameters, and provides monitoring and analysis cycle data through parameters. The C language program executes SQL statements, writes the monitoring results into the database, and then the customization of the monitoring cycle can be achieved by modifying the configuration file called by the Shell script.

[0053] The indicator information of different servers is as follows: CPU usage: call the Linux top command to obtain the CPU usage information. The obtained CPU usage information example is "Cpu(s): 0.3%us, 1.0%sy, 0.0%ni, 98.7%id, 0.0%wa, 0.0%hi, 0.0%si", where the meanings of different fields are: "0.3%us" means that the user control occupies 0.3% of the CPU; "0.1%sy" means that the kernel space occupies 0.1% of the CPU; "0.0%ni" means that the process with changed priority in the user process space occupies 0.0% of the CPU; "98.7id" means that the idle CPU percentage is 98.7%; "0.0%wa" means that the CPU percentage waiting for input and output is 0.0%; "hi" and "si" are the proportions of hard interrupts and soft interrupts respectively.

[0054] CPU load: CPU load information can be obtained by calling the linux uptime command. An example of CPU load information obtained is "00:06:41up 1:53,2users,load average:0.01,0.02,0.04," where the different fields mean: "00:06:41" means the current time, "up 1:53" means the system running time is 1:53, in the format of hours:minutes; "2users" means the number of currently logged-in users is 2; "load average:0.01,0.02,0.04" means the system load, that is, the average length of the task queue. The three values ​​are the average values ​​from 1 minute, 5 minutes, and 15 minutes ago to now.

[0055] Memory usage: Memory usage information can be obtained by calling the linux top command. An example of memory usage is "Mem:1024864k total,618964k used,405900k free,46716k buffers", where "Mem:1024864k total" means the total physical memory is 1024864k; "618964k used" means the total amount of physical memory used is 618964k; "405900k free" means the total amount of free memory is 405900k, and "46716k buffers" means the amount of memory used as kernel cache is 46716k.

[0056] Disk space usage: The df command is used by Linux to view the disk space system. The file system is viewed in units of disk partitions. You can add parameters to view the remaining disk space information. The command format is: df -hl.

[0057] Disk I / O: You can get disk I / O information by calling the linux iostat-dk command. TPS indicates the number of transmissions output to the physical disk per second, kb_read / s indicates the amount of data read from the disk per second, kb_wrtn / s indicates the amount of data written to the disk per second, kb_read indicates the total number of KB read, and kb_wrtn indicates the total number of KB written.

[0058] Network traffic: This part needs to capture all qualified data packets within a period of time according to the configuration. For example, capture all IP packets, TCP packets and UDP packets from 9 to 10 o'clock. This part is implemented using instructions (libpcap), which can capture packets for specific port segments and IP segments.

[0059] Specific system processes: This part detects specific system processes by executing Linux commands under super user (root) privileges. It can also remotely kill or restart processes.

[0060] Considering that the support vector machine (SVM) algorithm is mainly designed for binary classification problems, when dealing with the multi-classification problem of this fault data set, an indirect construction method can be used, that is, by combining multiple SVM binary classifiers to solve the multi-classification problem, the multi-classification problem can be decomposed into multiple binary classification problems to solve. In the way that SVM handles the multi-classification problem of multiple fault data, this application adopts the SVM one-versus-rest (OVR) method based on the linear kernel function. Due to the characteristics of the data set of the monitoring system, the normal data is far more than the abnormal data, so there is still a problem: how to quickly distinguish normal and abnormal data? For each new model of hardware, waiting until a certain amount of fault data appears and then training a classification model can solve the problem of fault auxiliary diagnosis. However, as an automated fault detection system, it is lagging behind. Referring to the good effect of the regression algorithm based on SVM in network abnormal data detection, in order to solve this problem and achieve more real-time fault prediction, this application designs a real-time collection method of time series data, and proposes to use SVR to regress the time series data, filter out abnormal values ​​beyond the offset range, and then use SVC to classify the abnormal data. In the multi-classification calculation method of SVM, K types of data are multi-classified. The one-to-many classifier (One VS RestClassifier) ​​algorithm needs to construct K classifiers, while the one-to-one classifier (One Vs OneClassifier) ​​requires K(K-1) / 2 classifiers. Using the SVM regression algorithm to determine whether the data is abnormal is essentially a binary problem. Therefore, it is reasonable to use the SVM regression algorithm instead of the SVM classification algorithm in this article to apply to normal data in terms of computational complexity.

[0061] After the system captures sensitive logs, it executes various commands through Shell scripts to obtain the basic performance of the server, and stores sensitive logs and the performance status of the server when abnormal logs occur in the database for later query of historical data at any time and for troubleshooting, diagnosis, self-learning and training.

[0062] This application proposes a method to judge and solve the failure of distributed cluster servers by combining the server operation status and abnormal sensitive logs, and replaces the fatigue, negligence or misjudgment that may occur in manual inspection. The machine learning model can perform detection continuously and stably, and is not affected by factors such as fatigue and emotions. The machine learning model used in this application can process a large amount of data in a short time. By using the machine learning model for automated detection, a large amount of human resources can be saved and the detection cost can be reduced.

[0063] Figure 2 is a schematic diagram of an optional server fault diagnosis processing process according to an embodiment of the present invention, such as Figure 2 As shown, when a fault occurs, the system log can be collected; the artificial neural network can be used for fault diagnosis and decision making; the decision execution queue can be obtained, the decision can be executed and whether the execution is successful can be judged; when the execution fails, the decision can be optimized and jump to decision making; when the execution is successful, a fault check can be performed to determine whether the fault is eliminated; when the fault is not eliminated, the artificial neural network can be used for fault diagnosis optimization and jump to fault diagnosis; when the fault is eliminated, the above process ends.

[0064] Figure 3 is a schematic diagram of an optional cluster server fault diagnosis processing process according to an embodiment of the present invention, such as Figure 3 As shown, at the client bottom layer, the stream data acquisition server can collect data from the operating systems corresponding to multiple servers such as server 1, server 2, server 3...server n, etc., to the client; at the data middle layer, the data collected by the stream data acquisition server can be sliced ​​and preprocessed, and the preprocessed data can be archived in the object storage server, the preprocessed data can be stored in a relational database, and prediction / classification can be performed based on the valid data after preprocessing, and the obtained data can be stored in the relational database. The data stored in the relational database can be trained regularly to achieve prediction / classification; at the visual user layer, information such as user login, server configuration management, and server flow management can be stored in the relational database, and an overview of the server status and a list of faults can also be obtained based on the relational database.

[0065] According to another aspect of an embodiment of the present application, a server fault diagnosis and processing device is also provided, which can execute the server fault diagnosis and processing method of the above embodiment. The specific implementation method and preferred application scenario are the same as the above embodiment and will not be repeated here.

[0066] Figure 4 is a schematic diagram of a server fault diagnosis and processing device according to an embodiment of the present application, such as Figure 4 As shown, the device includes the following: an acquisition module 402 , a diagnosis module 404 and a processing module 406 .

[0067] Among them, the acquisition module is used to obtain the operating status data of the server in response to receiving a status monitoring instruction; the diagnosis module is used to use a pre-trained fault diagnosis model to perform fault diagnosis on the server based on the operating status data to obtain the fault type of the server, wherein the pre-trained fault diagnosis model is used to represent a pre-trained neural network model for performing fault diagnosis on the server; the processing module is used to determine fault handling information based on the fault type to perform fault handling on the server, wherein the fault handling information is used to represent fault handling strategy information for handling faults on the server.

[0068] Among them, the diagnosis module is also used to extract features from the operating status data to obtain target meta-features; using the pre-trained fault diagnosis model, the target meta-features are compared with the preset fault type knowledge base to obtain the fault type, wherein the preset fault type knowledge base is used to represent a knowledge base constructed based on the correspondence between the sample operating status data of the server and the sample fault type.

[0069] Among them, the diagnosis module is also used to use the pre-trained fault diagnosis model to compare the target meta-features with the preset fault type knowledge base to obtain the confidence between the target meta-features and the sample fault type; and determine the fault type based on the confidence.

[0070] Among them, the diagnosis module is also used to obtain sample operation status data of the server; mark the fault type of the sample operation status data to obtain the sample fault type; and train a pre-trained fault diagnosis model based on the sample operation status data and the sample fault type.

[0071] Among them, the diagnosis module is also used to obtain the historical operation status data of the server, and extract features from the historical operation status data to obtain historical meta-features; the historical operation status data and the historical meta-features are spliced ​​to obtain sample operation status data.

[0072] Among them, the diagnosis module is also used to train the base learner in the preset fault diagnosis model based on the sample operating status data and the sample fault type to obtain a pre-trained fault diagnosis model, wherein the sample operating status data is used as the input of the base learner and the sample fault type is used as the output of the base learner.

[0073] An embodiment of the present application further provides an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of the present invention when running.

[0074] The above-mentioned memory may refer to a device inside a computer for storing data and programs, and may include memory, hard disk, etc., wherein the memory may be used to temporarily store running programs and data, the hard disk may be used to store programs and data for a long time, and the memory may be used to enable the computer to read and write data, and execute programs; the above-mentioned processor may be responsible for executing instructions in computer programs and performing data processing, and may be responsible for controlling and executing various operations, including arithmetic operations, logical operations, data transmission, etc.

[0075] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0076] The above-mentioned computer storage medium may refer to a medium in a computer memory used to store certain discontinuous physical quantities. Computer storage media mainly include semiconductors, magnetic cores, magnetic drums, magnetic tapes, laser disks, etc.; the stored program included in the computer-readable storage medium may be a set of instructions that can be recognized and executed by a computer, running on an electronic computer, and is an information tool that meets certain needs of people.

[0077] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present invention when executed by a processor.

[0078] The above-mentioned computer program product may refer to a software program that has been written, tested and released and can be run on a computer or other device. The computer program product may include an application, an operating system, tool software, etc., which is used to implement specific functions or solve specific problems.

[0079] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0080] The above-mentioned non-volatile computer-readable storage medium may refer to a medium for storing data. The non-volatile computer-readable storage medium can keep the data from being lost when the power is off, and can be used to store long-term data, such as operating systems, applications, and user files. The non-volatile storage medium may include hard disk drives, solid-state drives, optical disks, and flash memory storage devices, etc.

[0081] The embodiments of the present application further provide a computer program, which implements the methods in the above-mentioned embodiments of the present invention when executed by a processor.

[0082] The above-mentioned computer program may refer to a collection of instructions used to tell a computer to perform a specific task or operation. A computer program may be written by a programmer using a specific programming language and may include algorithms, data structures, logic, and control flows. A computer program may be used for a variety of purposes, including application software, operating systems, and the like.

[0083] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0086] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0087] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0088] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for diagnosing and processing a server fault, characterized in that: include: In response to receiving the status monitoring instruction, acquiring operation status data of the server; Using a pre-trained fault diagnosis model, based on the operating status data, a fault diagnosis is performed on the server to obtain a fault type of the server, wherein the pre-trained fault diagnosis model is used to represent a pre-trained neural network model for performing fault diagnosis on the server; Fault handling information is determined based on the fault type to perform fault handling on the server, wherein the fault handling information is used to represent fault handling strategy information for performing fault handling on the server.

2. The server fault diagnosis and processing method according to claim 1, characterized in that: Using the pre-trained fault diagnosis model, performing fault diagnosis on the server based on the operating status data to obtain the fault type of the server includes: Extracting features from the running status data to obtain target meta-features; Using the pre-trained fault diagnosis model, the target meta-feature is compared with a preset fault type knowledge base to obtain the fault type, wherein the preset fault type knowledge base is used to represent a knowledge base constructed based on the correspondence between the sample operating status data of the server and the sample fault type.

3. The server fault diagnosis and processing method according to claim 2, characterized in that: Using the pre-trained fault diagnosis model, the target meta-feature is compared with a preset fault type knowledge base to obtain the fault type, including: Using the pre-trained fault diagnosis model, the target meta-feature is compared with a preset fault type knowledge base to obtain a confidence level between the target meta-feature and the sample fault type; The fault type is determined based on the confidence level.

4. The server fault diagnosis and processing method according to claim 2, characterized in that: The method further comprises: Obtaining sample operation status data of the server; Marking the sample running status data with a fault type to obtain the sample fault type; The pre-trained fault diagnosis model is trained based on the sample operating status data and the sample fault type.

5. The server fault diagnosis and processing method according to claim 4, characterized in that: Obtaining sample operation status data of the server, including: Acquire historical operation status data of the server, and perform feature extraction on the historical operation status data to obtain historical meta-features; The historical running status data and the historical meta-features are concatenated to obtain the sample running status data.

6. The server fault diagnosis and processing method according to claim 4, characterized in that: Based on the sample operating state data and the sample fault type, the pre-trained fault diagnosis model is trained, including: Based on the sample operating status data and the sample fault type, a base learner in a preset fault diagnosis model is trained to obtain the pre-trained fault diagnosis model, wherein the sample operating status data serves as the input of the base learner and the sample fault type serves as the output of the base learner.

7. A server fault diagnosis and processing device, characterized in that: include: An acquisition module, configured to acquire the operation status data of the server in response to receiving the status monitoring instruction; A diagnosis module, used to perform fault diagnosis on the server based on the operation status data using a pre-trained fault diagnosis model to obtain a fault type of the server, wherein the pre-trained fault diagnosis model is used to represent a pre-trained neural network model for performing fault diagnosis on the server; A processing module is used to determine fault processing information based on the fault type to perform fault processing on the server, wherein the fault processing information is used to represent fault processing strategy information for performing fault processing on the server.

8. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program, when running, executes the server fault diagnosis and processing method described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the storage medium is located is controlled to execute the server fault diagnosis and processing method according to any one of claims 1 to 6.

10. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the fault diagnosis and processing method of the server according to any one of claims 1 to 6.

Images