Server state monitoring management system and method based on data analysis

Abstract

The invention relates to the technical field of computers, and provides a server state monitoring management system and method based on data analysis, and the system comprises a monitoring management middle platform, a data collection module, an operation mode determination module, a model matching module, a model prediction module and a state monitoring module. The data acquisition module acquires operation data of the target server at the current time; the operation mode determination module determines a current operation mode of the target server based on the network flow data and the process information; the model matching module is used for matching a target fault prediction model based on the current operation mode; the model prediction module inputs the target CPU usage rate, the target memory usage rate and the target disk read-write speed into a target fault prediction model to obtain a fault prediction result output by the target fault prediction model; the state monitoring module monitors the operation state of the target server based on the fault prediction result. According to the invention, the continuity of enterprise businesses is guaranteed, and the reliability and stability of server management are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a server status monitoring and management system based on data analysis. Background Art

[0002] In today's digital age, servers, as core components of enterprise information technology infrastructure, undertake a large number of key business operations and data storage tasks. The stable operation of servers is crucial to the daily operations, customer service and data security of enterprises.

[0003] On the one hand, traditional server status monitoring and management methods can only monitor a single indicator in isolation, and it is difficult to discover the potential correlation and coordinated change patterns between multiple indicators. However, the various performance indicators of the server are not independent of each other. The abnormal change of one indicator may trigger a chain reaction of other indicators. Therefore, only focusing on the threshold alarm of a single indicator cannot fully and accurately evaluate the overall health of the server. On the other hand, there is a lack of effective analysis and prediction capabilities for real-time trends in server status changes. When abnormal server performance indicators are found, they often have a certain impact on the business, and it is impossible to predict possible failures in advance and take corresponding preventive measures, which makes the enterprise passively respond to server failures, increasing the risk of business interruption. Summary of the invention

[0004] The present invention provides a server status monitoring management system and method based on data analysis, aiming to achieve a comprehensive assessment of the overall health status of the server, predict server failures in advance, ensure the continuity of enterprise business, and improve the reliability and stability of server management.

[0005] In a first aspect, the present invention provides a server status monitoring and management system based on data analysis, comprising a monitoring and management middle platform, a data acquisition module, an operation mode determination module, a model matching module, a model prediction module and a status monitoring module; the monitoring and management middle platform is respectively connected to the data acquisition module, the operation mode determination module, the model matching module, the model prediction module and the status monitoring module, and is used to manage each module; The data collection module is used to collect the operating data of the target server at the current time; the operating data includes the target CPU usage rate, the target memory usage rate, the target disk read and write speed, the network traffic data and the process information; The operation mode determination module is used to determine the current operation mode of the target server based on the network traffic data and the process information; The model matching module is used to match a target fault prediction model based on the current operation mode; the target fault prediction model is trained based on sample data pairs and their corresponding fault label results, and the sample data pairs include sample CPU usage, sample memory usage, and sample disk read and write speed; The model prediction module is used to input the target CPU usage rate, the target memory usage rate and the target disk read / write speed into the target fault prediction model to obtain the fault prediction result output by the target fault prediction model; The status monitoring module is used to monitor the operating status of the target server based on the fault prediction result.

[0006] In a second aspect, the present invention further provides a server status monitoring and management method based on data analysis, which is applied to the server status monitoring and management system based on data analysis described in the first aspect, and the server status monitoring and management method based on data analysis includes: Collect the target server's operating data at the current time; the operating data includes the target CPU usage, target memory usage, target disk read and write speed, network traffic data and process information; Determining a current operation mode of the target server based on the network traffic data and the process information; A target fault prediction model is matched based on the current operation mode; the target fault prediction model is trained based on sample data pairs and their corresponding fault label results, the sample data pairs including sample CPU usage, sample memory usage and sample disk read and write speed; Inputting the target CPU usage rate, the target memory usage rate and the target disk read / write speed into the target fault prediction model to obtain a fault prediction result output by the target fault prediction model; The operating status of the target server is monitored based on the fault prediction result.

[0007] According to the server status monitoring and management method based on data analysis provided by an embodiment of the present invention, the target fault prediction model includes a feature fusion layer, multiple intermediate hidden layers and an output layer; The step of inputting the target CPU usage rate, the target memory usage rate, and the target disk read / write speed into the target fault prediction model to obtain a fault prediction result output by the target fault prediction model includes: Input the target CPU usage rate, the target memory usage rate and the target disk read / write speed into the target fault prediction model, normalize and fuse the target CPU usage rate, the target memory usage rate and the target disk read / write speed based on the feature fusion layer, and output fusion features; Processing the fused features based on the first intermediate hidden layer, and outputting a first intermediate result; Processing the first intermediate result based on the second intermediate hidden layer, and outputting a second intermediate result; Processing the second intermediate result based on the third intermediate hidden layer, and outputting a third intermediate result; The third intermediate result is processed based on the output layer to output the fault prediction result; the fault prediction result represents the predicted probability that an abnormality exists in the target server.

[0008] According to the server status monitoring and management method based on data analysis provided by an embodiment of the present invention, the processing process of the first intermediate hidden layer is as follows: , ; in, represents the first intermediate result, Represents the vector corresponding to the fusion feature, represents the weight matrix of the first intermediate hidden layer, represents the bias vector of the first intermediate hidden layer, represents the activation function of the first intermediate hidden layer, Represents the base of the exponential function; The processing of the second intermediate hidden layer is as follows: , ; in, represents the second intermediate result, represents the weight matrix of the second intermediate hidden layer, represents the bias vector of the second intermediate hidden layer, represents the activation function of the second intermediate hidden layer; The processing process of the third intermediate hidden layer is as follows: , ; in, represents the third intermediate result, represents the weight matrix of the third intermediate hidden layer, represents the bias vector of the third intermediate hidden layer, represents the activation function of the third intermediate hidden layer; The processing of the output layer is as follows: ; in, represents the fault prediction result, express ,or, ,or The input variables.

[0009] According to the server status monitoring and management method based on data analysis provided by an embodiment of the present invention, the training process of the target fault prediction model specifically includes the following steps: Performing data division on the sample data pairs to obtain a plurality of training sample data pairs and a plurality of testing sample data pairs; The pre-trained model is trained based on multiple training sample data pairs and their corresponding fault label results to obtain an initial fault prediction model; Performing data prediction on each test sample data pair based on the initial fault prediction model to obtain a fault prediction result for each test sample data pair; The initial fault prediction model is optimized based on the fault label result and the fault prediction result of each test sample data pair to obtain the target fault prediction model.

[0010] According to the server status monitoring management method based on data analysis provided by an embodiment of the present invention, the initial fault prediction model is optimized based on the fault label result and the fault prediction result of each test sample data pair to obtain the target fault prediction model, including: Obtaining a first loss function value between a fault label result and a fault prediction result of a first test sample data pair, a second loss function value between a fault label result and a fault prediction result of a second test sample data pair, and a loss difference between the first loss function value and the second loss function value; If at least one of the first loss function value, the second loss function value and the loss difference value does not satisfy its corresponding preset condition, adjusting the model parameters in the initial fault prediction model; Based on the adjusted initial fault prediction model, the loss function value of each test sample data pair and the loss difference between two adjacent test sample data pairs are obtained, until the loss function value of each test sample data pair meets its corresponding preset conditions, and a continuous number of loss differences between two adjacent test sample data pairs meet their corresponding preset conditions, thereby completing the optimization of the initial fault prediction model and obtaining the target fault prediction model.

[0011] According to the server status monitoring and management method based on data analysis provided by an embodiment of the present invention, matching a target fault prediction model based on the current operation mode includes: Obtaining an operating load feature vector of the target server in the current operating mode and a model feature vector of each fault prediction model in a preset model library; Determining a matching coefficient of each fault prediction model to the current operation mode based on the operation load feature vector and the model feature vector of each fault prediction model; The fault prediction model corresponding to the largest matching coefficient in the preset model library is determined as the target fault prediction model of the target server in the current operation mode.

[0012] According to the server status monitoring and management method based on data analysis provided by an embodiment of the present invention, the network traffic data includes the number of incoming bytes, the number of outgoing bytes and the number of connections; the process information includes the number of processes, the amount of memory occupied by the process and the CPU time slice occupied by the process; The determining the current operation mode of the target server based on the network traffic data and the process information includes: Determining a traffic load ratio based on the number of incoming bytes, the number of outgoing bytes, and the number of connections; Determine a process busy coefficient and a process memory pressure coefficient based on the number of processes, the amount of memory occupied by the processes, and the CPU time slices occupied by the processes; Determining a comprehensive characteristic coefficient based on the traffic load rate, the process busy coefficient and the process memory pressure coefficient; The current operation mode of the target server at the current time is determined based on the comprehensive characteristic coefficient.

[0013] In a third aspect, the present invention further provides an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby implementing any of the above-mentioned server status monitoring and management methods based on data analysis.

[0014] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, wherein the storage medium stores a computer software program, and when the computer software program is executed by a processor, the server status monitoring and management method based on data analysis as described in any one of the above is implemented.

[0015] In a fifth aspect, the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned server status monitoring and management methods based on data analysis.

[0016] The server status monitoring and management system based on data analysis provided by the embodiment of the present invention, on the one hand, monitors and manages the target server status through the CPU usage rate, memory usage rate, disk read and write speed, network traffic data and process information of the target server, thereby realizing a comprehensive assessment of the overall health status of the target server. On the other hand, the target fault prediction model is matched through the current operation mode of the target server, and the operation status of the target server is monitored through the fault prediction results output by the target fault prediction model, so that the state change trend of the target server can be effectively analyzed and predicted in real time, so that the server failure can be predicted in advance, the continuity of the enterprise business is guaranteed, and the reliability and stability of server management are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a structural diagram of the server status monitoring and management system based on data analysis provided by the present invention; Figure 2 It is a flow chart of a server status monitoring and management method based on data analysis provided by the present invention; Figure 3 It is a flowchart of the fault prediction model training process provided by the present invention; Figure 4 An embodiment diagram of an electronic device provided by an embodiment of the present invention; Figure 5 An embodiment diagram of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0019] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0020] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0021] Optional, see Figure 1 As shown, Figure 1 It is a structural diagram of the server status monitoring and management system based on data analysis provided by the present invention. The server status monitoring and management system based on data analysis includes a monitoring and management middle station, a data acquisition module, an operation mode determination module, a model matching module, a model prediction module and a state monitoring module. In one embodiment, the monitoring and management middle station is respectively connected to the data acquisition module, the operation mode determination module, the model matching module, the model prediction module and the state monitoring module, so the monitoring and management middle station can manage each module.

[0022] Optionally, the operation data in the embodiment of the present invention includes CPU (Central Processing Unit) usage, memory usage, disk read / write speed, network traffic data and process information, wherein the network traffic data includes the number of incoming bytes, the number of outgoing bytes and the number of connections, and the process information includes the number of processes, the amount of memory occupied by the process and the CPU time slice occupied by the process. The network traffic data and process information can be collected through network monitoring tools (Wireshark's server-side agent) and system monitoring tools (top command under Linux or performance monitor under Windows). CPU usage, memory usage and disk read / write speed can be obtained by analyzing the operation log of the server.

[0023] Therefore, the data collection module collects the target CPU usage, target memory usage, target disk read and write speed, network traffic data and process information of any target server in the server cluster at the current time.

[0024] Further, the operation mode determination module calculates the traffic load rate of the target server according to the number of incoming bytes, the number of outgoing bytes and the number of connections, and calculates the process busy coefficient and the process memory pressure coefficient of the target server according to the number of processes, the amount of memory occupied by the processes and the CPU time slices occupied by the processes. Further, the operation mode determination module predicts the current operation mode of the target server according to the traffic load rate, the process busy coefficient and the process memory pressure coefficient of the target server.

[0025] The monitoring and management center of the embodiment of the present invention stores a pre-built preset model library, which includes multiple fault prediction models. Each fault prediction model is suitable for different operating modes. Therefore, the model matching module matches the best target fault prediction model under the current operating mode in the preset model library according to the matching coefficient between the current operating mode and the fault prediction model in the preset model library, wherein the target fault prediction model is trained based on sample data pairs and their corresponding fault label results, and the sample data pairs include sample CPU usage, sample memory usage and sample disk read and write speeds.

[0026] Furthermore, the model prediction module inputs the target CPU usage, target memory usage and target disk read and write speed into the target fault prediction model. The target fault prediction model makes predictions based on the target CPU usage, target memory usage and target disk read and write speed, and outputs fault prediction results. Thus, the fault prediction results output by the target fault prediction model can be obtained.

[0027] The fault prediction result in the embodiment of the present invention represents the predicted probability that the target server has an abnormality. Therefore, the state monitoring module compares the fault prediction result with the preset fault probability threshold value to determine whether the fault prediction result is greater than or equal to the preset fault probability threshold value. If the fault prediction result is greater than or equal to the preset fault probability threshold value, it is determined that the operating state of the target server is abnormal. If the fault prediction result is less than the preset fault probability threshold value, it is determined that the operating state of the target server is not abnormal.

[0028] In one embodiment, the abnormal behavior of the target server's running state is such as memory overflow, disk failure, network congestion, etc. For example, according to the fault prediction result, it is determined that the memory usage rate continues to rise and is close to saturation, and at the same time, the disk swap operations increase frequently, and it is predicted that the system performance degradation fault caused by insufficient memory is about to occur.

[0029] The server status monitoring and management system based on data analysis provided by the embodiment of the present invention, on the one hand, monitors and manages the target server status through the CPU usage rate, memory usage rate, disk read and write speed, network traffic data and process information of the target server, thereby realizing a comprehensive assessment of the overall health status of the target server. On the other hand, the target fault prediction model is matched through the current operation mode of the target server, and the operation status of the target server is monitored through the fault prediction results output by the target fault prediction model, so that the state change trend of the target server can be effectively analyzed and predicted in real time, so that the server failure can be predicted in advance, the continuity of the enterprise business is guaranteed, and the reliability and stability of server management are improved.

[0030] Optional, see Figure 2 , Figure 2 It is a flow chart of the server status monitoring and management method based on data analysis provided by the present invention.

[0031] The execution subject of the server status monitoring and management method based on data analysis in the embodiment of the present invention is a server status monitoring and management system based on data analysis. For the convenience of description, the following embodiment briefly describes the server status monitoring and management system based on data analysis as a monitoring and management system. Therefore, the server status monitoring and management method based on data analysis includes: Step 10: Collect the running data of the target server at the current time.

[0032] The operation data in the embodiment of the present invention includes CPU usage, memory usage, disk read and write speed, network traffic data and process information, wherein the network traffic data includes the number of incoming bytes, the number of outgoing bytes and the number of connections, and the process information includes the number of processes, the amount of memory occupied by the process and the CPU time slice occupied by the process. The network traffic data and process information can be collected through network monitoring tools (Wireshark's server-side agent) and system monitoring tools (top command under Linux or performance monitor under Windows). CPU usage, memory usage and disk read and write speed can be obtained by analyzing the operation log of the server. The monitoring and management system collects the target CPU usage, target memory usage, target disk read and write speed, network traffic data and process information of any target server in the server cluster at the current time.

[0033] Step 20: Determine the current operation mode of the target server based on the network traffic data and process information.

[0034] Further, the monitoring and management system calculates the traffic load rate of the target server according to the number of incoming bytes, the number of outgoing bytes and the number of connections, and calculates the process busy coefficient and the process memory pressure coefficient of the target server according to the number of processes, the amount of memory occupied by the processes and the CPU time slices occupied by the processes. Further, the monitoring and management system predicts the current operation mode of the target server according to the traffic load rate, the process busy coefficient and the process memory pressure coefficient of the target server, as described in detail in steps 201 to 204.

[0035] Step 30: Match a target fault prediction model based on the current operation mode.

[0036] The monitoring and management system of the embodiment of the present invention stores a pre-built preset model library, which includes multiple fault prediction models, each of which is applicable to a different operation mode.

[0037] Therefore, the monitoring and management system matches the best target fault prediction model under the current operating mode in the preset model library according to the matching coefficient between the current operating mode and the fault prediction model in the preset model library, as described in steps 301 to 303, wherein the target fault prediction model is trained based on sample data pairs and their corresponding fault label results, and the sample data pairs include sample CPU usage, sample memory usage, and sample disk read and write speeds.

[0038] Step 40, input the target CPU usage, the target memory usage and the target disk read / write speed into the target fault prediction model to obtain the fault prediction result output by the target fault prediction model.

[0039] Furthermore, the monitoring and management system inputs the target CPU usage, target memory usage and target disk read and write speed into the target fault prediction model. The target fault prediction model makes predictions based on the target CPU usage, target memory usage and target disk read and write speed, and outputs fault prediction results. Therefore, the monitoring and management system can obtain the fault prediction results output by the target fault prediction model, as specifically described in steps 401 to 405.

[0040] Step 50: monitor the operating status of the target server based on the fault prediction result.

[0041] The fault prediction result in the embodiment of the present invention represents the predicted probability of an abnormality in the target server. Therefore, the monitoring and management system compares the fault prediction result with a preset fault probability threshold to determine whether the fault prediction result is greater than or equal to the preset fault probability threshold.

[0042] If the fault prediction result is greater than or equal to the preset fault probability threshold, the monitoring management system determines that the operation status of the target server is abnormal. If the fault prediction result is less than the preset fault probability threshold, the monitoring management system determines that the operation status of the target server is not abnormal.

[0043] In one embodiment, the abnormal behavior of the target server's operating state is such as memory overflow, disk failure, network congestion, etc. For example, if it is determined according to the fault prediction result that the memory usage of the target server continues to rise and is close to saturation, and at the same time, the disk swap operations of the target server increase frequently, the monitoring management system predicts that the target server is about to have a system performance degradation fault caused by insufficient memory.

[0044] On the one hand, the embodiment of the present invention monitors and manages the state of the target server through the CPU usage rate, memory usage rate, disk read and write speed, network traffic data and process information of the target server, thereby achieving a comprehensive assessment of the overall health status of the target server. On the other hand, the target fault prediction model is matched through the current operation mode of the target server, and the operation status of the target server is monitored through the fault prediction results output by the target fault prediction model. Therefore, the state change trend of the target server can be effectively analyzed and predicted in real time, so that the server failure can be predicted in advance, the continuity of the enterprise business is guaranteed, and the reliability and stability of server management are improved.

[0045] In one embodiment, the description of steps 201 to 204 is as follows: Step 201, determine the traffic load rate based on the number of incoming bytes, the number of outgoing bytes and the number of connections.

[0046] Specifically, the monitoring management system obtains the sequence of the number of bytes incoming to the target server at the current time and the sequence of the number of bytes transmitted , where n represents the time point within the observation window, Indicates The number of bytes passed in at a time point , Indicates Number of outgoing bytes at a time point Therefore, the monitoring and management system calculates the fluctuation coefficient of the incoming traffic according to the sequence of the number of incoming bytes. The specific calculation formula is: ; in, represents the fluctuation coefficient of incoming traffic, The mean of the sequence representing the number of bytes passed in.

[0047] Furthermore, the monitoring management system calculates the fluctuation coefficient of the outgoing traffic based on the sequence of the number of outgoing bytes. The calculation formula is the same as the calculation formula for the fluctuation coefficient of the incoming traffic.

[0048] Furthermore, the monitoring and management system calculates the traffic load rate of the target server according to the fluctuation coefficient of the incoming traffic, the fluctuation coefficient of the outgoing traffic and the number of connections. The specific formula is as follows: ; in, Indicates the traffic load rate, Represents the preset weight coefficient, Indicates the number of connections. Indicates the network bandwidth of the target server.

[0049] Step 202, based on the number of processes, the amount of memory occupied by the processes, and the CPU time slices occupied by the processes, determine the process busy coefficient and the process memory pressure coefficient.

[0050] Furthermore, the monitoring and management system calculates the process busy coefficient of the target server according to the number of processes and the CPU time slices occupied by the processes. The specific calculation formula is as follows: ; in, Indicates the process busy coefficient, Indicates the number of processes. Indicates the CPU time slice occupied by the jth process; Indicates the maximum value of a single CPU time slice, which depends on the operating system and hardware configuration of the target server.

[0051] Furthermore, the monitoring and management system calculates the process memory pressure coefficient of the target server according to the amount of memory occupied by the process. The specific calculation formula is as follows: ; in, Indicates the process memory pressure coefficient, represents the amount of memory occupied by the jth process, Indicates the total memory of the target server.

[0052] Step 203, determining a comprehensive characteristic coefficient based on the flow load rate, the process busy coefficient and the process memory pressure coefficient.

[0053] Furthermore, the monitoring and management system calculates the comprehensive characteristic coefficient according to the traffic load rate, the process busy coefficient and the process memory pressure coefficient. The specific formula is as follows: ; in, represents the comprehensive characteristic coefficient, Represents the base of the exponential function.

[0054] Step 204, determining the current operation mode of the target server at the current time based on the comprehensive characteristic coefficient.

[0055] Furthermore, the monitoring and management system determines the comprehensive characteristic coefficient The data range in which the data is located includes the first data range , the second data range and the third data range ,in, and Indicates the preset coefficient, which is determined according to the performance of the target server.

[0056] If it is determined that the comprehensive characteristic coefficient is within the first data range, the monitoring and management system determines that the current operation mode of the target server at the current time is an idle mode, that is, a low-load mode.

[0057] If it is determined that the comprehensive characteristic coefficient is within the second data range, the monitoring and management system determines that the current operation mode of the target server at the current time is a medium load mode.

[0058] If it is determined that the comprehensive characteristic coefficient is within the third data range, the monitoring and management system determines that the current operation mode of the target server at the current time is a busy mode, that is, a high-load mode.

[0059] The embodiment of the present invention accurately determines the current operating mode of the target server at the current time through network traffic data and process information. Therefore, the target fault prediction model can be quickly and accurately matched according to the current operating mode. The fault prediction result of the target server can be accurately predicted through the target fault prediction model, so that the fault of the target server can be obtained in advance according to the fault prediction result, thereby ensuring the continuity of the enterprise business and improving the reliability and stability of server management.

[0060] In one embodiment, the description of steps 301 to 303 is as follows: Step 301: Obtain an operating load feature vector of a target server in a current operating mode, and a model feature vector of each fault prediction model in a preset model library.

[0061] The operating load characteristics in the embodiment of the present invention are the degree of deviation between the current performance indicator and the average performance indicator under the corresponding operating mode. Therefore, the monitoring and management system obtains the current performance indicator vector of the target server under the current operating mode, as well as the performance indicator average value vector under the current operating mode.

[0062] Furthermore, the monitoring and management system determines the operating load characteristic vector under the current operating mode according to the current performance indicator vector and the performance indicator average value vector under the current operating mode. In one embodiment, the performance indicator average value vector under the current operating mode is , the current performance indicator vector in the current operating mode is ,in, Represents the dimension of the performance indicator, the operating load feature vector under the current operating mode It can be expressed as: .

[0063] Among them, the operating load characteristic vector The Load characteristic vector Can be expressed .

[0064] Furthermore, the monitoring management system obtains the model feature vector of each fault prediction model in the preset model library pre-built in the monitoring management system ,in, Indicates The model feature vector of the fault prediction model, Represents the number of fault prediction models.

[0065] Step 302: Determine the matching coefficient of each fault prediction model to the current operation mode based on the operation load feature vector and the model feature vector of each fault prediction model.

[0066] Furthermore, the monitoring and management system calculates the matching coefficient of each fault prediction model to the current operation mode according to the operating load characteristic vector and the model characteristic vector of each fault prediction model. The specific calculation formula of the matching coefficient is as follows: ; in, Represents the operating load characteristic vector and the The matching coefficient between the model feature vectors of the fault prediction models is Indicates The model parameters of the fault prediction model are represents the base of the exponential function, Indicates The model feature vector of the fault prediction model feature vectors, represents the maximum value function; Indicates the preset value to prevent the denominator from being 0.

[0067] Step 303: determine the fault prediction model corresponding to the largest matching coefficient in the preset model library as the target fault prediction model of the target server in the current operation mode.

[0068] Furthermore, the monitoring and management system traverses the matching coefficients between the operating load feature vector in the preset model library and the model feature vector of each fault prediction model, and determines the fault prediction model corresponding to the largest matching coefficient as the target fault prediction model of the target server under the current operating mode.

[0069] The embodiment of the present invention accurately matches the target fault prediction model of the target server under the current operation mode by running the load feature vector and the model feature vector of each fault prediction model in the preset model library. Therefore, the fault prediction result of the target server can be accurately predicted through the target fault prediction model, so that the fault of the target server can be obtained in advance according to the fault prediction result, thereby ensuring the continuity of the enterprise business and improving the reliability and stability of server management.

[0070] In one embodiment, the target fault prediction model includes a feature fusion layer, multiple intermediate hidden layers and an output layer, the multiple intermediate hidden layers include a first intermediate hidden layer, a second intermediate hidden layer and a third intermediate hidden layer, and the loss functions of the three intermediate hidden layers are different. The descriptions of steps 401 to 405 are as follows: Step 401, input the target CPU usage, target memory usage and target disk read / write speed into the target fault prediction model, normalize and fuse the target CPU usage, target memory usage and target disk read / write speed based on the feature fusion layer, and output the fusion feature.

[0071] Specifically, the monitoring and management system inputs the target CPU usage, target memory usage and target disk read and write speed into the target fault prediction model, and the feature fusion layer normalizes the target CPU usage, target memory usage and target disk read and write speed to obtain the normalized CPU usage, normalized memory usage and normalized disk read and write speed, respectively. Among them, the normalization process is a conventional method and the embodiments of the present invention will not be repeated here.

[0072] Furthermore, the feature fusion layer calculates the mean of the normalized CPU usage, the normalized memory usage, and the normalized disk read and write speed to fuse the normalized CPU usage, the normalized memory usage, and the normalized disk read and write speed and outputs the fused feature.

[0073] Step 402: Process the fusion feature based on the first intermediate hidden layer and output a first intermediate result. Step 403: Process the first intermediate result based on the second intermediate hidden layer and output a second intermediate result. Step 404: Process the second intermediate result based on the third intermediate hidden layer and output a third intermediate result.

[0074] Furthermore, the first intermediate hidden layer processes the fusion features output by the feature fusion layer and outputs a first intermediate result, wherein the specific processing process of the first intermediate hidden layer is as follows: , ; in, represents the first intermediate result, Represents the vector corresponding to the fusion feature, represents the weight matrix of the first intermediate hidden layer, represents the bias vector of the first intermediate hidden layer, represents the activation function of the first intermediate hidden layer, Represents the base of the exponential function.

[0075] Furthermore, the second intermediate hidden layer processes the first intermediate result output by the first intermediate hidden layer and outputs a second intermediate result, wherein the specific processing process of the second intermediate hidden layer is as follows: , ; in, represents the second intermediate result, represents the weight matrix of the second intermediate hidden layer, represents the bias vector of the second intermediate hidden layer, represents the activation function of the second intermediate hidden layer.

[0076] Further, the third intermediate hidden layer processes the second intermediate result output by the second intermediate hidden layer, and outputs a third intermediate result, wherein the specific processing process of the third intermediate hidden layer is as follows: , ; in, represents the third intermediate result, represents the weight matrix of the third intermediate hidden layer, represents the bias vector of the third intermediate hidden layer, represents the activation function of the third intermediate hidden layer.

[0077] Step 405: Process the third intermediate result based on the output layer and output a fault prediction result.

[0078] Further, the output layer processes the third intermediate result output by the third intermediate hidden layer, and outputs a fault prediction result, wherein the fault prediction result represents the predicted probability that the target server has an abnormality. The specific processing process of the output layer is as follows: ; in, represents the fault prediction result, express ,or, ,or The input variables.

[0079] The embodiment of the present invention accurately predicts the fault prediction result of the target server through the target fault prediction model, so that the fault of the target server can be obtained in advance according to the fault prediction result, thereby ensuring the continuity of the enterprise business and improving the reliability and stability of server management.

[0080] Optional, see Figure 3 , Figure 3 : is a flow chart of the fault prediction model training process provided by the present invention. The training process of the target fault prediction model specifically includes the following steps: Step 60, partitioning the sample data pairs to obtain a plurality of training sample data pairs and a plurality of testing sample data pairs.

[0081] Specifically, the monitoring and management system divides the sample data pairs to obtain multiple training sample data pairs and multiple test sample data pairs, wherein the ratio between the number of training sample data pairs and the number of test sample data pairs is set according to actual conditions, such as 8:2; 7:3; 7.5:2.5, etc.

[0082] Step 70: training the pre-trained model based on a plurality of training sample data pairs and their corresponding fault label results to obtain an initial fault prediction model.

[0083] Furthermore, the monitoring and management system outputs multiple training sample data pairs and their corresponding fault label results to a pre-trained model, performs model training on the pre-trained model, and obtains an initial fault prediction analysis model, wherein the pre-trained model is such as a feedforward neural network, a graph neural network, and a feedback neural network, and the pre-trained model in the embodiment of the present invention is a feedforward neural network. Therefore, it can be understood that the monitoring and management system outputs multiple training sample data pairs and their corresponding fault label results to a feedforward neural network for model training to obtain an initial fault prediction analysis model, and it can be understood that the structure of the initial fault prediction analysis model is consistent with the structure of the target fault prediction model.

[0084] Step 80: Perform data prediction on each test sample data pair based on the initial fault prediction model to obtain a fault prediction result for each test sample data pair.

[0085] Step 90 , optimizing the initial fault prediction model based on the fault label result and the fault prediction result of each test sample data pair to obtain a target fault prediction model.

[0086] It should be noted that the parameters in the initial fault prediction analysis model are not yet optimal parameters, and the parameters in the initial fault prediction analysis model need to be optimized. Therefore, the initial fault prediction model performs data prediction on each test sample data pair to obtain the fault prediction result of each test sample data pair.

[0087] Furthermore, the monitoring and management system optimizes the initial fault prediction model according to the fault label result and the fault prediction result of each test sample data pair to obtain a target fault prediction model, as specifically described in steps 901 to 903 .

[0088] The embodiment of the present invention trains a target fault prediction model, which can subsequently accurately predict the fault prediction results of the target server, so that the fault of the target server can be obtained in advance, ensuring the continuity of the enterprise's business and improving the reliability and stability of server management.

[0089] In one embodiment, the description of step 901 to step 903 is as follows: Step 901, obtain the first loss function value between the fault label result and the fault prediction result of the first test sample data pair, the second loss function value between the fault label result and the fault prediction result of the second test sample data pair, and the loss difference between the first loss function value and the second loss function value.

[0090] Specifically, the monitoring and management system inputs the fault label result and the fault prediction result of the first test sample data pair into a preset loss function to obtain a first loss function value between the fault label result and the fault prediction result of the first test sample data pair. Similarly, the fault label result and the fault prediction result of the second test sample data pair are input into the preset loss function to obtain a second loss function value between the fault label result and the fault prediction result of the second test sample data pair, wherein the preset loss function is specifically: ; in, Indicates The loss function value of the test sample data pair is represents the number of test sample data pairs, represents the exponential function, Indicates The fault label results of the test sample data pairs are: Indicates The fault prediction results of the test sample data pairs are Represents the maximum value function.

[0091] Furthermore, the monitoring and management system calculates the loss difference between the first loss function value and the second loss function value, that is, the loss difference=|second loss function value-first loss function value|.

[0092] Step 902: If at least one of the first loss function value, the second loss function value and the loss difference value does not satisfy its corresponding preset condition, the model parameters in the initial fault prediction model are adjusted.

[0093] Furthermore, the monitoring and management system determines whether the first loss function value and the second loss function value are less than or equal to a preset loss threshold, and whether the loss difference is less than or equal to a preset difference threshold, wherein the preset loss threshold and the preset difference threshold are set according to actual conditions.

[0094] If it is determined that the first loss function value, the second loss function value and the loss difference all meet their corresponding preset conditions, that is, the first loss function value and the second loss function value are less than or equal to the preset loss threshold, and the loss difference is less than or equal to the preset difference threshold, the monitoring and management system determines that the model parameters in the initial fault prediction model are the optimal parameters, and determines the current initial fault prediction model as the target fault prediction model.

[0095] If it is determined that at least one of the first loss function value, the second loss function value and the loss difference does not meet the corresponding preset conditions, that is, the first loss function value is greater than the preset loss threshold, or / and, the second loss function value is greater than the preset loss threshold, or / and, the loss difference is greater than the preset difference threshold, the monitoring and management system adjusts the model parameters in the initial fault prediction model, such as adjusting the model parameters of one hidden layer or multiple hidden layers in the initial fault prediction model.

[0096] Step 903, based on the adjusted initial fault prediction model, obtain the loss function value of each test sample data pair and the loss difference between two adjacent test sample data pairs, until the loss function value of each test sample data pair meets its corresponding preset conditions, and a continuous number of loss differences between two adjacent test sample data pairs meet their corresponding preset conditions, thereby completing the optimization of the initial fault prediction model and obtaining the target fault prediction model.

[0097] Furthermore, the monitoring and management system inputs the test sample data pair into the adjusted initial fault prediction model to obtain the fault prediction result of each test sample data pair output by the adjusted initial fault prediction model.

[0098] Furthermore, the monitoring and management system inputs the fault label result and fault prediction result of each test sample data pair into a preset loss function to obtain the loss function value of each test sample data pair, and performs difference calculation on the loss function values ​​of two adjacent test sample data pairs to obtain the loss difference between the two adjacent test sample data pairs.

[0099] Furthermore, the monitoring and management system continuously optimizes the model parameters of the initial fault prediction model according to the loss function value of each test sample data pair and the loss difference between two adjacent test sample data pairs, until the loss function value of each test sample data pair satisfies its corresponding preset conditions, and the loss difference between a continuous number of adjacent test sample data pairs satisfies its corresponding preset conditions, that is, the loss function value of each test sample data pair is less than or equal to the preset loss threshold, and the loss difference between a continuous number of adjacent test sample data pairs is less than or equal to the preset difference threshold, thereby completing the optimization of the initial fault prediction model and obtaining the target fault prediction model, wherein the continuous number is set according to the number of test sample data pairs. In one embodiment, the number of test sample data pairs is 100, and the continuous number can be set to 50, 60, 70, etc.

[0100] The embodiment of the present invention continuously optimizes the model parameters in the initial fault prediction model to obtain a target fault prediction model with optimal model parameters. The target fault prediction model can then accurately predict the fault prediction results of the target server, so that the fault of the target server can be obtained in advance according to the fault prediction results, thereby ensuring the continuity of the enterprise business and improving the reliability and stability of server management.

[0101] See also Figure 4 , Figure 4 FIG. 1 is an embodiment diagram of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented: Collect the target server's operating data at the current time; the operating data includes the target CPU usage, target memory usage, target disk read and write speed, network traffic data and process information; Determine the current operation mode of the target server based on network traffic data and process information; A target fault prediction model is matched based on the current operation mode; the target fault prediction model is trained based on sample data pairs and their corresponding fault label results, and the sample data pairs include sample CPU usage, sample memory usage, and sample disk read and write speed; Input the target CPU usage, target memory usage and target disk read / write speed into the target fault prediction model to obtain the fault prediction result output by the target fault prediction model; Monitor the operating status of the target server based on the fault prediction results.

[0102] See also Figure 5 , Figure 5 Detailed description of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention. Figure 5 As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 411 is stored. When the computer program 411 is executed by a processor, the following steps are implemented: Collect the target server's operating data at the current time; the operating data includes the target CPU usage, target memory usage, target disk read and write speed, network traffic data and process information; Determine the current operation mode of the target server based on network traffic data and process information; A target fault prediction model is matched based on the current operation mode; the target fault prediction model is trained based on sample data pairs and their corresponding fault label results, and the sample data pairs include sample CPU usage, sample memory usage, and sample disk read and write speed; Input the target CPU usage, target memory usage and target disk read / write speed into the target fault prediction model to obtain the fault prediction result output by the target fault prediction model; Monitor the operating status of the target server based on the fault prediction results.

[0103] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the server status monitoring and management method based on data analysis provided by the above methods, the method includes: Collect the target server's operating data at the current time; the operating data includes the target CPU usage, target memory usage, target disk read and write speed, network traffic data and process information; Determine the current operation mode of the target server based on network traffic data and process information; A target fault prediction model is matched based on the current operation mode; the target fault prediction model is trained based on sample data pairs and their corresponding fault label results, and the sample data pairs include sample CPU usage, sample memory usage, and sample disk read and write speed; Input the target CPU usage, target memory usage and target disk read / write speed into the target fault prediction model to obtain the fault prediction result output by the target fault prediction model; Monitor the operating status of the target server based on the fault prediction results.

[0104] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A server status monitoring and management system based on data analysis, characterized in that: It includes a monitoring and management center, a data acquisition module, an operation mode determination module, a model matching module, a model prediction module and a state monitoring module; the monitoring and management center is respectively connected to the data acquisition module, the operation mode determination module, the model matching module, the model prediction module and the state monitoring module, and is used to manage each module; The data collection module is used to collect the operating data of the target server at the current time; the operating data includes the target CPU usage rate, the target memory usage rate, the target disk read and write speed, the network traffic data and the process information; The operation mode determination module is used to determine the current operation mode of the target server based on the network traffic data and the process information; The model matching module is used to match a target fault prediction model based on the current operation mode; The target fault prediction model is trained based on sample data pairs and their corresponding fault label results, wherein the sample data pairs include sample CPU usage, sample memory usage, and sample disk read and write speed; The model prediction module is used to input the target CPU usage rate, the target memory usage rate and the target disk read / write speed into the target fault prediction model to obtain the fault prediction result output by the target fault prediction model; The status monitoring module is used to monitor the operating status of the target server based on the fault prediction result.

2. A server status monitoring and management method based on data analysis, applied to the server status monitoring and management system based on data analysis as claimed in claim 1, characterized in that: The server status monitoring and management method based on data analysis includes: Collect the target server's operating data at the current time; the operating data includes the target CPU usage, target memory usage, target disk read and write speed, network traffic data and process information; Determining a current operation mode of the target server based on the network traffic data and the process information; A target fault prediction model is matched based on the current operation mode; the target fault prediction model is trained based on sample data pairs and their corresponding fault label results, the sample data pairs including sample CPU usage, sample memory usage and sample disk read and write speed; Inputting the target CPU usage rate, the target memory usage rate and the target disk read / write speed into the target fault prediction model to obtain a fault prediction result output by the target fault prediction model; The operating status of the target server is monitored based on the fault prediction result.

3. The server status monitoring and management method based on data analysis according to claim 2 is characterized in that: The target fault prediction model includes a feature fusion layer, multiple intermediate hidden layers and an output layer; The step of inputting the target CPU usage rate, the target memory usage rate, and the target disk read / write speed into the target fault prediction model to obtain a fault prediction result output by the target fault prediction model includes: Input the target CPU usage rate, the target memory usage rate and the target disk read / write speed into the target fault prediction model, normalize and fuse the target CPU usage rate, the target memory usage rate and the target disk read / write speed based on the feature fusion layer, and output fusion features; Processing the fused features based on the first intermediate hidden layer, and outputting a first intermediate result; Processing the first intermediate result based on the second intermediate hidden layer, and outputting a second intermediate result; Processing the second intermediate result based on the third intermediate hidden layer, and outputting a third intermediate result; The third intermediate result is processed based on the output layer to output the fault prediction result; the fault prediction result represents the predicted probability that an abnormality exists in the target server.

4. The server status monitoring and management method based on data analysis according to claim 3 is characterized in that: The processing of the first intermediate hidden layer is as follows: ， ； in, represents the first intermediate result, Represents the vector corresponding to the fusion feature, represents the weight matrix of the first intermediate hidden layer, represents the bias vector of the first intermediate hidden layer, represents the activation function of the first intermediate hidden layer, Represents the base of the exponential function; The processing of the second intermediate hidden layer is as follows: ， ； in, represents the second intermediate result, represents the weight matrix of the second intermediate hidden layer, represents the bias vector of the second intermediate hidden layer, represents the activation function of the second intermediate hidden layer; The processing process of the third intermediate hidden layer is as follows: ， ； in, represents the third intermediate result, represents the weight matrix of the third intermediate hidden layer, represents the bias vector of the third intermediate hidden layer, represents the activation function of the third intermediate hidden layer; The processing of the output layer is as follows: ； in, represents the fault prediction result, express ,or, ,or The input variables.

5. The server status monitoring and management method based on data analysis according to claim 2 is characterized in that: The training process of the target fault prediction model specifically includes the following steps: Performing data division on the sample data pairs to obtain a plurality of training sample data pairs and a plurality of testing sample data pairs; The pre-trained model is trained based on multiple training sample data pairs and their corresponding fault label results to obtain an initial fault prediction model; Performing data prediction on each test sample data pair based on the initial fault prediction model to obtain a fault prediction result for each test sample data pair; The initial fault prediction model is optimized based on the fault label result and the fault prediction result of each test sample data pair to obtain the target fault prediction model.

6. The server status monitoring and management method based on data analysis according to claim 5 is characterized in that: The optimizing the initial fault prediction model based on the fault label result and the fault prediction result of each test sample data pair to obtain the target fault prediction model includes: Obtaining a first loss function value between a fault label result and a fault prediction result of a first test sample data pair, a second loss function value between a fault label result and a fault prediction result of a second test sample data pair, and a loss difference between the first loss function value and the second loss function value; If at least one of the first loss function value, the second loss function value and the loss difference value does not satisfy its corresponding preset condition, adjusting the model parameters in the initial fault prediction model; Based on the adjusted initial fault prediction model, the loss function value of each test sample data pair and the loss difference between two adjacent test sample data pairs are obtained, until the loss function value of each test sample data pair meets its corresponding preset conditions, and a continuous number of loss differences between two adjacent test sample data pairs meet their corresponding preset conditions, thereby completing the optimization of the initial fault prediction model and obtaining the target fault prediction model.

7. The server status monitoring and management method based on data analysis according to claim 2 is characterized in that: The matching of a target fault prediction model based on the current operation mode includes: Obtaining an operating load feature vector of the target server in the current operating mode and a model feature vector of each fault prediction model in a preset model library; Determining a matching coefficient of each fault prediction model to the current operation mode based on the operation load feature vector and the model feature vector of each fault prediction model; The fault prediction model corresponding to the largest matching coefficient in the preset model library is determined as the target fault prediction model of the target server in the current operation mode.

8. The server status monitoring and management method based on data analysis according to any one of claims 2 to 7, characterized in that: The network traffic data includes the number of incoming bytes, the number of outgoing bytes and the number of connections; the process information includes the number of processes, the amount of memory occupied by the processes and the CPU time slices occupied by the processes; The determining the current operation mode of the target server based on the network traffic data and the process information includes: Determining a traffic load ratio based on the number of incoming bytes, the number of outgoing bytes, and the number of connections; Determine a process busy coefficient and a process memory pressure coefficient based on the number of processes, the amount of memory occupied by the processes, and the CPU time slices occupied by the processes; Determining a comprehensive characteristic coefficient based on the traffic load rate, the process busy coefficient and the process memory pressure coefficient; The current operation mode of the target server at the current time is determined based on the comprehensive characteristic coefficient.

9. An electronic device, comprising: A memory and a processor, characterized in that a computer software program is stored in the memory, and when the processor reads and executes the computer software program, the server status monitoring and management method based on data analysis as described in any one of claims 2 to 8 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that: The storage medium stores a computer software program, and when the computer software program is executed by the processor, the server status monitoring and management method based on data analysis as described in any one of claims 2 to 8 is implemented.

Images