Dynamic health management method of server cluster, electronic equipment and storage medium

Abstract

The invention discloses a dynamic health management method of a server cluster, electronic equipment and a storage medium, and relates to the technical field of server clusters, multi-source data is converted into a first feature sequence, and multi-modal data fusion is realized; the first feature sequence is input into a health management model to obtain health prediction results, the health prediction results at least comprise a first prediction result and a second prediction result, the first prediction result is used for representing a fault type of the target server, and the second prediction result is used for representing a potential fault propagation path of the target server in the target cluster, the health management model supports cross-layer fault positioning; and determining a dynamic adjustment strategy according to the health prediction result, and executing a corresponding dynamic adjustment operation according to the dynamic adjustment strategy to complete dynamic health management of the target cluster, so that the target server is adjusted in advance to ensure the health state of the target cluster.

Description

Technical Field

[0001] This application relates to the field of health management technology for server clusters, and in particular to dynamic health management methods, electronic devices, and storage media for server clusters. Background Technology

[0002] As data centers continue to grow in scale, fault diagnosis and prediction of server clusters can quickly identify and resolve potential data center problems, prevent system crashes and data loss, and ensure business continuity and stability. Related technologies perform fault diagnosis and prediction by collecting real-time data from server clusters and comparing it with fixed thresholds. However, this method has several problems: 1. Data fragmentation and insufficient real-time performance: Real-time data collection relies on independent hardware sensors and log analysis, resulting in scattered data and a lack of cross-layer correlation, leading to delays in fault location; 2. Weak generalization ability of static models: Fault diagnosis and prediction of server clusters based on fixed thresholds cannot adapt to dynamic scenarios such as hardware aging and environmental changes, resulting in a high false alarm rate; 3. Lack of multimodal data fusion: Related technologies do not fully integrate heterogeneous data such as hardware status, log semantics, and network traffic, leading to low prediction accuracy. Summary of the Invention

[0003] This application provides a dynamic health management method, electronic device, and storage medium for server clusters to at least address at least one of the following problems in related technologies: data fragmentation and insufficient real-time performance, weak generalization ability of static models, and lack of multimodal data fusion.

[0004] This application provides a dynamic health management method for server clusters. The dynamic health management method for server clusters includes: collecting multi-source data from a target server; preprocessing the multi-source data to obtain a first feature sequence; inputting the first feature sequence into a health management model to obtain a health prediction result, wherein the health prediction result includes at least a first prediction result and a second prediction result, the first prediction result being used to represent the fault type of the target server, and the second prediction result being used to represent the potential fault propagation path of the target server in the target cluster; determining a dynamic adjustment strategy based on the health prediction result, and executing the corresponding dynamic adjustment operation according to the dynamic adjustment strategy to complete the dynamic health management of the target cluster.

[0005] This application also provides a dynamic health management system for a server cluster. The dynamic health management system includes an edge agent cluster, a cloud agent platform, and a visualization management platform. The cloud agent platform includes at least a data fusion agent, a model training agent, and a decision execution agent. The edge agent cluster is used to convert collected multi-source data into a first feature sequence and transmit the first feature sequence to the cloud agent platform. The data fusion agent is used to aggregate the feature vectors of the edge agent cluster using a federated learning framework and construct an association graph through a graph neural network to identify potential fault propagation paths. The model training agent is used to construct a prediction model based on temporal convolutional layers and long short-term memory layers, and dynamically adjust the model parameters of the prediction model using a backpropagation algorithm. The decision execution agent is used to push health prediction results to a hardware management module or a cloud management platform to perform corresponding dynamic adjustment operations and adjust the parameters of the policy network based on the dynamic adjustment results. The visualization management platform is used to display the health status of the target cluster based on digital twin technology. The health status includes at least potential fault propagation paths, health prediction results, and dynamic adjustment strategies.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement at least the following steps of a dynamic health management method for a server cluster: collecting multi-source data from a target server; preprocessing the multi-source data to obtain a first feature sequence; inputting the first feature sequence into a health management model to obtain a health prediction result, wherein the health prediction result includes at least a first prediction result and a second prediction result, the first prediction result representing the fault type of the target server, and the second prediction result representing the potential fault propagation path of the target server in the target cluster; determining a dynamic adjustment strategy based on the health prediction result, and executing corresponding dynamic adjustment operations based on the dynamic adjustment strategy to complete the dynamic health management of the target cluster.

[0007] This application also provides a computer-readable storage medium storing a computer program, wherein when executed by a processor, the computer program implements at least the following steps: collecting multi-source data from a target server; preprocessing the multi-source data to obtain a first feature sequence; inputting the first feature sequence into a health management model to obtain a health prediction result, wherein the health prediction result includes at least a first prediction result and a second prediction result, the first prediction result representing the fault type of the target server and the second prediction result representing the potential fault propagation path of the target server in the target cluster; determining a dynamic adjustment strategy based on the health prediction result, and executing corresponding dynamic adjustment operations based on the dynamic adjustment strategy to complete the dynamic health management of the target cluster.

[0008] This application transforms multi-source data into a first feature sequence to achieve multimodal data fusion. The first feature sequence is then input into a health management model to obtain health prediction results. These results include at least a first prediction result and a second prediction result. The first prediction result represents the fault type of the target server, and the second prediction result represents the potential fault propagation path of the target server in the target cluster, enabling the health management model to support cross-layer fault location. Based on the health prediction results, a dynamic adjustment strategy is determined, and corresponding dynamic adjustment operations are executed according to the strategy to complete the dynamic health management of the target cluster, thereby enabling advance adjustment of the target server to ensure the health status of the target cluster. Attached Figure Description

[0009] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating the dynamic health management method for the server cluster in the first embodiment; Figure 2 This is a structural block diagram of the dynamic health management system for the server cluster in the second embodiment; Figure 3 This is a functional architecture diagram of the cloud-based intelligent agent platform in the second embodiment; Figure 4 This is a functional architecture diagram of the visualization management platform in the second embodiment; Figure 5 This is a diagram of the internal structure of the electronic device in the third embodiment. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0012] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0013] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] In the first embodiment, such as Figure 1 As shown, a dynamic health prediction method for server clusters is provided. This method includes: collecting multi-source data from the target server; preprocessing the multi-source data to obtain a first feature sequence; inputting the first feature sequence into a health management model to obtain a health prediction result, wherein the health prediction result includes at least a first prediction result and a second prediction result. The first prediction result represents the fault type of the target server, and the second prediction result represents the potential fault propagation path of the target server in the target cluster. A dynamic adjustment strategy is determined based on the health prediction result, and corresponding dynamic adjustment operations are executed according to the dynamic adjustment strategy to complete the dynamic health management of the target cluster.

[0015] Specifically, multi-source data is transformed into a first feature sequence to achieve multimodal data fusion; the first feature sequence is input into a health management model to obtain health prediction results, which include at least a first prediction result and a second prediction result. The first prediction result is used to represent the fault type of the target server, and the second prediction result is used to represent the potential fault propagation path of the target server in the target cluster, enabling the health management model to support cross-layer fault location; a dynamic adjustment strategy is determined based on the health prediction results, and the corresponding dynamic adjustment operation is executed according to the dynamic adjustment strategy to complete the dynamic health management of the target cluster, thereby enabling the target server to be adjusted in advance to ensure the health status of the target cluster.

[0016] In practice, hardware metrics such as processor temperature, memory voltage, and fan speed are collected in real time. Hardware fault logs, such as hard drive health data and memory ECC (Error Correcting Code) error counts, are obtained through the server controller module based on the intelligent platform management interface protocol. Network traffic is collected through the mirror port to extract network metrics such as TCP (Transmission Control Protocol) connection count and throughput.

[0017] Further, the multi-source data is preprocessed to obtain a first feature sequence, including: denoising the sensor data in the multi-source data to obtain a first feature sub-sequence; extracting log data in the multi-source data to obtain a second feature sub-sequence; normalizing the network data in the multi-source data to obtain a third feature sub-sequence; encoding the timestamps in the multi-source data to obtain a fourth feature sub-sequence; encoding the hardware types in the multi-source data to obtain a fifth feature sub-sequence; and concatenating the first, second, third, fourth, and fifth feature sub-sequences to obtain the first feature sequence.

[0018] Specifically, the feature subsequence includes at least multiple time steps and multiple feature sub-vectors, with each feature sub-vector corresponding one-to-one with a time step. The first feature sequence is composed of a first feature subsequence, a second feature subsequence, a third feature subsequence, a fourth feature subsequence, and a fifth feature subsequence. The first feature subsequence corresponds to sensor data, the second feature subsequence corresponds to log data, the third feature subsequence corresponds to network data, the fourth feature subsequence corresponds to timestamps, and the fifth feature subsequence corresponds to hardware type, thus achieving the fusion of multi-source data.

[0019] In the specific implementation, the code for preprocessing multi-source data is as follows: `def preprocess_data(sensor_data,log_data,network_data): sensor_features=DAE(sensor_data)# Sensor data noise reduction, input dimension 20, output dimension 16; log_tokens=BERT_tokenizer(log_data)# Log semantic extraction, generating a 50-dimensional semantic sequence; network_features=minmax_scale(network_data,0,1)# Network metric normalization, input dimension 10, output dimension 10; time_features=encode_timestamp(timestamp)# Time features (24 dimensions), hourly one-hot encoding (24); hardware_features=encode_hardware_type()# Hardware type features (28 dimensions), server model embedding sequence (28); return` `np.concatenate([sensor_features,log_tokens,network_features,time_feature,hardware_features])` # Concatenates features to generate a 128-dimensional feature sequence. A lightweight model is used to clean and extract features from multi-source data, generating the first feature sequence, which can be a 128-dimensional standardized feature sequence.

[0020] Further, the first feature sequence is input into the health management model to obtain a health prediction result, including: inputting the first feature sequence into the temporal convolutional layer of the health management model to obtain a second feature sequence; inputting the second feature sequence into the long short-term memory layer of the health management model to obtain a final hidden state; and determining a first prediction result based on the final hidden state. The first feature sequence includes at least multiple time steps and multiple first feature vectors, with each first feature vector corresponding to one of the multiple time steps, and the dimension of each first feature vector being a first dimension value. The second feature sequence includes at least multiple time steps and multiple second feature vectors, with each second feature vector corresponding to one of the multiple time steps, and the dimension of each second feature vector being a second dimension value, where the second dimension value is less than the first dimension value.

[0021] Specifically, the temporal convolutional layer is responsible for "seeing" the details of each instant and local pattern, while the long short-term memory layer is responsible for "thinking" about the connections between these details, ultimately forming an overall "view." Finally, a conclusion is drawn based on this "view," including the specific type of fault. This achieves multimodal data fusion and improves the predictive accuracy of the health management model.

[0022] In the specific implementation, the first dimension has a value of 128, and the second dimension has a value of 16. The input layer receives the first feature sequence, with a time window set to 72 hours, a resolution of 5 minutes, and a total of 864 time steps. The temporal convolutional layer consists of three layers with kernel sizes of 8, 4, and 2, and dilation coefficients of 2^0, 2^1, and 2^2, capturing hardware state changes at different time scales. The long short-term memory layer has 256 memory units, extracting long-distance dependent features, such as the impact of continuously increasing memory utilization on hardware lifespan. The output layer maps the 256-dimensional vector of the penultimate time step to a 4-dimensional vector through one or more fully connected layers, specifically including hard disk, memory, processor, and power dimensions. An activation function converts the values ​​of the 4-dimensional vector into a probability distribution, with the sum of the four probability values ​​being 1. Each probability value represents the probability that the sample belongs to a hard disk failure, memory failure, processor failure, or power failure. The category with the highest probability value is taken as the health prediction result, achieving multi-class fault diagnosis. This health management model supports prediction up to 72 hours in advance.

[0023] Further, the first feature sequence is input into the temporal convolutional layer of the health management model to obtain the second feature sequence, including: inputting the first feature sequence into the first temporal convolutional network of the temporal convolutional layer to obtain the first intermediate sequence, wherein the size of the first convolutional kernel corresponding to the first temporal convolutional network is the first convolutional value, the size of the corresponding first dilation coefficient is the first dilation value, and the size of the corresponding first number of channels is the first channel value; inputting the first intermediate sequence into the second temporal convolutional network of the temporal convolutional layer to obtain the second intermediate sequence, wherein the size of the second convolutional kernel corresponding to the second temporal convolutional network is the second convolutional value, and the size of the corresponding second dilation coefficient is the second... The second convolution value corresponds to the second channel number, and the second convolution value is less than the first convolution value, the second dilation value is greater than the first dilation value, and the number of second channels is less than the number of first channels. The second intermediate sequence is input into the third temporal convolutional network of the temporal convolutional layer to obtain the second feature sequence. In the third temporal convolutional network, the size of the third convolution kernel is the third convolution value, the size of the corresponding third dilation coefficient is the third dilation value, the size of the corresponding third channel number is the third channel value, and the third convolution value is less than the second convolution value, the third dilation value is greater than the second dilation value, the number of third channels is equal to the value of the second dimension, and the number of third channels is less than the number of second channels.

[0024] Specifically, by using dilated convolution, a large receptive field is obtained with only 3 layers of temporal convolutional network, avoiding the need for extremely deep networks like ordinary convolution, improving computational efficiency and effectively alleviating the gradient vanishing problem.

[0025] In the specific implementation, the first layer of the temporal convolutional network has a first convolution value of 8, a first dilation value of 1, and a first channel value of 64. Therefore, the receptive field is (8-1)*1+1=8, meaning this layer can see the original input range of 8 time steps, focusing on capturing short-term, fine-grained patterns, such as transient spikes and short-term cycles (I / O bursts within a few minutes). The second layer of the temporal convolutional network has a second convolution value of 4, a second dilation value of 2, and a second channel value of 32. Therefore, the receptive field is (4-1)*2+1=7. The first layer can see the original input range of 7*8=56 time steps, capturing intermediate patterns, such as a high load state lasting half an hour; the third layer is a temporal convolutional network: the third convolution value is 2, the third dilation value is 4, the third channel value is 16, and the receptive field is (2-1)*4+1=5, that is, the original input range that this layer can see is 5*7*8=280 time steps, capturing long-term, coarse-grained trend changes, such as the slow increase in utilization caused by memory leaks over several hours, and the gradual degradation of hard disk performance. That is, the temporal convolutional layer transforms the original 128-dimensional feature vector into a 16-dimensional high-level feature vector at each time step. These 16 features can be understood as abstract features extracted from the original 128 features that are more representative of the hardware state, such as: "short-term load pattern", "intermediate trend strength", "abnormal fluctuation index", etc. The temporal dependence in the sequence is preserved in the feature map.

[0026] Further, the second feature sequence is input into the long short-term memory layer of the health management model to obtain the final hidden state, including: transposing the second feature sequence to obtain a third feature sequence, wherein the third feature sequence includes at least multiple time steps and multiple third feature vectors, and the multiple third feature vectors correspond one-to-one with the multiple time steps; calculating the target hidden state and the target cell state based on the third feature sequence, wherein the target hidden state is the hidden state of the feature vector corresponding to the penultimate time step, and the target cell state is the cell state of the feature vector corresponding to the penultimate time step; obtaining the target feature vector in the third feature sequence, wherein the target feature vector is the feature vector corresponding to the penultimate time step; and calculating the final hidden state based on the target feature vector, the target hidden state, and the target cell state.

[0027] Specifically, the Long Short-Term Memory (LSTM) layer can be used to identify, for example, long-term correlations between “an anomalous memory access at time step 200” and “a surge in memory failure probability starting at time step 800”, integrating contextual information from the entire sequence.

[0028] In the specific implementation, the second feature sequence (1, 16, 864) is transposed into the third feature sequence (1, 864, 16). The Long Short-Term Memory (LSTM) layer processes the third feature sequence step by step from t=1 to t=864. For each time step t, the LSM layer does the following: a. Receive the current input: the 16-dimensional feature vector of the t-th time step; b. Receive the hidden state h_{t-1} and cell state c_{t-1} of the previous time step (for t=1, these states are usually initialized to zero); c. Calculate the new hidden state h_t and the new cell state c_t through its internal gating mechanism (input gate, forget gate, output gate); d. Output the hidden state h_t of the current time step; The final hidden state is generated when the LSM layer processes the last time step (t=864), completing the "reading" of the entire sequence. At this point, the cell state c_{864} inside the Long Short-Term Memory (LSTM) layer has accumulated all the important information from step 1 to step 864. Based on this final cell state and the final input, the LSM layer calculates the final hidden state h_{864}. h_{864} is the "final hidden state" we are looking for; it is a tensor of shape (1, 256).

[0029] Further, based on the final hidden state, the first prediction result is determined, including: inputting the final hidden state into the fully connected layer in the health management model to obtain a fourth feature vector, wherein the dimension of the final hidden state is the third dimension value, the dimension of the fourth feature vector is the fourth dimension value, the fourth dimension value is the same as the number of fault types of the target server, and the fourth dimension value is less than the third dimension value; converting the fourth feature vector into a probability distribution vector through an activation function, wherein the probability distribution vector includes at least multiple probability values, and the multiple probability values ​​correspond one-to-one with multiple fault types of the target server; and determining the fault type with the highest probability value as the first prediction result.

[0030] Specifically, the temporal convolutional layer plus the long short-term memory layer is used for vertical depth analysis. It answers the question, "Based on my own historical data, what is the probability of me experiencing a failure in the future? And the root cause of the failure is likely to be the hard drive, memory, processor, or power supply, so it is recommended to check these first."

[0031] Furthermore, inputting the first feature sequence into the health management model to obtain a health prediction result also includes: constructing an association graph based on the graph neural network layer in the health management model; calculating the first prediction result using a propagation algorithm based on the association graph to obtain a second prediction result, wherein the second prediction result includes at least a potential fault propagation path, the potential fault propagation path includes at least a potential fault propagation direction and a set of potential fault servers, the set of potential fault servers includes at least multiple potential fault servers, and the multiple potential fault servers are located in the same cluster as the target server; and obtaining a health prediction result based on the first prediction result and the second prediction result.

[0032] In practical implementation, a federated learning framework is used to aggregate feature sequences and feature vectors from multiple servers. This aggregated data is then used to train the health management model, enabling the creation of a powerful global health management model capable of predicting all types of hardware failures without aggregating any sensitive monitoring data. Health prediction results include the probability of a target server failing in the future, identifying the root cause (hard drive, memory, processor, or power supply), and determining whether the root cause of the target server's failure lies with other servers and whether the target server will affect other servers. Specifically, the federated learning framework combined with a graph neural network layer is used for lateral correlation analysis. This reveals the dependencies between servers in the cluster and simulates how failures propagate through these relationships, thereby locating the root cause and assessing the scope of impact, answering the questions: "Is the anomaly of other devices affecting me, and will I affect other devices?" Therefore, the temporal convolutional layer + long short-term memory layer and the federated learning framework + graph neural network layer are complementary and reinforcing.

[0033] In its implementation, the propagation algorithm is an improvement on the webpage ranking algorithm. If the current server meets the propagation algorithm trigger conditions, such as the current server's hardware temperature > 85℃ and fan speed > 90%, the risk score of adjacent nodes is updated through the propagation algorithm to identify potential fault propagation paths. The formula for the propagation algorithm is: R_new = d * A^T * R_old + (1-d) * S, where R_old and R_new represent the risk score vectors of the current and next iterations; A^T is the transpose of the normalized adjacency matrix, and normalization is to ensure that the sum of the weights of the outgoing edges is 1; d (damping coefficient): usually taken as 0.85, representing the probability that a node will propagate along an edge; S (personalized vector): S[i] = 1.0, if node i is the currently detected anomaly source, S[i] = 0, for all other nodes. In this way, the risk value will start from the anomaly source node S and "spread" to neighboring nodes along the high-weight edges (i.e., connections with large network bandwidth and strong business dependence). The above formula is iterated and calculated multiple times until the risk score vector R tends to stabilize. Ultimately, each node will receive a converged risk score, and the higher the score, the greater the risk of being affected by the initial anomaly source.

[0034] Furthermore, based on the graph neural network layer in the health management model, an association graph is constructed, including: identifying multiple servers in the target cluster as multiple graph nodes in the association graph, where the target server is located in the target cluster, and multiple servers correspond one-to-one with multiple graph nodes; determining multiple graph edges in the association graph based on the network connection relationship or business dependency relationship between multiple servers; and constructing the association graph based on the multiple graph nodes and multiple graph edges. In specific implementation, graph nodes at least include the attribute information of the corresponding server, including at least hardware model, runtime, and historical fault records. The weights of graph edges represent network bandwidth and business dependency, etc., and are stored using an adjacency matrix. Constructing the adjacency matrix A: Create an N*N matrix, where N is the total number of nodes. If there is an edge between nodes i and j, then A[i][j] = comprehensive weight; otherwise, it is 0. This matrix is ​​usually sparse. Constructing the node feature matrix X: Create an N*F matrix, where F is the number of features. Each row represents the feature vector of a node, and features at different scales need to be normalized. For example, the feature vector may include: [runtime, number of historical failures, current temperature, current fan speed, processor utilization], and these features need to be scaled to the [0, 1] range.

[0035] Furthermore, the health management method for server clusters also includes: calculating the prediction accuracy of the health management model and calculating the prediction accuracy reward value based on the prediction accuracy; calculating the power consumption reduction ratio of the health management model and calculating the energy efficiency reward value based on the power consumption reduction ratio; calculating the repair time reduction ratio of the health management model and calculating the operation and maintenance efficiency reward value based on the repair time reduction ratio; calculating the resource consumption ratio of the health management model and calculating the action stability penalty value based on the resource consumption ratio; calculating the reward function value of the health management model based on the prediction accuracy reward value, energy efficiency reward value, operation and maintenance efficiency reward value, and action stability penalty value; and adjusting the health management model based on the reward function value until the adjusted reward function value is the maximum value of the health management model's reward function. Specifically, by adjusting the parameters of the health management model through the reward function, the health management model can balance performance, energy consumption, operation and maintenance efficiency, and cost, achieving "optimal" decision-making.

[0036] In practical implementation, the formula for the reinforcement learning reward function is: R = 0.4 × (1 - Pfail) + 0.3 × Energy_saving + 0.2 × MTTR_reduction - 0.1 × Action_cost, where Pfail is the failure prediction error rate of the health management model on the validation set; Energy_saving is the power consumption reduction ratio, such as 0.1 if power consumption decreases by 10% after dynamic voltage adjustment; MTTR_reduction is the repair time reduction ratio, representing the average repair time reduced due to predictive maintenance, such as 0.5 if it decreases from 30 minutes to 15 minutes; and Action_cost is the resource consumption ratio for performing dynamic adjustment operations, such as 0.02 if virtual machine migration requires 2GB of network bandwidth.

[0037] In practical implementation, the power consumption reduction ratio can be indirectly defined as 1 - (inference time / baseline time). Define the baseline: select a reference model, such as the initial model or the best model in the previous round, and calculate the first average inference time on representative data, i.e., the baseline time. Measure the current value: use the new model after adjusting the parameters using reinforcement learning, and calculate the second average inference time on the same data, i.e., the inference time.

[0038] Furthermore, the health management method for server clusters also includes: executing corresponding dynamic adjustment operations according to a dynamic adjustment strategy and obtaining dynamic adjustment results at a preset frequency; comparing the dynamic adjustment results with the expected adjustment results in the dynamic adjustment strategy to obtain the strategy execution error; adjusting the model weights of the prediction model in the health management model according to the backpropagation algorithm based on the strategy execution error, wherein the prediction model consists of an input layer, a temporal convolutional layer, a long short-term memory layer, and an output layer; and updating the policy network in the health management model according to the reward function based on the strategy execution error, wherein the policy network includes at least multiple dynamic adjustment strategies.

[0039] Specifically, based on the policy execution error, the parameters of the prediction model and policy network are adjusted through the backpropagation algorithm and reward function to achieve closed-loop optimization of the health management model, thereby improving the prediction accuracy and policy effectiveness of the health management model.

[0040] In the specific implementation: 1. Hardware Deployment: 1) Edge agents were installed on 3,000 critical business servers to collect data on processor temperature, hard drive health status, network I / O throughput, and more than 30 other indicators; 2) A 50-node graphics processing unit cluster was deployed in the cloud to build a federated learning framework and aggregate the training parameters of each edge node. 2. Model Training: 1) Historical Data Preprocessing: 3 million hardware logs from the past year were cleaned, 5,000 hard drive failure samples were labeled, and a balanced dataset with a positive to negative sample ratio of 1:10 was constructed; 2) Model Optimization: The cloud model was compressed to 10MB through knowledge distillation, and the local inference latency of the edge agent was ≤10ms, meeting the real-time early warning requirements. 3. Strategy Execution: 1) Fault Warning: When the hard drive bad sector count exceeds the threshold for three consecutive cycles (15 minutes), virtual machine migration is triggered, reducing downtime due to hard drive failure by 78% year-on-year; 2) Energy Consumption Optimization: Fan speed is dynamically adjusted based on processor temperature and load, reducing average fan power consumption in summer from 40% to 25%, and data center energy efficiency from 1.5 to 1.32, saving 3.2 million yuan in electricity costs annually. The health management model improves fault prediction accuracy while reducing mean time to repair (MTBT), the number of invalid alarms, and data center energy efficiency.

[0041] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed, and they can be performed in other orders. Furthermore, Figure 1At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0042] In the second embodiment, as Figure 2 As shown, a dynamic health management system for server clusters is provided. This system includes an edge agent cluster, a cloud agent platform, and a visualization management platform. The cloud agent platform includes at least a data fusion agent, a model training agent, and a decision execution agent. The edge agent cluster converts collected multi-source data into a first feature sequence and transmits this sequence to the cloud agent platform. The data fusion agent aggregates feature vectors from the edge agent cluster using a federated learning framework and constructs an association graph using a graph neural network to identify potential fault propagation paths. The model training agent builds a prediction model based on temporal convolutional layers and long short-term memory layers, and dynamically adjusts the model parameters using a backpropagation algorithm. The decision execution agent pushes health prediction results to the hardware management module or cloud management platform to perform corresponding dynamic adjustment operations and adjusts the parameters of the policy network based on the dynamic adjustment results. The visualization management platform displays the health status of the target cluster based on digital twin technology. The health status includes at least potential fault propagation paths, health prediction results, and dynamic adjustment strategies.

[0043] In practical implementation, the edge agent cluster collects data at 50ms intervals, filters outliers through a noise reduction autoencoder, extracts log keywords through natural language processing, and generates feature vectors with uniform dimensions. The cloud agent also uses federated learning to collaboratively train the prediction model on edge nodes and in the cloud, and optimizes the policy network by combining reinforcement learning reward functions. The decision-making and execution agent automatically adjusts hardware parameters, such as dynamic voltage and frequency adjustment, based on the health prediction results, and updates the health management model through the error backpropagation algorithm, forming a closed-loop system of "monitoring-prediction-control-optimization".

[0044] In practical implementation, edge agent clusters are deployed at the edge of server hardware or server racks. Edge agents are designed as modular circuit boards, deployed via expansion cards, supporting hot-swappable replacement, and compatible with mainstream server models. The cloud agent platform architecture is as follows: Figure 3As shown, it includes: a computing layer, deploying a cluster of graphics processing units to support federated learning parameter aggregation and parallel training of graph neural networks, with computing power utilization dynamically scheduled through a container cluster management platform; a storage layer, employing a distributed storage system to store time-series data uploaded by the edge agent cluster, retaining 7 days of raw data and 30 days of feature data; a network layer, building a software-defined network, with the edge agent cluster and the cloud agent platform connected via dedicated lines, supporting encrypted data transmission; and a software stack, building a model training platform based on an open-source machine learning architecture, implementing distributed training through a deep learning framework, deploying model inference services through a remote procedure call framework, monitoring the status of the model training platform through open-source monitoring tools and a front-end display platform, orchestrating federated learning tasks through a scheduling and monitoring workflow platform, and managing the model lifecycle through a lifecycle management platform, supporting automatic model version iteration.

[0045] In specific implementation, the dynamic adjustment strategy includes: controller control, which sends fan speed adjustment commands through the intelligent platform management protocol, such as increasing the speed to 80% when the temperature is >80℃, or hardware self-test commands, which trigger a hard drive health status check once a week; cloud platform linkage, which calls the cloud platform interface to realize virtual machine hot migration, and when the predicted hard drive failure probability is >80%, the migration is completed within 5 minutes, and the migration interruption time is ≤200ms.

[0046] In practical implementation, closed-loop optimization includes: policy execution feedback, with edge agents reporting hardware status changes every 10 minutes, such as temperature fluctuations after fan speed adjustment, and cloud agents calculating policy execution errors, such as the deviation between the actual temperature drop and the expected drop; model parameter updates, adjusting the prediction model weights through the error backpropagation algorithm, with reinforcement learning agents updating the policy network according to the reward function, and completing a global model iteration every 24 hours; and dynamic threshold calibration, automatically adjusting fault warning thresholds based on historical data statistics to adapt to hardware aging characteristics, such as dynamically reducing the temperature warning threshold of old server processors from 85℃ to 80℃.

[0047] In practical implementation, a visual management platform, such as Figure 4 As shown, it includes a data layer, a data processing layer, a model service layer, a business logic layer, a visualization layer, and a user interaction layer. The visualization management platform is used to store data from the edge intelligent agent cluster and the cloud intelligent agent platform, provide fault warnings, and display the data to users.

[0048] Specific limitations regarding the dynamic health management system for server clusters can be found in the above description of the dynamic health management methods for server clusters, and will not be repeated here. Each module in the aforementioned dynamic health management system for server clusters can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in an electronic device, or stored in the memory of an electronic device in software form, so that the processor can call and execute the corresponding operations of each module.

[0049] In a third embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: collecting multi-source data from a target server; preprocessing the multi-source data to obtain a first feature sequence; inputting the first feature sequence into a health management model to obtain a health prediction result, wherein the health prediction result includes at least a first prediction result and a second prediction result, the first prediction result representing the fault type of the target server, and the second prediction result representing the potential fault propagation path of the target server in the target cluster; determining a dynamic adjustment strategy based on the health prediction result, and executing the corresponding dynamic adjustment operation according to the dynamic adjustment strategy to complete the dynamic health management of the target cluster.

[0050] When the program instructions are read and executed by one or more processors, they can also perform operations corresponding to the steps in the above method embodiments, as described above, and will not be repeated here. Reference Figure 5 This exemplifies the architecture of an electronic device, which may include a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, and a memory 520. The processor 510, video display adapter 511, disk drive 512, input / output interface 513, network interface 514, and memory 520 can communicate with each other via a communication bus 530.

[0051] The processor 510 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in this application.

[0052] The memory 520 can be implemented as a read-only memory (ROM), random access memory (RAM), static storage device, dynamic storage device, etc. The memory 520 can store an operating system 521 for controlling the operation of the electronic device 500, and a basic input / output system (BIOS) 522 for controlling the low-level operations of the electronic device 500. Additionally, it can store a web browser 523, data storage management 524, and an icon / font processing system 525, etc. The aforementioned icon / font processing system 525 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 520 and executed by the processor 510.

[0053] Input / output interface 513 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0054] Network interface 514 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0055] Bus 530 includes a pathway for transmitting information between various components of the device, such as processor 510, video display adapter 511, disk drive 512, input / output interface 513, network interface 514, and memory 520.

[0056] In addition, the electronic device 500 can also obtain information on specific acquisition conditions from a virtual resource object acquisition condition information database (not shown in the figure) for condition judgment, etc.

[0057] It should be noted that although the above-described electronic device 500 only shows a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, a memory 520, and a bus 530, in specific implementations, the electronic device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0058] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause an electronic device (which may be a personal computer, cloud server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0059] In a fourth embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: collecting multi-source data from a target server; preprocessing the multi-source data to obtain a first feature sequence; inputting the first feature sequence into a health management model to obtain a health prediction result, wherein the health prediction result includes at least a first prediction result and a second prediction result, the first prediction result being used to represent the fault type of the target server, and the second prediction result being used to represent the potential fault propagation path of the target server in the target cluster; determining a dynamic adjustment strategy based on the health prediction result, and executing the corresponding dynamic adjustment operation according to the dynamic adjustment strategy to complete the dynamic health management of the target cluster.

[0060] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0061] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0062] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application.

Claims

1. A method for dynamic health management of a server cluster, characterized in that, The method includes: Collect multi-source data from the target server; The multi-source data is preprocessed to obtain a first feature sequence; The first feature sequence is input into the health management model to obtain a health prediction result, wherein the health prediction result includes at least a first prediction result and a second prediction result, the first prediction result is used to represent the fault type of the target server, and the second prediction result is used to represent the potential fault propagation path of the target server in the target cluster; A dynamic adjustment strategy is determined based on the health prediction results, and corresponding dynamic adjustment operations are executed according to the dynamic adjustment strategy to complete the dynamic health management of the target cluster.

2. The method according to claim 1, characterized in that, The preprocessing of the multi-source data to obtain the first feature sequence includes: The sensor data in the multi-source data is subjected to noise reduction processing to obtain the first feature subsequence; The log data from the multi-source data is extracted and processed to obtain the second feature subsequence; The network data in the multi-source data is normalized to obtain the third feature subsequence; The timestamps in the multi-source data are encoded to obtain the fourth feature subsequence; The hardware type in the multi-source data is encoded to obtain the fifth feature subsequence; The first feature sequence is obtained by concatenating the first feature subsequence, the second feature subsequence, the third feature subsequence, the fourth feature subsequence, and the fifth feature subsequence.

3. The method according to claim 1, characterized in that, The step of inputting the first feature sequence into the health management model to obtain health prediction results includes: The first feature sequence is input into the temporal convolutional layer of the health management model to obtain the second feature sequence; The second feature sequence is input into the long short-term memory layer of the health management model to obtain the final hidden state; The first prediction result is determined based on the final hidden state; The first feature sequence includes at least multiple time steps and multiple first feature vectors, each of which corresponds to one of the multiple time steps. The dimension of each of the multiple first feature vectors is a first dimension value. The second feature sequence includes at least multiple time steps and multiple second feature vectors, each of which corresponds to one of the multiple time steps. The dimension of each of the multiple second feature vectors is a second dimension value, and the second dimension value is less than the first dimension value.

4. The method according to claim 3, characterized in that, The step of inputting the first feature sequence into the temporal convolutional layer of the health management model to obtain the second feature sequence includes: The first feature sequence is input into the first temporal convolutional network of the temporal convolutional layer to obtain the first intermediate sequence, wherein the size of the first convolutional kernel corresponding to the first temporal convolutional network is the first convolutional value, the size of the corresponding first dilation coefficient is the first dilation value, and the size of the corresponding first number of channels is the first channel value. The first intermediate sequence is input into the second temporal convolutional network of the temporal convolutional layer to obtain the second intermediate sequence. The size of the second convolutional kernel corresponding to the second temporal convolutional network is the second convolutional value, the size of the corresponding second dilation coefficient is the second dilation value, the size of the corresponding second number of channels is the second channel value, and the second convolutional value is less than the first convolutional value, the second dilation value is greater than the first dilation value, and the second number of channels is less than the first number of channels. The second intermediate sequence is input into the third temporal convolutional network of the temporal convolutional layer to obtain the second feature sequence. The size of the third convolutional kernel corresponding to the third temporal convolutional network is the third convolutional value, the size of the corresponding third dilation coefficient is the third dilation value, the size of the corresponding third number of channels is the third channel value, and the third convolutional value is less than the second convolutional value, the third dilation value is greater than the second dilation value, the third number of channels is equal to the value of the second dimension, and the third number of channels is less than the second number of channels.

5. The method according to claim 3, characterized in that, The step of inputting the second feature sequence into the long short-term memory layer of the health management model to obtain the final hidden state includes: The second feature sequence is transposed to obtain a third feature sequence, wherein the third feature sequence includes at least multiple time steps and multiple third feature vectors, and the multiple third feature vectors correspond one-to-one with the multiple time steps; Based on the third feature sequence, the target hidden state and the target cell state are calculated, wherein the target hidden state is the hidden state of the feature vector corresponding to the second to last time step, and the target cell state is the cell state of the feature vector corresponding to the second to last time step; Obtain the target feature vector in the third feature sequence, wherein the target feature vector is the feature vector corresponding to the penultimate time step; The final hidden state is calculated based on the target feature vector, the target hidden state, and the target cell state.

6. The method according to claim 3, characterized in that, Determining the first prediction result based on the final hidden state includes: The final hidden state is input into the fully connected layer in the health management model to obtain the fourth feature vector. The dimension of the final hidden state is the third dimension value, and the dimension of the fourth feature vector is the fourth dimension value. The fourth dimension value is the same as the number of fault types of the target server, and the fourth dimension value is less than the third dimension value. The fourth feature vector is transformed into a probability distribution vector by an activation function, wherein the probability distribution vector includes at least a plurality of probability values, and the plurality of probability values ​​correspond one-to-one with a plurality of fault types of the target server. The fault type with the highest probability value is determined as the first prediction result.

7. The method according to claim 3, characterized in that, The step of inputting the first feature sequence into the health management model to obtain health prediction results further includes: Based on the graph neural network layer in the health management model, an association graph is constructed; Based on the association graph, the first prediction result is calculated using a propagation algorithm to obtain the second prediction result. The second prediction result includes at least a potential fault propagation path, which includes at least a potential fault propagation direction and a set of potential fault servers. The set of potential fault servers includes at least multiple potential fault servers, and the multiple potential fault servers are located in the same cluster as the target server. The health prediction result is obtained based on the first prediction result and the second prediction result.

8. The method according to claim 7, characterized in that, The step of constructing an association graph based on the graph neural network layer in the health management model includes: determining multiple servers in the target cluster as multiple graph nodes in the association graph, wherein the target server is located in the target cluster, and the multiple servers correspond one-to-one with the multiple graph nodes; Based on the network connection relationship or business dependency relationship between the multiple servers, determine multiple graph edges in the association graph; The associated graph is constructed based on the plurality of graph nodes and the plurality of graph edges.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the dynamic health management method for a server cluster as described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the dynamic health management method for the server cluster as described in any one of claims 1 to 8.

Images