Cloud platform intelligent monitoring and fault early warning method based on real-time data flow

Abstract

The invention discloses a cloud platform intelligent monitoring and fault early warning method based on a real-time data stream, and the method comprises the steps: constructing a four-dimensional data collection architecture through employing a mixed collection mode of a distributed probe and an asynchronous queue, and constructing a multi-dimensional monitoring index system in combination with a data fusion engine with a three-stage data alignment mechanism; in a multi-dimensional monitoring index system, operation is executed as follows: S1, edge calculation preprocessing operation including dynamic abnormal value filtering and multi-dimensional feature screening optimization is executed on monitoring data; s2, constructing an LSTM-Attention network early warning model, and optimizing a model structure based on a dynamic threshold value; s3, performing three-dimensional topology visualization on the monitoring data by using the optimized network early warning model, and calculating the health degree of the monitoring data by using a state mapping rule; and S4, executing data stream automatic order sending operation based on a visualization result. The method can improve the recognition accuracy of the abnormal state of the system, reduces the false alarm rate, and reduces the invalid consumption of operation and maintenance resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of monitoring data processing, and in particular to a cloud platform intelligent monitoring and fault early warning method based on real-time data flow, a device and a storage medium. BACKGROUND

[0002] Under the background of rapid development of informatization and digitization, the stable operation of various business systems (such as financial transaction systems, industrial internet platforms, etc.) increasingly relies on effective monitoring mechanisms.

[0003] The existing monitoring system is mostly based on traditional threshold detection technology, which collects and compares the data generated by the system in real time by pre-setting the threshold range of each monitoring indicator (such as transaction response time, equipment operating parameters, etc.), and triggers an alarm when the data exceeds the threshold, thereby realizing the monitoring and early warning of system abnormal state and providing a basis for system operation and fault handling.

[0004] However, in actual use, the existing monitoring system still has the following technical bottlenecks: (1) Significant lag, the traditional threshold alarm mechanism has a delay of more than 500ms in identifying and responding to abnormal data, which is difficult to meet the millisecond-level response scenarios with extremely high real-time requirements such as financial transactions; (2) High false alarm rate, as only a single indicator threshold is used for judgment, there is a lack of comprehensive analysis of multi-dimensional data, resulting in a false alarm rate of more than 35%, for example, a bank system has more than 2000 daily false alarm times, which seriously consumes operation and maintenance resources; (3) Lack of prediction ability, the existing technology can only passively alarm the occurred abnormality, and cannot predict potential system faults in advance, resulting in a passive state of fault handling, such as the average fault repair time (MTTR) of an industrial internet platform being as long as 48 minutes, affecting the stable operation and business continuity of the system.

[0005] Therefore, the present application proposes a cloud platform intelligent monitoring and fault early warning method based on real-time data flow to solve the above technical problems. SUMMARY

[0006] The main purpose of the present application is to provide a cloud platform intelligent monitoring and fault early warning method based on real-time data flow to solve the technical problems proposed in the background art.

[0007] The present application solves the above technical problems by adopting the following technical solutions: A cloud platform intelligent monitoring and fault early warning method based on real-time data flow adopts a hybrid collection mode of distributed probes and asynchronous queues to construct a four-dimensional data collection architecture, combines a data fusion engine with a three-level data alignment mechanism to construct a multi-dimensional monitoring index system, and performs the following operations in the multi-dimensional monitoring index system: S1. Perform edge computing preprocessing operations such as dynamic outlier filtering and multi-dimensional feature screening optimization on the monitoring data; S2. Construct an LSTM-Attention network early warning model, and optimize the model structure based on a dynamic threshold; S3. Use the optimized network early warning model to perform three-dimensional topological visualization on the preprocessed monitoring data, and calculate the health degree of the monitoring data using state mapping rules; S4. Perform data stream automatic dispatching operations based on the visualization results.

[0008] Preferably, the multi-dimensional monitoring index system includes: A four-dimensional data collection architecture that collects four-dimensional data including the time consumed by the CPU in user mode or system mode, the total time length of the specified time slice, the data volume and the received volume of the GPU in the specified time window, and is used to calculate the CPU core utilization and the GPU memory bandwidth; A data fusion engine with a set of three-level data alignment mechanisms, including: (1) Time window alignment is achieved through sliding window aggregation; (2) Time sequence synchronization is achieved based on NTP protocol clock offset compensation; (3) Multi-feature fusion is achieved by using D-S evidence theory to integrate multi-source data. Preferably, the S1 step further performs a dynamic window adjustment operation before performing dynamic outlier filtering, and the window size is dynamically adjusted and outputted by inputting the current load level of the system using the following adjustment formula:

[0009] Wherein, The value range of is 0-2, 0 represents low load, and 2 represents high load. The window size is limited within a specified range to ensure the calculation stability under extreme load.

[0010] Preferably, the specific operation process of the dynamic outlier filtering in the S1 step includes: Based on the time sequence data in the sliding window, the mean value of the data in the window is calculated And the standard deviation , which are:

[0011] Wherein, The amount of data within the window. For the first in the window One data point; Outlier detection is based on the 3σ principle. If a data point meets the following condition: If an anomaly is detected within the window, the time-series data will be filtered out.

[0012] Preferably, the specific optimization operations for multidimensional feature screening in step S1 include: Construct multidimensional feature variables and use mutual information to evaluate features among the multidimensional feature variables. With target variable The shared information is used to filter redundant features, where the mutual information filtering formula is:

[0013] in, Indicates evaluation characteristics With target variable The joint probability distribution, To evaluate features The probability distribution, For target variable The probability distribution; The first 18 dimensions of features based on the MI value are retained during the screening process.

[0014] Preferably, the multidimensional feature variables include: (a) The second difference of the current acceleration derivative is used as a trend characteristic variable to capture sudden trends in the index and identify sudden faults. The calculation formula is as follows:

[0015] in, Representing the current time The second difference of the derivative of acceleration, It is a smoothing factor, and ; (b) Using the entropy value calculated from the dispersion of the current sample distribution as a statistical feature variable to quantify the dispersion of the data distribution and measure the randomness of the system behavior, we have:

[0016] in, The entropy value is represented by the calculated result of the current distribution dispersion. For the first The probability distribution of discrete values; (c) the current dynamic Pearson coefficient with time-varying window is used as the correlation feature variable, which is used to measure the time-varying correlation of two groups of specified variable indicators, to find the potential fault propagation path, and has:

[0017] wherein, denotes the current time the time-varying correlation of the specified variables X and Y, is the covariance of the specified variables X and Y, respectively, the standard deviation of the specified variables X and Y.

[0018] Preferably, the LSTM-Attention network early warning model structure in the S2 step comprises, in sequence and in layers: an input layer receiving time series data with a shape of (time step, feature number), wherein the feature number is the feature dimension contained in each time step; a first layer of LSTM containing 256 hidden units, used to output the result of each time step for use by the next layer; a Dropout layer used to randomly discard a specified proportion of neuron connections to prevent overfitting; a second layer of LSTM containing 256 hidden units, used to output the result of each time step for use by the next layer; an attention layer used to calculate attention weights for the output of the second layer of LSTM, and convert features through a fully connected layer to obtain intermediate scores, and then compress them into attention weights for each time step through a fully connected layer, normalize the weights using softmax, obtain the proportion of importance of each time step, and weight-sum the features of all time steps to output a context vector; a fully connected layer provided with 64 units and using a relu activation function, used to extract nonlinear features; an output layer provided with 1 unit and using a sigmoid activation function, used to output a fault probability.

[0019] Preferably, the specific operation process of the threshold dynamic optimization model structure in the S2 step comprises: L1. Perform feature variable standardization operation, which has:

[0020] wherein, , are the sliding window statistics at the time the feature variable observation value at the time after standardization; L2. Perform weight training through an Adam optimizer, which has:

[0021] wherein, is the learning rate, which is initially 0.01, and adopts a decay strategy of multiplying 0.95 per epoch; L3. The threshold value is dynamically adjusted, and the formula is:

[0022] wherein, and is the load adaptive adjustment value, and is a specified numerical value, represents the calculation of the hyperbolic tangent function, represents the load function changing over time t.

[0023] Preferably, the model training in the S2 step adopts a loss function combining Focal Loss and a custom timing loss, and the formula is:

[0024] wherein, is the probability of the model predicting as abnormal, and are both weights for suppressing easily classified samples, and respectively represent two groups of modulation parameters of Focal Loss, is the weight of the timing smoothing loss.

[0025] Preferably, the dispatching strategy in the S4 step includes: calculating the severity score based on the failure influence domain, and the formula is:

[0026] wherein is the node importance score, is the failure propagation probability; then performing service priority scheduling based on the list of pending failure work orders, wherein each work order contains a severity score severity and a creation timestamp create_time, and wherein: the work orders are primarily dispatched according to severity, with urgent work orders being prioritized; and secondarily dispatched according to creation time, with larger timestamps indicating newer work orders and higher priority.

[0027] In another aspect, the present application also discloses a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to make the processor execute the steps of the above method.

[0028] In still another aspect, the present application also discloses a computer device comprising a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to make the processor execute the steps of the above method.

[0029] From the above technical solution, the present application provides a cloud platform intelligent monitoring and fault early warning method based on real-time data flow. Compared with the prior art, the present application has the following advantages: 1. The present application can comprehensively collect and integrate real-time data of each layer of the system by constructing a multi-source data fusion architecture, setting a multi-dimensional monitoring index system covering computing, storage, network and tasks, breaking the limitation of single index analysis, thereby improving the identification accuracy of abnormal state of the system, reducing the false alarm rate and reducing the invalid consumption of operation and maintenance resources.

[0030] 2. The present application can realize real-time processing nearby by sinking part of data processing and analysis tasks to edge nodes through the construction of edge-center collaborative computing system, greatly reducing the delay caused by data transmission and centralized processing, so as to meet the demand of millisecond level response scene with extremely high real-time requirement such as financial transaction.

[0031] 3. The present application can clearly show the warning basis while predicting potential system failure by constructing an explainable early warning model based on attention mechanism, thereby assisting operation and maintenance personnel to quickly understand and judge the failure cause, facilitating to improve the fault prediction accuracy and operation and maintenance decision efficiency, so as to realize the active prediction and rapid processing of system failure, and guarantee the stable operation and business continuity of the system.

[0032] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. Of course, any product implementing the present application does not necessarily need to achieve all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0033] The drawings accompanying the specification of this application form a part thereof, serve to provide further understanding of the present application, and together with the description of the exemplary embodiments of the present application and their description serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings: Fig. 1 It is a schematic diagram of the method operation flow of the present application; Fig. 2 It is a schematic diagram of the hybrid collection mode of the distributed probe + asynchronous queue of the present application. DETAILED DESCRIPTION

[0034] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The embodiments in the present application and the features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0035] In the embodiments, refer to Figs. 1-2 .

[0036] As shown in Fig. 1 , the cloud platform intelligent monitoring and fault early warning method based on real-time data flow proposed in the embodiments of the present application adopts a hybrid collection mode of distributed probes and asynchronous queues to construct a four-dimensional data collection architecture, and combines a data fusion engine with a three-level data alignment mechanism to construct a multi-dimensional monitoring index system. In the multi-dimensional monitoring index system: (A) Four-dimensional data collection architecture, refer to Fig. 2 , collect four-dimensional data including the time spent by CPU in user state or system state, the total time length of the specified time slice, the data amount and the receiving amount of GPU in the specified time window, and calculate the CPU core utilization rate and GPU memory bandwidth, which has: (A1) CPU core utilization rate is calculated by analyzing time slice occupancy rate through / proc / stat, which is used to quantify the actual occupancy rate of CPU on the computing task and reflect the computing resource tension degree, which has:

[0037] Among them, is the time spent by CPU in user state or system state in the ith time slice; is the total time length of the ith time slice (usually 1 second).

[0038] Here, / proc is a virtual file system (procfs) implemented by Linux kernel, which exposes the runtime state of kernel and system hardware information to user space in the form of files, so further, / proc / stat is one of the core files, which records the statistical information of CPU time allocation (unit: clock ticks, usually jiffies, 1 second = 100 jiffies, depending on system configuration).

[0039] (A2) GPU memory bandwidth reads fbmem_used_rate based on the NVML library, where fbmem_used_rate can be used to describe the proportion of the used capacity of the GPU frame buffer (FB) memory to the total capacity, and fbmem_used_rate, as one of the core indicators for measuring GPU memory pressure, can also be used to evaluate the real-time use state of GPU memory (such as whether it is facing the risk of insufficient memory), and then through the calculation of fbmem_used_rate combined with the memory transaction period, it is used to measure the data transmission efficiency between GPU and memory, reflecting the load pressure of GPU, which has:

[0040] wherein, is the amount of data sent by the GPU in the time window Δt, is the data received by the GPU in the time window Δt, where Δt (time window) as one of the core parameters for calculating the memory bandwidth, refers to the time interval between two data sampling, is used to measure the actual amount of data transmitted between GPU and memory in this period of time, and then calculate the bandwidth utilization or transmission efficiency of memory, so it can directly affect the real-time and accuracy of the results.

[0041] At this time, the precision control in the calculation process of CPU core utilization and GPU memory bandwidth is realized through Kalman filtering, and the following state equation is used to smooth the instantaneous fluctuations of GPU memory bandwidth and suppress noise interference:

[0042]

[0043]

[0044]

[0045] wherein, represents the estimated value of the system state at time k, A represents the state transition matrix (describing the dynamic characteristics of the system), B represents the control input matrix (the influence of external control signals), represents the control input at time k (such as memory allocation instructions), represents the estimated error covariance matrix (quantifying the uncertainty of state estimation), represents the Kalman gain (trade-off between predicted value and observed value), represents the observed value at time k (actual measured memory bandwidth), represents the process noise covariance matrix, which is used to describe the noise (such as system dynamic modeling error, randomness of load mutation) that is not captured by the model in the state transition process, represents the measurement matrix, which is used to map the state vector to the actual measurable signal space (such as converting abstract states to actual sampling values of the memory bandwidth), represents the measurement noise covariance matrix, which is used to describe the noise introduced in the sensor or data acquisition process (such as random errors of memory bandwidth sampling, quantization errors of CPU utilization statistics), represents the updated state estimation vector, that is, the optimal estimation combining the prediction value and the measurement value (smoothed results after eliminating transient fluctuations).

[0046] In addition, it needs to be pointed out that the Kalman filter used here is a recursive algorithm for dynamic system state estimation, which extracts the true state trend from the observed sequence with noise by combining system model prediction and observation data update, thereby smoothing transient fluctuations and suppressing noise interference.

[0047] (B) Data fusion engine, with a set of three-level data alignment mechanisms, including: (1) Time window alignment is achieved by sliding window aggregation (window length = 500 ms, step = 100 ms); (2) Time sequence synchronization is achieved based on NTP protocol clock offset compensation, which is used to eliminate the time deviation between edge nodes and center nodes, and ensure the time alignment of multi-source data, which has:

[0048] where, is the local timestamp of the edge node, represents the time offset calculated by the NTP protocol (the time difference between the center node and the edge node); (3) Multi-feature fusion is achieved by using D-S evidence theory to synthesize multi-source data, which is used to solve the contradiction between multi-source data through conflict evidence synthesis rule, and improve the reliability of fault diagnosis, which has:

[0049] where, , respectively represent the basic probability distribution of two independent evidence sources to propositions B and C, and m(A) is the credibility of the fused proposition A.

[0050] Specifically, in the multi-dimensional monitoring index system, the following operations are performed: S1. Perform edge computing preprocessing operations including dynamic outlier filtering and multi-dimensional feature selection optimization on monitoring data.

[0051] It should also be noted that before performing dynamic outlier filtering, a dynamic window adjustment operation is also performed. Based on the current system load level, the window size is dynamically adjusted and output using the following formula:

[0052] in, The value range is 0 to 2, where 0 represents low load and 2 represents high load. , The window size is limited to a specified range of 200 to 1000 to ensure computational stability under extreme loads. The size of the output window is adjusted to serve as a sliding window for subsequent anomaly detection data.

[0053] In one specific embodiment, when the load level When the value is calculated, 100 × 0.5 + 50 = 100, but because the minimum value is limited to 200, the final window size is 200. When the load level When the value is outside the normal range, the calculated value is 100×3.0+50=350, but because the maximum value is limited to 1000, the final window size is 350.

[0054] Furthermore, the specific operational procedures for dynamic outlier filtering at this point include: Based on time series data within a sliding window ( (The length is the dynamically adjusted window size), calculate the mean of the data within the window. and standard deviation ,have:

[0055] Among them, The amount of data within the window. For the first in the window One data point; Outlier detection is based on the 3σ principle. If a data point meets the following condition: If an anomaly is detected within the window, the time series data is filtered using Boolean operations (True indicates the presence of an anomaly, False indicates the absence of an anomaly).

[0056] In one specific embodiment, if the window data is [2,3,4,5,4,3,2,100] (window size 8), then the mean of the data within the window is calculated. and standard deviation We have μ = 15.375, σ ≈ 33.6, 3σ ≈ 100.8, where data point 100 satisfies... This indicates no abnormalities. If the data point is 120, then... This is considered abnormal.

[0057] Furthermore, the specific optimization operations for multidimensional feature selection at this point include: Construct multidimensional feature variables and use mutual information to evaluate features among the multidimensional feature variables. With target variable The shared information is used to filter redundant features, where the mutual information filtering formula is:

[0058] in, Indicates evaluation characteristics With target variable The joint probability distribution, To evaluate features The probability distribution, For target variable The probability distribution; During the screening process, a greedy algorithm is used to optimize and retain the top 18 features of the MI value.

[0059] It should be further explained that the multidimensional feature variables include: (a) The second difference of the current acceleration derivative is used as a trend characteristic variable to capture sudden trends in the index and identify sudden faults. The calculation formula is as follows:

[0060] in, Representing the current time The second difference of the derivative of acceleration, It is a smoothing factor, and ; (b) Using the entropy value calculated from the dispersion of the current sample distribution as a statistical feature variable to quantify the dispersion of the data distribution and measure the randomness of the system behavior, we have:

[0061] in, The entropy value is represented by the calculated result of the current distribution dispersion. For the first The probability distribution of discrete values; (c) Using the current dynamic Pearson coefficient with a time-varying window as the correlation feature variable to measure the time-varying correlation between two sets of specified variable indicators in order to discover potential fault propagation paths, we have:

[0062] in, Indicates the current time time-varying correlation of specified variables X and Y, covariance of specified variables X and Y, standard deviation of specified variables X and Y, respectively.

[0063] S2. Construct an LSTM-Attention network early warning model and optimize the model structure based on a dynamic threshold.

[0064] The LSTM-Attention network early warning model structure combines LSTM (Long Short-Term Memory Network) and Attention Mechanism, and can be used to solve the fault early warning problem of time series data. In the actual structure, it specifically includes the following layers in sequence: Input layer, receiving time series data with shape (time steps, feature number), where the feature number is the feature dimension included in each time step, for example (128, 50), indicating 128 time steps, each containing 50-dimensional features; First layer LSTM, containing 256 hidden units, with return_sequences=True set in the network topology code to output the results of each time step for use by the next layer; Dropout layer, with a dropout rate of 0.3, used to randomly discard 30% of the specified neuron connections to prevent overfitting; Second layer LSTM, containing 256 hidden units, also with return_sequences=True set to output the results of each time step for use by the next layer; The attention layer is used to calculate the attention weight of the output of the second LSTM layer (features of each time step), and it is necessary to supplement that a set of implicit fully connected layers are arranged in the attention layer, which are used to map high-dimensional features to 64 dimensions and use tanh (Hyperbolic Tangent Function) activation to convert and process the time sequence features (in the shape of (batch, time_step, 256)) to obtain intermediate scores. The core function of the tanh function is to introduce a nonlinear transformation capability into the neural network, and at the same time, the input features are mapped to a specific value range (-1, 1). After the data activated by the tanh function, the implicit fully connected layer (1 unit) is used to compress the attention weight of each time step, and the weight is normalized by using the softmax function to obtain the importance proportion of each time step. The key information of all time steps is integrated, and the features of all time steps are weighted and summed to output the context vector, so as to focus on the key time step features and enhance the sensitivity of the model to abnormal patterns. The softmax activation function used for normalization weight can convert an arbitrary length of real number vector into a probability distribution (all elements are non-negative and the sum is 1). The fully connected layer is provided with 64 units and uses the relu activation function to further extract nonlinear features. The core function of the relu activation function is to introduce a nonlinear transformation capability into the neural network, and at the same time, to solve the gradient disappearance problem of the traditional activation function (such as Sigmoid and Tanh). The LSTM-Attention early warning model constructed in the present application is applied to the feature transformation of the intermediate layer or the fully connected layer to enhance the model's ability to capture complex patterns. The output layer is provided with 1 unit and uses the sigmoid activation function to output the fault probability (0-1, the closer to 1, the higher the fault risk). The sigmoid activation function is mainly used for the gating mechanism of the LSTM (such as the input gate, the forgetting gate and the output gate). By controlling the "retention" or "discard" proportion of information, the model can capture the long-term dependence of the time sequence.

[0065] In one specific embodiment, the input is monitoring data of a server at 128 time steps (each step includes 50-dimensional features such as CPU utilization and memory utilization): after processing by the first LSTM layer, the output is a feature sequence of (128, 256). After the Dropout layer randomly discards 30% of the connections, the feature sequence is still (128, 256). The second LSTM layer outputs a deepened feature sequence of (128, 256). Then, the attention layer calculates the weight of each time step (for example, the weight of time steps 60 to 80 is higher, corresponding to the period of sudden load increase), and outputs a context vector of (256). Finally, the output layer outputs 0.85, indicating that the server has an 85% failure risk.

[0066] The specific operational procedures for dynamically optimizing the model structure using thresholds at this point include: L1. Perform feature variable standardization operations, then:

[0067] in, , All Sliding window statistics at time points. For standardization The observed values ​​of the characteristic variables at time; L2. Weights are trained using the Adam optimizer, resulting in:

[0068] in, The learning rate is initially set to 0.01, and a decay strategy of multiplying by 0.95 per epoch is adopted. L3. To achieve dynamic threshold adjustment, the formula is as follows:

[0069] in, and This is the load adaptive adjustment value, which is a specified value. This represents the calculation of the hyperbolic tangent function. This represents the load function that varies with time t.

[0070] To further explain, the model training at this point employs a loss function combining Focal Loss and a custom temporal loss to balance the ratio of positive to negative samples and reduce false negatives and false positives. The formula is as follows:

[0071] in, This represents the probability that the model predicts an anomaly. and Both are aimed at suppressing the weights of easily classified samples. The weight of the time sequence smoothing loss.

[0072] It should also be pointed out that in the model training process, Gaussian noise (σ=0.05) and time shift (±3 time steps) can also be added for data augmentation.

[0073] S3. Use the optimized network early warning model to perform three-dimensional topological visualization on the preprocessed monitoring data, and calculate the health degree of the monitoring data using the state mapping rule.

[0074] In the three-dimensional topological visualization process, first use the WebGL fragment shader to process floating-point numbers with medium precision, receive the 3D position vector vPosition from the vertex shader and the texture sampler uTexture, obtain the color information of the current pixel through texture sampling, remove the pixels with transparency less than 0.1, and output the color component of the 3D position multiplied by the texture color, realizing the rendering of the texture color associated with the spatial position with transparency effect.

[0075] Here, vPosition is the screen space position passed by the vertex shader, which is the output variable of the vertex shader (VertexShader), representing the three-dimensional position of the current vertex in the screen space (clipping space or normalized device coordinates NDC). It is used as key information passed from the vertex shader to the fragment shader after processing by the vertex shader, to determine the position of the fragment (pixel) on the screen. uTexture as a set of texture samplers (TextureSampler) is a texture sampler variable in the fragment shader, used to obtain the color information corresponding to the current fragment from the bound texture object, to realize the construction of 3D model surface details (such as texture mapping, material color).

[0076] Then use the state mapping rule to calculate the health degree of the monitoring data, and use CPU, Memory, Network, and Task to form a multi-dimensional evaluation system of system health status, and realize the comprehensive diagnosis of the overall operation status of the system through weighted summation. The process of calculating the score formula is: S=0.4×CPU+0.3×Memory+0.2×Network+0.1×Task Where, CPU is used to reflect the load and efficiency of CPU resources, Memory is used to reflect the allocation and stability of memory resources, Network is used to reflect the reliability and efficiency of network communication, and Task is used to reflect the success rate and efficiency of task execution. At this time, there is a color gradient: S>0.8: #00FF00 (healthy), 0.6<S≤0.8: #FFFF00 (warning), S≤0.6: #FF0000 (fault).

[0077] S4. Perform data flow automatic dispatching operation based on the visualization result, at this time the dispatching strategy includes: Calculate severity score based on the fault impact domain, which has:

[0078] Wherein is the node importance score (such as the number of CPU cores, the number of connected services), is the fault propagation probability (calculated based on topological dependencies); Then perform service priority scheduling based on the list of pending fault work orders, where each work order contains severity (severity: 1-urgent, 2-important, 3-general) and create_time (creation timestamp), where: First, prioritize dispatching according to severity: severity (1>2>3, urgent work order first); Second, dispatch according to creation time, the larger the timestamp, the newer the work order, the higher the priority; Finally, output the work order list sorted from high to low priority.

[0079] For example, in one embodiment, the work orders to be sorted are: Work order A: severity=2, create_time=1620000000 (created earlier); Work order B: severity=1, create_time=1620001000 (created more recently); Work order C: severity=2, create_time=1620002000 (created most recently) Then the sorted result is: work order B (urgent) → work order C (important and most recent) → work order A (important but earlier).

[0080] Further, in a set of simulation experiment platforms (based on open source container orchestration platform: Kubernetes and open source system monitoring and alarm toolkit Prometheus, the core goal is to ensure that applications run stably and efficiently in complex environments through intelligent resource scheduling and self-healing capabilities), the technical effect verification of the above operation method has:

[0081] Wherein for the verification method: Kruskal-Wallis test is a non-parametric rank-sum test used to compare whether there is a significant difference in the median of three or more independent samples, generally as a non-parametric alternative to parametric tests (such as one-way ANOVA), without relying on the assumption of normal distribution of data, wherein if there is p<0.001, it means that there is a very small probability that McNemar test is a paired chi-square test used to compare the consistency of binary classification results (such as "success / failure" "correct / incorrect") of the same sample under two different conditions, and its focus is mainly on "whether the change in conditions leads to a systematic change in classification results". Wherein if there is a chi-square statistic x 2 =14.7, probability p=0.0001, it means that there is a very significant difference between the two schemes in the false alarm rate index, that is, the scheme of the present application is significantly superior to the traditional scheme in the control of false alarm rate.

[0082] DeLong test is a non-parametric test specifically used to compare the difference in the area under the ROC curve (AUC) of two correlated samples, which takes into account the correlation between the two ROC curves in the actual test process (such as two models on the same data set, or the performance of the same model under different conditions), and belongs to the classic method of evaluating the difference in the performance of classification models. Wherein if p=0.003, it means that under a small probability (0.3%), the fault prediction AUC values of the two schemes come from the same distribution, that is, the original hypothesis is rejected, and it is considered that the scheme of the present application is significantly superior to the traditional scheme in the performance of fault prediction.

[0083] The technical effects of the further embodiment application are summarized as follows:

[0084] In conclusion, in the actual use process of the multi-source data fusion architecture constructed in the application, by setting the multi-dimensional monitoring index system covering calculation, storage, network and task, four-dimensional data fusion of calculation / storage / network / task can be realized, real-time data of each layer of the system can be comprehensively collected and integrated, the limitation of single index analysis is broken, the identification accuracy of the abnormal state of the system is significantly improved, the false alarm rate is reduced by 65%, and finally the invalid consumption of operation and maintenance resources is effectively reduced. At this time, by building an edge-center collaborative computing system, part of the data processing and analysis tasks are sunk to the edge node to realize real-time processing nearby, the delay caused by data transmission and centralized processing can be greatly reduced, thereby the monitoring delay is reduced by 90%, the overall delay is reduced to 50ms (increased by 83% compared with the traditional scheme), and finally the demand of the millisecond level response scene with extremely high real-time requirement such as financial transaction is met. At the same time, by constructing an interpretable early warning model based on an attention mechanism, the feature importance visualization can be realized while predicting the potential system failure, the early warning basis is clearly displayed, the function of assisting operation and maintenance personnel to quickly understand and judge the failure cause is played, the fault prediction accuracy is improved to 82%, the operation and maintenance decision efficiency is improved by 60%, so as to realize the active prediction and rapid processing of system failure, and guarantee the stable operation and business continuity of the system.

[0085] In another aspect, the application further discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to make the processor execute the steps of the above method.

[0086] In another aspect, the application further discloses a computer device, which comprises a memory and a processor, and the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the above method.

[0087] In another embodiment provided in the application, a computer program product containing instructions is also provided, which makes a computer execute the cloud platform intelligent monitoring and fault early warning method based on real-time data flow in any of the above embodiments when running on the computer.

[0088] It can be understood that the system provided by the embodiments of the application corresponds to the method provided by the embodiments of the application, and the explanation, examples and beneficial effects of the related content can refer to the corresponding parts in the above method.

[0089] The application also provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus, The memory is used to store a computer program. The processor is configured to implement the intelligent monitoring and fault early warning method based on the real-time data stream of the cloud platform by executing the program stored in the memory.

[0090] The communication bus mentioned in the electronic device can be a peripheral component interconnect standard bus or an extended industry standard architecture bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0091] The communication interface is configured to communicate between the electronic device and other devices.

[0092] The memory can include a random access memory and can also include a non-volatile memory, such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the processor.

[0093] The processor mentioned above can be a general-purpose processor, including a central processing unit, a network processing unit, etc. It can also be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0094] It should be further noted that the electronic device also includes a terminal device, which can also be referred to as a terminal, a user equipment, a mobile station, a mobile terminal, etc. The terminal device can be a mobile phone, a smart television, a wearable device, a tablet computer, a computer with wireless transceiver function, a virtual reality terminal device, an augmented reality terminal device, a wireless terminal in industrial control, a wireless terminal in unmanned driving, a wireless terminal in remote surgery, a wireless terminal in smart power grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc. The embodiments of the present application do not limit the specific technology and specific device form of the terminal device.

[0095] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired or wireless manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available media sets. The available medium can be magnetic medium, optical medium or semiconductor medium, etc.

[0096] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0097] In addition, it needs to be explained that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, motion condition, etc. between components in a certain posture, and if the certain posture changes, the directional indications also change accordingly.

[0098] In addition, if the embodiments of the present application involve the description of "first", "second", etc., the description of "first", "second", etc. is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing throughout the text includes three parallel solutions. Taking "A and / or B" as an example, it includes A solution, or B solution, or A and B solution. In addition, in the embodiments of the present application, "multiple" means more than two. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the present application.

Claims

1. A cloud platform intelligent monitoring and fault early warning method based on real-time data flow, characterized in that, A four-dimensional data collection architecture is constructed by using a hybrid collection mode of distributed probes and asynchronous queues, and a multi-dimensional monitoring index system is constructed by combining a data fusion engine with a three-level data alignment mechanism. In the multi-dimensional monitoring index system, the following operations are performed: S1. Perform edge computing preprocessing operations on the monitoring data, including dynamic outlier filtering and multi-dimensional feature selection optimization; S2. Construct an LSTM-Attention network early warning model, and optimize the model structure based on a dynamic threshold; S3. Use the optimized network early warning model to perform three-dimensional topological visualization on the preprocessed monitoring data, and calculate the health degree of the monitoring data using state mapping rules; S4. Perform data stream automatic dispatching operations based on the visualization results.

2. The real-time data flow based cloud platform intelligent monitoring and fault early warning method of claim 1, wherein, The multi-dimensional monitoring index system includes: A four-dimensional data collection architecture that collects four-dimensional data including the time spent by the CPU in user mode or system mode, the total time length of the specified time slice, the data volume sent and received by the GPU in the specified time window, and is used to calculate the CPU core utilization and GPU memory bandwidth; A data fusion engine with a set of three-level data alignment mechanisms, including: (1) Time window alignment is achieved through sliding window aggregation; (2) Time sequence synchronization is achieved based on NTP protocol clock offset compensation; (3) Multi-feature fusion is achieved by using D-S evidence theory to integrate multi-source data. 3.The real-time data flow based cloud platform intelligent monitoring and fault early warning method according to claim 1, wherein, In the S1 step, a dynamic window adjustment operation is performed before dynamic outlier filtering, and the following adjustment formula is used to perform dynamic window size adjustment based on the current load level of the input system and output: wherein, The value range of the load factor is 0-2, 0 represents low load, and 2 represents high load. The window size is limited in the specified range to ensure the stability of the calculation under extreme load.

4. The method of claim 1, wherein the method further comprises: The specific operation process of dynamic outlier filtering in the S1 step includes: Based on the time series data within the sliding window, calculate the mean of the data within the window and the standard deviation , are: Among them, The amount of data within the window. For the first in the window One data point; Based on the 3σ principle to detect outliers, if there is a data point that satisfies the condition: Then determine that there are abnormalities in the window, filter the time series data.

5. The real-time data flow based cloud platform intelligent monitoring and fault early warning method of claim 1, wherein, The specific optimization operations of multi-dimensional feature selection optimization in the S1 step include: A multi-dimensional feature variable is constructed, and mutual information is used to evaluate the features in the multi-dimensional feature variable The shared information amount with the target variable Redundant features are screened, and the mutual information screening formula is: wherein, represents an evaluation feature a joint probability distribution of the target variable , is a probability distribution of the evaluation feature , is a probability distribution of the target variable ; During the filtering process, the first 18 dimensions of the MI value are retained. 6.The real-time data flow based cloud platform intelligent monitoring and fault early warning method according to claim 5, wherein, The multi-dimensional feature variables include: (a) The second-order difference of the current time acceleration derivative is used as a trend feature variable to capture the mutation trend of the index and identify sudden failures, and the calculation formula is: wherein represents the current time second difference of the acceleration derivative, is a smoothing factor, and ; (b) The entropy value calculation result of the current sample distribution dispersion is used as a statistical feature variable to quantify the dispersion degree of the data distribution and measure the randomness of the system behavior, and the formula is: wherein, represents the entropy value calculated as a result of the current distribution dispersion, is the probability distribution of the first dispersion value; (c) The dynamic Pearson coefficient with a time-varying window is used as a correlation feature variable to measure the time-varying correlation between two specified variable indexes to discover potential fault propagation paths, and the formula is: wherein, denotes the current time instant denotes the time-varying correlation of the variables X and Y, denotes the covariance of the variables X and Y, denotes the standard deviation of the variables X, Y, respectively.

7. The real-time data flow based cloud platform intelligent monitoring and fault early warning method of claim 1, wherein, The LSTM-Attention network early warning model structure in the S2 step includes: The input layer receives time series data with a shape of (time step, feature number), where the feature number is the feature dimension included in each time step; The first layer LSTM contains 256 hidden units and is used to output the results of each time step for use by the next layer; The Dropout layer is used to randomly discard a specified proportion of neuron connections to prevent overfitting; The second layer LSTM contains 256 hidden units and is used to output the results of each time step for use by the next layer; An attention layer is configured to calculate attention weights for the output of the second LSTM, convert features through a fully connected layer to obtain intermediate scores, and compress the intermediate scores into attention weights for each time step through a fully connected layer, normalize the weights by using a softmax function, obtain the importance proportion of each time step, and output a context vector by weighting and summing the features of all time steps; The fully connected layer is configured to have 64 units and use a relu activation function, and is configured to extract nonlinear features; The output layer is configured to have 1 unit and use a sigmoid activation function, and is configured to output a fault probability.

8. The real-time data flow based cloud platform intelligent monitoring and fault early warning method of claim 1, wherein, The specific operation process of dynamically optimizing the model structure by using the threshold in the S2 step includes: L1. Perform a feature variable standardization operation, which includes: wherein, , are the sliding window statistics at time t, are the normalized feature variable observations at time t. L2. Perform weight training by using an Adam optimizer, which includes: wherein, is the learning rate, which is initially 0.01 and adopts a decay strategy of multiplying by 0.95 per epoch; L3. Dynamically adjust the threshold, which has a formula as follows: wherein, with is a load adaptive adjustment value, is a specified value, denotes a hyperbolic tangent function calculation, denotes a load function that varies over time t. 9.The real-time data flow based cloud platform intelligent monitoring and fault early warning method of claim 6, wherein, The model training in the S2 step adopts a loss function combined by using a Focal Loss and a self-defined time sequence loss, and the formula is as follows: wherein, is the probability of the model predicting as abnormal, and are the weights of the easy-to-classify samples, and respectively represent two sets of modulation parameters of the Focal Loss, is the weight of the time smoothing loss.

10. The real-time data flow based cloud platform intelligent monitoring and fault early warning method of claim 6, wherein, The dispatching strategy in the S4 step includes: Calculate a severity classification severity score based on a fault influence domain, which includes: wherein is a node importance score, is a failure propagation probability; Then, service priority scheduling is performed based on a to-be-processed fault work order list, each work order including a severity classification severity and a creation timestamp create_time, wherein: First, dispatch work orders according to severity, and urgent work orders are given priority; Second, dispatch work orders according to creation time, and the larger the timestamp, the newer the work order, and the higher the priority.

Images