A Deep Learning-Based Method for Detecting Anomalies in Data Center Operations and Maintenance

Abstract

The application provides a data center machine room operation and maintenance anomaly detection method based on deep learning, comprising: for a low-dimensional feature set, using a long short-term memory network to analyze time series and capture subtle patterns in low-dimensional dynamic characteristics to obtain a behavior pattern sequence; extracting packet structure and time delay changes from an abnormal candidate set, using an isolation forest algorithm for secondary screening to determine a hidden abnormal behavior set; according to the abnormal pattern set, using a sliding window mechanism to analyze real-time data streams to determine whether abnormal patterns continue to appear to obtain an abnormal persistence score; according to the final abnormal detection result, updating the parameters of the behavior benchmark model, using an online learning mechanism to optimize the model to obtain an updated behavior benchmark model; for the updated behavior benchmark model, cyclically executing real-time data stream analysis to continuously capture subtle patterns to obtain continuous abnormal detection output.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method for detecting anomalies in data center operations and maintenance based on deep learning. Background Technology

[0002] Background of the problem:

[0003] Data center operation and maintenance is a critical area for ensuring the smooth operation of modern information society. Its stability and security directly affect business continuity and data integrity. With the rapid development of cloud computing and big data, data centers face increasingly complex network attacks and system failures, making anomaly detection technology a core element in maintaining stable data center operation. Traditional detection methods mainly rely on rule matching and threshold monitoring, which are effective in handling explicit faults but often fall short when dealing with subtle anomalies. These methods typically fail to capture minute and deliberately hidden changes in communication protocols. In one embodiment, carefully designed low-frequency attacks or abnormal patterns disguised as normal behavior make it difficult to detect potential threats in a timely manner.

[0004] In communication protocol anomaly detection, the core challenge lies in identifying subtle, carefully masked anomalous behaviors. Communication protocol data streams are high-dimensional and dynamic, and anomalous patterns are often hidden within massive amounts of data, manifesting as minute time delays or subtle changes in packet structure. These changes are typically designed to closely resemble normal behavior, making them difficult to distinguish using conventional analysis methods. Furthermore, due to the concealment of anomalous patterns, detection systems need to dynamically build baseline models of protocol behavior without relying on predefined rules. The construction of these baseline models depends on in-depth analysis of massive amounts of communication data, but the high dimensionality and real-time nature of the data require a balance between computational efficiency and detection accuracy. Identifying hidden anomalies requires capturing subtle patterns in the data, and the accuracy of pattern capture directly affects the system's ability to judge anomalies.

[0005] Therefore, how to dynamically construct behavioral benchmark models and accurately capture subtle abnormal patterns through in-depth analysis of communication protocol data streams has become a key issue in the detection of anomalies in data center operations and maintenance. Summary of the Invention

[0006] This invention provides a deep learning-based method for detecting operational anomalies in data center computer rooms, mainly including:

[0007] Real-time data streams are acquired from communication protocol data. Preprocessing is used to extract packet structure and time delay variation features, followed by principal component analysis (PCA) for dimensionality reduction, resulting in a low-dimensional feature set. For this low-dimensional feature set, a Long Short-Term Memory (LSTM) network is used to analyze the time series, capturing subtle patterns in the low-dimensional dynamic characteristics to obtain a behavior pattern sequence. Based on this behavior pattern sequence, an adaptive behavior baseline model is constructed. If the deviation between new data and the model exceeds a preset threshold, it is identified as a potential anomaly, resulting in an anomaly candidate set. Packet structure and time delay variations are extracted from the anomaly candidate set, and a secondary screening using the Isolation Forest algorithm is employed to determine a set of hidden anomalous behaviors. For this set, the similarity between each anomaly pattern and the behavior baseline model is calculated. If the similarity is below a preset threshold, it is identified as an anomalous pattern, resulting in an anomaly pattern set. Based on this anomaly pattern set, a sliding window mechanism is used to analyze the real-time data stream to determine whether the anomaly pattern persists, resulting in an anomaly persistence score. For the anomaly persistence score, a weighted average method is used to fuse the weights of packet structure and time delay variations. If the score exceeds a preset threshold, an anomaly is confirmed, yielding the final anomaly detection result.

[0008] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0009] First, this invention deeply analyzes communication protocol data streams, utilizes Long Short-Term Memory (LSTM) networks to capture subtle patterns in the data, and constructs an adaptive behavioral baseline model. Compared to traditional methods, it can more accurately identify hidden abnormal behaviors, effectively solving the problem of insufficient detection capability of traditional methods when facing complex and hidden anomalies, and significantly improving the accuracy and reliability of anomaly detection in data center operations and maintenance.

[0010] Secondly, this invention uses principal component analysis (PCA) to reduce the dimensionality of high-dimensional features, which can effectively reduce data dimensionality, reduce data complexity, and improve data processing efficiency. At the same time, it retains key information in the data and highlights the main features, making subsequent analysis and detection more efficient and accurate, thus solving the problem of difficult high-dimensional data processing.

[0011] Finally, this invention constructs an adaptive behavioral benchmark model that reflects normal behavioral patterns through time series analysis and clustering algorithms. This model can be dynamically updated as new data is continuously added, adapting to changes in data center operation and maintenance data. It effectively solves the problem that fixed thresholds and rules in traditional methods are difficult to adapt to dynamic data changes, and improves the adaptability and accuracy of anomaly detection. Attached Figure Description

[0012] Figure 1 This is a flowchart of a data center operation and maintenance anomaly detection method based on deep learning, provided for an embodiment of the present invention.

[0013] Figure 2 The flowchart illustrates a data center operation and maintenance anomaly detection method based on deep learning, as provided in another embodiment of the present invention.

[0014] Figure 3 The flowchart illustrates a data center operation and maintenance anomaly detection method based on deep learning, which is provided as another embodiment of the present invention. Detailed Implementation

[0015] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] like Figure 1 As shown in the figure, this embodiment of the invention provides a data center operation and maintenance anomaly detection method based on deep learning, which may specifically include the following steps:

[0017] S101. Obtain real-time data stream from communication protocol data, extract data packet structure and time delay change features through preprocessing, and use principal component analysis to reduce dimensionality to obtain a low-dimensional feature set.

[0018] Real-time data streams are acquired from communication protocol data. Raw data packets are parsed using a data stream capture tool to obtain a data packet sequence. The parsed data packet sequence is preprocessed to extract data packet structure and time delay features, resulting in a structured feature set. If the dimensionality of the structured feature set exceeds a preset threshold, principal component analysis is used to reduce the dimensionality, resulting in a low-dimensional feature set. Based on the low-dimensional feature set, the delay variation trend of the time delay features is calculated, resulting in a delay variation sequence. Cluster analysis is performed on the delay variation sequence to identify abnormal patterns in the delay variation trend, resulting in an abnormal pattern set. If a pattern in the abnormal pattern set matches a preset abnormal template, a feature subset of the matching pattern is extracted, resulting in an abnormal feature subset. The abnormal feature subset is compared with the low-dimensional feature set to determine the source of the abnormal features, resulting in an abnormal source distribution.

[0019] The main steps of principal component analysis (PCA) dimensionality reduction include:

[0020] Input: Structured feature set X m×n (m samples, n features), with a preset principal component analysis dimensionality reduction threshold k.

[0021] Standardization process: Standardize X to obtain Xi. ∗ For each feature j, the standardization formula can be:

[0022] , where μ j and σ j These are the mean and standard deviation of feature j, respectively.

[0023] Calculate the covariance matrix: Find X ∗ The covariance matrix C has elements that satisfy:

[0024] .

[0025] Eigenvalue decomposition: Performing eigenvalue decomposition on C yields the eigenvalues: λ1≥λ2≥…≥λ n and the corresponding feature vectors v1, v2, ..., v n .

[0026] Principal components are selected: the first k eigenvectors are used to form the projection matrix V = [v1, v2, ..., v]. k ].

[0027] Dimensionality reduction: Calculating X ∗ V, yielding the dimensionality-reduced feature set X. reduced .

[0028] In data center operations and maintenance, communication protocol data contains a large number of features, such as source IP, destination IP, port number, protocol type, load size, and time latency. This increase in data dimensionality leads to massive data storage requirements. PCA reduces the number of features by projecting the original high-dimensional data into a low-dimensional space. For example, the original data might have 100 dimensions; after PCA dimensionality reduction, only 10-20 principal components might be needed to explain most of the variance, thus significantly reducing data storage space.

[0029] Specifically, when acquiring real-time data streams from communication protocol data, TCP / IP packets on the network interface can be captured using protocol analysis tools such as Wireshark. Assuming 1000 packets are captured, each packet contains a source IP, destination IP, timestamp, and payload size, with the payload size ranging from 64 to 1500 bytes. The captured data stream is stored in PCAP format with a timestamp accuracy in the microsecond range; in one embodiment, 2025-05-13 19:09:23.123456. During preprocessing, the PCAP file is parsed to extract the packet structure. Each packet is read using Python's Scapy library, extracting fields such as source IP (192.168.1.10), destination IP (10.0.0.5), timestamp, and payload size (512 bytes). Simultaneously, the time delay between adjacent packets is calculated, with a delay value ranging from 0.1 to 10 milliseconds, resulting in a feature matrix X with dimensions 1000×4 (source IP encoding, destination IP encoding, payload size, and time delay). To analyze the characteristics of time delay variation, statistical features of the delay were calculated, such as a mean of 0.5 milliseconds and a standard deviation of 0.2 milliseconds. Fourier transforms of the time series were generated, and periodic variation features with a principal frequency of 50 Hz were extracted, supplementing matrix X to 1000×6. Principal Component Analysis (PCA) was used for dimensionality reduction. The sklearn.decomposition.PCA library was used, with the standardized matrix X (mean 0, variance 1) as input. The covariance matrix was calculated, and the first two principal components were extracted, contributing a cumulative 85% of the total. The output was a low-dimensional feature set Y with dimensions of 1000×2. The first principal component showed a correlation of 0.9 with load size, and the second principal component showed a correlation of 0.7 with time delay. The analysis process used eigenvalue decomposition to ensure that dimensionality reduction retained key information. The low-dimensional feature Y can be used for subsequent business applications, such as network anomaly detection, detecting time delay anomalies (points exceeding the mean ± 2 standard deviations, accounting for approximately 5%), forming a complete logical chain from data capture to feature extraction.

[0030] S102. For low-dimensional feature sets, long short-term memory networks are used to analyze time series, capture subtle patterns in low-dimensional dynamic characteristics, and obtain behavioral pattern sequences.

[0031] Time-series data is acquired from a low-dimensional feature set through a pre-defined data interface. The data is then normalized using a standardization method to obtain processed time-series data. A Long Short-Term Memory (LSTM) network is used to train the processed time-series data, with hidden layer units set to capture temporal dependencies, generating intermediate feature representations containing dynamic characteristics. Based on these intermediate feature representations, an attention mechanism is applied to weight the low-dimensional dynamic characteristics, extracting subtle patterns at key time steps to obtain a weighted feature vector. Specifically, assuming the input sequence is {x1, x2, ..., x...} t}, where x tThis is the feature vector at time step t. Attention mechanisms typically calculate an attention score e. t To measure the importance of each time step:

[0032] ;

[0033] Where f can be a function of various forms, such as a dot product, addition, or a feedforward neural network, h t-1 This refers to the hidden states output by the Long Short-Term Memory (LSTM) network model. This embodiment of the invention uses additive attention as an example for illustration:

[0034] ;

[0035] Among them, v a W is a learnable weight vector. a U a It is the weight matrix, b a This is the bias term, where ┬ represents the transpose of a vector or matrix, and tanh is the hyperbolic tangent function. Then, the attention score e is... t Normalized to attention weight α t ;

[0036] Finally, the input features are multiplied by the attention weights to obtain the weighted feature vector:

[0037] .

[0038] When analyzing time-series data from data center operations and maintenance, the data may contain numerous features, but not all features are useful for anomaly detection. Attention mechanisms can automatically learn and highlight subtle patterns at key time steps, more accurately capturing subtle changes that may lead to anomalies. For example, small changes in packet structure or latency at a specific point in time may indicate potential anomalous behavior; attention mechanisms can identify and reinforce these key features. Furthermore, attention mechanisms can reduce interference from irrelevant information. In data center operations and maintenance data, normal behavior patterns may dominate, while anomalous behavior may only manifest as localized, subtle changes. Attention mechanisms can focus on parts that may contain anomalous information, reducing the interference of normal behavior data on the anomaly detection process, thereby improving detection accuracy. Finally, by assigning different weights to each time step, attention mechanisms clarify which time step features are more critical for anomaly detection. This helps operations personnel understand how the model makes anomaly detection decisions. For example, if the attention mechanism assigns a high weight to a certain time point, operations personnel can focus on the data features at that time point, analyzing whether they correspond to actual anomalies, thus better understanding the model's decision-making basis.

[0039] If the dimension of the weighted feature vector exceeds a preset threshold, principal component analysis (PCA) is used for dimensionality reduction to obtain a dimensionality-reduced feature vector. A preset classifier is then used to perform pattern recognition on the dimensionality-reduced feature vector to generate a preliminary behavioral pattern sequence. This preliminary behavioral pattern sequence is then used to detect outliers using a sliding window method. If outliers are detected, interpolation is used to smooth the sequence, resulting in a smoothed behavioral pattern sequence. Based on the smoothed behavioral pattern sequence, clustering is applied to group the sequence, determining the final behavioral pattern sequence.

[0040] Specifically, in a low-dimensional feature set, suppose there is a time series dataset containing 10 features, each with a time step of 100. First, the data is normalized to between 0 and 1 using a normalization method to ensure consistent scale across different features. Next, a Long Short-Term Memory (LSTM) network model is constructed, consisting of two LSTM layers, each with 50 hidden units, and using the ReLU activation function. The model's input shape is (100, 10), representing 100 time steps and 10 features. The Adam optimizer is used, with a learning rate set to 0.001, and mean squared error (MSE) is used as the loss function during training. During training, the batch size is set to 32, and the number of training epochs is 50. After training, the trained model is used to predict the test set to obtain the behavioral pattern sequence. To capture subtle patterns in low-dimensional dynamic characteristics, in other embodiments, the prediction results can be further clustered. The K-means algorithm is used to divide the behavioral pattern sequences into 5 categories, and the clustering effect is evaluated by the silhouette coefficient. A silhouette coefficient of 0.75 indicates that the clustering results have good discriminative power. Finally, the clustering results are displayed using visualization tools to more intuitively understand the differences between different behavioral patterns.

[0041] S103. Based on the behavioral pattern sequence, construct an adaptive behavioral baseline model. If the deviation between the new data and the model exceeds a preset threshold, it is judged as a potential anomaly, and an anomaly candidate set is obtained.

[0042] An initial adaptive behavioral baseline model is constructed by means of a behavioral pattern sequence. Time series analysis is used to extract sequence features to obtain the baseline model. The deviation between the new data and the baseline model is calculated. In one embodiment, the mean squared error method can be used to quantify the deviation value to obtain the deviation result. If the deviation result exceeds a preset threshold, the new data is marked as a potential anomaly, resulting in a potential anomaly set. Cluster analysis is used to group similar anomalies into a candidate anomaly set. The adaptive behavioral baseline model is updated by comparing the candidate anomaly set with historical data to obtain an updated model. The deviation value of the new data is recalculated based on the updated model using the mean squared error method to obtain a new deviation result. If the new deviation result still exceeds the preset threshold, the candidate anomaly is confirmed as an anomaly, resulting in the final anomaly set.

[0043] Specifically, when constructing an adaptive behavior baseline model, it is first necessary to collect historical behavior pattern sequence data, such as user login time, operation frequency, and access path in one embodiment. Assume 1000 historical data entries are collected, each containing 10 features. The K-means clustering algorithm is used to cluster this data, with a cluster size of 5. The distance between each data point and its cluster center is calculated using Euclidean distance to obtain the category label for each data point. Next, a baseline model is constructed based on the clustering results, and the centroid of each category is calculated as the baseline value.

[0044] In one embodiment, the centroids of category 1 are [0.5, 0.7, 0.3, 0.8, 0.6, 0.4, 0.9, 0.2, 0.1, 0.5]. When new data arrives, its distance to each category centroid is calculated using Mahalanobis distance as the metric, with a threshold of 2.0. If the minimum distance between the new data and all category centroids exceeds 2.0, it is considered a potential anomaly.

[0045] In one embodiment, the Mahalanobis distance between the new data [0.6, 0.8, 0.4, 0.9, 0.7, 0.5, 1.0, 0.3, 0.2, 0.6] and the center point of category 1 is 2.5, which exceeds the threshold, so it is added to the anomaly candidate set. In this way, the baseline model can be dynamically updated and anomalies in new data can be continuously monitored.

[0046] S104. Extract data packet structure and time delay changes from the abnormal candidate set, and use the isolated forest algorithm for secondary screening to determine the set of hidden abnormal behaviors.

[0047] Data packet structure features and time delay variation features are extracted from the anomaly candidate set. Preprocessing methods are used to normalize these features, resulting in a standardized feature set. If the distribution of data packet structure features in the standardized feature set deviates from a preset threshold, dimensionality reduction is applied to obtain a dimensionality-reduced feature set. Anomaly detection is performed on the dimensionality-reduced feature set using the isolated forest algorithm to obtain anomaly scores and determine a preliminary set of anomalies. Based on the time delay variation features of the preliminary anomaly set, a clustering algorithm is used to group the anomalies, resulting in anomalous behavior clusters. If the number of anomalous behavior clusters is below a preset threshold, time series analysis is performed on the time delay variation features of each cluster to identify hidden anomalous behaviors.

[0048] In one embodiment, the ARIMA model is used for hidden anomalous behavior analysis. Specifically,

[0049] Input: Time series of time delay variation features for each cluster {y1, y2, ..., y} t The process involves: establishing an ARIMA model: performing differencing on the time series to make it stationary, and determining the parameters p (number of autoregressive terms), d (number of differencing terms), and q (number of moving average terms) of the ARIMA model; model fitting and prediction: fitting the ARIMA model based on historical data and predicting values ​​for future times; calculating residuals: comparing the residuals between the actual and predicted values. If the residuals exceed a certain confidence interval, it is determined that there is hidden abnormal behavior at that time. The ARIMA model can effectively capture trends, seasonality, and periodicity in time series data. In data center operations, time delay variations may exhibit certain patterns over time. For example, latency may increase due to network traffic peaks during specific time periods. The ARIMA model can accurately extract these patterns by fitting historical data, thus providing strong evidence for anomaly detection. Furthermore, the parameters p, d, and q of the ARIMA model can be flexibly adjusted according to the specific characteristics of the time series. For different time delay variation characteristics in different data center operation scenarios, adjusting the model parameters can better fit the actual data, improving the model's accuracy and applicability. Furthermore, the magnitude of the residual can also reflect the severity of the anomaly to some extent. A larger residual means a greater deviation between the actual and predicted values, which may indicate a more serious anomaly. This helps maintenance personnel take appropriate measures based on the severity of the anomaly and allocate resources reasonably for troubleshooting and handling.

[0050] By performing correlation analysis on the structural features and delayed change features of concealed abnormal behaviors, a set of concealed abnormal behaviors is determined. Based on this set of concealed abnormal behaviors, a rule matching method is used to verify the behavior set, resulting in the final set of abnormal behaviors.

[0051] Specifically, in the anomaly candidate set, the structural information of each data packet is first extracted using a data packet parsing tool, including fields such as source IP, destination IP, port number, and protocol type, and the timestamp of each data packet is recorded.

[0052] In one embodiment, 200 packets with a source IP of 192.168.1.1 and 300 packets with a destination port of 80 are extracted from 1000 packets. Next, the time delay variation of these packets is calculated, assuming an average delay of 0.05 seconds and a standard deviation of 0.01 seconds within one second. Then, the isolated forest algorithm is used for secondary filtering of these data. The isolated forest algorithm detects outliers by constructing multiple binary trees. Each tree randomly selects features for segmentation, and outliers, due to their different distribution from normal points, are isolated in shallower nodes.

[0053] In one embodiment, the number of trees in the isolated forest is set to 100, and the number of samples per tree is 256. By calculating the anomaly score of each data point, data points with anomaly scores greater than 0.6 are selected, and the set of hidden abnormal behaviors is finally determined.

[0054] In one embodiment, 50 abnormal data packets are selected from 1,000 data packets. These packets have significantly different source IPs and destination ports from normal data packets, and their time delays also exceed the normal range. They are identified as a set of hidden abnormal behaviors.

[0055] S105. For the set of hidden abnormal behaviors, calculate the similarity between each abnormal pattern and the behavioral baseline model. If the similarity is lower than the preset threshold, it is determined to be an abnormal pattern, and an abnormal pattern set is obtained.

[0056] Through data preprocessing, behavioral data is extracted from concealed abnormal behaviors to obtain a pattern feature set. Principal component analysis (PCA) is used to reduce the dimensionality of the pattern feature set, resulting in a low-dimensional feature set. Based on the low-dimensional feature set and the behavioral baseline model, a similarity metric is calculated for each abnormal pattern, resulting in a similarity set. If the value in the similarity set is below a preset threshold, the corresponding pattern is determined to be abnormal, resulting in a preliminary abnormal pattern set. For the preliminary abnormal pattern set, time-series features of its behavioral data are obtained, resulting in a time-series feature set. An isolated forest algorithm is used to detect abnormal patterns from the time-series feature set, resulting in a refined abnormal pattern set. Based on the refined abnormal pattern set, the deviation between its pattern features and the behavioral baseline model is extracted, resulting in an abnormal pattern deviation set.

[0057] Specifically, for the set of concealed abnormal behaviors, a behavioral baseline model is first constructed. 1000 normal user behavior records are extracted from historical data. Each record contains three features: login time, operation frequency, and number of pages visited, normalized to the interval [0,1]. The feature mean vector M = [0.5, 0.2, 0.3] is calculated. The K-means clustering algorithm (K=3) is used to cluster the normal behaviors, resulting in three cluster centers: C1 = [0.4, 0.15, 0.25], C2 = [0.55, 0.25, 0.35], and C3 = [0.45, 0.2, 0.3]. The mean is taken as the baseline model B = [0.47, 0.2, 0.3]. For the set of concealed abnormal behaviors, assuming it contains 500 records of behaviors to be detected, three features are extracted from each record and normalized. In one embodiment, a record A = [0.1, 0.8, 0.9]. The similarity between model A and the baseline model B is calculated using the cosine similarity formula: cosθ=(A·B) / (||A||·||B||). The formulas are: A·B=0.1×0.47+0.8×0.2+0.9×0.3=0.477, ||A||=√(0.1²+0.8²+0.9²)=1.204, ||B||=√(0.47²+0.2²+0.3²)=0.592, cosθ=0.477 / (1.204×0.592)=0.669. A similarity threshold of 0.8 is set. Since 0.669<0.8, A is determined to be an anomalous pattern. This process is repeated for each of the 500 records to obtain a set of records with a similarity lower than 0.8. In one embodiment, this set includes 100 records representing an anomalous pattern. To ensure logical rigor, the feature distribution of the abnormal pattern was further analyzed, and the feature mean of the abnormal set [0.15, 0.75, 0.85] was calculated. Compared with the benchmark model, the operation frequency and page count were found to be significantly higher, suggesting that it may be an automated script attack. Related business functions can trigger real-time alarms and notify the security system to intercept it.

[0058] S106. Based on the set of abnormal patterns, a sliding window mechanism is used to analyze the real-time data stream, determine whether the abnormal patterns continue to occur, and obtain an abnormality persistence score.

[0059] The process involves acquiring real-time data streams and performing pattern matching using a pre-established set of abnormal patterns to determine the initial abnormal event. If the initial abnormal event is detected, a sliding window mechanism is used to divide the time window, obtaining a subset of data within that window. For each subset of data within the time window, the frequency of the abnormal pattern is calculated to obtain its frequency distribution characteristics. These characteristics are compared with a preset anomaly detection threshold to determine if the abnormal pattern persists. If the abnormal pattern persists, an anomaly persistence score is obtained based on a scoring formula, combined with the frequency and duration of the abnormal pattern. The persistence score is used to update the pattern weights in the abnormal pattern set, resulting in an optimized set of abnormal patterns. The optimized set of abnormal patterns is then used to re-analyze the real-time data stream to determine the next round of abnormal events. These steps form a logical closed loop, with the outputs and inputs of each step closely connected. All steps revolve around the goal of anomaly persistence scoring, based on attributes such as the abnormal pattern set, the sliding window mechanism, and the real-time data stream, avoiding irrelevant attributes such as data stream rate, and ensuring a tightly integrated technical process.

[0060] Specifically, in real-time data stream analysis, a sliding window mechanism is first adopted, with a window size of 100 data points and a sliding step size of 10 data points to ensure the continuity and coverage of the data stream. The data within each window is analyzed in the frequency domain using Fast Fourier Transform (FFT) to extract the main frequency components, and then matched with a predefined set of abnormal patterns.

[0061] In one embodiment, when an abnormal pattern with a frequency of 50Hz is detected within a window, the system records the number of occurrences and the duration of the anomaly. Next, by calculating the frequency of the abnormal pattern across consecutive windows, a weighted average algorithm is used, assigning a higher weight to the most recent window, to calculate the anomaly persistence score. Assuming the abnormal pattern occurs 3, 4, 5, 6, and 7 times in 5 consecutive windows, with weights of 0.1, 0.2, 0.3, 0.4, and 0.5 respectively, the anomaly persistence score is (3×0.1+4×0.2+5×0.3+6×0.4+7×0.5) / (0.1+0.2+0.3+0.4+0.5) = 5.5. Finally, based on the score, the system determines whether the abnormal pattern persists and triggers a corresponding early warning mechanism to ensure timely response and handling of potential problems.

[0062] S107. For the anomaly persistence score, a weighted average method is used to integrate the weights of data packet structure and time delay changes. If the score exceeds the preset threshold, the anomaly is confirmed and the final anomaly detection result is obtained.

[0063] A data acquisition module is used to obtain packet structure and latency variation information from network traffic, extracting structural features and latency variation features to obtain an initial feature set. The structural features and latency variation features in the initial feature set are fused using a pre-defined weighted average method to calculate a weighted feature vector, resulting in an anomaly persistence score. If the anomaly persistence score exceeds a pre-defined threshold, it is determined to be an anomaly, an anomaly label is generated, and a preliminary anomaly detection result is obtained. Based on the preliminary anomaly detection result, the K-Means algorithm is used to perform cluster analysis on the anomaly labels, classifying anomaly types to obtain a set of classified anomaly types. Representative features of each type are extracted from the classified anomaly type set, and a decision tree algorithm is used to construct an anomaly type classification model, resulting in an anomaly type prediction model. The anomaly type prediction model is used to analyze newly collected network traffic data in real time, generating anomaly type labels to obtain the final anomaly detection result. After obtaining the final anomaly detection result, the result is stored in a pre-defined database, and the anomaly detection log is updated to obtain an anomaly detection record.

[0064] Specifically, for the scoring of anomaly persistence, a weighted average method is used to integrate the weights of packet structure and time delay changes. The specific implementation method is as follows: First, collect network data streams, assuming 1000 packets are captured per second, and extract packet structure features (such as packet length and protocol type) and time delay features (such as inter-packet time intervals).

[0065] In one embodiment, in a data stream, the average packet length is 500 bytes, and the standard deviation is 50 bytes; the average time delay is 0.02 seconds, and the standard deviation is 0.005 seconds. Next, an anomaly score for the packet structure is calculated using a statistical anomaly detection algorithm. A packet length exceeding the mean ± 2 times the standard deviation is considered an anomaly. The percentage of anomaly packets is calculated; assuming 50 out of 1000 packets have a length exceeding 550 bytes or less than 450 bytes, the anomaly percentage is 5%. For time delay, the delay change rate is calculated using the formula |current delay - average delay| / average delay. If the change rate is greater than 0.25 (i.e., 0.005 / 0.02), it is marked as an abnormal delay; assuming 100 packets meet this condition, the anomaly percentage is 10%. Then, the two anomaly scores are combined using a weighted average method, setting the packet structure weight at 0.6 and the time delay weight at 0.4, to calculate the comprehensive score: 0.6 × 5% + 0.4 × 10% = 7%. A threshold of 6% is set; since 7% > 6%, an anomaly is confirmed. Finally, by comparing the comprehensive score with the threshold, the anomaly detection results are output, the timestamps and characteristics of the abnormal data streams are recorded, and the data is stored in a database for subsequent network security analysis, such as correlating with intrusion detection systems to confirm whether it is a DDoS attack. This process is implemented through automated scripts, with rigorous logic and clear numerical calculations, ensuring efficient and accurate anomaly detection.

[0066] Please see Figure 2As shown, in one embodiment, as a further improvement, the data center operation and maintenance anomaly detection method may further include:

[0067] S108. Based on the final anomaly detection results, update the parameters of the behavioral baseline model, optimize the model using an online learning mechanism, and obtain the updated behavioral baseline model.

[0068] Anomaly features are extracted from anomaly detection results. A feature extraction process is used to determine the deviation features between the anomaly and normal data, resulting in a deviation feature set. Using this deviation feature set, an online learning mechanism is employed to iteratively calculate the behavioral baseline model, updating the model parameters to obtain a temporary updated model. If the performance metrics of the temporary updated model fall below a preset threshold, new data is obtained from the training data input, and a real-time feedback adjustment strategy is used to re-iterate the model calculation, resulting in an adjusted model. Based on the adjusted model, a parameter adjustment strategy is used to optimize the model weight allocation, determining the final model parameters to obtain an optimized model. Updated behavioral pattern features are extracted from the optimized model, and data stream processing technology is used to update the behavioral baseline model in real time, resulting in an updated model. Using the updated model, a real-time data stream is acquired, and combined with the feature extraction process, it is determined whether the model is adapted to the current data distribution, resulting in an adaptability assessment result. If the adaptability assessment result shows that the model deviation exceeds a preset threshold, an online learning mechanism, combined with a real-time feedback adjustment strategy, is used to update the model parameters again, resulting in the final updated model.

[0069] Specifically, in the anomaly detection stage, by collecting the user behavior data stream in real time, the Local Outlier Factor (LOF) algorithm based on density is used to calculate the local outlier factor of each data point. The threshold is set to 1.5. When the LOF value of a certain login behavior reaches 2.3, an anomaly flag is triggered. For the detected abnormal behaviors, the online gradient descent algorithm is used to update the behavior benchmark model. The learning rate is set to 0.01, and the weight decay coefficient λ is taken as 0.001. The model parameter w is iteratively optimized as w = w - η(∇L(w) + λw), where the loss function L uses the Huber loss, and the δ parameter is set to 1.2 to balance the influence of outliers. During the parameter update process, the mean μ and standard deviation σ of the last 1000 normal behavior samples are retained through the sliding window mechanism. When the new data point x satisfies μ - 2σ < x < μ + 3σ, it is included in the training set. For the feature engineering link, PCA dimensionality reduction is used to retain 85% of the variance contribution rate, and the original 23-dimensional features are compressed into 8-dimensional principal components. The model performance is evaluated using the dynamic AUC metric. When the window period is 50 batches, the AUC value increases from 0.82 to 0.89, proving the effectiveness of the parameter update. At the same time, a feature importance ranking mechanism is established, and it is found that the feature weight of "operation interval standard deviation" increases from 0.15 to 0.22, reflecting the increased contribution of this indicator to anomaly recognition. During the whole process, a double-check mechanism is adopted. When the loss function decline rate of three consecutive parameter updates is less than 1%, the early stopping strategy is automatically triggered.

[0070] Please refer to Figure 3 As shown, in one embodiment, as a further improvement, the data center computer room operation and maintenance anomaly detection method may further include:

[0071] S109. For the updated behavior benchmark model, continuously perform real-time data stream analysis in a loop to continuously capture subtle patterns and obtain continuous anomaly detection outputs.

[0072] Obtain the real-time data stream, parse the data through the streaming processing technology to obtain a structured data sequence. If the structured data sequence meets the preset integrity conditions, perform pattern matching based on the behavior benchmark model to determine the initial matching pattern. Use the sliding window technology to process the initial matching pattern to capture weak patterns and obtain a pattern feature vector. If the deviation between the pattern feature vector and the behavior benchmark model exceeds the preset threshold, perform anomaly detection through the Isolation Forest algorithm to judge the anomaly points. For the anomaly points, combine the time series context and generate an anomaly identification sequence through dynamic analysis. According to the anomaly identification sequence, update the behavior benchmark model to obtain an optimized benchmark model. Through the optimized benchmark model, generate a continuous anomaly output sequence.

[0073] Specifically, in real-time data stream analysis, a sliding window mechanism is first used to collect data streams at 1-second intervals, with a window size of 1000 records. A Kalman filter algorithm is used to denoise the raw data, with filter parameters set to Q=0.01 and R=0.1. The preprocessed data is then input into an LSTM-based time-series prediction model. This model contains three hidden layers, each with 128 neurons, using the tanh activation function. The Adam optimizer is used during training, with a learning rate of 0.001. The model updates its weights online every 5 minutes. An anomaly alert is triggered when the residual between the predicted and actual values ​​exceeds three times the standard deviation. For detected anomalies, the DBSCAN clustering algorithm is used for pattern analysis, with a neighborhood radius eps=0.5 and a minimum sample size min_samples=10, aggregating similar anomalies into events. Simultaneously, the percentage of anomalies within the sliding window is calculated; when the percentage exceeds 15% within 10 seconds, a critical alert is issued. All detection results are pushed in real time via a Kafka message queue. The message format uses Protocol Buffers serialization and includes fields such as timestamp, anomaly score, and cluster label. The system generates a statistical report every hour, including metrics such as the number of anomalies, type distribution, and trend changes, and stores them in Elasticsearch for visualization.

[0074] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A deep learning-based data center machine room operation and maintenance anomaly detection method, characterized in that, The method comprises: Real-time data stream is obtained from communication protocol data, data packet structure and time delay variation characteristics are extracted by preprocessing, dimension reduction is performed by principal component analysis to obtain a low-dimensional feature set; for the low-dimensional feature set, time series is analyzed by using a long short-term memory network to capture subtle patterns in low-dimensional dynamic characteristics to obtain a behavior pattern sequence; For the low-dimensional feature set, the long short-term memory network is used to analyze the time series, capture the subtle patterns in the low-dimensional dynamic characteristics, and obtain the behavior pattern sequence, comprising: Time series data is obtained from the low-dimensional feature set through a preset data interface, and the data is normalized by using a standardization processing method to obtain processed time series data; The processed time series data is trained by using a long short-term memory network, the hidden layer unit is set to capture the time dependence, and the intermediate feature representation containing dynamic characteristics is generated; According to the intermediate feature representation, the attention mechanism is applied to weight the low-dimensional dynamic characteristics, the subtle patterns of the key time steps are extracted, and the weighted feature vector is obtained; If the dimension of the weighted feature vector exceeds a preset threshold, dimension reduction processing is performed by a principal component analysis method to obtain a reduced feature vector; The reduced feature vector is subjected to pattern recognition by a preset classifier to generate a preliminary behavior pattern sequence; The preliminary behavior pattern sequence is obtained, a sliding window method is used to detect abnormal points in the sequence, and if an abnormal point is detected, an interpolation method is used for smoothing to obtain a smoothed behavior pattern sequence; According to the smoothed behavior pattern sequence, a clustering method is applied to group the sequence to determine the final behavior pattern sequence; According to the behavior pattern sequence, an adaptive behavior benchmark model is constructed, if the deviation between new data and the model exceeds a preset threshold, it is judged as a potential anomaly to obtain an anomaly candidate set; the data packet structure and time delay variation are extracted from the anomaly candidate set, and a second screening is performed by using an isolation forest algorithm to determine a hidden abnormal behavior set; for the hidden abnormal behavior set, the similarity of each abnormal mode to the behavior benchmark model is calculated, if the similarity is lower than a preset threshold, it is determined as an abnormal mode to obtain an abnormal mode set; according to the abnormal mode set, a sliding window mechanism is used to analyze the real-time data stream to determine whether the abnormal mode is continuously present to obtain an abnormal persistence score; According to the abnormal mode set, the sliding window mechanism is used to analyze the real-time data stream to determine whether the abnormal mode is continuously present to obtain an abnormal persistence score, comprising: Step 1, obtaining a real-time data stream, performing pattern matching by using a pre-established abnormal mode set to determine an initial abnormal event; Step 2, if the initial abnormal event is detected, a sliding window mechanism is used to divide a time window to obtain a data subset in the time window; Step 3, for the data subset in the time window, the frequency of occurrence of the abnormal mode is calculated to obtain a frequency distribution characteristic; Step 4, comparing the frequency distribution characteristic with a preset abnormal detection threshold to determine whether the abnormal mode is continuously present; Step 5, if the abnormal mode is continuously present, the abnormal persistence score is obtained according to a scoring calculation formula in combination with the frequency of occurrence of the abnormal mode and the duration. Step 6, update the pattern weight in the abnormal pattern set by using the persistent score, and obtain an optimized abnormal pattern set; Step 7, reanalyze the real-time data stream by using the optimized abnormal pattern set, and determine the next round of abnormal events; The above steps 1-7 form a logical closed loop, and the output and input of the upper and lower steps are closely connected, which are based on the abnormal pattern set, the sliding window mechanism and the real-time data stream, and are developed around the goal of the abnormal persistent score, thereby avoiding irrelevant attributes and ensuring that the technical process is closely related; For the abnormal persistent score, a weighted average method is used to fuse the weights of the data packet structure and the time delay change, and if the score exceeds the preset threshold, the abnormality is confirmed, and the final abnormality detection result is obtained. 2.The data center machine room operation and maintenance anomaly detection method based on deep learning of claim 1, characterized in that, The real-time data stream is obtained from the communication protocol data, the data packet structure and the time delay change characteristics are extracted by preprocessing, the principal component analysis is used for dimension reduction, and a low-dimensional feature set is obtained, including: The real-time data stream is obtained from the communication protocol data, the data packet structure and the time delay change characteristics are extracted by preprocessing, the principal component analysis is used for dimension reduction, and a low-dimensional feature set is obtained, including: The data packet sequence is obtained by analyzing the original data packet by using a data stream capture tool; The data packet sequence is analyzed by preprocessing to extract the data packet structure and time delay characteristics, and a structured feature set is obtained; If the dimension of the structured feature set is higher than the preset threshold, the principal component analysis is used for dimension reduction on the feature set to obtain a low-dimensional feature set; According to the low-dimensional feature set, the delay change trend of the time delay feature is calculated, and a delay change sequence is obtained; The abnormal pattern set is obtained by clustering analysis on the delay change sequence to determine the delay change trend. If there is a pattern in the abnormal pattern set that matches the preset abnormal template, the feature subset of the matching pattern is extracted to obtain an abnormal feature subset; 3.The data center machine room operation and maintenance anomaly detection method based on deep learning of claim 1, characterized in that, The abnormal feature subset and the low-dimensional feature set are compared and analyzed to determine the source of the abnormal feature, and an abnormal source distribution is obtained. The adaptive behavior benchmark model is constructed according to the behavior pattern sequence, and if the deviation of the new data from the model exceeds the preset threshold, it is judged as a potential abnormality, and an abnormal candidate set is obtained, including: An initial adaptive behavior benchmark model is constructed by using the behavior pattern sequence, and the sequence characteristics are extracted by using the time series analysis method to obtain the benchmark model; The deviation of the new data from the benchmark model is calculated, and the deviation value is quantified by using the mean square error method to obtain the deviation result; If the deviation result exceeds the preset threshold, the new data is marked as a potential abnormality, and a potential abnormality set is obtained; For the potential abnormality set, a clustering analysis method is used to group similar abnormalities to obtain an abnormal candidate set; The adaptive behavior benchmark model is updated by comparing the abnormal candidate set with the historical data to obtain an updated model; The deviation value of the new data is recalculated according to the updated model, and the new deviation result is obtained by using the mean square error method.

4. The deep learning-based data center machine room operation and maintenance anomaly detection method of claim 1, wherein, If the new deviation result still exceeds the preset threshold, the abnormal candidate is confirmed as an abnormality, and a final abnormality set is obtained. The data packet structure and time delay change are extracted from the abnormal candidate set, and a hidden abnormal behavior set is determined by using the isolation forest algorithm for secondary screening, including: The data packet structure feature and the time delay change feature are extracted from the abnormal candidate set, and the features are normalized by using a preprocessing method to obtain a standardized feature set; If the distribution of the data packet structure feature in the standardized feature set deviates from the preset threshold value, dimensionality reduction processing is performed on the feature to obtain a reduced feature set; Anomaly detection is performed on the reduced feature set by an isolation forest algorithm to obtain an anomaly score and determine a preliminary anomaly point set; According to the time delay change feature of the preliminary anomaly point set, a clustering algorithm is used to group the anomaly points to obtain an anomaly behavior cluster; If the number of the anomaly behavior cluster is lower than a preset threshold value, time series analysis is performed on the time delay change feature of each cluster to determine a hidden anomaly behavior; By performing correlation analysis on the structure feature and the delay change feature of the hidden anomaly behavior, a hidden anomaly behavior set is determined; According to the hidden anomaly behavior set, a rule matching method is used to verify the behavior set to obtain a final anomaly behavior set.

5. The deep learning-based data center machine room operation and maintenance anomaly detection method of claim 1, wherein, The similarity between each anomaly mode and the behavior benchmark model is calculated for the hidden anomaly behavior set, and if the similarity is lower than a preset threshold value, it is determined as an anomaly mode to obtain an anomaly mode set, including: By data preprocessing, behavior data is extracted from the hidden anomaly behavior to obtain a mode feature set; A principal component analysis algorithm is used to reduce the dimensionality of the mode feature set to obtain a low-dimensional feature set; According to the low-dimensional feature set and the behavior benchmark model, the similarity of each anomaly mode is calculated to obtain a similarity set; If the value in the similarity set is lower than a preset threshold value, the corresponding mode is determined as an anomaly to obtain a preliminary anomaly mode set; For the preliminary anomaly mode set, the time series feature of its behavior data is obtained to obtain a time series feature set; By an isolation forest algorithm, anomaly modes are detected from the time series feature set to obtain a refined anomaly mode set; According to the refined anomaly mode set, the deviation of its mode feature and the behavior benchmark model is extracted to obtain an anomaly mode deviation set.

6. The deep learning-based data center machine room operation and maintenance anomaly detection method of claim 1, wherein, For anomaly persistence scoring, a weighted average method is used to fuse the weights of the data packet structure and the time delay change, and if the score exceeds a preset threshold value, the anomaly is confirmed to obtain a final anomaly detection result, including: A data acquisition module is used to obtain data packet structure and time delay change information from network traffic, extract structure features and delay change features, and obtain an initial feature set; The structure features and delay change features in the initial feature set are fused by a preset weighted average method to calculate a weighted feature vector and obtain an anomaly persistence score; If the anomaly persistence score exceeds a preset threshold value, it is determined as an anomaly to generate an anomaly label and obtain a preliminary anomaly detection result; According to the preliminary anomaly detection result, a K-Means algorithm is used to perform clustering analysis on the anomaly label to divide the anomaly types and obtain a classified anomaly type set; Representative features of each type are extracted from the classified anomaly type set, and a decision tree algorithm is used to construct an anomaly type classification model to obtain an anomaly type prediction model; The anomaly type prediction model is used to perform real-time analysis on newly collected network traffic data to generate an anomaly type label and obtain a final anomaly detection result; After obtaining the final anomaly detection result, the result is stored in a preset database, and the anomaly detection log is updated to obtain an anomaly detection record.

7. The deep learning-based data center machine room operation and maintenance anomaly detection method of claim 1, wherein, The data center machine room operation and maintenance anomaly detection method further comprises: updating the parameters of the behavior benchmark model according to the final anomaly detection result, optimizing the model by using an online learning mechanism to obtain an updated behavior benchmark model. 8.The data center machine room operation and maintenance anomaly detection method based on deep learning of claim 7, characterized in that, The data center machine room operation and maintenance anomaly detection method further comprises: cyclically executing real-time data stream analysis for the updated behavior benchmark model, continuously capturing subtle patterns, and obtaining continuous anomaly detection output.

Images