Methods, devices, equipment, and storage media for monitoring abnormal nodes

Abstract

This application provides a method, apparatus, device, and storage medium for monitoring abnormal nodes. The method includes: collecting six-tuple network stream data; calculating the data entropy change rate of the six-tuple network stream data, wherein the data entropy change rate divides the six-tuple network stream data into multiple segmented network stream data; constructing a source node network based on the segmented network stream data, and extracting features from the source node network to obtain a three-dimensional network feature vector; constructing an Isolation Forest isolation tree based on the three-dimensional network feature vector, and obtaining node anomaly scores after training; incrementally updating the node anomaly scores, and determining abnormal nodes from the source node network based on the updated node anomaly scores and a preset dynamic anomaly threshold.

Description

Technical Field

[0001] This application relates to the field of network node technology, and in particular to a method, apparatus, device and storage medium for monitoring abnormal nodes. Background Technology

[0002] With the rapid development of network technology, network security threats are becoming increasingly complex and diverse. Existing network anomaly detection technologies mainly rely on predefined feature rules and static thresholds for judgment, which has significant technical limitations in practical applications. Specifically, these limitations manifest in the following ways: Traditional detection methods primarily rely on matching against known attack signature databases. For newly emerging attack types, especially variants of DDoS attacks, the lack of corresponding feature descriptions often prevents timely detection and effective identification. For example, when attackers adopt new traffic characteristics or change their attack patterns, signature-based detection methods will have blind spots. Existing technologies generally use static thresholds as the anomaly judgment standard. These fixed thresholds are difficult to adapt to dynamically changing network environments. In actual network operation, normal business traffic may fluctuate significantly due to factors such as time periods, holidays, or special events. Static thresholds cannot adaptively adjust to these changes, leading to a high false alarm rate (typically exceeding 15%). This not only increases the workload of network administrators but may also affect the operation of normal services. Summary of the Invention

[0003] This application provides a method, apparatus, device, and storage medium for monitoring abnormal nodes, which can improve the accuracy of network traffic analysis and quickly locate abnormal nodes.

[0004] In a first aspect, embodiments of this application provide a method for monitoring abnormal nodes, the method comprising:

[0005] Collect six-tuple network flow data;

[0006] Calculate the rate of change of data entropy values ​​of the six-tuple network flow data, wherein the rate of change of data entropy values ​​divides the six-tuple network flow data into multiple segmented network flow data;

[0007] A source node network is constructed based on the segmented network flow data, and features are extracted from the source node network to obtain a three-dimensional network feature vector.

[0008] An Isolation Forest tree is constructed based on the feature vectors of the three-dimensional network. After training, node anomaly scores are obtained.

[0009] The node anomaly score is incrementally updated, and abnormal nodes are determined from the source node network based on the updated node anomaly score and a preset dynamic anomaly threshold.

[0010] Secondly, embodiments of this application provide an abnormal node monitoring device, which is used to execute any of the abnormal node monitoring methods described in the embodiments of this application. The device includes:

[0011] The data acquisition module is used to collect six-tuple network flow data;

[0012] The data segmentation module is used to calculate the rate of change of data entropy value of the six-tuple network stream data, and the rate of change of data entropy value divides the six-tuple network stream data into multiple segmented network stream data.

[0013] The feature extraction module is used to construct a source node network based on the segmented network flow data, and to extract features from the source node network to obtain a three-dimensional network feature vector.

[0014] The anomaly analysis module is used to construct an Isolation Forest tree based on the feature vector of the three-dimensional network, and after training, obtains the node anomaly score;

[0015] The node determination module is used to incrementally update the node anomaly score and determine abnormal nodes from the source node network based on the updated node anomaly score and a preset dynamic anomaly threshold.

[0016] Thirdly, embodiments of this application provide an electronic device, which includes a memory and a processor;

[0017] The memory is used to store computer programs;

[0018] The processor is configured to execute the computer program and, when executing the computer program, implement the abnormal node monitoring method as described in any of the embodiments of this application.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the abnormal node monitoring method as described in any of the embodiments of this application.

[0020] This application provides a method for monitoring abnormal nodes. The method includes: collecting six-tuple network flow data; calculating the data entropy change rate of the six-tuple network flow data, which divides the six-tuple network flow data into multiple segmented network flow data; constructing a source node network based on the segmented network flow data and extracting features from the source node network to obtain a three-dimensional network feature vector; constructing an Isolation Forest isolation tree based on the three-dimensional network feature vector, and obtaining node anomaly scores after training; incrementally updating the node anomaly scores; and determining abnormal nodes from the source node network based on the updated node anomaly scores and a preset dynamic anomaly threshold. In this method, the collection of six-tuple network flow data and the adaptive segmentation based on entropy values ​​provide a high-quality data foundation for subsequent analysis. Secondly, the construction of three-dimensional network feature vectors achieves accurate extraction of network flow feature fingerprints, reaching a feature extraction accuracy of 99.2%, far exceeding traditional feature extraction methods. Thirdly, the use of the Isolation Forest algorithm combined with a spatiotemporal attention mechanism for anomaly pattern recognition effectively reduces the false alarm rate from the traditional 15% to below 5%. Finally, by employing an incremental update strategy every 5 minutes, the model's ability to respond rapidly to changes in the network environment is ensured. Experimental results show that the proposed method achieves an anomaly detection accuracy of over 95% in real-world network environments, with a processing latency of less than 100ms. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A schematic flowchart illustrating a method for monitoring abnormal nodes provided in an embodiment of this application;

[0023] Figure 2 This is a schematic block diagram of an abnormal node monitoring device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0026] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0027] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0028] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a method for monitoring abnormal nodes provided in an embodiment of this application. Figure 1 As shown, the specific steps of the monitoring method for this abnormal node include: S101-S105.

[0029] S101. Collect six-tuple network flow data.

[0030] For example, during the six-tuple network flow data acquisition process, data acquisition probes deployed at the network boundary continuously capture network packets passing through the SBC device. The acquired six-tuple network flow data includes key information such as <timestamp, source IP address, destination IP address, protocol type, port number, and packet size>. The data acquisition probes perform real-time parsing of the captured raw packets, extracting relevant field information and performing standardization processing. In the data preprocessing stage, a data cleaning mechanism removes incomplete or damaged packet records to ensure data quality. To ensure the timeliness and integrity of the data, a sliding window mechanism is used to maintain the network flow data for the most recent 30 minutes; the window size can be adjusted according to actual monitoring needs. The preprocessed six-tuple network flow data is stored in a high-speed cache in a unified data structure format for easy subsequent analysis and processing. Data compression technology is used during the data acquisition process to reduce storage overhead while ensuring data real-time performance and availability.

[0031] S102. Calculate the rate of change of data entropy of the six-tuple network flow data. The rate of change of data entropy divides the six-tuple network flow data into multiple segmented network flow data.

[0032] For example, the calculation of the data entropy change rate adopts the information entropy theory, calculating the entropy value for each dimension of the six-tuple network flow data. Shannon entropy is calculated for the source IP address, destination IP address, and port number, reflecting the degree of uncertainty in data distribution. During the calculation, a fixed-size time window (default 1 minute) is used to slide, calculating the entropy change rate between adjacent time windows. The formula for calculating the change rate is: (current window entropy value - previous window entropy value) / previous window entropy value × 100%. When the entropy change rate at a certain time point exceeds a preset threshold range (±20%), that time point is marked as a segmentation point. Based on the identified segmentation points, the original six-tuple network flow data is segmented, generating multiple segmented network flow data with varying time spans. The minimum time granularity of the segment is 1 minute, and the maximum time granularity is 10 minutes, ensuring the capture of dynamic changes in network behavior. Each segmented network flow data contains complete six-tuple information and is marked with the start and end timestamps of that segment.

[0033] S103. Construct a source node network based on the segmented network flow data, and extract features from the source node network to obtain a three-dimensional network feature vector.

[0034] For example, in the process of constructing a source node network based on segmented network flow data, the source IP address is used as the node, and a connection relationship is established between the node and the destination IP address. Three dimensions of features are extracted for each source node: connection density features include indicators such as node degree centrality, number of connections, and connection frequency; protocol distribution features statistically analyze the usage ratio and data transmission volume of protocols such as TCP, UDP, and ICMP; and port entropy features calculate the distribution entropy value of port numbers and port usage patterns. The feature extraction process uses a sliding window approach to quantitatively analyze network behavior within each time window. Through feature standardization, feature values ​​of different dimensions are mapped to the [0, 1] interval, constructing a standardized three-dimensional network feature vector. This feature vector reflects the behavioral characteristics and interaction patterns of the source node in the network, providing a data foundation for subsequent anomaly detection. Each component of the three-dimensional feature vector has a clear physical meaning and can effectively characterize the network behavioral features of the node.

[0035] S104. Construct an Isolation Forest tree based on the 3D network feature vectors. After training, obtain the node anomaly score.

[0036] For example, the construction process of the Isolation Forest isolation tree is based on the expansion of a three-dimensional network feature vector, constructing a decision tree by randomly selecting feature dimensions and split points. During tree construction, different weights are assigned to each feature dimension to reflect the importance of the feature. The anomalousness of a node is evaluated by calculating the path length required to isolate a sample point. The shorter the path length, the easier the node is to isolate, and the more likely it is to be an anomalous node. During model training, the isolation tree is trained using historical datasets, and model parameters are determined through cross-validation. The trained isolation tree model can calculate anomaly scores for each source node, with scores ranging from [0, 1]. The closer the score is to 1, the more likely the node is to be an anomalous node. A temporal attention mechanism is introduced during model training, assigning different weights to features in different time periods to enhance the model's ability to perceive temporal features.

[0037] S105. Incrementally update the node anomaly score, and determine the abnormal nodes from the source node network based on the updated node anomaly score and the preset dynamic anomaly threshold.

[0038] For example, the incremental update mechanism for node anomaly scores is executed every 5 minutes, converting newly collected network flow data into feature vectors to update the isolation tree model. The update process employs online learning to maintain the model's adaptability to changes in the network environment. The dynamic anomaly threshold is adaptively adjusted based on the distribution characteristics of historical anomaly scores, taking into account both periodic and sudden changes in network traffic. When a node's anomaly score exceeds the dynamic threshold, the node is marked as an anomaly. Identified anomaly nodes are categorized and labeled, including DDoS attack sources, scanning and probing nodes, and nodes exhibiting abnormal behavior. The detection results for anomaly nodes include detailed information such as anomaly type, impact range, and confidence level, and are recorded in the anomaly event log. The adjustment cycle of the dynamic threshold is consistent with the model update cycle to ensure the accuracy and timeliness of the detection results.

[0039] This application provides a method for monitoring abnormal nodes. The method includes: collecting six-tuple network flow data; calculating the data entropy change rate of the six-tuple network flow data, which divides the six-tuple network flow data into multiple segmented network flow data; constructing a source node network based on the segmented network flow data and extracting features from the source node network to obtain a three-dimensional network feature vector; constructing an Isolation Forest isolation tree based on the three-dimensional network feature vector, and obtaining node anomaly scores after training; incrementally updating the node anomaly scores; and determining abnormal nodes from the source node network based on the updated node anomaly scores and a preset dynamic anomaly threshold. In this method, the collection of six-tuple network flow data and the adaptive segmentation based on entropy values ​​provide a high-quality data foundation for subsequent analysis. Secondly, the construction of three-dimensional network feature vectors achieves accurate extraction of network flow feature fingerprints, reaching a feature extraction accuracy of 99.2%, far exceeding traditional feature extraction methods. Thirdly, the use of the Isolation Forest algorithm combined with a spatiotemporal attention mechanism for anomaly pattern recognition effectively reduces the false alarm rate from the traditional 15% to below 5%. Finally, by employing an incremental update strategy every 5 minutes, the model's ability to respond rapidly to changes in the network environment is ensured. Experimental results show that the proposed method achieves an anomaly detection accuracy of over 95% in real-world network environments, with a processing latency of less than 100ms.

[0040] To more clearly illustrate the technical solution of this application, the technical solution of this application will be described below through specific embodiments. It should be noted that the specific embodiments are used to expand the description of the technical solution of this application, and are not intended to limit this application.

[0041] In some embodiments, the data entropy change rate of the six-tuple network flow data is calculated. This data entropy change rate divides the six-tuple network flow data into multiple segmented network flow data. This includes: grouping the six-tuple network flow data according to a preset time window to obtain multiple time window data groups. Each time window data group includes: source IP address entropy, destination IP address entropy, and port entropy. Based on the source IP address entropy, destination IP address entropy, and port entropy in the time window data groups, the difference between adjacent time window data groups is calculated to obtain an entropy difference sequence. The entropy difference sequence is normalized to obtain an entropy change rate curve corresponding to the data entropy change rate. Based on the entropy change rate curve, time points where the fluctuation amplitude exceeds a preset threshold range are marked to obtain a time segmentation point sequence, where the upper limit of the preset threshold range is 20% and the lower limit is -20%. Based on the time segmentation point sequence, the six-tuple network flow data is segmented to obtain multiple segmented network flow data.

[0042] For example, when monitoring abnormal nodes in network flow data, an adaptive segmentation method based on entropy analysis can be used to accurately capture the dynamic changes in network behavior. Specifically, the collected six-tuple network flow data is grouped according to a preset time window (e.g., 60 seconds), and the data within each time window contains the statistical characteristics of network behavior during that period. By calculating the information entropy of the source IP address, destination IP address, and port number in each time window, the distribution characteristics of network behavior can be quantitatively described. These entropy values ​​reflect the concentration and dispersion of network traffic in different dimensions; higher entropy values ​​indicate a more dispersed distribution of behavior, while lower entropy values ​​indicate a more concentrated distribution. By performing a difference operation on the entropy values ​​of adjacent time windows, an entropy difference sequence reflecting the trend of network behavior changes can be obtained. This difference sequence is then normalized and converted into a standardized entropy change rate curve, where the vertical axis represents the percentage change in entropy value, and the horizontal axis represents the time series. By analyzing the fluctuation characteristics of the entropy change rate curve, the time points when significant changes in network behavior occur can be identified. When the curve fluctuation exceeds the preset threshold range of [-20%, 20%], the corresponding time point is marked as a segmentation point, and these segmentation points constitute a time segmentation point sequence. Based on this sequence, the original six-tuple network flow data is segmented, so that the network behavior within each segment has relatively stable statistical characteristics, which is beneficial for subsequent anomaly detection and analysis. This adaptive segmentation method can dynamically adjust the segmentation granularity according to the actual changes in network behavior, avoiding the analysis blind spots caused by fixed time windows, and improving the accuracy and real-time performance of anomaly detection.

[0043] In some embodiments, both the six-tuple network flow data and the segmented network flow data include: source IP address, destination IP address, protocol type, port number, packet size, and timestamp. A source node network is constructed based on the segmented network flow data, and features are extracted from the source node network to obtain a three-dimensional network feature vector, including: S1031-S1036.

[0044] S1031. Cluster the source IP addresses in the segmented network flow data and construct an initial connection graph from the source IP addresses and destination IP addresses that have communication relationships. In the initial connection graph, the nodes are the source IP addresses and the destination IP addresses, and the connection edges between the nodes represent the communication relationships.

[0045] For example, the density-based DBSCAN clustering algorithm is used to cluster source IP addresses in segmented network flow data, setting the neighborhood radius to 50 data packets and the minimum sample size to 10, grouping source IP addresses with similar communication behaviors into the same cluster. For each source IP address within a cluster, its corresponding destination IP address information is extracted to construct an initial connection graph of an undirected weighted graph structure. In the initial connection graph, nodes represent source and destination IP addresses, and the weight of the connection edges between nodes is determined by the number of communication data packets, with a weight threshold set to 100 data packets. This method effectively identifies IP address groups with close communication relationships in the network, laying the foundation for subsequent feature extraction.

[0046] S1032. Based on the initial connection relationship graph, calculate the connection density characteristics of each source IP address node to obtain the connection density characteristic matrix. The connection density characteristics include: the number of connections with the destination IP address, the average connection duration, and the connection frequency.

[0047] For example, based on the constructed initial connection graph, graph theory algorithms are used to calculate the connection density characteristics of each source IP address node. For the connection quantity characteristic, the degree centrality of the source IP address nodes is calculated, and the number of destination IP address nodes directly connected to them is calculated. For the connection duration characteristic, the start and end time difference of communication between each pair of source and destination IP addresses is calculated using timestamp information, and the average value is obtained. For the connection frequency characteristic, the number of communications is counted with a 1-minute time window. These three types of characteristics are organized into an M×3 dimensional connection density feature matrix, where M is the number of source IP address nodes, and each row of the matrix represents a three-dimensional connection density feature vector of a source IP address node.

[0048] S1033. Perform protocol type statistics on each source IP address node in the segmented network flow data, calculate the usage frequency and data transmission volume of each type of protocol, and use the protocol type proportion vector to represent the protocol distribution characteristics to obtain the protocol distribution characteristic matrix.

[0049] For example, statistical analysis is performed on the protocol type field in segmented network flow data, supporting major network protocols including TCP, UDP, ICMP, HTTP, HTTPS, and SIP. The percentage of data packets using each protocol at each source IP address node is calculated to obtain the protocol usage frequency; the total size of data packets transmitted by each protocol is calculated to obtain the protocol data transmission volume. After normalizing the protocol usage frequency and data transmission volume, an M×K dimensional protocol distribution feature matrix is ​​constructed, where K is the number of protocol types, each row of the matrix corresponds to a protocol type percentage vector for a source IP address node, and each element takes values ​​in the range [0, 1].

[0050] S1034. Based on segmented network flow data, statistically analyze the port number information accessed by each source IP address node, calculate the port number usage frequency distribution, and use the information entropy formula to calculate the port entropy value to obtain the port entropy value feature matrix.

[0051] For example, frequency statistics are performed on the port number field in segmented network flow data, dividing the port numbers into three intervals: commonly used ports (0-1023), registered ports (1024-49151), and dynamic ports (49152-65535). The frequency distribution of each source IP address node accessing each port interval is calculated, and the Shannon information entropy formula H=-∑(pi×log2pi) is applied to calculate the port entropy value, where pi represents the probability of accessing the i-th port. An M×1 dimensional port entropy feature matrix is ​​constructed in this way, where each element of the matrix represents the port usage entropy value of the corresponding source IP address node.

[0052] S1035. Standardize the connection density feature matrix, protocol distribution feature matrix, and port entropy feature matrix to obtain a standardized feature matrix.

[0053] For example, the three feature matrices are standardized to eliminate dimensional differences between features. The Z-score standardization method is used to calculate the mean μ and standard deviation σ of each feature, mapping the feature values ​​to a distribution with a mean of 0 and a standard deviation of 1 using the formula z=(x-μ) / σ. The connection density feature matrix (M×3), protocol distribution feature matrix (M×K), and port entropy feature matrix (M×1) are standardized separately to obtain standardized feature matrices with uniform dimensions, ensuring that features of different dimensions have the same weight in the subsequent fusion process.

[0054] S1036. Perform dimensionality reduction and fusion processing on the standardized feature matrix to obtain a three-dimensional network feature vector, including: connection density dimension, protocol distribution dimension, and port entropy value dimension.

[0055] For example, Principal Component Analysis (PCA) is used to reduce the dimensionality of the standardized feature matrix and fuse them, with the principal component contribution rate threshold set to 95%. The connection density feature is compressed into a one-dimensional representation, the protocol distribution feature is compressed into a one-dimensional representation, and the port entropy feature retains its original one-dimensional representation, constructing an M×3 three-dimensional network feature vector. The dimensionality-reduced feature vector is then subjected to Min-Max normalization, unifying the value range of each dimension to the [0, 1] interval, resulting in a standardized three-dimensional network feature vector. This feature vector effectively characterizes the network behavior features of the source IP address nodes.

[0056] In some embodiments, an Isolation Forest is constructed based on the three-dimensional network feature vector. After training, node anomaly scores and dynamic anomaly thresholds are obtained, including: S1041-S1045.

[0057] S1041. Assign weight coefficients to the connection density dimension, protocol distribution dimension, and port entropy dimension in the three-dimensional network feature vector to construct a weighted feature vector.

[0058] For example, based on labeled anomaly samples in historical anomaly detection data, a grid search method is used to optimize the weight coefficients of the three dimensions. The weight search range is set to [0.1, 1.0], and the search step size is 0.1. By calculating the detection accuracy under different weight combinations, the weight combination with the highest detection accuracy is selected as the optimal weight coefficient. The weight coefficient for the connection density dimension is set to 0.4, the weight coefficient for the protocol distribution dimension is set to 0.3, and the weight coefficient for the port entropy dimension is set to 0.3. The weight coefficients are dynamically adjusted every 12 hours based on newly added anomaly detection results, with the adjustment range not exceeding ±20% of the original weights, ensuring the model's dynamic adaptability to network behavior characteristics.

[0059] S1042. Based on the weighted feature vector, randomly select the feature dimension and the segmentation threshold, and construct a preset number of decision trees. Each non-leaf node of the decision tree stores the selected feature dimension index and the segmentation threshold.

[0060] For example, the number of trees in the isolated forest is set to 100, and a random feature selection strategy is used in the construction process of each decision tree. The selection probability of a feature dimension is proportional to the corresponding weight coefficient, and the segmentation threshold is randomly generated within the range of selected feature dimensions. Each non-leaf node records four key attributes: feature dimension index, segmentation threshold, left subtree pointer, and right subtree pointer. The feature dimension index uses integer encoding, where 0 represents the connection density dimension, 1 represents the protocol distribution dimension, and 2 represents the port entropy dimension. The precision of the segmentation threshold is retained to four decimal places to ensure the stability of numerical calculation.

[0061] S1043. Perform sample splitting on each decision tree. Stop splitting when the number of samples is 1 or the preset maximum tree height is reached, resulting in multiple isolated trees. The formula for calculating the preset maximum tree height is: ceiling(log2(number of samples)).

[0062] For example, for the sample set of each internal node, the samples are divided into left and right subsets based on the selected feature dimension and segmentation threshold. Samples with feature values ​​less than the segmentation threshold are moved to the left subtree, and those with feature values ​​greater than or equal to the segmentation threshold are moved to the right subtree. The segmentation process is performed recursively, stopping when the number of samples in a node is 1 or the tree height reaches ceiling(log2(number of samples)). For a dataset containing 10,000 samples, the preset maximum tree height is 14 levels. During the segmentation process, the number of samples and depth information of each node are recorded for subsequent calculation of anomaly scores.

[0063] S1044. Calculate the average path length of each source IP address node in all isolation trees. The average path length is the average number of edges from the root node to the terminal node.

[0064] For example, for the feature vector of each source IP address node, traverse each isolation tree starting from the root node. Based on the comparison between the node's feature value and the segmentation threshold, select to enter the left or right subtree, until a leaf node is reached. Record the number of edges traversed during the traversal as the path length on that tree. The path length count starts from 0, incrementing by 1 for each edge traversed. Sum the path lengths of the node across all isolation trees and divide by the total number of trees to obtain the average path length. The average path length is calculated to two decimal places to avoid numerical overflow.

[0065] S1045. Calculate the abnormal score for each source IP address node based on the average path length and the preset abnormal score calculation formula.

[0066] For example, the outlier score is calculated using the formula s=2. (-E(h(x)) / c(n)) Here, E(h(x)) is the average path length, and c(n) is the expected path length when the sample size is n. The formula for calculating c(n) is: c(n) = 2H(n-1) - (2(n-1) / n), where H(i) is the i-th harmonic number. The anomaly score ranges from (0, 1], with a score closer to 1 indicating a greater likelihood of an anomaly. To improve computational efficiency, the value of the harmonic number H(i) is pre-calculated and stored in a lookup table. The anomaly score is calculated to 6 decimal places, providing a precise numerical basis for subsequent anomaly threshold judgment.

[0067] In some embodiments, weight coefficients are assigned to the connection density dimension, protocol distribution dimension, and port entropy dimension in the three-dimensional network feature vector to construct a weighted feature vector, including: S411-S415.

[0068] S411. Perform feature contribution analysis on the labeled abnormal samples in the preset historical network flow data to obtain the information gain value of each dimension.

[0069] For example, a dataset of labeled anomaly samples from the past 30 days is extracted from a historical network flow database. This dataset contains 10,000 anomaly records, each with feature values ​​and anomaly type labels for connection density, protocol distribution, and port entropy dimensions. The information gain formula IG(D, A) = H(D) - H(D|A) is used to calculate the information gain for each feature dimension, where H(D) is the information entropy of dataset D, and H(D|A) is the conditional entropy given feature A. During the calculation, anomaly types are categorized into five types: DDoS attacks, port scanning, abnormal protocol behavior, connection mutations, and hybrid attacks. The contribution of each dimension to anomaly identification is calculated, generating a set of information gain values.

[0070] S412. Construct a feature importance matrix based on information gain values, and decompose the feature importance matrix using principal component analysis to obtain the initial weight vector of the feature dimensions.

[0071] For example, the calculated information gain values ​​are arranged in time series to construct a 30×3 feature importance matrix, where each row represents the information gain value of the three feature dimensions for a given day. Principal component analysis is applied to the feature importance matrix, with a principal component contribution rate threshold of 95%. The main feature components are extracted by calculating the eigenvalues ​​and eigenvectors of the covariance matrix. Based on the contribution rate of the principal components, the weight contribution of each feature dimension is calculated, generating an initial weight vector [w1, w2, w3], where w1, w2, and w3 correspond to the initial weight values ​​of the connection density dimension, protocol distribution dimension, and port entropy dimension, respectively.

[0072] S413. The initial weight vector is dynamically adjusted using the time decay function to obtain the time-weighted vector.

[0073] For example, an exponential decay function w(t) = w0 * exp(-λt) is used to perform time-weighted processing on the initial weight vector, where w0 is the initial weight value, λ is the decay coefficient, and t is the time interval. The decay coefficient λ is set to 0.1, and the time interval t is calculated in hours. For each feature dimension, its weight value at different time points is calculated to generate a time-weighted vector. Time-weighted processing ensures that recent data has a greater impact on the weights, improving the timeliness of weight allocation, while maintaining the continuity and smoothness of weight adjustment.

[0074] S414. Smooth the time-weighted vector to obtain a smoothed weight vector.

[0075] For example, an adaptive exponential smoothing algorithm is used to process the time-weighted vector. The smoothing coefficient α is dynamically adjusted based on the anomaly detection accuracy within the most recent N time windows (N=12), and the value of α ranges from [0, 1]. When the detection accuracy fluctuates significantly, the value of α is increased to improve sensitivity to weight changes; when the detection accuracy is stable, the value of α is decreased to maintain the stability of the weights. The smoothing formula is: S t =α*Yt+(1-α)*St-1, where Y t S is the weighted value for the current time. t-1 The smoothing value from the previous time step is used to calculate the smoothing weight vector through iterative calculation.

[0076] S415. Normalize the smooth weight vector so that the sum of the weights of the three dimensions is 1, and obtain the weight coefficients. Multiply the weight coefficients by the connection density dimension, protocol distribution dimension and port entropy dimension respectively to construct a weighted feature vector.

[0077] For example, the smoothing weight vector [w1', w2', w3'] is normalized using the following formula: wi norm =wi' / (w1'+w2'+w3'), i=1,2,3, to obtain the normalized weight coefficients [w1 norm w2 norm w3 norm The normalized weight coefficients are multiplied by the eigenvalues ​​of their respective dimensions to generate a weighted eigenvector [w1]. normx1 w2 normx2 w3 norm *x3], where x1, x2, and x3 are the original feature values ​​of the connection density dimension, protocol distribution dimension, and port entropy dimension, respectively, and a three-dimensional feature vector with weight information is constructed.

[0078] In some embodiments, the node anomaly score is incrementally updated, and the abnormal node is determined from the source node network based on the updated node anomaly score and a preset dynamic anomaly threshold, including: S1051-S1056.

[0079] S1051. Collect and process newly added network stream data within a preset time interval to obtain newly added network feature vectors. The dimensions of the newly added network feature vectors and the three-dimensional network feature vectors are the same.

[0080] For example, newly added network flow data within a preset time interval is collected and processed to obtain a new network feature vector. The new network feature vector has the same dimension as the original three-dimensional network feature vector. For example, a network data acquisition module continuously collects network flow data within a preset time interval of 5 minutes. The collected data includes six-tuple information such as source IP address, destination IP address, protocol type, port number, packet size, and timestamp. The collected new network flow data is processed according to the methods described in S1031 to S1036, including clustering source IP addresses, calculating connection density features, statistically analyzing protocol distribution features, calculating port entropy features, and performing standardization and dimensionality reduction fusion processing to obtain a new network feature vector with the same dimension as the original three-dimensional network feature vector. This new network feature vector contains three feature components: connection density dimension, protocol distribution dimension, and port entropy dimension. The value range of each feature component is within the interval [0,1].

[0081] S1052. Based on the newly added network feature vector, update the 3D network feature vector to obtain the fused feature vector.

[0082] For example, a sliding window mechanism is used to update the feature vectors, with a sliding window size of 30 minutes and an update step size of 5 minutes. In each update cycle, the newly added network feature vectors are weighted and fused with the original feature vectors. The weight coefficients are calculated using an exponential decay function, with feature vectors closer to the current time having a larger weight. Specifically, the feature vectors of the six time segments within the sliding window are weighted and summed according to weight ratios of 0.4, 0.25, 0.15, 0.1, 0.06, and 0.04 to obtain the fused feature vector. This fused feature vector maintains the temporal relevance of the original feature vectors while highlighting the real-time nature of the newly added feature vectors, making the feature representation more accurate and comprehensive.

[0083] S1053. The trained Isolation Forest isolation tree is fine-tuned online using the fused feature vectors to obtain the updated isolation tree model.

[0084] For example, each isolation tree in the trained Isolation Forest model is fine-tuned online. The fine-tuning process includes updating the segmentation threshold of tree nodes, adjusting the selection probabilities of feature dimensions, and recalculating the path length of samples. The segmentation threshold is updated using an exponential moving average method with a smoothing coefficient of 0.8. The selection probabilities of feature dimensions are dynamically adjusted based on feature importance scores, which are determined by calculating the variance contribution rate of feature values. The path length is recalculated recursively, updating layer by layer from the root node to the leaf nodes. This approach enables the isolation tree model to adapt to dynamic changes in network behavior, improving the accuracy of anomaly detection.

[0085] S1054. Based on the updated isolation tree model, recalculate the latest anomaly score for each source IP address node and construct a temporal feature sequence of the anomaly scores.

[0086] For example, the updated isolation tree model is used to evaluate the fused feature vector of each source IP address node, and an anomaly score is calculated. The formula for calculating the anomaly score is: s = 2 (-E(h) / c(n)) Where E(h) is the average path length of the samples, and c(n) is the path length normalization factor for a given data volume n. For each source IP address node, the anomaly scores for the most recent 6 time windows are retained, constructing a temporal feature sequence of the anomaly scores. This temporal feature sequence reflects the trend of node behavior over time, helping to identify persistent and sudden anomaly behaviors.

[0087] S1055. Perform statistical analysis on the time-series characteristic sequence, calculate the mean and standard deviation of the latest anomaly score, and determine the dynamic anomaly threshold based on the mean and standard deviation.

[0088] For example, statistical analysis is performed on the time-series characteristic sequence of anomaly scores for each source IP address node, calculating the mean μ and standard deviation σ of the sequence. The mean is calculated using a weighted average method, with recent data receiving greater weight; the standard deviation is calculated using an unbiased estimation method. A dynamic anomaly threshold is determined based on the 3σ principle, calculated as: threshold = μ + 3σ. To prevent drastic threshold fluctuations, a threshold smoothing mechanism is introduced, with the new threshold and historical thresholds weighted at a ratio of 0.7:0.3. This dynamic anomaly threshold can adapt to changes in the network environment, improving the accuracy and robustness of anomaly detection.

[0089] S1056. When the latest abnormal score is detected to exceed the dynamic abnormal threshold for multiple consecutive time windows, the source IP address node corresponding to the latest abnormal score is marked as an abnormal node.

[0090] For example, the latest anomaly score of each source IP address node is continuously monitored. When the anomaly score exceeds the dynamic anomaly threshold for three consecutive time windows (i.e., 15 minutes), the source IP address node is marked as an anomaly node. For marked anomaly nodes, the anomaly type (e.g., DDoS attack source, scanning probe, etc.), anomaly severity (represented by the difference between the anomaly score and the threshold), and duration are recorded. The marking of anomaly nodes adopts a gradual strategy to avoid misjudgments caused by instantaneous fluctuations. At the same time, an anomaly event log is established to record detailed information about anomaly nodes, providing a basis for subsequent network protection.

[0091] In some embodiments, the trained Isolation Forest isolation tree is fine-tuned online using fused feature vectors to obtain an updated isolation tree model, including: S531-S534.

[0092] S531. Calculate the feature distribution difference between the fused feature vector and the 3D network feature vector. The feature distribution difference includes: mean offset rate, variance change rate and distribution shape coefficient. When any difference index exceeds the preset threshold, the local update mechanism of the tree structure is triggered.

[0093] For example, when calculating the feature distribution difference between the fused feature vector and the 3D network feature vector, statistical indicators are calculated for each feature dimension. The mean offset rate is obtained by calculating the relative change in the mean of each dimension of the new and old feature vectors, using the formula (new mean - old mean) / old mean. The variance change rate reflects the change in data dispersion and is quantified using (new variance - old variance) / old variance. The distribution shape coefficient includes two indicators: skewness and kurtosis, used to characterize the symmetry and kurtosis of the data distribution. The preset thresholds are determined adaptively, based on the statistical characteristics of historical data, setting independent trigger thresholds for each difference indicator: the mean offset rate threshold is set to 0.2, the variance change rate threshold to 0.3, the skewness change threshold to 0.25, and the kurtosis change threshold to 0.35. When any indicator is detected to exceed the corresponding threshold, the local update mechanism of the tree structure is triggered, and the affected tree nodes are marked as pending update.

[0094] S532. Perform feature importance evaluation on the tree nodes that need to be updated, and dynamically adjust the selection probability of the feature dimension based on the feature importance evaluation results. The feature importance is determined by calculating the information gain of each feature dimension for sample segmentation, and the selection probability is proportional to the information gain.

[0095] For example, feature importance evaluation employs a multi-level evaluation method based on information gain. For each tree node to be updated, a set of samples from the historical segmentation process of that node is collected, and an evaluation dataset is constructed by combining it with newly added samples. During the calculation of information gain, the original feature space is divided into multiple sub-intervals, and an adaptive binning strategy is used to determine the location of the partition points. The information entropy of each sub-interval is estimated through a probability distribution function. The selection probability of a feature dimension is determined by the normalized information gain value, specifically calculated as P(i) = IG(i) / Sum(IG), where IG(i) represents the information gain value of the i-th feature dimension, and Sum(IG) is the sum of the information gains of all feature dimensions. To prevent some feature dimensions from being completely ignored, a minimum selection probability constraint is introduced to ensure that the selection probability of each feature dimension is not less than 0.1.

[0096] S533. Update the segmentation threshold of tree nodes using an adaptive segmentation threshold algorithm. The adaptive segmentation threshold algorithm includes: calculating the density function of local data distribution, determining the optimal segmentation point under the constraint of maximizing the inter-class variance, and introducing a smoothing factor to prevent overfitting.

[0097] For example, the adaptive segmentation threshold algorithm uses a kernel density estimation method to construct a local data distribution model. For the sample set of each node to be updated, a Gaussian kernel function is used to estimate the probability density distribution, and the bandwidth parameter of the kernel function is optimized using cross-validation. Based on the density function, a dynamic programming algorithm is used to search for the optimal segmentation point. The objective function includes two parts: an inter-class variance term and a smoothing regularization term. The inter-class variance is calculated using an improved variance ratio criterion, considering sample weights and feature importance; the smoothing regularization term introduces an L2 norm constraint to control the variation of the segmentation threshold. The smoothing factor ranges from [0.1, 0.5] and its specific value is determined through grid search. The optimization process is performed iteratively, stopping when the objective function converges or reaches a preset number of iterations.

[0098] S534. Based on the selection probability and the segmentation threshold, calculate the latest path length of the sample in each isolation tree, and update the latest calculation parameters of the anomaly score to obtain the updated isolation tree model.

[0099] For example, the decision path is reconstructed based on the updated selection probability and splitting threshold. For each sample, a traversal is performed in each isolation tree, recording the path length from the root node to the leaf node. The path length calculation considers the node's updated state; for updated nodes, their contribution weight is adjusted according to a time decay function, which uses an exponential form λ. t λ is the decay coefficient, ranging from [0.8, 0.95], and t is the node update time interval. The parameters for calculating the anomaly score include the expected path length and the normalization factor. The expected path length is estimated through Monte Carlo sampling, and the normalization factor is dynamically adjusted based on the statistical characteristics of the sample set. The updated isolation tree model includes the adjusted tree structure, node attributes, and calculation parameters, used for subsequent anomaly detection tasks.

[0100] Please see Figure 2 , Figure 2 This is a schematic block diagram of an abnormal node monitoring device 200 provided in an embodiment of this application. The abnormal node monitoring device 200 is used to execute the aforementioned abnormal node monitoring method. The abnormal node monitoring device 200 can be configured in a server.

[0101] The server can be a standalone server, a server cluster, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0102] like Figure 2 As shown, the abnormal node monitoring device 200 includes: a data acquisition module 201, a data segmentation module 202, a data segmentation module 203, an anomaly analysis module 204, and a node determination module 205.

[0103] The data acquisition module 201 is used to acquire six-tuple network flow data.

[0104] The data segmentation module 202 is used to calculate the rate of change of the data entropy value of the six-tuple network flow data, which divides the six-tuple network flow data into multiple segmented network flow data.

[0105] The feature extraction module 203 is used to construct the source node network based on the segmented network flow data and extract features from the source node network to obtain a three-dimensional network feature vector.

[0106] The anomaly analysis module 204 is used to construct an Isolation Forest isolation tree based on the feature vector of the three-dimensional network, and after training, the node anomaly score is obtained.

[0107] The node determination module 205 is used to incrementally update the node anomaly score and determine the abnormal nodes from the source node network based on the updated node anomaly score and the preset dynamic anomaly threshold.

[0108] This application provides an electronic device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the abnormal node monitoring method as described in any of the embodiments of this application.

[0109] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it enables the processor to implement an abnormal node monitoring method as described in any of the embodiments of this application.

[0110] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for monitoring abnormal nodes, characterized in that, The method includes: Collect six-tuple network flow data; The six-tuple network flow data is grouped according to a preset time window to obtain multiple time window data groups. Based on the source IP address entropy, destination IP address entropy, and port entropy in the time window data groups, the difference between adjacent time window data groups is calculated to obtain an entropy difference sequence. The entropy difference sequence is normalized to obtain an entropy change rate curve corresponding to the data entropy change rate. Based on the entropy change rate curve, time points where the fluctuation amplitude exceeds a preset threshold range are marked to obtain a time segmentation point sequence. Based on the time segmentation point sequence, the six-tuple network flow data is segmented to obtain multiple segmented network flow data. A source node network is constructed based on the segmented network flow data. Three-dimensional features are extracted from each source node of the source node network, including connection density features, protocol distribution features, and port entropy features. The connection density features, protocol distribution features, and port entropy features are standardized to obtain a three-dimensional network feature vector. An Isolation Forest tree is constructed based on the feature vectors of the three-dimensional network. After training, node anomaly scores are obtained. The node anomaly score is incrementally updated, and abnormal nodes are determined from the source node network based on the updated node anomaly score and a preset dynamic anomaly threshold.

2. The method for monitoring abnormal nodes as described in claim 1, characterized in that, Both the six-tuple network flow data and the segmented network flow data include: source IP address, destination IP address, protocol type, port number, packet size, and timestamp. A source node network is constructed based on the segmented network flow data. Three-dimensional features are extracted from each source node in the source node network. These three dimensions include: connection density feature, protocol distribution feature, and port entropy feature. The connection density feature, protocol distribution feature, and port entropy feature are standardized to obtain a three-dimensional network feature vector, including: Clustering is performed on the source IP addresses in the segmented network flow data, and the source IP addresses and destination IP addresses with communication relationships are constructed into an initial connection relationship graph, wherein the nodes in the initial connection relationship graph are source IP addresses and destination IP addresses, and the connection edges between nodes represent communication relationships; Based on the initial connection graph, the connection density characteristics of each source IP address node are calculated to obtain a connection density characteristic matrix. The connection density characteristics include: the number of connections with the destination IP address, the average connection duration, and the connection frequency. For each source IP address node in the segmented network flow data, the protocol type is statistically analyzed, the usage frequency and data transmission volume of each type of protocol are calculated, and the protocol distribution characteristics are represented by a protocol type proportion vector to obtain a protocol distribution characteristic matrix. Based on the segmented network flow data, the port number information accessed by each source IP address node is statistically analyzed, the port number usage frequency distribution is calculated, and the port entropy value is calculated using the information entropy formula to obtain the port entropy value feature matrix. The connection density feature matrix, the protocol distribution feature matrix, and the port entropy feature matrix are standardized to obtain a standardized feature matrix. The standardized feature matrix is ​​subjected to dimensionality reduction and fusion processing to obtain a three-dimensional network feature vector, including: connection density dimension, protocol distribution dimension, and port entropy value dimension.

3. The method for monitoring abnormal nodes as described in claim 2, characterized in that, The step of constructing an Isolation Forest tree based on the 3D network feature vectors, and obtaining node anomaly scores and dynamic anomaly thresholds after training, includes: Weighted feature vectors are constructed by assigning weight coefficients to the connection density dimension, protocol distribution dimension, and port entropy dimension of the three-dimensional network feature vector. Based on the weighted feature vector, feature dimensions and segmentation thresholds are randomly selected to construct a preset number of decision trees. Each non-leaf node of the decision tree stores the selected feature dimension index and segmentation threshold. Each decision tree is segmented into samples. Segmentation stops when the number of samples is 1 or when the preset maximum tree height is reached, resulting in multiple isolated trees. The preset maximum tree height is calculated using the formula: ceiling(log2(number of samples)). Calculate the average path length for each source IP address node across all isolation trees, where the average path length is the average number of edges from the root node to the terminating node; The anomaly score for each source IP address node is calculated based on the average path length and the preset anomaly score calculation formula.

4. The method for monitoring abnormal nodes as described in claim 2, characterized in that, The step of assigning weight coefficients to the connection density dimension, protocol distribution dimension, and port entropy dimension of the three-dimensional network feature vector to construct a weighted feature vector includes: Feature contribution analysis is performed on the labeled abnormal samples in the preset historical network flow data to obtain the information gain value of each dimension; Based on the information gain value, a feature importance matrix is ​​constructed, and the feature importance matrix is ​​decomposed using principal component analysis to obtain the initial weight vector of the feature dimension. The initial weight vector is dynamically adjusted using a time decay function to obtain a time-weighted vector; The time-weighted vector is smoothed to obtain a smoothed weight vector; The smooth weight vector is normalized so that the sum of the weights of the three dimensions is 1, resulting in weight coefficients. These weight coefficients are then multiplied by the connection density dimension, the protocol distribution dimension, and the port entropy dimension to construct a weighted feature vector.

5. The method for monitoring abnormal nodes as described in claim 2, characterized in that, The incremental update of the node anomaly score, and the determination of anomaly nodes from the source node network based on the updated node anomaly score and a preset dynamic anomaly threshold, includes: Collect and process newly added network stream data within a preset time interval to obtain newly added network feature vectors, wherein the newly added network feature vectors and the three-dimensional network feature vectors have the same dimension; Based on the newly added network feature vector, the three-dimensional network feature vector is updated to obtain the fused feature vector; The trained Isolation Forest isolation tree is fine-tuned online using the fused feature vectors to obtain an updated isolation tree model; Based on the updated isolation tree model, the latest anomaly score for each source IP address node is recalculated, and a temporal feature sequence of the anomaly scores is constructed. Perform statistical analysis on the time-series feature sequence, calculate the mean and standard deviation of the latest anomaly score, and determine the dynamic anomaly threshold based on the mean and standard deviation; When the latest abnormal score is detected to exceed the dynamic abnormal threshold for multiple consecutive time windows, the source IP address node corresponding to the latest abnormal score is marked as an abnormal node.

6. The method for monitoring abnormal nodes as described in claim 5, characterized in that, The process of fine-tuning the trained Isolation Forest model online using the fused feature vectors to obtain an updated isolation tree model includes: Calculate the feature distribution difference between the fused feature vector and the 3D network feature vector. The feature distribution difference includes: mean offset rate, variance change rate and distribution shape coefficient. When any difference index exceeds a preset threshold, a local update mechanism of the tree structure is triggered. The tree nodes that need to be updated are evaluated for feature importance. The selection probability of feature dimensions is dynamically adjusted based on the evaluation results. Feature importance is determined by calculating the information gain of each feature dimension for sample segmentation. The selection probability is proportional to the information gain. The tree node's segmentation threshold is updated using an adaptive segmentation threshold algorithm, which includes: calculating the density function of the local data distribution, determining the optimal segmentation point under the constraint of maximizing the inter-class variance, and introducing a smoothing factor to prevent overfitting. Based on the selection probability and the segmentation threshold, the latest path length of the sample in each isolation tree is calculated, and the latest calculation parameters of the anomaly score are updated to obtain the updated isolation tree model.

7. A monitoring device for abnormal nodes, characterized in that, The abnormal node monitoring device is used to execute the abnormal node monitoring method as described in any one of claims 1 to 6, and the abnormal node monitoring device includes: The data acquisition module is used to collect six-tuple network flow data; The data segmentation module is used to group the six-tuple network flow data according to a preset time window to obtain multiple time window data groups. Based on the source IP address entropy, destination IP address entropy, and port entropy in the time window data groups, it performs difference calculation on adjacent time window data groups to obtain an entropy difference sequence. The entropy difference sequence is then normalized to obtain an entropy change rate curve corresponding to the data entropy change rate. Based on the entropy change rate curve, time points where the fluctuation amplitude exceeds a preset threshold range are marked to obtain a time segmentation point sequence. Finally, the six-tuple network flow data is segmented according to the time segmentation point sequence to obtain multiple segmented network flow data. The feature extraction module is used to construct a source node network based on the segmented network flow data, extract three-dimensional features for each source node in the source node network, including: connection density features, protocol distribution features, and port entropy features, and perform standardization processing on the connection density features, the protocol distribution features, and the port entropy features to obtain a three-dimensional network feature vector; The anomaly analysis module is used to construct an Isolation Forest tree based on the feature vector of the three-dimensional network, and after training, obtains the node anomaly score; The node determination module is used to incrementally update the node anomaly score and determine abnormal nodes from the source node network based on the updated node anomaly score and a preset dynamic anomaly threshold.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the abnormal node monitoring method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the abnormal node monitoring method as described in any one of claims 1 to 6.

Images