A DDoS attack detection method based on heteroscedastic unscented Kalman filter

Abstract

This invention discloses a DDoS attack detection method based on heteroscedastic unscented Kalman filtering. Addressing the issues of limited resources in edge networks and low anomaly detection accuracy and high latency in low signal-to-noise ratio environments, this invention utilizes Sketch for full-scale, unsampled traffic feature extraction in the programmable data plane and reports observations to the control plane through differential processing. In the control plane, a third-order state-space model incorporating instantaneous traffic rate, rate of change, and acceleration is innovatively constructed. An additive heteroscedastic noise model is proposed to adaptively handle Sketch hash collisions and traffic shot noise, and unscented Kalman filtering is employed for state estimation. Simultaneously, an intelligent hybrid control mechanism is introduced to overcome filtering lag, and finally, NIS and CUSUM are combined for dual-modal anomaly detection, outputting the system state. This invention enables high-precision real-time detection of DDoS attacks with extremely low resource overhead, effectively solving the trade-off between measurement and detection in existing technologies.

Description

Technical Field

[0001] This invention belongs to the field of computer network security, specifically relating to a DDoS attack detection method based on heteroscedastic unscented Kalman filtering. Background Technology

[0002] Distributed Denial of Service (DDoS) attacks are attacks designed to exhaust the resources of a target network through massive amounts of malicious traffic. With the development of edge computing, resource-constrained edge networks have become a major target for capacity-exhaustion DDoS attacks.

[0003] In resource-constrained edge networks, existing traffic detection solutions generally face the following problems: First, traditional machine learning-based anomaly detection solutions have high computational resource overhead, making them difficult to deploy on inexpensive network switches. Second, lightweight statistical algorithms are slow to respond to sudden traffic changes, resulting in high detection latency. Third, to monitor massive data streams in limited memory, existing methods use the Sketch data structure to perform lossy compression on traffic, introducing hash collision noise. Furthermore, the random fluctuations in edge network traffic itself introduce shot noise, and existing time-series tracing algorithms such as homoscedastic Kalman filtering cannot effectively distinguish between real attack signals and mixed heteroscedastic noise, leading to high false positives or false negatives in low signal-to-noise ratio environments. Finally, traditional detection methods mostly rely on a single traffic threshold for judgment, which cannot distinguish between normal burst traffic with acceleration approaching zero and DDoS attacks with sustained positive acceleration, and are prone to inertial hysteresis and prediction overshoot under nonlinear step signals.

[0004] Therefore, there is an urgent need for a detection method that can operate efficiently on a resource-constrained programmable data plane, accurately adapt to low signal-to-noise ratio and heteroscedastic noise environments, and distinguish between normal bursts and DDoS attacks in real time. Summary of the Invention

[0005] To address the problems of high computational resource consumption, high detection latency, insufficient accuracy under low signal-to-noise ratio, and poor adaptability to nonlinear burst traffic in existing technologies, this invention proposes a DDoS attack detection method based on heteroscedastic unscented Kalman filtering.

[0006] To achieve the above objectives, the technical solution adopted in this invention is as follows: a DDoS attack detection method based on heteroscedastic unscented Kalman filtering. This method performs unsampling feature extraction on the entire data stream using a Sketch data structure in the programmable data plane, and constructs a third-order state-space model in the control plane, including instantaneous traffic rate, rate of change, and acceleration. It utilizes an additive heteroscedastic unscented Kalman filter (UKF) algorithm that integrates hash collision errors and Poisson shot noise for high-precision state estimation and feature extraction. Finally, it combines a dual-modal detection mechanism to determine anomalies and output the current network security status. The main steps include:

[0007] Step 1: Global feature extraction of network data flow: In the data plane, based on the global monitoring strategy of data flow without sampling, the Sketch data structure is used to statistically analyze network data packet information in real time, and the flow feature data is periodically reported to the control plane within a fixed time window;

[0008] Step 2, Flow Rate Differential Observation Processing: In the control plane, the reported flow characteristic data is differentially processed to obtain the flow rate observation values ​​within each time window;

[0009] Step 3: Flow state estimation based on heteroscedasticity UKF: A third-order state-space model is constructed in the control plane to model the instantaneous flow rate, flow rate of change, and flow acceleration, respectively. The heteroscedasticity UKF algorithm is used to estimate the state and extract features from the flow rate observations. In the observation update phase, the UKF algorithm combines Poisson shot noise and Sketch hash collision error to establish an additive heteroscedasticity noise model to adaptively calculate the observation noise covariance within the current time window.

[0010] Step 4: Dual-modal collaborative anomaly detection and judgment: Based on the state estimation results and Normalized Innovation Squared (NIS) output by the UKF algorithm, combined with the Cumulative Sum (CUSUM) algorithm, dual-modal anomaly detection of instantaneous features and cumulative features is performed. The risk score is calculated to determine whether the current network traffic is under DDoS attack and to trigger the corresponding switching of the detection system state machine. Beneficial effects

[0011] The DDoS attack detection method based on heteroscedastic unscented Kalman filtering of the present invention has the following beneficial effects:

[0012] First, it breaks through the "measurement-detection paradox," achieving a balance between extremely low resource overhead and sub-second real-time detection. In the data plane, a compact Sketch data structure is used for full, unsampled packet processing. In the control plane, complex nonlinear dynamic detection algorithms are abstracted into UKF filtering and bimodal decision-making. This enables extremely low-latency detection while meeting the stringent resource constraints of edge devices.

[0013] Second, an additive heteroscedastic noise model is proposed to improve detection accuracy in low signal-to-noise ratio environments. By establishing a heteroscedastic model in UKF, the observation noise covariance matrix is ​​adaptively and dynamically adjusted with the background flow scale, accurately separating the mixed noise generated by the superposition of hash collision error and Poisson arrival shot noise.

[0014] Third, a third-order state-space model is constructed to effectively distinguish between normal bursts and DDoS attacks. Traffic change rate and traffic acceleration are introduced as state variables, and the Verhulst-Pearl logistic growth model is used to describe the stochastic differential equation evolution of traffic, keenly capturing the essential characteristic that the acceleration of a DDoS attack is consistently positive.

[0015] Fourth, an intelligent hybrid control mechanism is proposed to address the hysteresis and overshoot issues of the filtering algorithm under abrupt signals. This is achieved by forcibly compressing the noise covariance through an observation covariance compression mechanism to eliminate upward hysteresis, and by cutting off inertial predictions through a momentum reset mechanism to prevent downward oscillations.

[0016] Fifth, it introduces dual-modal collaborative detection to comprehensively cover different attack modes. It integrates NIS-based instantaneous anomaly detection and CUSUM-based cumulative anomaly detection, achieving comprehensive coverage and accurate capture of both volumetric high-volume burst attacks and slow resource exhaustion attacks. Attached Figure Description

[0017] Figure 1 This is a system architecture diagram of a DDoS attack detection method based on heteroscedastic unscented Kalman filtering according to the present invention.

[0018] Figure 2 This is a flowchart of state estimation using an unscented Kalman filter based on additive heteroscedastic noise.

[0019] Figure 3 This is a flowchart of the dual-modal anomaly detection logic. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the following detailed description, in conjunction with the accompanying drawings, provides a further detailed explanation of a DDoS attack detection method based on heteroscedastic unscented Kalman filtering. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the scope of the invention.

[0021] like Figure 1 As shown, this invention proposes a DDoS attack detection method based on heteroscedastic unscented Kalman filtering. The system is divided into two parts: a data plane (physical layer) and a control plane (interface layer, prediction layer, fusion layer, and decision layer). The method mainly includes the following steps:

[0022] Step 1: Global Feature Extraction of Network Data Flow. In the data plane, based on a non-sampling global monitoring strategy for the data flow, the Sketch data structure is used to collect real-time data packet information and periodically report flow feature data to the control plane within a fixed time window. Specifically, the traditional probability sampling mechanism is removed, and a full data packet processing strategy is adopted. InputSalting technology is used, and basic verification algorithms such as CRC32 are reused to generate different mappings by changing the suffix of the input hash calculation unit. After the flow features of the network data packets are hashed, they are updated to the counter corresponding to the Count-Min Sketch data structure.

[0023] Step 2: Flow Rate Differential Observation Processing. In the control plane, the reported flow characteristic data is differentially processed to obtain the flow rate observation values ​​within each time window. Specifically, addressing the difficulty of atomically clearing registers in the data plane, the control plane records the global counter state matrix read in the current time window. and the matrix read in the previous time window By calculating the difference matrix and divide it by the length of the time window. This yields the observed flow rates of each network flow within the current time window. .

[0024] Step 3: Flow state estimation based on heteroscedasticity UKF. A third-order state-space model is constructed in the control plane to model the instantaneous flow rate, rate of change of flow rate, and flow acceleration, respectively. The heteroscedasticity UKF algorithm is then used to estimate the state and extract features from the observed flow rates. Figure 2 As shown, a third-order state vector is defined to address the essential characteristic of DDoS attacks as "continuously accelerating" traffic. ,in Instantaneous flow rate For the rate of change of flow, For flow acceleration. Flow evolution follows the following stochastic differential equation (SDE):

[0025] Among them, the drift term The Verhulst-Pearl logistic growth model is adopted to encode the prior knowledge of system recovery and stability, as well as the physical bandwidth constraints, including damping terms, saturation factors, maximum physical bandwidth limits of links, and system gain parameters.

[0026] diffusion term Using a shot noise model, since the arrival of network data packets approximately follows a Poisson process, the internal noise of the system increases with increasing signal strength.

[0027] In the state extrapolation prediction phase of the UKF, a fourth-order Runge-Kutta (RK4) method is used for integral calculation in order to perform high-precision conversion between the continuous-time SDE model and discrete-time observations: During the UKF observation update phase, an additive heteroscedastic noise model is established to adaptively calculate the observation noise covariance for the current time window k. :

[0028] Among them, hash collision noise variance Incremental background flow within the window Proportional, that is Poisson noise variance Reflecting the random fluctuation characteristics of network traffic itself, and the current observation count. Related, that is ; Noise floor variance This model is used to ensure the positive definiteness of the system's numerical values ​​under extremely low traffic conditions. Through this model, when a surge in background traffic leads to increased Sketch hash collisions, Automatically increasing, the UKF algorithm dynamically reduces the confidence level in current unreliable observations.

[0029] Observation noise covariance based on dynamic calculation UKF calculates the Kalman gain matrix This updates the state vector. With error covariance matrix :

[0030] Where H is the observation matrix. The covariance update process uses Joseph Form, and during the quiescent period, it updates the predicted covariance. A small amount of empirical noise is injected to prevent the system from becoming overconfident due to prolonged periods without attacks. An intelligent hybrid control mechanism is introduced during the UKF update phase: when the NIS exceeds a first preset threshold and the new information... If the value is greater than zero, an up-jump is determined, and the current effective observation noise is forcibly compressed. Greatly improves Kalman gain This causes the state vector to converge instantaneously to the observed value; when the information is much less than zero, the velocity in the state vector is forced to converge. and acceleration Reset to zero and disable inertial prediction.

[0031] Step 4: Dual-modal collaborative anomaly detection and judgment. Based on the state estimation results output by the UKF algorithm and NIS, combined with the CUSUM algorithm, dual-modal anomaly detection is performed to determine whether the current network traffic is under a DDoS attack. For example... Figure 3 As shown, in order to comprehensively cover attacks with different energy distributions, a comprehensive risk scoring model that integrates instantaneous and cumulative features is constructed:

[0032] The NIS index reflects the degree of deviation between current observations and model predictions, and is extremely sensitive to large-scale, sudden bursts of traffic. This represents the tolerance threshold for instantaneous, volumetric, large-volume bursts; the CUSUM algorithm accumulates persistent, small, abnormal offsets. This is the cumulative threshold; and These are the weighting coefficients. The activation function is the comprehensive risk score. When the attack exceeds a preset threshold, the detection system state machine switches from the normal state to the attack state and outputs an alarm. After the attack ends, the system state machine returns to the normal state after a cooldown period.

Claims

1. A DDoS attack detection method based on heteroscedastic unscented Kalman filtering, characterized in that, This method performs unsampled feature extraction on the entire data stream using the Sketch data structure in the programmable data plane. In the control plane, it constructs a third-order state-space model incorporating instantaneous flow rate, rate of change, and acceleration. It then utilizes an additive heteroscedastic unscented Kalman filter algorithm that fuses hash collision errors and Poisson shot noise for state estimation and feature extraction. Based on this, a dual-modal detection mechanism is used for anomaly detection, and the current network security status is output. The method specifically includes the following steps: Step 1, Traffic Feature Acquisition: In the programmable data plane, the Sketch data structure is adopted. Different hash mappings are achieved by using multiple independent hash functions and changing the input unit suffix to reuse the CRC32 check algorithm. The full network data stream is subjected to sampling-free feature extraction and statistics. Network data packet information is recorded in real time, and the flow feature data is periodically reported to the control plane within a fixed time window. Step 2, Construction of flow rate observation values: In the control plane, the flow feature data of adjacent time windows are differentially processed, the global count values ​​read from the Sketch data structure of the current time window and the previous time window are recorded, and the difference between the two is divided by the length of the time window to obtain the flow rate observation value in the current time window; Step 3: Construct a third-order flow state-space model in the control plane, with its state vector as follows: ,in Instantaneous flow rate For the rate of change of flow, The flow rate is measured by acceleration. Simultaneously, a heteroscedastic unscented Kalman filter algorithm is used to estimate the state and extract features from the flow rate observations. During the observation update phase, an additive heteroscedastic observation noise model is constructed, and the observation noise covariance is adaptively calculated based on the observation data within the current time window. ,in For hash collision measurement error term, For shot noise, This is the basic noise floor term; at the same time, an intelligent hybrid control mechanism is introduced based on the normalized innovation square. When the normalized innovation square is greater than the first preset threshold and the innovation is positive, the current observation noise covariance is compressed. When the innovation shows a negative abrupt change, the flow rate change rate and flow acceleration are set to zero. Step 4, Anomaly Detection and State Determination: Based on the state estimation results output by the heteroscedastic unscented Kalman filter algorithm described above, and combined with the normalized innovation squared cumulative sum control chart, dual-modal anomaly detection is achieved by jointly analyzing instantaneous and cumulative features; a comprehensive risk score is calculated based on the detection results. When the comprehensive risk score is greater than a preset decision threshold, it is determined that there is a distributed denial-of-service attack on the current network, and the current network security status is output.

2. The DDoS attack detection method based on heteroscedastic unscented Kalman filtering according to claim 1, characterized in that, In step 3, the flow evolution of the third-order state-space model follows the following stochastic differential equation: Among them, the drift term The Verhulst-Pearl logistic growth model is used to describe the dynamic changes in flow, including a saturation factor determined by the system's physical bandwidth limitations, a damping term representing the system's tendency to stabilize, and a diffusion term. A shot noise model is adopted, whose variance increases with the increase of signal strength and provides a basic noise floor when the flow rate is close to zero.

3. The DDoS attack detection method based on heteroscedastic unscented Kalman filtering according to claim 1, characterized in that, In step 3, the hash collision measurement error term in the observation noise covariance of the current time window Shot noise item and basic noise level Calculate according to the following formulas respectively: Among them, the hash collision measurement error term This is used to characterize the measurement error caused by lossy compression of the Sketch data structure, and its variance is related to the background traffic increment; the shot noise term This is used to characterize the random fluctuations caused by the Poisson arrival process of network traffic, and its variance is related to the current observation count; the basic noise floor term , is a preset positive value used to ensure the positive definiteness of the observation noise covariance value under low flow conditions.

4. The DDoS attack detection method based on heteroscedastic unscented Kalman filtering according to claim 1, characterized in that, The dual-modal anomaly detection and determination in step 4 is achieved by calculating a comprehensive risk score: in, and These are the weights for instantaneous features and cumulative features, respectively. A preset normalized innovation squared tolerance threshold is used to detect instantaneous volumetric bursts of large flow rates; The corresponding thresholds for accumulation and control graphs are used to detect slow resource exhaustion attacks; For activation functions; When the comprehensive risk score When the threshold value exceeds the preset threshold, a DDoS attack is detected and the system state machine switches from the normal state to the attack state. After the attack ends, the state machine returns to the normal state after a cooldown period.

Images