Link-level network tomography and attack detection method based on conditional random field

Abstract

A link-level network tomography and attack detection method based on a conditional random field relates to the field of communication, and comprises the following steps: acquiring a routing matrix and a link prior congestion probability, acquiring a path state containing a time sequence rule from samples conforming to link-level first-order Markov property, and taking the path state as a time sequence sample pool; performing feature modeling by combining the routing matrix, the link prior congestion probability and the path state to generate a link-level feature vector; the link-level feature vectors and the corresponding actual link state tags are transmitted into a CRF model for training, and state transition weight parameters and state feature corresponding weight parameters are obtained through training; performing network state prediction by using the trained CRF model; and carrying out attack judgment on the predicted network state. According to the method, the space-time relevance between state labels is obviously modeled, and the data relevance between time steps can be better processed by fusing topology perception characteristics and time sequence state transition modeling, so that the high-precision detection of link-level anomalies is realized, and the calculation complexity is reduced.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and specifically to a link-level network tomography and attack detection method based on conditional random fields. Background Technology

[0002] With the widespread adoption of cloud computing and IoT technologies, the network link layer faces increasingly complex security threats. Malicious attacks such as Distributed Denial-of-Service (DDoS) attacks and link flooding attacks intentionally create link congestion, leading to the interruption of critical services. These attacks exhibit two main trends: increased stealth (low-speed persistent attacks simulating normal traffic patterns) and topology awareness (targeted flooding of core links).

[0003] Traditional detection methods, such as the single-moment diagnostic method SCFS / CLINK, cannot identify cross-time step attacks such as LDoS because they ignore the temporal correlation of states. Attack detection methods that combine deep learning require training with massive amounts of attack samples, and the scarcity of actual attack data and the black-box decision-making make it difficult to trace the root cause of false alarms. Summary of the Invention

[0004] To address the technical problems of traditional detection schemes, this invention provides a link-level network tomography and attack detection method based on conditional random fields.

[0005] This invention overcomes two key challenges. First, it solves the underdetermined inverse problem of inferring high-dimensional link states from a small number of path observation sequences. Second, it models spatiotemporal correlations, moving away from single-moment diagnosis as in traditional detection schemes. Instead, it combines state transition patterns based on temporal characteristics and achieves collaborative reasoning of link states in both topological and temporal dimensions by fusing spatiotemporal sensing signals such as topological load and neighbor congestion propagation.

[0006] The technical solution adopted by this invention to solve the technical problem is as follows:

[0007] This invention provides a link-level network tomography and attack detection method based on conditional random fields, which specifically includes the following steps:

[0008] Step S1: Obtain the routing matrix and the prior congestion probability of the links, and obtain the path state containing the temporal pattern from the samples that conform to the first-order Markov property of the link level. Use the routing matrix, the prior congestion probability of the links, and the path state as the temporal sample pool.

[0009] Step S2: Combine the routing matrix, prior link congestion probability, and path state to perform feature modeling and generate link-level feature vectors;

[0010] Step S3: Input the link-level feature vector and its corresponding actual link state label into the CRF model for training, and obtain the state transition weight parameter μ and the weight parameter λ corresponding to the state feature.

[0011] Step S4: Predict the network state using the trained CRF model;

[0012] Step S5: Determine if the predicted network state is vulnerable to attack.

[0013] Furthermore, in step S1, the path state is obtained by constructing time-series samples that satisfy the first-order Markov property at the link level using the routing matrix and the prior link congestion probability; the link state sequence is first processed... The first-order Markov property model is constructed as follows t represents the current time step, and T represents the total time steps; the state of link l at each update. Depends on the state of the previous time step And by the transition probability α l β l The decision is made as follows: when the difference between the empirical congestion probability and the prior congestion probability of all links is less than the tolerance, the congestion distribution of the sample data is considered to satisfy the prior congestion probability distribution of the links; the difference between the empirical congestion probability and the prior congestion probability of each link is calculated, and it is checked whether the difference of all links is less than the tolerance.

[0014] Furthermore, the specific implementation process of step S2 is as follows:

[0015] S2.1: The mathematical model for the prior congestion probability characteristics of each link is as follows:

[0016]

[0017] The link prior congestion probability is used to integrate link-level prior knowledge and help the CRF model understand whether a link is prone to congestion.

[0018] S2.2: The mathematical model for the path's quantitative characteristics is as follows:

[0019]

[0020] R pl Indicates whether path p passes through link l; the path has a quantity characteristic, and the link load is evaluated by counting the number of times link l passes through path P at time step t;

[0021] S2.3: The mathematical model for the quantitative characteristics of path congestion is as follows:

[0022]

[0023] This indicates whether path p is congested at time step t; the path congestion quantity feature captures path-level congestion accumulation, i.e., abnormal load conditions of the link, by counting the number of paths P that pass through link l and are congested.

[0024] S2.4: The mathematical model for the link congestion ratio is as follows:

[0025]

[0026] The link congestion ratio is obtained by using the characteristics of the number of congested paths and the characteristics of the number of paths.

[0027] S2.5: The mathematical model for the average congestion ratio of neighbor links is as follows:

[0028]

[0029] P n Let ω represent the set of paths on neighbor link n. l This represents the set of neighboring links of link l; in the average congestion ratio of neighboring links, links that follow the same path are grouped as neighboring links. By statistically analyzing the congestion ratio of neighboring links, the congestion situation of the links in the vicinity of the network is reflected.

[0030] Furthermore, the specific implementation process of step S3 is as follows:

[0031] S3.1: The input data consists of link-level feature vectors and corresponding actual link status labels;

[0032] S3.2: The CRF model processes the input data;

[0033] S3.3: Model the state characteristic function and state transition function on a per-link basis;

[0034] S3.4: Model the conditional probability distribution;

[0035] S3.5: Solve for the conditional probability using the L-BFGS optimization method;

[0036] S3.6: Obtain a set of state transition weight parameters μ and state feature corresponding weight parameters λ, and the CRF model training ends.

[0037] Furthermore, the specific implementation process of step S3.2 is as follows:

[0038] S3.2.1: The CRF model pairs the link-level feature vectors with the corresponding actual link state labels;

[0039] S3.2.2: The CRF model needs to model the time series samples, taking the original input feature vector... and the current link status. Convert to sequence In form, This represents the link state at the previous moment.

[0040] Furthermore, the specific implementation process of step S3.4 is as follows:

[0041] S3.4.1: After feature modeling, the problem is transformed into inferring the true state of the link based on link-level feature vectors; the mathematical model of the conditional random field is as follows:

[0042]

[0043] in, The state sequence of link l, Let λ be the link-level feature vector corresponding to link l. k f represents the weight parameter of the k-th feature. k This represents the Boolean indicator function corresponding to the k-th feature, used to indicate the observation at the current position t. With candidate tags Mapped to a 0 / 1 eigenvalue, where K represents the number of features and μ represents the total time steps. ij This represents the weights for transitioning from state i to state j. Z(F) represents an indicator function, the symbol ∧ represents logical AND, and Z(F) represents a logical AND function. l ) is the partition function, used to ensure probability normalization, and its mathematical expression is:

[0044]

[0045] in, To determine the state label value of a link at time step t during normalized summation, The status label value is taken for the same link at time step t-1;

[0046] S3.4.2: After modeling with a Conditional Random Field (CRF), the problem is transformed into finding a set of parameters that maximizes the probability of the training data appearing in the model given the known samples; this is equivalent to solving for maximum likelihood estimation. Log-likelihood modeling of the linear chain CRF is as follows:

[0047]

[0048] Among them, X (t) F represents the hypothetical label state at time t. (t) Let θ represent the characteristic state at time t, and let θ represent the parameters λ and μ. and Here, c1 and c2 are L2 regularization terms, and both are regularization coefficients. Since the CRF model is trained by minimizing, the objective function is to solve for...

[0049] Furthermore, the specific implementation process of step S3.5 is as follows:

[0050] S3.5.1: Perform first-order differentiation based on the objective:

[0051] S3.5.2: Construct an approximate Hessian inverse matrix based on historical information:

[0052] S3.5.3: Use a line search strategy to find the optimal step size; after finding the optimal step size and updating the parameters, repeat S3.5.2 and S3.5.3 until the number of iterations is reached, then stop the loop and obtain a set of optimal parameters.

[0053] Furthermore, the specific implementation process of step S4 is as follows:

[0054] S4.1: Obtain the routing matrix and the prior congestion probability of the links, and extract the path states containing temporal patterns from the samples that conform to the first-order Markov property of the links. Use the routing matrix, the prior congestion probability of the links, and the path states as a temporal sample pool. Combine the routing matrix, the prior congestion probability of the links, and the path states to perform feature modeling and generate link-level feature vectors.

[0055] S4.2: Loading the CRF model means loading the state transition weight parameter μ and the weight parameter λ corresponding to the state features;

[0056] S4.3: Use the Viterbi method to find the optimal link state solution;

[0057] S4.4: Output the state inference sequence of all links within the total time step T.

[0058] Furthermore, the specific implementation process of step S4.3 is as follows:

[0059] S4.3.1: Combining the feature scoring function and the state transition scoring function, the problem model is as follows:

[0060]

[0061] in, The state sequence of link l, Let λ be the link-level feature vector corresponding to link l. k f represents the weight parameter of the k-th feature. k This represents the Boolean indicator function corresponding to the k-th feature, used to indicate the observation at the current position t. With candidate tags Mapped to a 0 / 1 eigenvalue, where T represents the total time steps. Represents the state from time t-1. Transition to the state at time t The weights;

[0062] S4.3.2: Initialize the maximum path score parameter in the first time step:

[0063] S4.3.3: Recursively determine the optimal path for state transitions and save it to the next state. The path refers to the direction of the state transition.

[0064] S4.3.4: Record the path source of the best path;

[0065] S4.3.5: Continuously iterate through S4.3.3 and S4.3.4 until all time steps of data have been traversed to find the final optimal path;

[0066] S4.3.6: By tracing back the entire path from the path source of the recorded best path, complete state transition information can be obtained.

[0067] Furthermore, the specific implementation process of step S5 is as follows:

[0068] S5.1: Compare the state inference sequence of all links within the total time step T obtained in step S4 with the actual link state labels to obtain the attack prediction result;

[0069] S5.2: Compare the attack prediction results with the actual attack labels to obtain the accuracy, detection rate, F1 score and false positive rate of the detection method, so as to verify the detection effect of the CRF model.

[0070] The beneficial effects of this invention are:

[0071] This invention provides a link-level network tomography and attack detection method based on conditional random fields. It designs five types of dynamic features through multi-dimensional feature modeling to provide spatiotemporal awareness input, uses a first-order chain CRF to model the joint probability of state sequences, maximizes the regularized log-likelihood through the L-BFGS optimization method, and uses the forward-backward algorithm to efficiently calculate the model expectation in the gradient, which greatly reduces the computational complexity.

[0072] This invention is studied in a distributed network topology, and its feasibility and usability in the scenario are verified by simulating the network traffic collaborative reasoning process:

[0073] During the training phase, this invention uses unattacked time-series samples to learn state transition patterns that conform to temporal characteristics. By learning from the states of unattacked sample data, the CRF model can master the state transition patterns and the correspondence between state features under normal conditions.

[0074] During the inference phase, this invention calls the trained CRF model parameters, uses the Viterbi algorithm to achieve optimal decoding of the state sequence, and finally returns the link state prediction value that conforms to the time sequence pattern.

[0075] Finally, by combining the actual link state data to output the attack detection judgment results and comparing them with the actual attack labels, evaluation indicators such as attack judgment accuracy and false positive rate are obtained. The experiment verifies the effectiveness of the present invention in terms of inference result accuracy compared with traditional tomographic detection technologies SCFS and CLINK under different attack scenarios.

[0076] This invention solves the underdetermined inverse problem of inferring link congestion status from path observation sequences while achieving high-precision link attack detection. Unlike traditional single-moment link detection methods, this invention explicitly models the spatiotemporal correlation between state labels. By fusing topology-aware features with temporal state transition modeling, it can better handle data correlations between time steps, achieving high-precision detection of link-level anomalies. Attached Figure Description

[0077] Figure 1 The flowchart illustrates a link-level network tomography and attack detection method based on conditional random fields, as provided in this invention.

[0078] Figure 2 This is a schematic diagram of the feature modeling of the present invention.

[0079] Figure 3 This is a flowchart of the CRF model training process of the present invention.

[0080] Figure 4 This is a schematic diagram of CRF modeling for the present invention.

[0081] Figure 5 This is a schematic diagram of the CRF model inference logic of the present invention.

[0082] Figure 6 This is a flowchart of the CRF model inference process of the present invention.

[0083] Figure 7 This is a flowchart of the detection result evaluation process of the present invention.

[0084] Figure 8 This chart compares the accuracy of the present invention with that of traditional tomographic detection methods in identifying link attacks under different attack scenarios.

[0085] Figure 9 This is a comparison chart showing the detection rate of the present invention with that of traditional tomographic detection methods for link attack identification under different attack scenarios.

[0086] Figure 10This is a comparison chart of the F1 scores of the present invention and traditional tomographic detection methods for link attack identification under different attack scenarios.

[0087] Figure 11 This is a comparison chart showing the false positive rates of the present invention and traditional tomographic detection methods for link attack identification under different attack scenarios. Detailed Implementation

[0088] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0089] This invention discloses a link-level network tomography and attack detection method based on conditional random fields (CRF). Utilizing the spatiotemporal correlation characteristics of distributed network topologies, it jointly models path observation sequences based on five-dimensional dynamic features to efficiently solve the underdetermined inverse problem of link state inversion. In this invention, a first-order chain-like CRF model is trained using only non-attacked samples to explicitly learn the state transition rules under normal traffic conditions. Furthermore, spatiotemporal sensing signals such as topology load and neighbor congestion propagation are integrated to achieve accurate separation of attack behavior from natural congestion. Subsequently, the optimal state sequence is decoded based on the Viterbi method, and gradient calculation is accelerated using a forward-backward algorithm to overcome the computational bottleneck of real-time inference of high-dimensional link states. This process is achieved by maximizing the regularized log-likelihood. By independently mining spatiotemporal correlations, this invention enables lightweight attack detection for large-scale distributed networks. Experiments on multi-region links of the ChinaNet backbone network show that the pure transfer learning mechanism and spatiotemporal joint modeling of this invention achieve higher link-level detection accuracy, lower false positive rate, and higher F1 score compared to traditional tomographic detection methods in different attack rate scenarios.

[0090] See Figure 1 The present invention provides a link-level network tomography and attack detection method based on conditional random fields, the specific implementation process of which is as follows:

[0091] Step S1: Dataset construction;

[0092] Obtain the routing matrix and link prior congestion probability from the environment (no input required), and obtain the path state from samples that meet the first-order Markov property of the link level. The path state contains temporal patterns; use the above three data as a temporal sample pool.

[0093] According to the present invention, the specific implementation process of step S1 is as follows:

[0094] S1.1: From the ChinaNet public Internet topology, extract a portion of geographically representative links starting from a specific city, mark the topology structure with dark colors, and form a routing matrix R∈{0,1}.P×L Where L is the number of links, P is the number of paths, and P << L.

[0095] S1.2: The prior congestion probability of the link is preset to follow a uniform distribution. Where l represents the link, P k This represents the prior congestion probability, with the subscript k used to distinguish it from the conditional probability.

[0096] S1.3: Using the routing matrix from step S1.1 and the prior link congestion probability from step S1.2, construct time-series samples that satisfy the first-order Markov property at the link level to obtain the path state Y∈{0,1}. T×P This means that the current link state depends only on the link state of the previous moment and is independent of the link state of earlier moments.

[0097] The specific implementation process of step S1.3 is as follows:

[0098] S1.3.1: Link State Sequence The first-order Markov property model is constructed as follows:

[0099]

[0100] Where t represents the current time step and T represents the total time steps.

[0101] The state of link l during each update Depends on the state of the previous time step And through the transition probability α l and β l To determine. As shown in the following formula, where This represents the state transition situation where the same link is in a normal state at time t-1 but an abnormal state at time t. This represents the state transition situation where the same link is in an abnormal state at time t-1 but recovers to normal at time t:

[0102]

[0103] Under long-term stable operation, the state transition matrix will equal the prior congestion probability of the link, that is:

[0104]

[0105] Among them, Π k (0), Π k (1) All are state transition matrices.

[0106] S1.3.2: In addition to satisfying the link-level first-order Markov property, it should also satisfy full sample coverage, that is, it needs to ensure that the generated link state combinations cover all possible link states. L(L is the total number of links, 2) L This represents the total number of all possible link states, thereby improving the reasonableness of the dataset samples.

[0107] S1.3.3: The congestion distribution of the sample data should satisfy the prior congestion probability distribution of the links. Here, a tolerance ∈ is defined. When the difference between the empirical congestion probability and the prior congestion probability of all links is less than the tolerance ∈ , the congestion distribution of the sample data is considered to satisfy the prior congestion probability distribution of the links. For each link l, its empirical congestion probability... It is estimated based on the samples generated so far:

[0108]

[0109] Where n represents the total number of samples generated. Indicates an indicator function, when Returns 1 if the condition is met, otherwise returns 0.

[0110] Then calculate the empirical congestion probability for each link l. Link prior congestion probability The differences are checked, and it is determined whether the differences across all links are less than the tolerance ∈:

[0111]

[0112] Step S2: Combine the routing matrix, prior link congestion probability, and path state to model five types of features and generate link-level feature vectors. Each link contains 5 types of features;

[0113] like Figure 2 As shown, This represents the prior congestion probability of link l. The figure assumes the prior congestion probabilities of the first link and the Nth link. This is used to understand the presentation of prior congestion probabilities. The input link-level time-series vector is modeled using five types of features and transformed into a link-level time-series feature vector. That is, the characteristics of link l at time t.

[0114] The specific implementation process of step S2 is as follows:

[0115] S2.1: Link prior congestion probability feature f l ;

[0116] The first feature integrates link-level prior knowledge, using the link prior congestion probability to help the CRF model understand whether a link is prone to congestion. The link prior congestion probability feature f for each link... l The mathematical model is as follows:

[0117]

[0118] in,

[0119] S2.2: The path has a quantitative characteristic f2;

[0120] The second characteristic assesses link load by counting the number of paths traversed by the link l. A higher number of paths generally indicates a greater load on the link. The mathematical model for the path quantity characteristic f2 is as follows:

[0121]

[0122] The path existence count represents the number of times link l traverses path P at time step t. Where R... pl Indicates whether path p passes through link l.

[0123] S2.3: Path congestion quantity characteristics f3;

[0124] The third feature captures path-level congestion accumulation, i.e., abnormal load conditions of the link, by statistically analyzing the number of congestions along the path traversed by link l. The mathematical model for the path congestion quantity feature f3 is as follows:

[0125]

[0126] The path congestion count represents the number of paths P that pass through link l and are congested, where... This indicates whether path p is congested at time step t.

[0127] S2.4: Link congestion ratio f4;

[0128] The fourth feature uses the path congestion quantity feature f3 and the path existence quantity feature f2 to obtain the link load congestion ratio. The higher the congestion ratio, the more likely the link is to be congested. The mathematical model for the link congestion ratio f4 is as follows:

[0129]

[0130] S2.5: Average congestion ratio of neighboring links, f5;

[0131] The fifth feature categorizes links along the same path as neighbor links. By statistically analyzing the congestion ratio of neighbor links, the congestion situation of links in the network's vicinity can be reflected, providing a better understanding of the network. The mathematical model for the average congestion ratio f5 of neighbor links is as follows:

[0132]

[0133] Among them, P n Let ω represent the set of paths on neighbor link n. lThis represents the set of neighboring links of link l.

[0134] Step S3: Link-level feature vector F of total time step T l and its corresponding link actual status label X l As input, it is passed into the CRF model for training;

[0135] Based on the training method of Conditional Random Fields (CRF), the link-level feature vector F is... l As a feature value input, the corresponding link actual state label X l As input, the time-series pattern and state feature mapping relationship are learned from normal data (pure, non-attack data), and the state transition weight parameter μ and the weight parameter λ corresponding to the state feature are obtained through training.

[0136] like Figure 3 As shown, the specific implementation process of step S3 is as follows:

[0137] S3.1: The input data is the link-level feature vector F l and the corresponding link actual status label X l ;

[0138] S3.2: The CRF model processes the input data;

[0139] S3.2.1: For the input link-level feature vector and the corresponding link actual status label The CRF model first pairs states with features, that is...

[0140] S3.2.2: The samples are sequential inputs, and the CRF model needs to model the time series. Since the scenario is set as a first-order Markov sample, the input of the CRF model, in addition to the link state at the current time step, is... In addition, the link status from the previous moment is also required. Therefore, when training a CRF model, the original input feature vector will be... and link status Convert to sequence In the form of.

[0141] S3.3: Model the state characteristic function and state transition function;

[0142] like Figure 4 As shown, the specific implementation process of step S3.3 is as follows:

[0143] S3.3.1: Perform state feature modeling on a link-by-link basis, as follows:

[0144]

[0145] Where, λ k f represents the weight parameter of the k-th feature; k This represents the Boolean indicator function corresponding to the k-th feature, used to indicate the observation at the current position t. With candidate tags Mapping to a 0 / 1 feature value lets the model know what's happening in the current observation. Below, candidate tags Whether this clue is triggered, and thus the corresponding weight λ. k To increase or decrease the local potential score of the label; K represents the number of features; T represents the total time steps.

[0146] S3.3.2: State transition modeling is performed on a per-link basis, with the specific modeling details as follows:

[0147]

[0148] Where, μ ij This represents the weights for transitioning from state i to state j. This indicates an indicator function, and the symbol ∧ represents logical AND.

[0149] S3.4: The core of the training part, namely, modeling the conditional probability distribution;

[0150] S3.4.1: The essence of the CRF model is modeling the conditional probability distribution. In the problem of link-level network tomography attack detection based on conditional random fields, the essence is based on the path state Y∈{0,1}. T×P Inferring the true state of the link X∈{0,1} L After feature modeling, the problem is transformed into one based on link-level feature vectors. Inferring the true state of the link X∈{0,1} L Combining the modeling of the two functions in step S3.3, the modeling of the conditional random field is as follows:

[0151]

[0152] in, The state sequence of link l, Z(F) is the link-level feature vector corresponding to link l. l Let be the partition function, ensuring probability normalization. Its mathematical expression is:

[0153]

[0154] in, To determine the state label value of a link at time step t during normalized summation, The value of the status label for the same link at time step t-1 is determined.

[0155] S3.4.2: After modeling the Conditional Random Field (CRF), the training goal is to maximize the desired conditional probability on the training set. Therefore, the problem is transformed into finding a set of parameters that maximizes the probability of the training data appearing in the model, given the known samples. This is equivalent to solving for maximum likelihood estimation. To facilitate data processing, log-likelihood modeling is performed on the linear chain CRF, as follows:

[0156]

[0157] Among them, X (t) Indicates the status label, F (t) Let θ represent the eigenvalues, and let θ represent the parameters λ and μ. and Here, c1 and c2 are L2 regularization terms, and both are regularization coefficients. Since the CRF model is trained by minimizing, the objective function is to solve for...

[0158] S3.5: Solve for the conditional probabilities using the L-BFGS optimization method;

[0159] S3.5.1: In the L-BFGS optimization method, the first step is to perform first-order differentiation based on the above-mentioned objective. The mathematical model is as follows:

[0160]

[0161] in, Indicates L2 regularization terms. This refers to minimizing the negative log-likelihood as mentioned in S3.4.2.

[0162] Specifically, the calculation formula for the first derivative is as follows:

[0163]

[0164]

[0165] S3.5.2: Construct an approximate Hessian inverse matrix based on historical information. The specific construction process is as follows:

[0166] S k =θ k+1 -θ (k) ;

[0167] Where, θ k+1 Let θ represent the parameter vector for the (k+1)th iteration (this iteration). (k) This represents the parameter vector for the k-th iteration (the previous one).

[0168] Record the parameter difference between the previous and current values;

[0169]

[0170] Record the gradient difference between the two measurements;

[0171]

[0172] Where, γ k This represents the scaling factor used to scale the identity matrix, where I represents the identity matrix (a square matrix with the same dimension as the parameter). Indicates the parameter difference S k The transpose of y k Represents the gradient difference vector. This represents the transpose of the gradient difference vector.

[0173] Construct a diagonal initial approximation matrix;

[0174]

[0175] The bidirectional update yields the descent direction; among which, In L-BFGS, the Hessian inverse matrix is ​​approximated by a double-loop recursive formula, denoted by [symbol missing]. Let be the gradient operator, indicating that the objective function J is at θ. (k) The gradient vector at that point.

[0176] S3.5.3: The optimal step size is found using a line search strategy. The specific construction process is as follows:

[0177] θ (k+1) =θ (k) +ηd k ;

[0178] Where η is the step size parameter;

[0179] θ (k+1) =θ (k) +α k d k ;

[0180] Where, α k This is the optimal step size.

[0181] Find the optimal step size α k After updating the parameters, increment k by 1, and repeat steps S3.5.2 and S3.5.3 until k = iteration, i.e., k equals the number of iterations, then stop the loop. At this point, a set of optimal parameters θ is obtained.

[0182] S3.6: Obtain a set of state transition weight parameters μ and weight parameters λ corresponding to the state features. Since the sample is two-state, the state transition weight parameter μ has 4 values, namely μ 00 μ 01 μ 10 and μ 11 The number of weight parameters λ corresponding to the state features depends on the number of features K. According to this invention, each link contains 5 types of features, i.e., λ = K = 5. At this point, a set of global parameters has been obtained, and the CRF model training is complete.

[0183] Step S4: Predict the network state using the trained CRF model;

[0184] Load the trained model parameters, call the CRF model to predict the link state, and output the predicted link state as an attack-free link state that satisfies the mapping relationship between the temporal pattern and the state feature.

[0185] like Figure 5 As shown, This represents the prior congestion probability of link l, which is prior knowledge already possessed by the system. Figure 5 The prior congestion probabilities of the first link and the Nth link are assumed. This is used to understand the presentation of prior congestion probabilities. The input link-level time-series vector is modeled using five types of features and transformed into a link-level time-series feature vector. That is, the characteristics of link l at time t. The link-level temporal feature vector is fed into the CRF model to obtain the optimal state solution.

[0186] The specific implementation process of step S4 is as follows:

[0187] S4.1: Obtain the path state from samples that satisfy the first-order Markov property at the link level, obtain the routing matrix and the link prior congestion probability from the environment (no input required), and perform the same five-class feature modeling on the above input and environment data as in the training process to generate link-level feature vectors.

[0188] S4.2: Load the CRF model, i.e., load the parameters: state transition weight parameter μ and state feature corresponding weight parameter λ;

[0189] S4.3: Use the Viterbi method to find the optimal link state solution;

[0190] like Figure 6 As shown, the specific implementation process of step S4.3 is as follows:

[0191] S4.3.1: The problem that the Viterbi method needs to solve is Combining the feature scoring function and the state transition scoring function, the problem is modeled as follows:

[0192]

[0193] in, Let's consider the state from time t-1. Transition to the state at time t The weight.

[0194] S4.3.2: Initialize the maximum path score parameter in the first time step:

[0195]

[0196] Where δ1(j) represents the maximum path score (in log form) from the starting point to the first time step, ending in state j; This represents the feature value of the input link l at time t=1.

[0197] S4.3.3: Recursive state transition, saving the best path to the next state. The path refers to the direction of the state transition:

[0198]

[0199] Where, δ t (j) represents the maximum path score (in log form) from the starting point to time step t, ending with state j; Denotes the state space, δ t-1 (i) represents the maximum path score (in log form) from the starting point to time step t-1, ending at state i.

[0200] S4.3.4: Record the source of the optimal path for easy backtracking later:

[0201]

[0202] in, The path source for the best path to record.

[0203] S4.3.5: Continuously recursively apply steps S4.3.3 and S4.3.4 until t = T, i.e., after traversing the data of all time steps, to find the final optimal path:

[0204]

[0205] Where, δ T (j) represents the maximum path score (in log form) from the starting point to the total time step T, ending with state j.

[0206] S4.3.6: Path source through the recorded best path By tracing back the entire "path", we can obtain complete state transition information:

[0207]

[0208] S4.4: Output the state inference sequence of all links within the total time step T.

[0209] Step S5: Determine if the predicted network state is vulnerable to attack.

[0210] The predicted link state from step S4 is compared with the actual link state to output an attack determination. The predicted attack determination is then compared with the actual attack label to obtain various evaluation indicators such as accuracy, thus verifying the detection effect of the CRF model.

[0211] like Figure 7 As shown, the specific implementation process of step S5 is as follows:

[0212] S5.1: Infer the state sequence of all links within the total time step T obtained in step S4. With real link state label By comparing the results, we can obtain the attack prediction results.

[0213] S5.2: Compare the attack prediction results with the actual attack labels to obtain the accuracy, detection rate (DR), F1 score, and false positive rate (FPR) of the detection method.

[0214] Traditional tomographic detection methods are limited by a single-moment diagnostic framework, fragmenting the continuous network state evolution process into isolated time slices. This fails to capture the long-step behavior patterns of low-speed persistent attacks, making it difficult to distinguish between natural anomalies and attack-induced anomalies, resulting in low attack detection accuracy. This invention provides a link-level network tomography and attack detection method based on Conditional Random Fields (CRFs). It proposes an innovative paradigm driven by temporal state transition laws, explicitly learning the natural evolution laws of link congestion states through a first-order chained CRF. The state transition matrix is ​​trained using only unattacked samples, allowing the CRF model to deeply internalize the dynamic changes in network traffic. During Viterbi decoding, a globally optimal state sequence is output, avoiding misjudgments of bursty traffic by single-moment diagnoses. Under different attack scenarios, this temporal modeling mechanism can achieve an attack identification accuracy of around 80%.

[0215] From the ChinaNet public Internet topology, a portion of regionally representative links are extracted, starting from a specific city and marked with dark colors. This extracted sub-topology contains 8 paths and 12 links. Time-series samples satisfying the link-level first-order Markov property are constructed using this sub-topology. An attack parameter ξ represents the random attack rate. For the time-series samples, attack samples with different attack rates (ξ∈[0,0.55]) are constructed. The experiment uses data samples with different attack parameters ξ for attack detection and compares the detection performance with that of traditional tomographic detection methods. Accuracy represents the proportion of correctly classified samples out of all identified samples; Detection Rate (DR) represents the proportion of detected positive samples out of all positive samples; False Positive Rate (FPR) is the proportion of actually negative samples out of positive samples; and the F1 score considers both recall and precision. Recall is the proportion of attacked links identified, and precision is the proportion of links predicted as attacked by the model that were actually attacked.

[0216] A comparison of the accuracy of a link-level network tomography and attack detection method based on conditional random fields (CRRF) of this invention with traditional tomography detection methods under different attack scenarios: Figure 8 As shown in the figure: the horizontal axis represents the attack parameters, and the vertical axis represents the attack identification accuracy. It can be seen that among all seven attack scenarios, this invention (CRF) has the highest accuracy, maintaining nearly 80% attack detection accuracy even in high-intensity attack scenarios, such as when more than half of the attacks have ξ = 0.55. CLINK maintains a relatively high attack detection accuracy due to its prior knowledge of congestion probability, but SCFS exhibits poor detection accuracy.

[0217] At the same time, Figure 9 In the detection rate evaluation, it can be seen that the detection rate of this invention (CRF) is at the highest level, and is far higher than that of traditional tomographic detection methods. CLINK has a slightly higher accuracy rate than SCFS, which is around 40%. It is evident that the detection rate of traditional tomographic detection methods is very poor. Therefore, it is clear that single-moment link diagnosis is not effective in detecting attacks under prolonged attacks.

[0218] pass Figure 10 The F1 score evaluation results show that the present invention (CRF) also scores the highest among the three methods, demonstrating the reliability of the attack detection results and the ability to identify more attack links.

[0219] pass Figure 11The false positive rate assessment results show that both CLINK and CRF have low values, meaning fewer false positives. The false positive rate of this invention (CRF) is slightly higher than CLINK in some scenarios, but still below 20%. Overall, the attack detection method implemented in this invention performs better.

[0220] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A link-level network tomography and attack detection method based on conditional random fields, characterized in that, Includes the following steps: Step S1: Obtain the routing matrix and the prior congestion probability of the links, and obtain the path state containing the temporal pattern from the samples that conform to the first-order Markov property of the link level. Use the routing matrix, the prior congestion probability of the links, and the path state as the temporal sample pool. Step S2: Combine the routing matrix, prior link congestion probability, and path state to perform feature modeling and generate link-level feature vectors; Step S3: Input the link-level feature vector and its corresponding actual link state label into the CRF model for training, and obtain the state transition weight parameter μ and the weight parameter λ corresponding to the state feature. Step S4: Predict the network state using the trained CRF model; Step S5: Determine if the predicted network state is vulnerable to attack.

2. The link-level network tomography and attack detection method based on conditional random fields according to claim 1, characterized in that, In step S1, the path state is obtained by constructing time-series samples that satisfy the first-order Markov property at the link level using the routing matrix and the prior link congestion probability; firstly, the link state sequence is... The first-order Markov property model is constructed as follows t represents the current time step, and T represents the total time steps; the state of link l at each update. Depends on the state of the previous time step And by the transition probability α l β l The decision is made as follows: when the difference between the empirical congestion probability and the prior congestion probability of all links is less than the tolerance, the congestion distribution of the sample data is considered to satisfy the prior congestion probability distribution of the links; the difference between the empirical congestion probability and the prior congestion probability of each link is calculated, and it is checked whether the difference of all links is less than the tolerance.

3. The link-level network tomography and attack detection method based on conditional random fields according to claim 1, characterized in that, The specific implementation process of step S2 is as follows: S2.1: The mathematical model for the prior congestion probability characteristics of each link is as follows: The link prior congestion probability is used to integrate link-level prior knowledge and help the CRF model understand whether a link is prone to congestion. S2.2: The mathematical model for the path's quantitative characteristics is as follows: R pl Indicates whether path p passes through link l; the path has a quantity characteristic, and the link load is evaluated by counting the number of times link l passes through path P at time step t; S2.3: The mathematical model for the quantitative characteristics of path congestion is as follows: Indicates whether path p is congested at time step t; The path congestion quantity feature captures path-level congestion accumulation, i.e., abnormal load conditions of links, by statistically analyzing the number of congested paths P passing through link l. S2.4: The mathematical model for the link congestion ratio is as follows: The link congestion ratio is obtained by using the characteristics of the number of congested paths and the characteristics of the number of paths. S2.5: The mathematical model for the average congestion ratio of neighbor links is as follows: P n Let ω represent the set of paths on neighbor link n. l This represents the set of neighboring links of link l; In the average congestion ratio of neighbor links, links that follow the same path are grouped as neighbor links. By statistically analyzing the congestion ratio of neighbor links, the congestion situation of the links in the vicinity of the network is reflected.

4. The link-level network tomography and attack detection method based on conditional random fields according to claim 1, characterized in that, The specific implementation process of step S3 is as follows: S3.1: The input data consists of link-level feature vectors and corresponding actual link status labels; S3.2: The CRF model processes the input data; S3.3: Model the state characteristic function and state transition function on a per-link basis; S3.4: Model the conditional probability distribution; S3.5: Solve for the conditional probability using the L-BFGS optimization method; S3.6: Obtain a set of state transition weight parameters μ and state feature corresponding weight parameters λ, and the CRF model training ends.

5. The link-level network tomography and attack detection method based on conditional random fields according to claim 4, characterized in that, The specific implementation process of step S3.2 is as follows: S3.2.1: The CRF model pairs the link-level feature vectors with the corresponding actual link state labels; S3.2.2: The CRF model needs to model the time series samples, taking the original input feature vector... and the current link status. Convert to sequence In form, This represents the link state at the previous moment.

6. The link-level network tomography and attack detection method based on conditional random fields according to claim 4, characterized in that, The specific implementation process of step S3.4 is as follows: S3.4.1: After feature modeling, the problem is transformed into inferring the true state of the link based on link-level feature vectors; the mathematical model of the conditional random field is as follows: in, The state sequence of link l, Let λ be the link-level feature vector corresponding to link l. k f represents the weight parameter of the k-th feature. k This represents the Boolean indicator function corresponding to the k-th feature, used to indicate the observation at the current position t. With candidate tags Mapped to a 0 / 1 eigenvalue, where K represents the number of features, T represents the total time steps, and μ ij This represents the weights for transitioning from state i to state j. This indicates an indicator function, the symbol ∧ represents logical AND, z(F) l ) is the partition function, used to ensure probability normalization, and its mathematical expression is: in, To determine the state label value of a link at time step t during normalized summation, The status label value is taken for the same link at time step t-1; S3.4.2: After modeling with a Conditional Random Field (CRF), the problem is transformed into finding a set of parameters that maximizes the probability of the training data appearing in the model given the known samples; this is equivalent to solving for maximum likelihood estimation. Log-likelihood modeling of the linear chain CRF is as follows: Among them, X (t) F represents the hypothetical label state at time t. (t) Let θ represent the characteristic state at time t, and let θ represent the parameters λ and μ. and Here, c1 and c2 are L2 regularization terms, and both are regularization coefficients. Since the CRF model is trained by minimizing, the objective function is to solve for...

7. The link-level network tomography and attack detection method based on conditional random fields according to claim 4, characterized in that, The specific implementation process of step S3.5 is as follows: S3.5.1: Perform first-order differentiation based on the objective: S3.5.2: Construct an approximate Hessian inverse matrix based on historical information: S3.5.3: Use a line search strategy to find the optimal step size; after finding the optimal step size and updating the parameters, repeat S3.5.2 and S3.5.3 until the number of iterations is reached, then stop the loop and obtain a set of optimal parameters.

8. The link-level network tomography and attack detection method based on conditional random fields according to claim 1, characterized in that, The specific implementation process of step S4 is as follows: S4.1: Obtain the routing matrix and the prior congestion probability of the links, and extract the path states containing temporal patterns from the samples that conform to the first-order Markov property of the links. Use the routing matrix, the prior congestion probability of the links, and the path states as a temporal sample pool. Combine the routing matrix, the prior congestion probability of the links, and the path states to perform feature modeling and generate link-level feature vectors. S4.2: Loading the CRF model means loading the state transition weight parameter μ and the weight parameter λ corresponding to the state features; S4.3: Use the Viterbi method to find the optimal link state solution; S4.4: Output the state inference sequence of all links within the total time step T.

9. The link-level network tomography and attack detection method based on conditional random fields according to claim 1, characterized in that, The specific implementation process of step S4.3 is as follows: S4.3.1: Combining the feature scoring function and the state transition scoring function, the problem model is as follows: in, The state sequence of link l, Let λ be the link-level feature vector corresponding to link l. k f represents the weight parameter of the k-th feature. k This represents the Boolean indicator function corresponding to the k-th feature, used to indicate the observation at the current position t. With candidate tags Mapped to a 0 / 1 eigenvalue, where T represents the total time steps. Represents the state from time t-1. Transition to the state at time t The weights; S4.3.2: Initialize the maximum path score parameter in the first time step: S4.3.3: Recursively determine the optimal path for state transitions and save it to the next state. The path refers to the direction of the state transition. S4.3.4: Record the path source of the best path; S4.3.5: Continuously iterate through S4.3.3 and S4.3.4 until all time steps of data have been traversed to find the final optimal path; S4.3.6: By tracing back the entire path from the path source of the recorded best path, complete state transition information can be obtained.

10. The link-level network tomography and attack detection method based on conditional random fields according to claim 1, characterized in that, The specific implementation process of step S5 is as follows: S5.1: Compare the state inference sequence of all links within the total time step T obtained in step S4 with the actual link state labels to obtain the attack prediction result; S5.2: Compare the attack prediction results with the actual attack labels to obtain the accuracy, detection rate, F1 score and false positive rate of the detection method, so as to verify the detection effect of the CRF model.

Images