Artificial intelligence enhanced distributed denial of service attack defense method and system

Abstract

The invention provides an artificial intelligence enhanced distributed denial of service attack defense method and system, and relates to the technical field of artificial intelligence, and the method comprises the steps: collecting a data packet through a network flow collection module, extracting basic feature information, and obtaining a behavior feature vector through a deep learning feature extraction model; inputting the behavior feature vector into a pre-trained multilayer attention mechanism neural network model, calculating an attack probability score and marking potential attack traffic; performing clustering analysis on the potential attack traffic to calculate a hazard level, and generating a defense strategy according to the hazard level; and executing the defense strategy through the defense equipment, and dynamically optimizing the defense strategy by using a reinforcement learning algorithm based on defense effect feedback. According to the method, the attack traffic can be accurately identified, the defense strategy is adaptively optimized, and the accuracy and efficiency of distributed denial of service attack defense are effectively improved.

Description

Technical Field

[0001] The present invention relates to artificial intelligence technology, and in particular to an artificial intelligence-enhanced distributed denial of service attack defense method and system. Background Art

[0002] Distributed denial of service (DDoS) attacks remain a major challenge in the cybersecurity field. With the rapid development of the internet and the continuous evolution of cyberattack techniques, DDoS attacks are increasing in scale and complexity, making traditional defense methods ineffective. Traditional DDoS defense methods primarily rely on rule-based detection and prevention technologies, such as filtering malicious traffic based on predefined rules, limiting bandwidth, or blocking the attack source IP address.

[0003] Rule-based defense methods require pre-defined rules based on known attack patterns, making them ineffective against new, unknown DDoS attacks. Attackers can circumvent these rules by constantly changing their attack methods and spoofing source IP addresses, rendering defense ineffective. Because the distinction between DDoS attack traffic and normal traffic is blurred, rule-based defense methods can easily misclassify normal traffic as attack traffic, disrupting normal network services.

[0004] When a large number of users simultaneously access a website, it may be mistakenly identified as a DDoS attack, rendering the website inaccessible. Traditional DDoS defense methods typically require manual intervention and configuration, making automated and intelligent defense difficult to achieve. When faced with large-scale DDoS attacks, defenses are slow to respond, making it difficult to quickly and effectively mitigate the impact. Summary of the Invention

[0005] The embodiments of the present invention provide an artificial intelligence-enhanced distributed denial of service attack defense method and system, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention,

[0007] Provides AI-enhanced DDoS attack defense methods, including:

[0008] The network traffic collection module collects data packets from the network and parses the data packets to obtain basic feature information of the data packets, including the source IP address, destination IP address, packet size, protocol type, and arrival time; the deep learning feature extraction model is used to extract features from the basic feature information to obtain a behavioral feature vector of the data packet, including the time series features, traffic pattern features, and protocol interaction features of the data packet;

[0009] The behavioral feature vector is input into a pre-trained multi-layer attention mechanism neural network model. The multi-layer attention mechanism neural network model calculates an attack probability score by performing a weighted analysis on the time series features, traffic pattern features, and protocol interaction features in the behavioral feature vector. Data packets are classified according to the attack probability score, and data packets with attack probability scores higher than a preset threshold are marked as potential attack traffic. Cluster analysis is performed on the data packets marked as potential attack traffic. Based on the clustering results, groups of data packets with similar attack features are identified, and the hazard level of each data packet group is calculated.

[0010] Based on the hazard level of each data packet group, the corresponding defense module is called to generate a defense strategy, which includes traffic filtering rules, bandwidth limitation parameters and firewall configuration; the defense strategy is sent to the defense devices in the network for execution, and real-time defense processing is performed on data packets marked as potential attack traffic; based on the feedback information of the defense effect, the defense strategy is dynamically optimized using a reinforcement learning algorithm. The reinforcement learning algorithm continuously adjusts and optimizes the parameter configuration of the defense strategy through the objective functions of maximizing network service quality and minimizing defense resource consumption.

[0011] The behavior feature vector is input into a pre-trained multi-layer attention mechanism neural network model. The multi-layer attention mechanism neural network model performs a weighted analysis on the time series features, traffic pattern features, and protocol interaction features in the behavior feature vector to calculate the attack probability score including:

[0012] The behavior feature vector is input into a pre-trained multi-layer attention mechanism neural network model. The multi-layer attention mechanism neural network model is pre-trained using a large-scale real network traffic dataset and learns the intrinsic correlation between features through self-supervised learning. The multi-layer attention mechanism neural network model pre-processes the time series features in the behavior feature vector using a position-aware encoder. The position-aware encoder consists of a sinusoidal position code and a learnable linear transformation. The traffic pattern features are encoded through a densely connected feature transformation network. The protocol interaction features are mapped to a fixed-dimensional vector representation using an attention-based sequence encoder to obtain an encoded feature vector.

[0013] The encoded feature vector is input into a feature importance evaluation module, which calculates the importance score by similarity calculation between the query vector and the feature vector based on the attention mechanism, and the importance score is input into a feature interaction modeling module. The feature interaction modeling module adopts a cross-attention mechanism to realize information interaction between different types of features, and obtains the interaction information between features by calculating the product of the feature vector and the parameter matrix; the interaction information between features is input into a dynamic weight allocation module, and the dynamic weight allocation module adopts a soft attention mechanism to fuse the importance score and the interaction information between features through a learnable parameter matrix, and obtains a comprehensive weight coefficient through a sigmoid activation function;

[0014] The multi-layer attention mechanism neural network model inputs the comprehensive weight coefficient and the encoded feature vector into a multi-layer perceptron structure for weighted analysis. The multi-layer perceptron structure introduces a residual connection structure and a layer normalization layer, and sets a Dropout regularization unit between adjacent layers. At the same time, a combined loss function of feature reconstruction loss and cross entropy loss is used to optimize the model. Adversarial samples are generated based on the fast gradient sign method for adversarial training. The robustness of the model is improved by minimizing the classification error of the adversarial samples, and the attack probability score is finally calculated.

[0015] Data packets are classified according to the attack probability scores, and data packets with attack probability scores higher than a preset threshold are marked as potential attack traffic. Cluster analysis is performed on the data packets marked as potential attack traffic, and based on the clustering results, data packet groups with similar attack characteristics are identified, and the hazard level of each data packet group is calculated, including:

[0016] Classify data packets according to their attack probability scores, search for the optimal initial threshold on the validation set using the Bayesian optimization method, divide the validation set into multiple time windows to calculate the true positive rate and false positive rate, select the threshold that maximizes the F1 score as the preset threshold, update the preset threshold using the exponential moving average method based on the real-time traffic feature distribution of the current data packet to obtain the current threshold, compare the attack probability score of the current data packet with the current threshold, and mark data packets with attack probability scores higher than the current threshold as potential attack traffic;

[0017] Extracting time features consisting of packet arrival intervals and burstiness, traffic features consisting of byte rate and packet size distribution, and behavioral features consisting of protocol type and port change pattern from the potential attack traffic, performing Z-score normalization on the time features, traffic features, and behavioral features to obtain normalized data, and performing principal component analysis on the normalized data to obtain normalized features after dimensionality reduction;

[0018] A density peak clustering algorithm is used to perform cluster analysis on the standardized features. Clustering is performed by calculating the local density and minimum distance of each data point in the standardized features. The local density is calculated using an adaptive cutoff distance. A density estimation correction term based on K-nearest neighbors is introduced. Points with local density values ​​greater than a preset density threshold are selected as cluster centers, and packets with similar attack characteristics are clustered into the same packet group.

[0019] Calculate the internal similarity and external difference of each data packet group, where the internal similarity is calculated based on the average cosine similarity of the member points within the data packet group to the cluster center point, and the external difference is measured by the distance between the member points and the cluster center point. Perform a multi-dimensional evaluation of each data packet group based on the internal similarity and the external difference, calculate the average attack probability score of the data packets in each data packet group to obtain the attack intensity, calculate the proportion of the number of data packets to obtain the attack scale, calculate the time span of the data packets to obtain the attack persistence, calculate the characteristic vector variance to obtain the attack complexity, and calculate the matching degree with the historical attack pattern to obtain the attack similarity.

[0020] The attack intensity, attack scale, attack persistence, attack complexity, and attack similarity are subjected to Min-Max normalization processing to obtain normalized index values, the weight coefficients of each index are determined through the hierarchical analysis method, and the normalized index values ​​and the weight coefficients are weighted to obtain the initial value of the hazard level of each data packet group; a long short-term memory network is used to perform time series correlation analysis on the attack behavior of each data packet group, and the attack trend is predicted to obtain the attack trend coefficient, and the initial value of the hazard level and the attack trend coefficient are weighted to obtain the final hazard level of each data packet group.

[0021] Based on the threat level of each packet group, the corresponding defense module is called to generate a defense strategy, which includes traffic filtering rules, bandwidth limit parameters, and firewall configuration. The defense strategy is then sent to defense devices in the network for execution. Real-time defense processing is performed on packets marked as potential attack traffic, including:

[0022] Receive a hazard level of a data packet group, divide the hazard level into five interval values: [0-0.2), [0.2-0.4), [0.4-0.6), [0.6-0.8), and [0.8-1.0], select a corresponding defense module based on the hazard level interval value, select a monitoring module for traffic feature extraction when the hazard level belongs to the interval [0-0.2), select a light defense module for traffic filtering when the hazard level belongs to the interval [0.2-0.4), select a moderate defense module for bandwidth restriction when the hazard level belongs to the interval [0.4-0.6), select a strict defense module for in-depth defense when the hazard level belongs to the interval [0.6-0.8), and select an emergency defense module for traffic isolation when the hazard level belongs to the interval [0.8-1.0];

[0023] Extracting five-tuple information from the data packet group to obtain basic features, the five-tuple information including source IP address, destination IP address, source port number, destination port number, and protocol type; calculating the flow size of the data packet group within a detection time window to obtain flow features; calculating information entropy and flow burstiness based on the basic features and the flow features to obtain statistical features; inputting the basic features, the flow features, and the statistical features into a pre-trained decision tree model, classifying the features based on the decision tree model, and generating a flow filtering rule set including address matching rules, protocol filtering rules, and port control rules;

[0024] A hierarchical token bucket algorithm is used to control the flow of the data packet group. A flow baseline value is calculated based on the bandwidth usage record of the data packet group within a historical time window. A priority coefficient is determined based on the service type of the data packet group, and the service type includes critical service, ordinary service, and non-core service. A dynamic adjustment coefficient is calculated by monitoring the load status of the current network device. The flow baseline value, the priority coefficient, and the dynamic adjustment coefficient are weighted to obtain a multi-level bandwidth limitation parameter.

[0025] Performing application layer inspection on the data packet group using deep packet inspection technology, extracting HTTP payload features, DNS query features, and SSL handshake features to obtain application layer attack features, using a session tracking mechanism to record the TCP connection status, connection duration, and data transmission mode of the data packet group to obtain session state information, and generating a firewall configuration including concurrent connection control rules, session duration limit rules, and application layer feature filtering rules based on the application layer attack features and the session state information;

[0026] The traffic filtering rule set, the multi-level bandwidth restriction parameters, and the firewall configuration are combined to generate a defense strategy. A defense device topology map is constructed by collecting location information and connection relationships of all defense devices in the network. An optimal delivery path for the defense strategy is calculated based on the Dijkstra shortest path algorithm. The defense strategy is delivered to network defense devices in a hierarchical delivery method with a priority on core devices. Rule compression is achieved by merging overlapping rules, and the execution efficiency of the defense strategy is improved by utilizing a multi-threaded parallel processing mechanism.

[0027] The execution effect of the defense strategy is monitored in real time, the strategy execution status on each defense device is collected to obtain execution status information, the attack traffic interception rate, strategy response delay, processor occupancy rate, and memory occupancy rate in the execution status information are calculated to obtain execution effect indicators. When the execution effect indicators do not reach the preset defense targets, the parameter configuration of the defense strategy is dynamically adjusted according to the execution effect indicators, and the adjusted defense strategy is used to implement real-time defense processing of the data packet group marked as potential attack traffic.

[0028] Based on defense effectiveness feedback, the defense strategy is dynamically optimized using a reinforcement learning algorithm. This algorithm continuously adjusts and optimizes the defense strategy's parameter configuration, including the following, based on the objective functions of maximizing network service quality and minimizing defense resource consumption:

[0029] receiving defense effect feedback information from a network device, the defense effect feedback information including network service quality information and defense resource consumption information, the network service quality information including service response delay, network throughput, packet loss rate, and number of service session interruptions, the defense resource consumption information including processor occupancy, memory usage, bandwidth occupancy, and the total number of active rules, normalizing the network service quality information to obtain a service quality score, and normalizing the defense resource consumption information to obtain a resource consumption score;

[0030] Taking the service quality score as a maximization objective and the resource consumption score as a minimization objective, constructing a multi-objective optimization function for reinforcement learning, wherein an output value of the multi-objective optimization function is composed of a weighted combination of the service quality score and the resource consumption score, and calculating a temporal difference error based on the service quality score and the resource consumption score;

[0031] Based on the defense effect feedback information, the number of currently active connections, the size of various types of business traffic, the device load status, and the abnormal traffic distribution are extracted to construct an environment state vector, and the environment state vector is input into the Actor neural network. The Actor neural network uses a long short-term memory unit to process the time series characteristics of the environment state vector and output a defense action, which includes bandwidth limitation parameters, rule priority parameters, firewall configuration parameters, and resource allocation parameters.

[0032] Extract features from the environment state vector via a state feature extraction branch and from the defense action via an action feature extraction branch, fuse the extracted state features with the action features to obtain state-action features, input the state-action features into a critic neural network, and predict a Q value based on the state-action features.

[0033] Constructing a priority experience replay pool to store a state transition sequence, the state transition sequence including the environment state vector, the defense action, the output value of the multi-objective optimization function, and the new environment state vector after executing the defense action; calculating a priority weight of the state transition sequence based on the temporal difference error; and sampling the state transition sequence according to the priority weight to obtain a training sample;

[0034] A deep deterministic policy gradient algorithm is used to update the parameters of the Critic neural network based on the training samples to minimize the temporal difference error, and the parameters of the Actor neural network are updated to maximize the output value of the multi-objective optimization function. The parameters of the target network are updated using a soft update method. The trained Actor neural network and the Critic neural network are obtained through iterative optimization of the deep deterministic policy gradient algorithm.

[0035] The trained Actor neural network is deployed to the defense system, and the current environment state vector is constructed based on the defense effect feedback information received in real time. The current environment state vector is input into the Actor neural network to generate a defense action, and the parameter configuration of the defense strategy is adjusted according to the defense action. The network defense effect is improved by continuously optimizing the parameter configuration.

[0036] A deep deterministic policy gradient algorithm is used to update the parameters of the Critic neural network based on the training samples to minimize the temporal difference error, and the parameters of the Actor neural network are updated to maximize the output value of the multi-objective optimization function. The parameters of the target network are updated using a soft update method. The trained Actor neural network and the Critic neural network are obtained through iterative optimization of the deep deterministic policy gradient algorithm, including:

[0037] Constructing an Actor neural network and a Critic neural network. The Actor neural network includes a state input layer, an action output layer, and a multi-layer feature extraction network connecting the state input layer and the action output layer. The action output layer uses a tanh activation function to map the output to the interval [-1, 1] to obtain an action value. The Critic neural network includes a state input layer, an action input layer, a state-action feature fusion layer, and a value output layer. The action input layer is used to receive the action value output by the Actor neural network.

[0038] Constructing an experience replay buffer pool for storing state transition sequence data, wherein the state transition sequence data includes the current state, current action, immediate reward, and next state; inputting the current state into the Actor neural network to obtain a predicted action value; and inputting the current state and the predicted action value into the Critic neural network to obtain an evaluation Q value of the current state-action pair;

[0039] Calculating a temporal difference error based on the evaluated Q value, calculating an importance weight of each state transition sequence data according to the temporal difference error, and performing priority sampling on the state transition sequence data in the experience replay buffer pool based on the importance weight to obtain a training sample;

[0040] Constructing a target Actor neural network and a target Critic neural network having the same network structure as the Actor neural network and the Critic neural network, inputting the next state in the training sample into the target Actor neural network to obtain a target action value, and inputting the next state and the target action value into the target Critic neural network to obtain a target Q value;

[0041] The target Q value and the immediate reward in the training sample are weighted to obtain a temporal difference target value, the current state and current action in the training sample are input into the critic neural network to obtain a current Q value, and the difference between the temporal difference target value and the current Q value is calculated to obtain a temporal difference error for optimization;

[0042] Inputting the current state in the training sample into the Actor neural network to obtain a training predicted action value, adding Gaussian noise to the training predicted action value to obtain a noisy predicted action value, and inputting the current state and the noisy predicted action value into the Critic neural network to obtain a noisy evaluation Q value;

[0043] A multi-objective optimization function is constructed based on the noisy evaluation Q value, and a deep deterministic policy gradient algorithm is used to update the parameters of the Critic neural network through back propagation based on the optimization temporal difference error to minimize the optimization temporal difference error. Based on the multi-objective optimization function, the parameters of the Actor neural network are updated through policy gradient ascent to maximize the output value of the multi-objective optimization function; and a soft update coefficient is used to control the smooth update of the updated parameters of the Actor neural network and the Critic neural network to the target Actor neural network and the target Critic neural network.

[0044] According to a second aspect of the embodiments of the present invention,

[0045] Provides an AI-enhanced distributed denial of service attack defense system, including:

[0046] The first unit collects data packets in the network through a network traffic collection module, parses the data packets to obtain basic feature information of the data packets, the basic feature information includes the source IP address, destination IP address, packet size, protocol type and arrival time; extracts features from the basic feature information based on a deep learning feature extraction model to obtain a behavior feature vector of the data packet, the behavior feature vector includes the time series features, traffic pattern features and protocol interaction features of the data packet;

[0047] The second unit is configured to input the behavioral feature vector into a pre-trained multi-layer attention mechanism neural network model. The multi-layer attention mechanism neural network model calculates an attack probability score by performing a weighted analysis on the time series features, traffic pattern features, and protocol interaction features in the behavioral feature vector; classifies data packets according to the attack probability score, and marks data packets with attack probability scores higher than a preset threshold as potential attack traffic; performs cluster analysis on the data packets marked as potential attack traffic, identifies groups of data packets with similar attack features based on the clustering results, and calculates the hazard level of each data packet group;

[0048] The third unit is used to call the corresponding defense module to generate a defense strategy based on the hazard level of each data packet group. The defense strategy includes traffic filtering rules, bandwidth limitation parameters and firewall configuration; the defense strategy is sent to the defense devices in the network for execution, and real-time defense processing is performed on data packets marked as potential attack traffic; based on the defense effect feedback information, the defense strategy is dynamically optimized using a reinforcement learning algorithm. The reinforcement learning algorithm continuously adjusts and optimizes the parameter configuration of the defense strategy through the objective functions of maximizing network service quality and minimizing defense resource consumption.

[0049] According to a third aspect of the embodiments of the present invention,

[0050] An electronic device is provided, comprising:

[0051] processor;

[0052] a memory for storing processor-executable instructions;

[0053] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0054] According to a fourth aspect of the embodiments of the present invention,

[0055] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0056] The beneficial effects of this application are as follows:

[0057] 1. Improve DDoS attack detection accuracy: Through deep learning feature extraction and a multi-layer attention mechanism, it is possible to more finely analyze the behavioral characteristics of data packets and identify attack traffic hidden in normal traffic, thereby improving the accuracy of DDoS attack detection and reducing false positives and missed reports.

[0058] 2. Accurately defend against DDoS attacks: Based on attack probability scores and cluster analysis, it can identify groups of packets with similar attack characteristics and formulate corresponding defense strategies based on the level of damage, achieving precise defense against DDoS attacks and avoiding impacts on normal traffic.

[0059] 3. Adaptive optimization of defense strategies: Utilizing reinforcement learning algorithms, the defense strategy parameters are dynamically adjusted and optimized based on defense effectiveness feedback. This allows the defense system to adaptively respond to evolving DDoS attack methods, improving its effectiveness and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flow chart of an artificial intelligence-enhanced distributed denial of service attack defense method according to an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of the structure of an artificial intelligence-enhanced distributed denial of service attack defense system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0063] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0064] Figure 1 This is a flow chart of the artificial intelligence enhanced distributed denial of service attack defense method according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0065] S11. Collect data packets from the network using a network traffic collection module, parse the data packets to obtain basic feature information of the data packets, including source IP address, destination IP address, packet size, protocol type, and arrival time; perform feature extraction on the basic feature information based on a deep learning feature extraction model to obtain a behavioral feature vector of the data packets, including time series features, traffic pattern features, and protocol interaction features of the data packets;

[0066] S12. Input the behavioral feature vector into a pre-trained multi-layer attention mechanism neural network model. The multi-layer attention mechanism neural network model calculates an attack probability score by performing a weighted analysis on the time series features, traffic pattern features, and protocol interaction features in the behavioral feature vector. Data packets are classified according to the attack probability score, and data packets with attack probability scores above a preset threshold are marked as potential attack traffic. Cluster analysis is performed on the data packets marked as potential attack traffic. Based on the clustering results, groups of data packets with similar attack features are identified, and the hazard level of each data packet group is calculated.

[0067] S13. Based on the hazard level of each data packet group, the corresponding defense module is called to generate a defense strategy, which includes traffic filtering rules, bandwidth limitation parameters, and firewall configuration; the defense strategy is sent to the defense devices in the network for execution, and real-time defense processing is performed on data packets marked as potential attack traffic; based on the defense effect feedback information, the defense strategy is dynamically optimized using a reinforcement learning algorithm. The reinforcement learning algorithm continuously adjusts and optimizes the parameter configuration of the defense strategy through the objective functions of maximizing network service quality and minimizing defense resource consumption.

[0068] In an optional embodiment, the behavior feature vector is input into a pre-trained multi-layer attention mechanism neural network model. The multi-layer attention mechanism neural network model performs a weighted analysis on the time series features, traffic pattern features, and protocol interaction features in the behavior feature vector to calculate an attack probability score including:

[0069] The behavior feature vector is input into a pre-trained multi-layer attention mechanism neural network model. The multi-layer attention mechanism neural network model is pre-trained using a large-scale real network traffic dataset and learns the intrinsic correlation between features through self-supervised learning. The multi-layer attention mechanism neural network model pre-processes the time series features in the behavior feature vector using a position-aware encoder. The position-aware encoder consists of a sinusoidal position code and a learnable linear transformation. The traffic pattern features are encoded through a densely connected feature transformation network. The protocol interaction features are mapped to a fixed-dimensional vector representation using an attention-based sequence encoder to obtain an encoded feature vector.

[0070] The encoded feature vector is input into a feature importance evaluation module, which calculates the importance score by similarity calculation between the query vector and the feature vector based on the attention mechanism, and the importance score is input into a feature interaction modeling module. The feature interaction modeling module adopts a cross-attention mechanism to realize information interaction between different types of features, and obtains the interaction information between features by calculating the product of the feature vector and the parameter matrix; the interaction information between features is input into a dynamic weight allocation module, and the dynamic weight allocation module adopts a soft attention mechanism to fuse the importance score and the interaction information between features through a learnable parameter matrix, and obtains a comprehensive weight coefficient through a sigmoid activation function;

[0071] The multi-layer attention mechanism neural network model inputs the comprehensive weight coefficient and the encoded feature vector into a multi-layer perceptron structure for weighted analysis. The multi-layer perceptron structure introduces a residual connection structure and a layer normalization layer, and sets a Dropout regularization unit between adjacent layers. At the same time, a combined loss function of feature reconstruction loss and cross entropy loss is used to optimize the model. Adversarial samples are generated based on the fast gradient sign method for adversarial training. The robustness of the model is improved by minimizing the classification error of the adversarial samples, and the attack probability score is finally calculated.

[0072] A network attack detection method based on a multi-layer attention neural network calculates the attack probability score of network behavior. The core of this method is to use a multi-layer attention mechanism to perform a weighted analysis of multiple characteristics of network behavior and enhance the robustness of the model through adversarial training.

[0073] First, the collected raw network traffic data is preprocessed to extract three key features: time series features, traffic pattern features, and protocol interaction features. For example, time series features can be the number of visits from a certain IP address in different time periods, traffic pattern features can be the statistical distribution of packet size and packet interval time, and protocol interaction features can be the different protocol types used between the client and server and the interaction sequence.

[0074] The extracted feature vectors are then fed into a pre-trained multi-layer attention neural network model. This model, pre-trained using a large-scale dataset of real network traffic, learns the inherent relationships between features through self-supervised learning. Specifically, the pre-training process uses a masked language model to learn the relationships between features by predicting the values ​​of masked features.

[0075] Next, different types of features are encoded. Time series features are preprocessed using a position-aware encoder, for example, by converting timestamps into sine and cosine function values. This is combined with a learnable linear transformation to enable the model to understand the sequential information of the time series. Traffic pattern features are encoded using a densely connected feature transformer network, for example, using multiple fully connected layers to map features to a high-dimensional space. Protocol interaction features are mapped into fixed-dimensional vector representations using an attention-based sequence encoder, for example, using a Transformer model to capture the contextual information of protocol interactions. For example, suppose the time series features are [10, 20, 30]. After passing through the position-aware encoder, they become [10.2, 20.5, 30.8]. The traffic pattern features are [100, 500, 200]. After passing through the feature transformer network, they become [0.1, 0.5, 0.2]. The protocol interaction features are ["TCP", "HTTP", "DNS"]. After passing through the sequence encoder, they become [0.7, 0.2, 0.1].

[0076] The encoded feature vectors are then fed into the feature importance assessment module. This module uses the attention mechanism to evaluate the importance of each feature by calculating the similarity between the query vector and the feature vector. For example, if the query vector is [0.2, 0.3, 0.5], the similarity calculation yields an importance score of 0.8 for the time series feature, 0.6 for the traffic pattern feature, and 0.7 for the protocol interaction feature.

[0077] At the same time, the encoded feature vector is input into the feature interaction modeling module. This module uses a cross-attention mechanism to capture the interaction information between different types of features by computing the product of the feature vector and the parameter matrix. For example, the interaction information between time series features and traffic pattern features can reflect changes in traffic patterns within a specific time period.

[0078] Next, the feature importance scores and feature interaction information are fed into the dynamic weight assignment module. This module uses a soft attention mechanism to fuse the importance scores and interaction information using a learnable parameter matrix and a sigmoid activation function to derive a comprehensive weight coefficient for each feature. Assume that the comprehensive weight coefficient for time series features is 0.75, for traffic pattern features is 0.65, and for protocol interaction features is 0.8.

[0079] Finally, the combined weight coefficients and encoded feature vectors are input into a multilayer perceptron architecture for weighted analysis. This architecture incorporates residual connections and layer normalization layers, with Dropout regularization units placed between adjacent layers to improve the model's generalization capabilities. The model is optimized using a combination of feature reconstruction loss and cross-entropy loss, and adversarial training is performed using the fast gradient sign method to generate adversarial examples to enhance the model's robustness. Finally, the model outputs an attack probability score; for example, 0.9 indicates that the network behavior has a high probability of attack.

[0080] The beneficial effects of this method can be summarized in the following three aspects:

[0081] 1. Improve detection accuracy: Through multi-layer attention mechanisms and feature interaction modeling, the characteristics of network attacks can be more accurately captured, thereby improving detection accuracy.

[0082] 2. Enhance model robustness: Adversarial training can make the model more resistant to adversarial samples, thereby improving the robustness of the model.

[0083] 3. Reduce false alarm rate: Through dynamic weight allocation, the false alarm rate of normal network behavior can be effectively reduced.

[0084] In an optional embodiment, data packets are classified according to the attack probability scores, and data packets with attack probability scores higher than a preset threshold are marked as potential attack traffic; cluster analysis is performed on the data packets marked as potential attack traffic, and based on the clustering results, groups of data packets with similar attack characteristics are identified, and the hazard level of each data packet group is calculated, including:

[0085] Classify data packets according to their attack probability scores, search for the optimal initial threshold on the validation set using the Bayesian optimization method, divide the validation set into multiple time windows to calculate the true positive rate and false positive rate, select the threshold that maximizes the F1 score as the preset threshold, update the preset threshold using the exponential moving average method based on the real-time traffic feature distribution of the current data packet to obtain the current threshold, compare the attack probability score of the current data packet with the current threshold, and mark data packets with attack probability scores higher than the current threshold as potential attack traffic;

[0086] Extracting time features consisting of packet arrival intervals and burstiness, traffic features consisting of byte rate and packet size distribution, and behavioral features consisting of protocol type and port change pattern from the potential attack traffic, performing Z-score normalization on the time features, traffic features, and behavioral features to obtain normalized data, and performing principal component analysis on the normalized data to obtain normalized features after dimensionality reduction;

[0087] A density peak clustering algorithm is used to perform cluster analysis on the standardized features. Clustering is performed by calculating the local density and minimum distance of each data point in the standardized features. The local density is calculated using an adaptive cutoff distance. A density estimation correction term based on K-nearest neighbors is introduced. Points with local density values ​​greater than a preset density threshold are selected as cluster centers, and packets with similar attack characteristics are clustered into the same packet group.

[0088] Calculate the internal similarity and external difference of each data packet group, where the internal similarity is calculated based on the average cosine similarity of the member points within the data packet group to the cluster center point, and the external difference is measured by the distance between the member points and the cluster center point. Perform a multi-dimensional evaluation of each data packet group based on the internal similarity and the external difference, calculate the average attack probability score of the data packets in each data packet group to obtain the attack intensity, calculate the proportion of the number of data packets to obtain the attack scale, calculate the time span of the data packets to obtain the attack persistence, calculate the characteristic vector variance to obtain the attack complexity, and calculate the matching degree with the historical attack pattern to obtain the attack similarity.

[0089] The attack intensity, attack scale, attack persistence, attack complexity, and attack similarity are subjected to Min-Max normalization processing to obtain normalized index values, the weight coefficients of each index are determined through the hierarchical analysis method, and the normalized index values ​​and the weight coefficients are weighted to obtain the initial value of the hazard level of each data packet group; a long short-term memory network is used to perform time series correlation analysis on the attack behavior of each data packet group, and the attack trend is predicted to obtain the attack trend coefficient, and the initial value of the hazard level and the attack trend coefficient are weighted to obtain the final hazard level of each data packet group.

[0090] A network attack detection method based on packet attack probability and cluster analysis can effectively identify and assess the severity of network attacks. The core concept of this method is to perform a preliminary screening based on the attack probability score of packets, then perform cluster analysis on the filtered potential attack traffic, and finally evaluate the severity of each packet group based on multiple dimensions.

[0091] First, the attack probability score of the data packet is calculated. This step can be implemented using existing machine learning models or deep learning models. For example, a classifier based on network traffic features is trained to output an attack probability score for each data packet.

[0092] Next, the threshold of the attack probability score is determined. In order to find the optimal initial threshold, the Bayesian optimization method is used to search on the validation set. The validation set is divided into multiple time windows, and the true positive rate and false positive rate under different thresholds are calculated in each time window, and the threshold that maximizes the F1 score is selected as the preset threshold. In order to adapt to changes in network traffic, the exponential moving average method is used to dynamically update the preset threshold according to the real-time traffic feature distribution of the current data packet to obtain the current threshold. The attack probability score of each data packet is compared with the current threshold, and the data packets above the current threshold are marked as potential attack traffic. For example, assuming the preset threshold is 0.8 and the attack probability score of the current data packet is 0.9, since 0.9 is greater than 0.8, the data packet is marked as potential attack traffic.

[0093] Next, feature extraction is performed on packets marked as potential attack traffic. The extracted features include temporal features (such as packet arrival interval and burstiness), traffic features (such as byte rate and packet size distribution), and behavioral features (such as protocol type and port change pattern). For example, a packet may have an arrival interval of 0.1 seconds, a burstiness of 10, a byte rate of 1000 bytes / second, a packet size of 500 bytes, a TCP protocol, and port 80.

[0094] In order to eliminate the dimensional differences between different features, the extracted features are subjected to Z-score normalization, and the value of each feature is converted into standardized data with a mean of 0 and a standard deviation of 1. Then, the principal component analysis method is used to reduce the dimension of the standardized data to obtain the standardized features after dimensionality reduction.

[0095] Cluster analysis is performed on the standardized features after dimensionality reduction. A density peak clustering algorithm is used to cluster data points by calculating the local density and minimum distance of each data point. Local density is calculated using an adaptive cutoff distance, and a K-nearest neighbor-based density estimation correction term is introduced. Points with local density values ​​greater than a preset density threshold are selected as cluster centers. Packets with similar attack characteristics are clustered into the same packet group.

[0096] Calculate the hazard level of each data packet group. First, calculate the internal similarity and external difference of each data packet group. The internal similarity is calculated based on the average cosine similarity of the member points within the data packet group to the cluster center point. The external difference is measured by the distance between the nearest data packet group. Then, based on the internal similarity and external difference, a multi-dimensional evaluation is performed on each data packet group to calculate the following indicators: attack intensity (average attack probability score of data packets within the data packet group), attack scale (number of data packets as a percentage), attack duration (time span of data packets), attack complexity (feature vector variance) and attack similarity (matching degree with historical attack patterns). For example, the attack intensity of a data packet group is 0.9, the attack scale is 0.1, the attack duration is 10 seconds, the attack complexity is 0.5, and the attack similarity is 0.8.

[0097] The above indicators are normalized using the Min-Max method to obtain normalized indicator values. The weight coefficients for each indicator are determined using the Analytic Hierarchy Process (AHP). The normalized indicator values ​​are weighted with the weight coefficients to obtain the initial hazard level for each data packet group.

[0098] A long short-term memory network is used to analyze the temporal correlation of attack behaviors for each data packet group, predict attack trends, and obtain an attack trend coefficient. The initial hazard level value and the attack trend coefficient are weighted to obtain the final hazard level for each data packet group.

[0099] Beneficial effects:

[0100] 1. Improved accuracy: By combining attack probability scores and multi-dimensional feature cluster analysis, potential attack traffic can be more accurately identified, reducing false positives and missed negatives.

[0101] 2. Enhanced adaptability: The exponential moving average method is used to dynamically update the threshold and the local density-based clustering algorithm can better adapt to changes in network traffic and improve the robustness of detection.

[0102] 3. More comprehensive hazard assessment: By evaluating the attack intensity, scale, persistence, complexity, and similarity in multiple dimensions, the hazard level of the attack can be more comprehensively assessed, providing security personnel with more effective decision-making support.

[0103] In an optional embodiment, based on the threat level of each data packet group, a corresponding defense module is invoked to generate a defense strategy, the defense strategy including traffic filtering rules, bandwidth limiting parameters, and firewall configuration; the defense strategy is distributed to defense devices in the network for execution, and real-time defense processing is performed on data packets marked as potential attack traffic, including:

[0104] Receive a hazard level of a data packet group, divide the hazard level into five interval values: [0-0.2), [0.2-0.4), [0.4-0.6), [0.6-0.8), and [0.8-1.0], select a corresponding defense module based on the hazard level interval value, select a monitoring module for traffic feature extraction when the hazard level belongs to the interval [0-0.2), select a light defense module for traffic filtering when the hazard level belongs to the interval [0.2-0.4), select a moderate defense module for bandwidth restriction when the hazard level belongs to the interval [0.4-0.6), select a strict defense module for in-depth defense when the hazard level belongs to the interval [0.6-0.8), and select an emergency defense module for traffic isolation when the hazard level belongs to the interval [0.8-1.0];

[0105] Extracting five-tuple information from the data packet group to obtain basic features, the five-tuple information including source IP address, destination IP address, source port number, destination port number, and protocol type; calculating the flow size of the data packet group within a detection time window to obtain flow features; calculating information entropy and flow burstiness based on the basic features and the flow features to obtain statistical features; inputting the basic features, the flow features, and the statistical features into a pre-trained decision tree model, classifying the features based on the decision tree model, and generating a flow filtering rule set including address matching rules, protocol filtering rules, and port control rules;

[0106] A hierarchical token bucket algorithm is used to control the flow of the data packet group. A flow baseline value is calculated based on the bandwidth usage record of the data packet group within a historical time window. A priority coefficient is determined based on the service type of the data packet group, and the service type includes critical service, ordinary service, and non-core service. A dynamic adjustment coefficient is calculated by monitoring the load status of the current network device. The flow baseline value, the priority coefficient, and the dynamic adjustment coefficient are weighted to obtain a multi-level bandwidth limitation parameter.

[0107] Performing application layer inspection on the data packet group using deep packet inspection technology, extracting HTTP payload features, DNS query features, and SSL handshake features to obtain application layer attack features, using a session tracking mechanism to record the TCP connection status, connection duration, and data transmission mode of the data packet group to obtain session state information, and generating a firewall configuration including concurrent connection control rules, session duration limit rules, and application layer feature filtering rules based on the application layer attack features and the session state information;

[0108] The traffic filtering rule set, the multi-level bandwidth restriction parameters, and the firewall configuration are combined to generate a defense strategy. A defense device topology map is constructed by collecting location information and connection relationships of all defense devices in the network. An optimal delivery path for the defense strategy is calculated based on the Dijkstra shortest path algorithm. The defense strategy is delivered to network defense devices in a hierarchical delivery method with a priority on core devices. Rule compression is achieved by merging overlapping rules, and the execution efficiency of the defense strategy is improved by utilizing a multi-threaded parallel processing mechanism.

[0109] The execution effect of the defense strategy is monitored in real time, the strategy execution status on each defense device is collected to obtain execution status information, the attack traffic interception rate, strategy response delay, processor occupancy rate, and memory occupancy rate in the execution status information are calculated to obtain execution effect indicators. When the execution effect indicators do not reach the preset defense targets, the parameter configuration of the defense strategy is dynamically adjusted according to the execution effect indicators, and the adjusted defense strategy is used to implement real-time defense processing of the data packet group marked as potential attack traffic.

[0110] Implementation method:

[0111] First, the received data packet group is evaluated for its risk level. This evaluation result is expressed as a value between 0 and 1, with higher values ​​indicating a higher risk level. The risk level is divided into five intervals: [0-0.2], [0.2-0.4], [0.4-0.6], [0.6-0.8], and [0.8-1.0].

[0112] Select the corresponding defense module based on the hazard level range of the packet group. For example, if a packet group with a hazard level of 0.1 falls within the range of [0-0.2), select the monitoring module. If a packet group with a hazard level of 0.3 falls within the range of [0.2-0.4), select the light defense module. If a packet group with a hazard level of 0.5 falls within the range of [0.4-0.6), select the moderate defense module. If a packet group with a hazard level of 0.7 falls within the range of [0.6-0.8), select the strict defense module. If a packet group with a hazard level of 0.9 falls within the range of [0.8-1.0], select the emergency defense module.

[0113] Then, the five-tuple information of the packet group is extracted, including the source IP address, destination IP address, source port number, destination port number, and protocol type, as the basic features. For example, the five-tuple information of a packet group is: source IP address 192.168.1.10, destination IP address 8.8.8.8, source port number 56789, destination port number 53, and protocol type UDP.

[0114] Next, the flow size of the data packet group within a specific time window is calculated. For example, in the past minute, the flow size of the data packet group is 10MB, which is used as the flow feature.

[0115] Based on basic features and traffic characteristics, information entropy and traffic burstiness are calculated as statistical features. For example, the information entropy value is 0.8 and the traffic burstiness is 1.2.

[0116] Basic features, traffic features, and statistical features are input into a pre-trained decision tree model for classification. The decision tree model determines the type of packet group based on the input features and generates a corresponding set of traffic filtering rules, including address matching rules, protocol filtering rules, and port control rules. For example, a generated rule might be to block UDP traffic with a source IP address of 192.168.1.10 and a destination port number of 53.

[0117] Next, a hierarchical token bucket algorithm is used to control the flow of the packet group. The traffic baseline value is calculated based on the bandwidth usage record of the packet group within the historical time window. For example, the average bandwidth usage of the packet group in the past hour was 2Mbps. The priority coefficient is determined based on the service type of the packet group (for example, critical service, ordinary service, or non-core service). For example, the priority coefficient of critical service is 0.9, the priority coefficient of ordinary service is 0.5, and the priority coefficient of non-core service is 0.1. The dynamic adjustment coefficient is calculated by monitoring the load status of the current network equipment. For example, if the current network load is high, the dynamic adjustment coefficient is 0.8. The traffic baseline value, priority coefficient, and dynamic adjustment coefficient are weighted to obtain the multi-level bandwidth limit parameters. For example, the calculation result is: the bandwidth limit of this packet group is 1.44Mbps (2Mbps*0.9*0.8).

[0118] Use deep packet inspection technology to perform application layer inspection on data packet groups. Extract HTTP payload features, DNS query features, and SSL handshake features as application layer attack features. For example, detect that the HTTP payload contains malicious code. Use a session tracking mechanism to record the TCP connection status, connection duration, and data transmission mode of the data packet group as session state information. For example, it is recorded that a TCP connection lasted for 10 minutes and transmitted a large amount of data. Generate firewall configuration based on application layer attack features and session state information, including concurrent connection control rules, session duration limit rules, and application layer feature filtering rules. For example, limit the number of concurrent connections from the same source IP to no more than 10.

[0119] A defense strategy is generated by combining traffic filtering rule sets, multi-level bandwidth limiting parameters, and firewall configurations. A topology map of all defense devices in the network is constructed by collecting their location and connectivity information. The optimal delivery path for the defense strategy is calculated using the Dijkstra shortest path algorithm, and the strategy is distributed to network defense devices in a hierarchical manner, prioritizing core devices. Overlapping rules are merged to achieve rule compression, and multi-threaded parallel processing is used to improve the execution efficiency of the defense strategy.

[0120] Finally, the effectiveness of the defense policy execution is monitored in real time. The policy execution status on each defense device is collected, and execution performance metrics such as attack traffic interception rate, policy response latency, processor utilization, and memory utilization are calculated. If the execution performance metrics do not meet the preset defense targets, the defense policy parameter configuration is dynamically adjusted based on the execution performance metrics.

[0121] Beneficial effects:

[0122] 1. Improve defense accuracy: Through multi-dimensional feature extraction and decision tree models, potential attack traffic can be more accurately identified, avoiding misjudgments and missed judgments, thereby improving defense accuracy.

[0123] 2. Enhanced defense flexibility: Dynamically select defense modules and adjust defense strategy parameters according to the hazard level, enabling flexible response to different types of attacks and adaptively adjusting defense strength according to changes in the network environment.

[0124] 3. Optimize resource utilization: The hierarchical token bucket algorithm and multi-level bandwidth limit parameters can effectively control the bandwidth consumption of attack traffic and avoid network congestion. At the same time, the core device-prioritized policy delivery method and rule compression mechanism can optimize network resource utilization.

[0125] In an optional embodiment, based on defense effect feedback information, a reinforcement learning algorithm is used to dynamically optimize the defense strategy. The reinforcement learning algorithm continuously adjusts and optimizes the parameter configuration of the defense strategy by using the objective functions of maximizing network service quality and minimizing defense resource consumption. The parameters include:

[0126] receiving defense effect feedback information from a network device, the defense effect feedback information including network service quality information and defense resource consumption information, the network service quality information including service response delay, network throughput, packet loss rate, and number of service session interruptions, the defense resource consumption information including processor occupancy, memory usage, bandwidth occupancy, and the total number of active rules, normalizing the network service quality information to obtain a service quality score, and normalizing the defense resource consumption information to obtain a resource consumption score;

[0127] Taking the service quality score as a maximization objective and the resource consumption score as a minimization objective, constructing a multi-objective optimization function for reinforcement learning, wherein an output value of the multi-objective optimization function is composed of a weighted combination of the service quality score and the resource consumption score, and calculating a temporal difference error based on the service quality score and the resource consumption score;

[0128] Based on the defense effect feedback information, the number of currently active connections, the size of various types of business traffic, the device load status, and the abnormal traffic distribution are extracted to construct an environment state vector, and the environment state vector is input into the Actor neural network. The Actor neural network uses a long short-term memory unit to process the time series characteristics of the environment state vector and output a defense action, which includes bandwidth limitation parameters, rule priority parameters, firewall configuration parameters, and resource allocation parameters.

[0129] Extract features from the environment state vector via a state feature extraction branch and from the defense action via an action feature extraction branch, fuse the extracted state features with the action features to obtain state-action features, input the state-action features into a critic neural network, and predict a Q value based on the state-action features.

[0130] Constructing a priority experience replay pool to store a state transition sequence, the state transition sequence including the environment state vector, the defense action, the output value of the multi-objective optimization function, and the new environment state vector after executing the defense action; calculating a priority weight of the state transition sequence based on the temporal difference error; and sampling the state transition sequence according to the priority weight to obtain a training sample;

[0131] A deep deterministic policy gradient algorithm is used to update the parameters of the Critic neural network based on the training samples to minimize the temporal difference error, and the parameters of the Actor neural network are updated to maximize the output value of the multi-objective optimization function. The parameters of the target network are updated using a soft update method. The trained Actor neural network and the Critic neural network are obtained through iterative optimization of the deep deterministic policy gradient algorithm.

[0132] The trained Actor neural network is deployed to the defense system, and the current environment state vector is constructed based on the defense effect feedback information received in real time. The current environment state vector is input into the Actor neural network to generate a defense action, and the parameter configuration of the defense strategy is adjusted according to the defense action. The network defense effect is improved by continuously optimizing the parameter configuration.

[0133] A dynamic optimization method for the defense system, which is based on defense effect feedback information and uses reinforcement learning algorithms to dynamically optimize the defense strategy to improve network defense effectiveness.

[0134] First, we collect defense effectiveness feedback from network devices. This information consists of two parts: network service quality (QoS) and defense resource consumption. QoS information includes metrics such as service response latency, network throughput, packet loss rate, and number of service session interruptions. For example, consider a service response latency of 10ms, network throughput of 1Gbps, packet loss rate of 0.1%, and two service session interruptions. Defense resource consumption information includes metrics such as processor utilization, memory utilization, bandwidth utilization, and the total number of active rules. For example, consider a processor utilization of 60%, memory utilization of 70%, bandwidth utilization of 50%, and a total number of 100 active rules.

[0135] Next, the collected information is normalized. For example, a 10ms service response delay is normalized to 0.1, a 1Gbps network throughput is normalized to 0.9, a 0.1% packet loss rate is normalized to 0.001, and two service session interruptions are normalized to 0.02. Similarly, a 60% processor utilization rate is normalized to 0.6, a 70% memory utilization rate is normalized to 0.7, a 50% bandwidth utilization rate is normalized to 0.5, and the total number of active rules of 100 is normalized to 0.8. The normalized network service quality information is combined into a service quality score. The normalized defense resource consumption information is combined into a resource consumption score, for example, a resource consumption score of (0.6 + 0.7 + 0.5 + 0.8) / 4 = 0.65.

[0136] Next, a multi-objective optimization function for reinforcement learning is constructed. The service quality score is maximized, and the resource consumption score is minimized. The output of the multi-objective optimization function is a weighted combination of the service quality score and the resource consumption score. The temporal difference error is calculated based on the current and previous service quality and resource consumption scores, as well as their corresponding multi-objective optimization function outputs.

[0137] At the same time, an environment state vector is extracted based on defense effectiveness feedback. For example, if the current number of active connections is 1000, web browsing traffic is 500Mbps, video streaming traffic is 200Mbps, file transfer traffic is 100Mbps, device load is medium, and abnormal traffic distribution is concentrated in a specific IP address segment, this information is combined into an environment state vector.

[0138] The environment state vector is input into the Actor Neural Network. The Actor Neural Network uses long short-term memory units to process the temporal characteristics of the environment state vector and outputs defensive actions. Defensive actions include parameters such as bandwidth limit, rule priority, firewall configuration, and resource allocation. For example, the bandwidth limit for web browsing traffic can be set to 100 Mbps, the bandwidth limit for video streaming traffic can be set to 50 Mbps, and the bandwidth limit for file transfer traffic can be set to 20 Mbps; the priority of rules for specific IP address segments can be adjusted; firewall configuration can be modified, such as blocking access to specific ports; and resource allocation can be adjusted, such as allocating more computing resources to critical business operations.

[0139] The environment state vector and defense action are fed into the state feature extraction branch and action feature extraction branch, respectively, to extract features. The extracted features are then fused to obtain the state-action feature. The state-action feature is then fed into the critic neural network, which predicts the Q value based on the state-action feature.

[0140] A priority experience replay pool is constructed to store state transition sequences. The state transition sequence includes the environment state vector, the defensive action, the output of the multi-objective optimization function, and the new environment state vector after the defensive action is executed. The priority weight of the state transition sequence is calculated based on the temporal difference error, and the state transition sequence is sampled according to the priority weight to obtain training samples.

[0141] A deep deterministic policy gradient algorithm is used to update the parameters of the critic neural network based on training samples to minimize the temporal difference error. The parameters of the actor neural network are also updated to maximize the output of the multi-objective optimization function. The parameters of the target network are updated using a soft update method. Through iterative optimization using the deep deterministic policy gradient algorithm, the trained actor and critic neural networks are obtained.

[0142] Finally, the trained Actor neural network is deployed to the defense system. Based on the real-time defense effect feedback, the current environment state vector is constructed. This state vector is then fed into the Actor neural network to generate defense actions. The defense strategy parameters are then adjusted based on these actions.

[0143] Beneficial effects:

[0144] 1. Improve network defense effectiveness: By dynamically adjusting defense strategy parameter configurations, effectively respond to ever-changing network attacks and improve the adaptability and robustness of the defense system.

[0145] 2. Optimize resource utilization efficiency: Minimize defense resource consumption and improve resource utilization efficiency while ensuring network service quality.

[0146] 3. Automated defense strategy configuration: Defense strategy parameter configuration can be automatically adjusted without manual intervention, reducing the burden on administrators and improving operation and maintenance efficiency.

[0147] In an optional embodiment, a deep deterministic policy gradient algorithm is used to update the parameters of the Critic neural network based on the training samples to minimize the temporal difference error, and the parameters of the Actor neural network are updated to maximize the output value of the multi-objective optimization function. The parameters of the target network are updated using a soft update method. The Actor neural network and the Critic neural network that are trained are obtained through iterative optimization of the deep deterministic policy gradient algorithm, including:

[0148] Constructing an Actor neural network and a Critic neural network. The Actor neural network includes a state input layer, an action output layer, and a multi-layer feature extraction network connecting the state input layer and the action output layer. The action output layer uses a tanh activation function to map the output to the interval [-1, 1] to obtain an action value. The Critic neural network includes a state input layer, an action input layer, a state-action feature fusion layer, and a value output layer. The action input layer is used to receive the action value output by the Actor neural network.

[0149] Constructing an experience replay buffer pool for storing state transition sequence data, wherein the state transition sequence data includes the current state, current action, immediate reward, and next state; inputting the current state into the Actor neural network to obtain a predicted action value; and inputting the current state and the predicted action value into the Critic neural network to obtain an evaluation Q value of the current state-action pair;

[0150] Calculating a temporal difference error based on the evaluated Q value, calculating an importance weight of each state transition sequence data according to the temporal difference error, and performing priority sampling on the state transition sequence data in the experience replay buffer pool based on the importance weight to obtain a training sample;

[0151] Constructing a target Actor neural network and a target Critic neural network having the same network structure as the Actor neural network and the Critic neural network, inputting the next state in the training sample into the target Actor neural network to obtain a target action value, and inputting the next state and the target action value into the target Critic neural network to obtain a target Q value;

[0152] The target Q value and the immediate reward in the training sample are weighted to obtain a temporal difference target value, the current state and current action in the training sample are input into the critic neural network to obtain a current Q value, and the difference between the temporal difference target value and the current Q value is calculated to obtain a temporal difference error for optimization;

[0153] Inputting the current state in the training sample into the Actor neural network to obtain a training predicted action value, adding Gaussian noise to the training predicted action value to obtain a noisy predicted action value, and inputting the current state and the noisy predicted action value into the Critic neural network to obtain a noisy evaluation Q value;

[0154] A multi-objective optimization function is constructed based on the noisy evaluation Q value, and a deep deterministic policy gradient algorithm is used to update the parameters of the Critic neural network through back propagation based on the optimization temporal difference error to minimize the optimization temporal difference error. Based on the multi-objective optimization function, the parameters of the Actor neural network are updated through policy gradient ascent to maximize the output value of the multi-objective optimization function; and a soft update coefficient is used to control the smooth update of the updated parameters of the Actor neural network and the Critic neural network to the target Actor neural network and the target Critic neural network.

[0155] Specific implementation of the deep deterministic policy gradient algorithm

[0156] This embodiment provides a reinforcement learning method based on a deep deterministic policy gradient algorithm for training an intelligent agent to make decisions in a continuous action space.

[0157] First, we build the Actor and Critic neural networks. The Actor network receives the state as input and outputs a deterministic action value. Its internal structure consists of a multi-layer feature extraction network. The final action output layer uses a tanh activation function to map the action value to the range -1 to 1. The Critic network receives the state and action as input and outputs a value assessment of the state-action pair, or Q-value. Its internal structure includes a state input layer, an action input layer, a state-action feature fusion layer for fusing state and action features, and a value output layer.

[0158] Next, create an experience replay buffer. This buffer stores historical data about the agent's interactions with the environment, namely, state transition sequences. Each state transition sequence contains the current state, the agent's action, the immediate reward received, and the next state reached. The current state is fed into the Actor network to generate a predicted action value. The current state and this predicted action value are fed into the Critic network to generate a Q-value for the current state-action pair.

[0159] Next, the temporal difference error (TDE) is calculated. TDE measures the difference between the estimated Q-value of the current state-action pair and the expected Q-value. The expected Q-value is calculated by inputting the next state into the target network and combining it with the immediate reward. The target network is a replica of the actor and critic networks, with its parameters updated less frequently than the original networks. Based on the TDE, each state transition sequence data point is assigned an importance weight. The importance weight reflects the importance of the data point to learning.

[0160] Based on importance weights, a batch of training samples is prioritized from the experience replay buffer. The next state of the training sample is input into the target actor network to obtain the target action value. The next state and target action value are input into the target critic network to obtain the target Q value. The target Q value and the immediate reward of the training sample are weighted and summed to obtain the temporal difference target value. The current state and current action of the training sample are input into the critic network to obtain the current Q value. The difference between the temporal difference target value and the current Q value is the temporal difference error used for optimization.

[0161] The current state in the training sample is fed into the Actor network to obtain the training predicted action value. To encourage exploration, Gaussian noise is added to the training predicted action value to obtain the noisy predicted action value. The current state and the noisy predicted action value are fed into the Critic network to obtain the noisy evaluation Q value.

[0162] A multi-objective optimization function is constructed based on noisy Q-value evaluations. Using a deep deterministic policy gradient algorithm, the parameters of the critic network are updated via backpropagation based on the optimization temporal difference error, minimizing the optimization temporal difference error. Simultaneously, the parameters of the actor network are updated via policy gradient ascent based on the multi-objective optimization function, maximizing the output of the multi-objective optimization function. The parameters of the target network are updated via a soft update method, using a soft update coefficient to smoothly update the updated parameters of the actor and critic networks to the target actor and critic networks.

[0163] For example, suppose the current state is a two-dimensional vector [0.5, 0.2], the action value output by the actor network is 0.8, the immediate reward is 1, and the next state is [0.6, 0.3]. Suppose the target Q value output by the target network is 10. The TD target value can be set to 1 + 0.95 * 10 = 10.5 (assuming a discount factor of 0.95). If the current Q value is 9, the TD error used for optimization is 10.5 - 9 = 1.5.

[0164] Beneficial effects

[0165] 1. Improve learning efficiency: By replaying experiences with priority, more important experience samples are used for training, thereby improving learning efficiency.

[0166] 2. Enhanced stability: Soft updating of target network parameters can avoid drastic fluctuations during training and improve the stability of the algorithm.

[0167] 3. Adaptation to continuous action space: The deep deterministic policy gradient algorithm can directly output continuous action values ​​and is suitable for reinforcement learning tasks in continuous action space.

[0168] Figure 2 This is a schematic diagram of the structure of the artificial intelligence enhanced distributed denial of service attack defense system according to an embodiment of the present invention. Figure 2 As shown, the system includes:

[0169] The first unit collects data packets in the network through a network traffic collection module, parses the data packets to obtain basic feature information of the data packets, the basic feature information includes the source IP address, destination IP address, packet size, protocol type and arrival time; extracts features from the basic feature information based on a deep learning feature extraction model to obtain a behavior feature vector of the data packet, the behavior feature vector includes the time series features, traffic pattern features and protocol interaction features of the data packet;

[0170] The second unit is configured to input the behavioral feature vector into a pre-trained multi-layer attention mechanism neural network model. The multi-layer attention mechanism neural network model calculates an attack probability score by performing a weighted analysis on the time series features, traffic pattern features, and protocol interaction features in the behavioral feature vector; classifies data packets according to the attack probability score, and marks data packets with attack probability scores higher than a preset threshold as potential attack traffic; performs cluster analysis on the data packets marked as potential attack traffic, identifies groups of data packets with similar attack features based on the clustering results, and calculates the hazard level of each data packet group;

[0171] The third unit is used to call the corresponding defense module to generate a defense strategy based on the hazard level of each data packet group. The defense strategy includes traffic filtering rules, bandwidth limitation parameters and firewall configuration; the defense strategy is sent to the defense devices in the network for execution, and real-time defense processing is performed on data packets marked as potential attack traffic; based on the defense effect feedback information, the defense strategy is dynamically optimized using a reinforcement learning algorithm. The reinforcement learning algorithm continuously adjusts and optimizes the parameter configuration of the defense strategy through the objective functions of maximizing network service quality and minimizing defense resource consumption.

[0172] According to a third aspect of the embodiments of the present invention,

[0173] An electronic device is provided, comprising:

[0174] processor;

[0175] a memory for storing processor-executable instructions;

[0176] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0177] According to a fourth aspect of the embodiments of the present invention,

[0178] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0179] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An artificial intelligence enhanced distributed denial of service attack defense method, characterized in that: include: The network traffic collection module collects data packets in the network, and parses the data packets to obtain basic feature information of the data packets, wherein the basic feature information includes the source IP address, the destination IP address, the size of the data packet, the protocol type, and the arrival time; based on the deep learning feature extraction model, the basic feature information is extracted to obtain the behavior feature vector of the data packet, wherein the behavior feature vector includes the time series feature, the traffic pattern feature, and the protocol interaction feature of the data packet; The behavior feature vector is input into a pre-trained multi-layer attention mechanism neural network model, and the multi-layer attention mechanism neural network model calculates an attack probability score by weighted analysis of the time series features, traffic pattern features, and protocol interaction features in the behavior feature vector; data packets are classified according to the attack probability score, and data packets with attack probability scores higher than a preset threshold are marked as potential attack traffic; cluster analysis is performed on the data packets marked as potential attack traffic, and data packet groups with similar attack features are identified based on the clustering results, and the hazard level of each data packet group is calculated; According to the hazard level of each data packet group, the corresponding defense module is called to generate a defense strategy, which includes traffic filtering rules, bandwidth limitation parameters and firewall configuration; the defense strategy is sent to the defense devices in the network for execution, and real-time defense processing is performed on the data packets marked as potential attack traffic; Based on the defense effect feedback information, the defense strategy is dynamically optimized using a reinforcement learning algorithm. The reinforcement learning algorithm continuously adjusts and optimizes the parameter configuration of the defense strategy through the objective functions of maximizing network service quality and minimizing defense resource consumption.

2. The method according to claim 1, characterized in that The behavior feature vector is input into a pre-trained multi-layer attention mechanism neural network model. The multi-layer attention mechanism neural network model calculates the attack probability score by weighted analysis of the time series features, traffic pattern features and protocol interaction features in the behavior feature vector: The behavior feature vector is input into a pre-trained multi-layer attention mechanism neural network model, which is pre-trained using a large-scale real network traffic data set and learns the intrinsic correlation between features through self-supervised learning. The multi-layer attention mechanism neural network model pre-processes the time series features in the behavior feature vector using a position-aware encoder, which is composed of a sinusoidal position code and a learnable linear transformation, encodes the traffic pattern features through a densely connected feature transformation network, and maps the protocol interaction features to a fixed-dimensional vector representation using an attention-based sequence encoder to obtain an encoded feature vector. The encoded feature vector is input into a feature importance evaluation module, which calculates the importance score by the similarity between the query vector and the feature vector based on the attention mechanism, and the importance score is input into a feature interaction modeling module, which adopts a cross attention mechanism to realize information interaction between different types of features, and obtains the interaction information between features by calculating the product of the feature vector and the parameter matrix; the interaction information between features is input into a dynamic weight allocation module, and the dynamic weight allocation module adopts a soft attention mechanism to fuse the importance score and the interaction information between features through a learnable parameter matrix, and obtains a comprehensive weight coefficient through a sigmoid activation function; The multi-layer attention mechanism neural network model inputs the comprehensive weight coefficient and the encoded feature vector into a multi-layer perceptron structure for weighted analysis. The multi-layer perceptron structure introduces a residual connection structure and a layer normalization layer, and sets a Dropout regularization unit between adjacent layers. At the same time, a combined loss function of feature reconstruction loss and cross entropy loss is used to optimize the model. Adversarial samples are generated based on the fast gradient symbol method for adversarial training. The robustness of the model is improved by minimizing the classification error of the adversarial samples, and the attack probability score is finally calculated.

3. The method according to claim 1, characterized in that The data packets are classified according to the attack probability scores, and the data packets with attack probability scores higher than a preset threshold are marked as potential attack traffic; cluster analysis is performed on the data packets marked as potential attack traffic, and data packet groups with similar attack characteristics are identified based on the clustering results, and the hazard level of each data packet group is calculated, including: Classify data packets according to the attack probability score, search for the optimal initial threshold on the validation set by using the Bayesian optimization method, divide the validation set into multiple time windows to calculate the true positive rate and the false positive rate, select the threshold that maximizes the F1 score as the preset threshold, update the preset threshold by the exponential moving average method based on the real-time traffic feature distribution of the current data packet to obtain the current threshold, compare the attack probability score of the current data packet with the current threshold, and mark the data packet with an attack probability score higher than the current threshold as potential attack traffic; Extracting the time characteristics consisting of the packet arrival interval and the burst degree, the flow characteristics consisting of the byte rate and the packet size distribution, and the behavior characteristics consisting of the protocol type and the port change mode from the potential attack traffic, performing Z-score standardization on the time characteristics, the flow characteristics, and the behavior characteristics to obtain standardized data, and performing principal component analysis on the standardized data to obtain standardized characteristics after dimensionality reduction; The standardized features are clustered by using a density peak clustering algorithm. The local density and the minimum distance of each data point in the standardized features are calculated for clustering, wherein the local density is calculated by an adaptive cutoff distance, and a density estimation correction term based on K nearest neighbors is introduced. Points with local density values ​​greater than a preset density threshold are selected as cluster centers, and data packets with similar attack characteristics are clustered into the same data packet group. Calculate the internal similarity and external difference of each data packet group, the internal similarity is calculated based on the average cosine similarity of the member points in the data packet group to the cluster center point, and the external difference is measured by the distance between the nearest data packet group; perform a multi-dimensional evaluation on each data packet group based on the internal similarity and the external difference, calculate the average attack probability score of the data packets in each data packet group to obtain the attack intensity, calculate the proportion of the number of data packets to obtain the attack scale, calculate the time span of the data packets to obtain the attack persistence, calculate the variance of the feature vector to obtain the attack complexity, and calculate the matching degree with the historical attack pattern to obtain the attack similarity; The attack intensity, the attack scale, the attack duration, the attack complexity, and the attack similarity are subjected to Min-Max normalization processing to obtain normalized index values, the weight coefficients of each index are determined through a hierarchical analysis method, and the normalized index values ​​and the weight coefficients are weighted to obtain an initial value of the hazard level of each of the data packet groups; a long short-term memory network is used to perform a time series correlation analysis on the attack behavior of each of the data packet groups, and the attack trend is predicted to obtain an attack trend coefficient, and the initial value of the hazard level and the attack trend coefficient are weighted to obtain a final hazard level of each of the data packet groups.

4. The method according to claim 1, characterized in that: According to the hazard level of each data packet group, calling the corresponding defense module to generate a defense strategy, the defense strategy includes traffic filtering rules, bandwidth limitation parameters and firewall configuration; The defense strategy is sent to the defense device in the network for execution, and real-time defense processing is performed on the data packets marked as potential attack traffic, including: Receive a hazard level of a data packet group, divide the hazard level into five interval values ​​of [0-0.2), [0.2-0.4), [0.4-0.6), [0.6-0.8), and [0.8-1.0], select a corresponding defense module based on the hazard level interval value, select a monitoring module to extract traffic features when the hazard level belongs to the interval [0-0.2), select a light defense module to filter traffic when the hazard level belongs to the interval [0.2-0.4), select a moderate defense module to limit bandwidth when the hazard level belongs to the interval [0.6-0.8), select a strict defense module to perform in-depth defense when the hazard level belongs to the interval [0.8-1.0], and select an emergency defense module to isolate traffic when the hazard level belongs to the interval [0.8-1.0]; Extracting five-tuple information from the data packet group to obtain basic features, the five-tuple information including source IP address, destination IP address, source port number, destination port number, and protocol type, calculating the flow size of the data packet group within the detection time window to obtain flow features, and calculating information entropy and flow burstiness based on the basic features and the flow features to obtain statistical features; inputting the basic features, the flow features, and the statistical features into a pre-trained decision tree model, classifying the features based on the decision tree model, and generating a flow filtering rule set including address matching rules, protocol filtering rules, and port control rules; A hierarchical token bucket algorithm is used to control the flow of the data packet group, a flow baseline value is calculated according to the bandwidth usage record of the data packet group in a historical time window, a priority coefficient is determined based on the service type of the data packet group, and the service type includes critical service, ordinary service, and non-core service. A dynamic adjustment coefficient is calculated by monitoring the load state of the current network device, and the flow baseline value, the priority coefficient, and the dynamic adjustment coefficient are weighted to obtain a multi-level bandwidth limitation parameter; Perform application layer inspection on the data packet group using deep packet inspection technology, extract HTTP payload features, DNS query features, and SSL handshake features to obtain application layer attack features, use a session tracking mechanism to record the TCP connection state, connection duration, and data transmission mode of the data packet group to obtain session state information, and generate a firewall configuration including concurrent connection control rules, session duration limit rules, and application layer feature filtering rules based on the application layer attack features and the session state information; The traffic filtering rule set, the multi-level bandwidth restriction parameters, and the firewall configuration are combined to generate a defense strategy, a defense device topology map is constructed by collecting the location information and connection relationship of all defense devices in the network, an optimal delivery path of the defense strategy is calculated based on the Dijkstra shortest path algorithm, and the defense strategy is delivered to the network defense device in a hierarchical delivery method with a priority on core devices, rule compression is achieved by merging overlapping rules, and the execution efficiency of the defense strategy is improved by using a multi-threaded parallel processing mechanism; The execution effect of the defense strategy is monitored in real time, the strategy execution status on each defense device is collected to obtain execution status information, the attack traffic interception rate, strategy response delay, processor occupancy rate, and memory occupancy rate in the execution status information are calculated to obtain execution effect indicators; when the execution effect indicators fail to reach the preset defense targets, the parameter configuration of the defense strategy is dynamically adjusted according to the execution effect indicators, and real-time defense processing of the data packet group marked as potential attack traffic is achieved through the adjusted defense strategy.

5. The method according to claim 1, characterized in that Based on the defense effect feedback information, the defense strategy is dynamically optimized using a reinforcement learning algorithm. The reinforcement learning algorithm continuously adjusts and optimizes the parameter configuration of the defense strategy through the objective function of maximizing network service quality and minimizing defense resource consumption, including: Receive defense effect feedback information of the network device, the defense effect feedback information includes network service quality information and defense resource consumption information, the network service quality information includes service response delay, network throughput, data packet loss rate, and service session interruption times, the defense resource consumption information includes processor occupancy rate, memory usage rate, bandwidth occupancy rate, and the total number of active rules, normalize the network service quality information to obtain a service quality score, and normalize the defense resource consumption information to obtain a resource consumption score; Taking the service quality score as the maximization target and the resource consumption score as the minimization target, constructing a multi-objective optimization function of reinforcement learning, wherein the output value of the multi-objective optimization function is composed of a weighted combination of the service quality score and the resource consumption score, and calculating the time difference error based on the service quality score and the resource consumption score; Based on the defense effect feedback information, the current number of active connections, the size of various types of business traffic, the equipment load status, and the abnormal traffic distribution are extracted to construct an environment state vector, and the environment state vector is input into the Actor neural network. The Actor neural network uses a long short-term memory unit to process the time series characteristics of the environment state vector and outputs a defense action, which includes a bandwidth limitation parameter, a rule priority parameter, a firewall configuration parameter, and a resource allocation parameter; Extract features from the environment state vector through a state feature extraction branch and from the defense action through an action feature extraction branch, respectively, fuse the extracted state features with the action features to obtain state-action features, input the state-action features into a Critic neural network, and the Critic neural network predicts a Q value based on the state-action features; Constructing a priority experience replay pool to store a state transfer sequence, wherein the state transfer sequence includes the environment state vector, the defense action, the output value of the multi-objective optimization function, and the new environment state vector after executing the defense action, calculating the priority weight of the state transfer sequence based on the temporal difference error, and sampling the state transfer sequence according to the priority weight to obtain a training sample; A deep deterministic policy gradient algorithm is used to update the parameters of the Critic neural network based on the training samples to minimize the temporal difference error, and the parameters of the Actor neural network are updated to maximize the output value of the multi-objective optimization function. The parameters of the target network are updated in a soft update manner, and the trained Actor neural network and the Critic neural network are obtained through iterative optimization of the deep deterministic policy gradient algorithm. The trained Actor neural network is deployed to the defense system, and the current environment state vector is constructed based on the defense effect feedback information received in real time. The current environment state vector is input into the Actor neural network to generate a defense action, and the parameter configuration of the defense strategy is adjusted according to the defense action. The network defense effect is improved by continuously optimizing the parameter configuration.

6. The method according to claim 5, characterized in that The deep deterministic policy gradient algorithm is used to update the parameters of the Critic neural network based on the training samples to minimize the temporal difference error, the parameters of the Actor neural network are updated to maximize the output value of the multi-objective optimization function, the parameters of the target network are updated in a soft update manner, and the Actor neural network and the Critic neural network that are trained are obtained through iterative optimization of the deep deterministic policy gradient algorithm, including: Constructing an Actor neural network and a Critic neural network, wherein the Actor neural network includes a state input layer, an action output layer, and a multi-layer feature extraction network connecting the state input layer and the action output layer, wherein the action output layer uses a tanh activation function to map the output to the interval [-1,1] to obtain an action value, and the Critic neural network includes a state input layer, an action input layer, a state-action feature fusion layer, and a value output layer, wherein the action input layer is used to receive the action value output by the Actor neural network; Construct an experience replay buffer pool for storing state transition sequence data, wherein the state transition sequence data includes a current state, a current action, an immediate reward, and a next state, input the current state into the Actor neural network to obtain a predicted action value, and input the current state and the predicted action value into the Critic neural network to obtain an evaluation Q value of the current state-action pair; Calculating a time difference error based on the evaluation Q value, calculating an importance weight of each state transition sequence data according to the time difference error, and performing priority sampling on the state transition sequence data in the experience replay buffer pool based on the importance weight to obtain a training sample; Constructing a target Actor neural network and a target Critic neural network having the same network structure as the Actor neural network and the Critic neural network, inputting the next state in the training sample into the target Actor neural network to obtain a target action value, and inputting the next state and the target action value into the target Critic neural network to obtain a target Q value; The target Q value and the instant reward in the training sample are weighted to obtain a temporal difference target value, the current state and the current action in the training sample are input into the Critic neural network to obtain a current Q value, and the difference between the temporal difference target value and the current Q value is calculated to obtain a temporal difference error for optimization; Inputting the current state in the training sample into the Actor neural network to obtain a predicted action value for training, adding Gaussian noise to the predicted action value for training to obtain a predicted action value with noise, and inputting the current state and the predicted action value with noise into the Critic neural network to obtain a noisy evaluation Q value; A multi-objective optimization function is constructed based on the noisy evaluation Q value, and a deep deterministic policy gradient algorithm is used to update the parameters of the Critic neural network through back propagation based on the optimization temporal difference error to minimize the optimization temporal difference error. Based on the multi-objective optimization function, the parameters of the Actor neural network are updated through policy gradient ascent to maximize the output value of the multi-objective optimization function; and a soft update coefficient is used to control the smooth update of the updated parameters of the Actor neural network and the Critic neural network to the target Actor neural network and the target Critic neural network.

7. An artificial intelligence enhanced distributed denial of service attack defense system, used to implement the method described in any one of claims 1 to 6, characterized in that: include: The first unit collects data packets in the network through a network traffic collection module, and parses the data packets to obtain basic feature information of the data packets, wherein the basic feature information includes source IP address, destination IP address, data packet size, protocol type and arrival time; extracts features from the basic feature information based on a deep learning feature extraction model to obtain a behavior feature vector of the data packet, wherein the behavior feature vector includes time series features, traffic pattern features and protocol interaction features of the data packet; The second unit is used to input the behavior feature vector into a pre-trained multi-layer attention mechanism neural network model, and the multi-layer attention mechanism neural network model calculates an attack probability score by performing weighted analysis on the time series features, traffic pattern features, and protocol interaction features in the behavior feature vector; classifies data packets according to the attack probability score, and marks data packets with attack probability scores higher than a preset threshold as potential attack traffic; performs cluster analysis on data packets marked as potential attack traffic, identifies data packet groups with similar attack features based on the clustering results, and calculates the hazard level of each data packet group; The third unit is used to call the corresponding defense module to generate a defense strategy according to the hazard level of each data packet group, wherein the defense strategy includes traffic filtering rules, bandwidth restriction parameters and firewall configuration; the defense strategy is sent to the defense device in the network for execution, and the data packets marked as potential attack traffic are subjected to real-time defense processing; Based on the defense effect feedback information, the defense strategy is dynamically optimized using a reinforcement learning algorithm. The reinforcement learning algorithm continuously adjusts and optimizes the parameter configuration of the defense strategy through the objective functions of maximizing network service quality and minimizing defense resource consumption.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images