An artificial intelligence-based cloud data anomaly detection and security response system

Abstract

The application discloses a cloud data anomaly detection and security response system based on artificial intelligence, relates to the technical field of data processing, and solves the technical problems that firstly, it is difficult to store and preprocess multi-source heterogeneous data by using an incremental compression algorithm; secondly, it is difficult to fuse a statistical analysis model and a deep learning model, locate an outlier, analyze abnormal semantics in unstructured data, and further effectively predict a potential attack path; thirdly, it is difficult to realize correlation analysis of cross-node anomalies to identify a distributed attack under the premise of guaranteeing data security and traceability; and finally, it is difficult to construct an attack-defense confrontation model to perform security response and evaluate the security response effect. The application processes cloud environment data through an adaptive probe cluster, generates anomaly detection features by fusing multiple models, predicts an attack path, cooperatively detects by using federated learning, responds by using reinforcement learning and digital twinning, and evaluates.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and specifically is a cloud data anomaly detection and security response system based on artificial intelligence. Background Art

[0002] With the rapid development of cloud computing technology, the scale of data in cloud environments has grown exponentially, and the types of data have expanded from traditional structured data to semi-structured and unstructured data. The widespread use of distributed architectures such as virtual machines, containers, and microservices has made data interaction paths increasingly complex, forming a crisscrossing network topology. In this environment, data flows between different nodes and services at extremely high speeds and with extremely high interaction frequency, which also poses potential security risks.

[0003] The existing technology has the following problems: First, it is difficult to use incremental compression algorithms to store and preprocess multi-source heterogeneous data; second, it is difficult to integrate statistical analysis and deep learning models to locate outliers and parse abnormal semantics in unstructured data, making it difficult to effectively predict potential attack paths; then, it is difficult to achieve cross-node anomaly correlation analysis to identify distributed attacks while ensuring data security and traceability; finally, it is difficult to build an attack and defense confrontation model for security response, and it is difficult to evaluate the effectiveness of the security response. Summary of the Invention

[0004] To address the problems in the prior art, the first aspect of the present invention provides an artificial intelligence-based cloud data anomaly detection and security response system, comprising the following modules:

[0005] Perception data acquisition and processing module: This module uses an adaptive probe cluster to capture multi-source heterogeneous data in the cloud environment in real time; it uses an incremental compression algorithm to store and pre-process data;

[0006] Data Analysis Module: This module integrates statistical analysis with deep learning models to locate outliers using the Isolation Forest algorithm. It also uses the BERT model to analyze anomaly semantics in unstructured data, generating multi-dimensional anomaly feature vectors and anomaly detection feature sets. It also uses the anomaly detection feature set in conjunction with a graph neural network model to predict potential attack paths.

[0007] Federated Collaborative Anomaly Detection Module: Based on anomaly detection feature sets, it uses a federated learning architecture combined with differential privacy and blockchain to ensure security and traceability. It also correlates and analyzes cross-node anomalies to identify distributed attacks.

[0008] Security response module: Build deep reinforcement learning and use game theory to construct an attack and defense model for security response; deploy digital twin verification and evaluate the effectiveness of security response through counterfactual reasoning.

[0009] Furthermore, we integrate statistical analysis with deep learning models and locate outliers using the isolation forest algorithm, which includes the following steps:

[0010] The structured data in the preprocessed multi-source heterogeneous data is input into the isolation forest model training. The isolation forest model randomly selects data points to construct a binary tree. The maximum depth of each tree is , where n is the sample size;

[0011] For sample i, calculate the path length from the root node to the leaf node in each isolated tree , when the sample size of the leaf node is greater than 1, a correction term is introduced , the corrected single tree path length is ; Among them, s is the sample size of leaf nodes, is the Euler constant; then the average path length of sample i in the forest is: ;

[0012] The formula for calculating the anomaly score for each data point is: ;

[0013] When the anomaly score is greater than the preset threshold, it is determined to be an outlier.

[0014] Furthermore, the BERT model is used to parse the anomaly semantics in unstructured data and generate a multi-dimensional anomaly feature vector and anomaly detection feature set, including the following steps:

[0015] Segment the unstructured data of multi-source heterogeneous data in cloud environments into semantic units;

[0016] Using the pre-trained BERT model, we input the segmented text fragments to obtain context-aware word embedding vectors and extract semantic features;

[0017] Build entity relationship graphs based on semantic dependency analysis to identify key entities and interaction relationships in logs;

[0018] Calculate the node centrality and edge weight in the graph, combine it with the semantic vector output by BERT, and generate a multi-dimensional feature vector containing entity features, semantic relevance, and anomaly probability;

[0019] The multi-dimensional feature vector is normalized and fused with the outlier features of structured data to form an anomaly detection feature set.

[0020] Furthermore, we use the anomaly detection feature set in conjunction with the graph neural network model to predict potential attack paths, including the following steps:

[0021] Collect anomaly detection feature sets, corresponding attack events and attack path information, as well as historical attack event records and historical data on known attack path patterns;

[0022] Perform time series correlation analysis on outlier features and multi-dimensional anomaly feature vectors in the anomaly detection feature set, sort them by timestamp and divide them into time windows, construct a time series of abnormal events with time decay factors, and vectorize them;

[0023] The time series anomaly feature vector is integrated with the entity relationship graph. The graph neural network nodes are defined as key entities for identification. The initial edge weights are dynamically calculated based on the semantic relevance and anomaly score. The weight value formula is: ;in, is the anomaly score, is the semantic relevance; is the adjustment coefficient;

[0024] The entity feature vectors and structured outlier features within each time window are input into the graph neural network model as node attributes. Through the graph neural network convolutional layer and attention mechanism, the entity temporal collaborative anomaly pattern is learned.

[0025] The temporal interaction sequences of entities in historical attack paths are used as label training models to output the probability distribution and path confidence of the attack path in each time window.

[0026] Furthermore, the federated collaborative anomaly detection module includes the following steps:

[0027] Build a federated learning architecture, set up a central coordination node and distributed cloud nodes, store data locally on each node, and only share model parameters and intermediate gradient results with the central coordination node to protect data privacy;

[0028] Each distributed cloud node trains a local anomaly detection sub-model based on the local anomaly detection feature set. The sub-model uses a lightweight graph neural network to output local anomaly scores and feature importance.

[0029] The central coordination node receives the parameters of each sub-model, aggregates the parameters using the federated averaging algorithm to generate a global collaborative model, and introduces a differential privacy mechanism to process the aggregated parameters.

[0030] The central coordination node sends the global model to each distributed cloud node. Each node performs incremental training based on the latest local anomaly data and dynamically updates the model weights based on the latest local anomaly data.

[0031] The distributed ledger of the blockchain records the model parameter update log, the characteristic hash value and timestamp of each node's anomaly detection results, ensuring that the collaborative process is traceable and tamper-proof;

[0032] Correlation analysis is performed on abnormal events across nodes. When the anomaly score of a single node is lower than the preset isolated anomaly threshold, but multiple nodes have similar anomaly feature vectors within a continuous time window, the global model determines a distributed attack based on temporal features and spatial correlation, and generates a cross-node collaborative anomaly report.

[0033] Furthermore: the similar abnormal feature vector comprises the following steps:

[0034] Calculate the cosine similarity of any two node anomaly feature vectors: ,in , are the abnormal feature vectors of the two nodes respectively, represents the L2 norm of the vector;

[0035] Compute the Euclidean distance of eigenvectors: , where n is the dimension of the feature vector, 、 is the i-th component of the vector;

[0036] The comprehensive matching degree is calculated as: ,in, is the weight coefficient, is the maximum feature distance threshold;

[0037] Setting the threshold ,when When , there are similar abnormal feature vectors.

[0038] Furthermore, we build deep reinforcement learning and adopt game theory to build an attack and defense model for security response, including the following steps:

[0039] Formalize the cloud environment status, including anomaly detection results, attack paths, resource status, attack and defense actions, and benefit matrix, to quantify the costs and benefits of both parties.

[0040] A defense strategy model is built based on the architecture of a reinforcement learning algorithm. The environment state is input and the action distribution is output. The reward function combines the threat level, response cost, and misjudgment loss. The strategy is optimized through adversarial training against simulated attackers.

[0041] Adopting the Stackelberg game framework, combining historical equilibrium solutions with real-time reinforcement learning output, it selects the most robust defense action against the attacker's optimal response.

[0042] Automatically execute targeted measures based on attack types, including single-point, distributed, and potential paths, and achieve automated responses through API docking with the cloud platform.

[0043] Furthermore, digital twin verification is deployed to evaluate the effectiveness of security responses through counterfactual reasoning, including the following steps:

[0044] Build a virtual image of the cloud environment, reproduce the response process, and simulate attacks in multiple scenarios. The response evaluation index formula is calculated as follows: ;

[0045] Among them, C is the attack blocking success rate, A is the system availability, and F is the false positive rate. 、 and is the corresponding weight, The time when the attack occurred, The time it takes to complete the safety response;

[0046] The effectiveness of the strategy is evaluated through counterfactual reasoning, and the evaluation results are fed back to optimize the parameters of the reinforcement learning model to form an iterative closed loop.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention uses an adaptive probe cluster to capture multi-source heterogeneous data in real time, integrates statistical analysis with deep learning models, locates outliers and parses abnormal semantics in unstructured data to form an anomaly detection feature set, and combines it with a graph neural network model to predict potential attack paths, thereby achieving comprehensive perception and accurate anomaly detection of structured and unstructured data in cloud environments.

[0049] This paper constructs a federated collaborative anomaly detection module by combining a federated learning architecture with differential privacy and blockchain technology. While ensuring data privacy, security, and traceability, it effectively correlates and analyzes cross-node anomalies, enhancing the collaboration and security of overall anomaly detection in the cloud environment.

[0050] This invention constructs an attack-defense confrontation model through deep reinforcement learning and game theory, and combines digital twin verification and counterfactual reasoning to evaluate the response effect, thereby realizing the dynamic optimization and precise execution of security response strategies, improving the efficiency and effectiveness of responses to different types of attacks, and forming a closed-loop security mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0053] The technical solutions of the present application will be described clearly and completely below in connection with the embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0054] Please refer to Figure 1 The first aspect embodiment of the present application provides an artificial intelligence-based cloud data anomaly detection and security response system, comprising the following modules:

[0055] The perception data acquisition and processing module: real-time capture of multi-source heterogeneous data in the cloud environment through adaptive probe cluster; incremental compression algorithm is used for storage and preprocessing;

[0056] The data analysis module: fusion of statistical analysis and deep learning model, positioning outliers through the isolated forest algorithm; using BERT model to analyze the abnormal semantics in unstructured data, generating multi-dimensional abnormal feature vector and abnormal detection feature set; using the abnormal detection feature set combined with the graph neural network model to predict the potential attack path;

[0057] The federal collaborative anomaly detection module: based on the anomaly detection feature set, through the federal learning architecture combined with differential privacy and blockchain to guarantee security and traceability, and at the same time, to analyze cross-node anomalies to identify distributed attacks;

[0058] The security response module: construct deep reinforcement learning, and use game theory to construct attack and defense confrontation model for security response; deploy digital twin verification, and evaluate the effect of security response through counterfactual reasoning.

[0059] Specifically, the physical servers, virtual machines, containers and network nodes of the cloud environment deploy probes, support real-time capture of structured data such as system logs, CPU / memory usage, network traffic packets, and unstructured data such as user operation text logs, API call descriptions, and abnormal alarm texts, and the probe can dynamically adjust the sampling frequency according to the data flow, such as increasing the sampling rate to 100% at the peak of traffic flow and reducing it to 50% at the low peak.

[0060] An improved LZW compression algorithm is used to incrementally store collected data. Preprocessing includes data cleaning, format conversion, and normalization. The isolation forest algorithm is used to process structured data, such as packet size and login times in network traffic, and to isolate abnormal data by constructing multiple binary trees. The pre-trained BERT model is used to process text data, extract semantic features, and generate abnormal feature vectors. An entity relationship graph is constructed based on the graph neural network (GNN). Entities include users, processes, files, and IP addresses. By learning the temporal interaction patterns between entities, potential attack paths are predicted. Central coordination nodes and distributed nodes, such as edge computing nodes and sub-cloud platforms, are set up. Each node only shares model parameters and does not leak original data. Data privacy is protected through differential privacy technology, and blockchain is used to record model update logs, abnormal feature hash values, and timestamps to ensure that data cannot be tampered with.

[0061] Correlate abnormal data from distributed nodes to identify distributed attacks. Based on deep reinforcement learning and game theory, simulate attacker strategies and optimize response decisions. Build a virtual mirror of the cloud environment, evaluate response effectiveness through counterfactual reasoning, and feed this into the model for iterative optimization.

[0062] In this embodiment, statistical analysis and deep learning models are integrated to locate outliers using the isolation forest algorithm, including the following steps:

[0063] The structured data in the preprocessed multi-source heterogeneous data is input into the isolation forest model training. The isolation forest model randomly selects data points to construct a binary tree. The maximum depth of each tree is , where n is the sample size;

[0064] For sample i, calculate the path length from the root node to the leaf node in each isolated tree , when the sample size of the leaf node is greater than 1, a correction term is introduced , the corrected single tree path length is ; Among them, s is the sample size of leaf nodes, is the Euler constant; then the average path length of sample i in the forest is: ;

[0065] The formula for calculating the anomaly score for each data point is: ;

[0066] When the anomaly score is greater than the preset threshold, it is determined to be an outlier.

[0067] Specifically, the structured data from the pre-processed multi-source heterogeneous data is input into the isolation forest model training. The isolation forest model randomly selects data points to construct a binary tree. Input the pre-processed structured data, such as the number of logins and CPU usage of a server within 1 hour, with a sample size of n=1000. Randomly select data points to construct 100 binary trees, and the maximum depth of each tree is , when n=1000, d≈10. For each sample i, calculate its original path length from the root node to the leaf node in each tree. When the number of leaf node samples is greater than 1, a correction term is introduced to compensate for the deviation caused by the incomplete isolation of the node. The average path length of sample i is calculated based on the corrected single tree path length, and then the anomaly score of each data point is calculated. The threshold of the anomaly score is set dynamically, and the initial threshold is set to 0.7. Through sliding window statistics, such as the 95% quantile of the most recent 1000 samples, the threshold is dynamically adjusted to adapt to data distribution drift. Samples with anomaly scores > threshold 0.7 are marked as outliers, and their original data, such as user ID, timestamp, and anomaly type such as traffic surge and permission violation, are recorded.

[0068] In this embodiment, the BERT model is used to parse the anomaly semantics in unstructured data to generate a multi-dimensional anomaly feature vector and anomaly detection feature set, including the following steps:

[0069] Segment the unstructured data of multi-source heterogeneous data in cloud environments into semantic units;

[0070] Using the pre-trained BERT model, we input the segmented text fragments to obtain context-aware word embedding vectors and extract semantic features;

[0071] Build entity relationship graphs based on semantic dependency analysis to identify key entities and interaction relationships in logs;

[0072] Calculate the node centrality and edge weight in the graph, combine it with the semantic vector output by BERT, and generate a multi-dimensional feature vector containing entity features, semantic relevance, and anomaly probability;

[0073] The multi-dimensional feature vector is normalized and fused with the outlier features of structured data to form an anomaly detection feature set.

[0074] Specifically, unstructured data is semantically segmented into short sentences.

[0075] For example, "2023-10-01 14:00 user admin entered the wrong password five times in a row and tried to log in to server A", the semantic segmentation is: "user admin entered the wrong password five times in a row" and "tried to log in to server A".

[0076] The segmented text is input into a pre-trained BERT model, and a context-aware word embedding vector with a dimension of 768 is output, capturing abnormal semantics such as "wrong password" and "continuous attempts".

[0077] Key entities such as "user admin", "server A", and "14:00 timestamp" are identified, and entity relationships such as "user admin - login - server A" are determined through semantic dependency analysis to build an entity relationship graph.

[0078] The centrality of nodes in the graph, such as the frequency of "user admin", and the edge weight, such as the abnormality of "login" behavior, are calculated, and a feature vector containing entity types, semantic correlation, and abnormal probability is generated based on BERT semantic vectors. The semantic correlation is quantified based on the semantic interaction between entities, and the abnormal probability is generated based on the abnormal semantic analysis and entity relationship analysis of unstructured data.

[0079] The multi-dimensional feature vector is normalized, combined with the outlier features of structured data, and the outlier scores output by the isolation forest to form an anomaly detection feature set.

[0080] In this embodiment, the anomaly detection feature set is combined with a graph neural network model to predict potential attack paths, including the following steps:

[0081] Collect anomaly detection feature sets, corresponding attack event and attack path information, and historical attack event records and known attack path pattern historical data;

[0082] Perform time series correlation analysis on the outlier features and multi-dimensional anomaly feature vectors in the anomaly detection feature set, sort and divide the time windows according to the timestamp, and construct an anomaly event time series sequence with a time decay factor, and vectorize the processing;

[0083] Fuse the time series anomaly feature vector with the entity relationship graph, define the graph neural network node as the identified key entity, and dynamically calculate the edge initial weight based on the semantic correlation and abnormal score. The weight value formula is: ; Wherein, is the abnormal score, is the semantic correlation; is the adjustment coefficient;

[0084] Input the entity feature vector and structured outlier features in each time window into the graph neural network model as node attributes; learn the entity time series collaborative anomaly pattern through the graph neural network graph convolution layer and attention mechanism;

[0085] The time series interaction sequence of entities in the historical attack path is used as a label to train the model, and the probability distribution and path confidence of the attack path in each time window are output.

[0086] Specifically, historical data is collected: including anomaly detection feature sets, known attack events, and attack path records. Time windows are divided by timestamps, time decay factors are added to anomaly features, and time series anomaly sequences are constructed and vectorized. , is the time decay factor, is the attenuation coefficient, is the time interval between the current time window and the time when the target abnormal event occurs. The decay coefficient can be adjusted according to the attack type and actual application. The time series of abnormal events with a time decay factor is converted into a fixed-length vector. For example, 20-dimensional features are extracted for each window, and 10 windows generate a 200-dimensional vector, where the vector elements are the product of the eigenvalue and the decay factor. GNN nodes are defined as key entities, and edge weights are calculated according to the formula, where the adjustment coefficient is 0.5. Node attributes such as the entity's abnormal feature vector and structured outlier score are input. Through the GNN's graph convolution layer and attention mechanism, the entities corresponding to high-weight edges are focused on to learn the temporal coordination pattern. The model is trained using the entity interaction sequence of the historical attack path as a label, and the probability distribution and path confidence of the attack path in each time window are output.

[0087] In this embodiment, the federated collaborative anomaly detection module includes the following steps:

[0088] Build a federated learning architecture, set up a central coordination node and distributed cloud nodes, store data locally on each node, and only share model parameters and intermediate gradient results with the central coordination node to protect data privacy;

[0089] Each distributed cloud node trains a local anomaly detection sub-model based on the local anomaly detection feature set. The sub-model uses a lightweight graph neural network to output local anomaly scores and feature importance.

[0090] The central coordination node receives the parameters of each sub-model, aggregates the parameters using the federated averaging algorithm to generate a global collaborative model, and introduces a differential privacy mechanism to process the aggregated parameters.

[0091] The central coordination node sends the global model to each distributed cloud node. Each node performs incremental training based on the latest local anomaly data and dynamically updates the model weights based on the latest local anomaly data.

[0092] The distributed ledger of the blockchain records the model parameter update log, the characteristic hash value and timestamp of each node's anomaly detection results, ensuring that the collaborative process is traceable and tamper-proof;

[0093] Correlation analysis is performed on abnormal events across nodes. When the anomaly score of a single node is lower than the preset isolated anomaly threshold, but multiple nodes have similar anomaly feature vectors within a continuous time window, the global model determines a distributed attack based on temporal features and spatial correlation, and generates a cross-node collaborative anomaly report.

[0094] Specifically, the central coordination node, deployed on a secure cloud server, is responsible for aggregating model parameters. Distributed nodes consist of five edge nodes, which store data locally and train sub-models. Each node uses a lightweight GNN to train a sub-model based on a local anomaly feature set, outputting a local anomaly score. The central node uses a federated averaging algorithm, weighting each node's data volume. For example, a node with 30% of the data volume is given a weight of 0.3 to aggregate sub-model parameters. Differential privacy noise is added to generate a global model. The global model is distributed to each node, where it is incrementally trained with the latest anomaly data, dynamically updating model weights. Smart contracts are used to write hashes of model parameters and timestamps of anomaly detection results into the blockchain to ensure traceability. If a single node's anomaly score falls below the threshold of 0.7, but the similarity of anomaly feature vectors for three or more nodes within three consecutive time windows exceeds the threshold of 0.8, a distributed attack is identified.

[0095] In this embodiment, the similar abnormal feature vector includes the following steps:

[0096] Calculate the cosine similarity of any two node anomaly feature vectors: ,in , are the abnormal feature vectors of the two nodes respectively, represents the L2 norm of the vector;

[0097] Compute the Euclidean distance of eigenvectors: , where n is the dimension of the feature vector, 、 is the i-th component of the vector;

[0098] The comprehensive matching degree is calculated as: ,in, is the weight coefficient, is the maximum feature distance threshold;

[0099] Setting the threshold ,when When , there are similar abnormal feature vectors.

[0100] Specifically, the abnormal feature vector to be compared is a multi-dimensional abnormal feature vector output by the distributed cloud node, the vector dimension n is determined by the abnormal detection feature set, and each component is a normalized feature value. The direction consistency is measured by the vector cosine value, wherein the numerator is the inner product of the vector, and the denominator is the product of the L2 norms of the two vectors; the numerical difference weight coefficient W=0.6 is measured by the straight line distance between two points in the vector space, and the cosine similarity is higher than the Euclidean distance, so the vector direction consistency is given priority; the maximum feature distance threshold is 2, which is set according to the maximum distance of the normal feature vector in the historical data, and the Euclidean distance contribution to the matching degree is 0 when the value exceeds the threshold. The minimum matching degree of the distributed attack feature vector in the historical attack case is used to determine the comprehensive matching degree threshold, and the threshold is set to 0.8. When M≥0.8, it is determined that the abnormal feature vectors are similar.

[0101] In the present embodiment, a deep reinforcement learning is constructed, and a game theory is used to construct an attack-defense confrontation model for security response, including the following steps:

[0102] The cloud environment state is formalized to include: abnormal detection result, attack path, resource state, attack-defense action and profit matrix, and the cost and benefit of both parties are quantified;

[0103] A defense strategy model is constructed based on the reinforcement learning algorithm, the environment state is input and the action distribution is output, the reward function combines the threat level, response cost and misjudgment loss, and the strategy is optimized by training against the simulated attacker;

[0104] The Stackelberg game framework is used, the historical equilibrium solution and the real-time reinforcement learning output are combined, and the most robust defense action that optimally responds to the attacker is selected;

[0105] According to the attack type, including single-point, distributed and potential path, the targeted measures are automatically executed, and the API is connected to the cloud platform to realize automatic response.

[0106] Specifically, the cloud environment state is represented as a multidimensional vector S = [R, P, Res, Ad, Aa, U], where R represents the anomaly detection result, such as the anomaly score τ and outlier labels; P represents attack path information, such as the predicted attack path sequence and path confidence; Res represents resource status, such as server CPU usage, memory utilization, and network bandwidth; Ad represents the set of defender actions, such as isolating nodes, enabling firewall rules, sending alerts, and pre-deploying defense policies, represented by discrete integers 0, 1, 2, and 3; Aa represents the set of possible simulated attacker actions, such as continuously attacking the current node, switching attack targets, and pausing the attack, represented by discrete integers 0, 1, and 2; and U represents the payoff matrix, which quantifies the costs and benefits of both attackers and defenders. The payoff matrix quantifies the costs and benefits of both attackers and defenders: the defender's payoff is equal to the reward for a successful block minus the response resource consumption minus the loss from misjudgment; the attacker's payoff is equal to the reward for a successful attack minus the attack cost minus the loss from being blocked.

[0107] The defense strategy model is constructed using the PPO algorithm in deep reinforcement learning. The model inputs the environment state vector and outputs the probability distribution of the defense action. The reward function R(S,Ad,Aa) integrates the threat level, response cost and misjudgment loss: , where: T is the threat level, based on the anomaly score and attack path confidence weighted calculation; C is the response cost, such as the resource consumption of the isolated node and the API call fee; L is the misjudgment loss, which is quantified as [0,1] if the response action is for normal behavior and 0 for correct response); α=0.5,β=0.3, =0.2 is a weight coefficient that can be adjusted based on the actual scenario. A rule base is constructed based on historical attack data to simulate attacker actions. The defense model engages in multiple rounds of confrontation with the simulated attacker. In each round, the defender outputs action Ad based on the current state S, and the attacker outputs a counter-action Aa based on S and Ad. The environment provides feedback on the new state S′ and reward R. The model optimizes the strategy using gradient descent to maximize the cumulative reward. Training is repeated for 10,000 rounds, each consisting of 100 time steps. A Stackelberg game is used, with the defender as the leader and the attacker as the follower. The strategy is adjusted based on the defender's actions. A Nash equilibrium strategy combination is obtained by solving the problem of maximizing the defender's own payoff while also considering the attacker's optimal response to the defense action. High-frequency optimal defense actions are extracted from the past 1,000 games. Real-time reinforcement learning outputs the probability distribution of the action recommended by the reinforcement learning model in the current state. The historical equilibrium solution and the real-time recommended actions are weighted and fused, with a weight of 0.5 each. The action with the highest overall score is selected as the most robust defense action. Attack type and targeted measures. For single-point attacks, such as abnormal logins to a single node, we freeze the account and trace the local logs. We trigger the account freeze command through the cloud platform API and call the log analysis interface to extract the attacking IP. For distributed attacks, we perform traffic cleaning and cross-node coordinated interception. We adjust network routing through the SDN controller API to divert abnormal traffic to the cleaning center, notify associated nodes to activate firewall rules, and configure the cloud platform API to add access whitelists between path nodes, triggering the snapshot backup mechanism. Automated response is achieved through API gateway integration with mainstream cloud platforms such as Alibaba Cloud and Azure, converting selected defense actions into standardized API calls with response latency under 1 second. Response process logs are written to the blockchain in real time, and the ledger is shared with the federated collaboration module to ensure traceability.

[0108] In this embodiment, digital twin verification is deployed to evaluate the security response effect through counterfactual reasoning, including the following steps:

[0109] Build a virtual image of the cloud environment, reproduce the response process and simulate multi-scenario attacks,

[0110] The formula for calculating the response assessment index is: ;

[0111] Among them, C is the attack blocking success rate, A is the system availability, and F is the false positive rate. 、 and is the corresponding weight, is the time when the attack occurs, and t is the time when the security response is completed;

[0112] The effectiveness of the strategy is evaluated through counterfactual reasoning, and the evaluation results are fed back to optimize the parameters of the reinforcement learning model to form an iterative closed loop.

[0113] Specifically, a digital twin platform is used to build a virtual image based on the cloud environment's physical topology, simulating characteristics such as hardware performance and network latency. Using the actual cloud environment's physical topology as a benchmark, for example, including three application servers, one firewall, one relational database, and two virtual machine nodes, a 1:1 virtual image is constructed using the digital twin platform. This includes: hardware simulations: server CPU model, memory capacity, disk I / O rate; firewall rules; database version and table structure; and network simulations: inter-node network latency, bandwidth limitations, and packet loss rate. Resource utilization and process status are synchronized with the physical environment in real time at a frequency of 1 second, ensuring a consistency error of ≤5% between the virtual image and the physical environment. The defensive actions output by the security response module are converted into executable instructions for the virtual image, and the execution of these actions is replicated along the physical environment's timeline.

[0114] The attack scenario library is predefined in the virtual image, including: single-point attacks, such as brute force attacks on the server, simulating 50 login attempts per second; distributed attacks, such as DDoS attacks on the firewall, simulating 100,000 PPS abnormal traffic; potential path attacks, such as injection attacks based on predicted paths, simulating the time sequence of SQL injection statements. The attack blocking success rate is equal to the number of blocked attacks divided by the total number of attacks, ranging from [0,1]. The system availability is equal to the number of nodes that provide normal services during the response period divided by the total number of nodes, ranging from [0,1]. The false positive rate is equal to the number of normal events misjudged as abnormal divided by the total number of normal events, ranging from [0,1]. Adjust the weights according to the security requirements of the cloud environment and actual applications, and set 、 and The corresponding weights are 0.4, 0.3 and 0.3, 、 and The sum is 1. Substitute the above parameters into the formula to calculate the response evaluation index. When the response evaluation index is higher, the response effect is better. Compare the actual response with the hypothetical scenario, and calculate the difference between the response evaluation index of the original scenario and the response evaluation index of the counter-scenario through counterfactual reasoning. The preset threshold is 0.3, which means that the defensive action is effective, that is, the response effect after executing the action is significantly better than that without executing it; otherwise, it means that the defensive action is invalid or has a negative effect. The threshold of 0.3 can be adjusted according to the actual situation. Feedback the counterfactual results to the PPO model through the API to adjust the policy network parameters. Redeploy the optimized model in the digital twin environment and repeat the above steps until the response evaluation index converges.

[0115] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An artificial intelligence-based cloud data anomaly detection and security response system, characterized by: Includes the following modules: Perception data acquisition and processing module: This module uses an adaptive probe cluster to capture multi-source heterogeneous data in the cloud environment in real time; it uses an incremental compression algorithm to store and pre-process data; Data Analysis Module: This module integrates statistical analysis with deep learning models to locate outliers using the Isolation Forest algorithm. It also uses the BERT model to analyze anomaly semantics in unstructured data, generating multi-dimensional anomaly feature vectors and anomaly detection feature sets. It also uses the anomaly detection feature set in conjunction with a graph neural network model to predict potential attack paths. Federated Collaborative Anomaly Detection Module: Based on anomaly detection feature sets, it uses a federated learning architecture combined with differential privacy and blockchain to ensure security and traceability. It also correlates and analyzes cross-node anomalies to identify distributed attacks. Security response module: Builds deep reinforcement learning and uses game theory to construct an attack and defense model for security response; Deploy digital twin verification to evaluate security response effectiveness through counterfactual reasoning; The BERT model is used to parse the anomaly semantics in unstructured data and generate a multi-dimensional anomaly feature vector and anomaly detection feature set. The following steps are included: Segment the unstructured data of multi-source heterogeneous data in cloud environments into semantic units; Using the pre-trained BERT model, we input the segmented text fragments to obtain context-aware word embedding vectors and extract semantic features; Build entity relationship graphs based on semantic dependency analysis to identify key entities and interaction relationships in logs; Calculate the node centrality and edge weight in the graph, combine it with the semantic vector output by BERT, and generate a multi-dimensional feature vector containing entity features, semantic relevance, and anomaly probability; Normalize the multi-dimensional feature vector and fuse it with the outlier features of the structured data to form an anomaly detection feature set; Using anomaly detection feature sets combined with a graph neural network model to predict potential attack paths involves the following steps: Collect anomaly detection feature sets, corresponding attack events and attack path information, as well as historical attack event records and historical data on known attack path patterns; Perform time series correlation analysis on outlier features and multi-dimensional anomaly feature vectors in the anomaly detection feature set, sort them by timestamp and divide them into time windows, construct a time series of abnormal events with time decay factors, and vectorize them; The time series anomaly feature vector is integrated with the entity relationship graph. The graph neural network nodes are defined as key entities for identification. The initial edge weights are dynamically calculated based on the semantic relevance and anomaly score. The weight value formula is: ;in, is the anomaly score, is the semantic relevance; is the adjustment coefficient; The entity feature vectors and structured outlier features within each time window are input into the graph neural network model as node attributes. Through the graph neural network convolutional layer and attention mechanism, the entity temporal collaborative anomaly pattern is learned. The temporal interaction sequences of entities in historical attack paths are used as label training models to output the probability distribution and path confidence of the attack path in each time window.

2. The artificial intelligence-based cloud data anomaly detection and security response system according to claim 1, characterized in that: Combining statistical analysis with deep learning models, we use the isolation forest algorithm to locate outliers, which includes the following steps: The structured data in the preprocessed multi-source heterogeneous data is input into the isolation forest model training. The isolation forest model randomly selects data points to construct a binary tree. The maximum depth of each tree is , where n is the sample size; For sample i, calculate the path length from the root node to the leaf node in each isolated tree , when the sample size of the leaf node is greater than 1, a correction term is introduced , the corrected single tree path length is + ; Among them, s is the sample size of leaf nodes, is the Euler constant; then the average path length of sample i in the forest is: ; The formula for calculating the anomaly score for each data point is: ; When the anomaly score is greater than the preset threshold, it is determined to be an outlier.

3. The artificial intelligence-based cloud data anomaly detection and security response system according to claim 1, characterized in that: The federated collaborative anomaly detection module includes the following steps: Build a federated learning architecture, set up a central coordination node and distributed cloud nodes, store data locally on each node, and only share model parameters and intermediate gradient results with the central coordination node to protect data privacy; Each distributed cloud node trains a local anomaly detection sub-model based on the local anomaly detection feature set. The sub-model uses a lightweight graph neural network to output the local anomaly score and feature importance. The central coordination node receives the parameters of each sub-model, aggregates the parameters using the federated averaging algorithm to generate a global collaborative model, and introduces a differential privacy mechanism to process the aggregated parameters. The central coordination node sends the global model to each distributed cloud node. Each node performs incremental training based on the latest local anomaly data and dynamically updates the model weights based on the latest local anomaly data. The distributed ledger of the blockchain records the model parameter update log, the characteristic hash value and timestamp of each node's anomaly detection results, ensuring that the collaborative process is traceable and tamper-proof; Correlation analysis is performed on abnormal events across nodes. When the anomaly score of a single node is lower than the preset isolated anomaly threshold, but multiple nodes have similar anomaly feature vectors within a continuous time window, the global model determines a distributed attack based on temporal features and spatial correlation, and generates a cross-node collaborative anomaly report.

4. The artificial intelligence-based cloud data anomaly detection and security response system according to claim 3 is characterized in that: The similar abnormal feature vector comprises the following steps: Calculate the cosine similarity of any two node anomaly feature vectors: ,in are the abnormal feature vectors of the two nodes respectively, represents the L2 norm of the vector; Compute the Euclidean distance of eigenvectors: , where n is the dimension of the feature vector, 、 is the i-th component of the vector; The comprehensive matching degree is calculated as: ,in, is the weight coefficient, is the maximum feature distance threshold; Setting the threshold ,when When , there are similar abnormal feature vectors.

5. The artificial intelligence-based cloud data anomaly detection and security response system according to claim 1, characterized in that: Build deep reinforcement learning and use game theory to build an attack and defense model for security response, including the following steps: Formalize the cloud environment status, including anomaly detection results, attack paths, resource status, attack and defense actions, and benefit matrix, to quantify the costs and benefits of both parties. A defense strategy model is built based on the architecture of a reinforcement learning algorithm. The environment state is input and the action distribution is output. The reward function combines the threat level, response cost, and misjudgment loss. The strategy is optimized through adversarial training against simulated attackers. Adopting the Stackelberg game framework, combining historical equilibrium solutions with real-time reinforcement learning output, it selects the most robust defense action against the attacker's optimal response. Automatically execute targeted measures based on attack types, including single-point, distributed, and potential paths, and achieve automated responses through API docking with the cloud platform.

6. The artificial intelligence-based cloud data anomaly detection and security response system according to claim 5, characterized in that: Deploy digital twin verification and evaluate security response effectiveness through counterfactual reasoning, including the following steps: Build a virtual image of the cloud environment, reproduce the response process and simulate multi-scenario attacks, The formula for calculating the response assessment index is: ; Among them, C is the attack blocking success rate, A is the system availability, and F is the false positive rate. 、 and is the corresponding weight, is the time when the attack occurs, and t is the time when the security response is completed; The effectiveness of the strategy is evaluated through counterfactual reasoning, and the evaluation results are fed back to optimize the parameters of the reinforcement learning model to form an iterative closed loop.

Images