Data security event real-time monitoring method and system

Abstract

The invention relates to the field of internet attack detection, in particular to a data security event real-time monitoring method and system, and the method comprises data collection, multi-modal fusion, threat detection, causal reasoning, dynamic response and feedback optimization. Compared with the traditional security analysis which depends on isolated data dimension or simple rule association, is difficult to capture a cross-data-source complex attack mode, and is faced with the limitations of low calculation efficiency, slow link traceability, storage access bottleneck and the like in mass data association analysis, the scheme integrates multi-source heterogeneous data into a dynamic association network through graph structure modeling, so that the security analysis efficiency is improved. Hidden association and behavior patterns among users, equipment and IPs are deeply mined by utilizing a GNN framework, the suspicious degree among entities can be accurately quantified, and hidden attack chains can be identified; and meanwhile, a hybrid storage architecture and a query optimization technology are adopted, so that a security analyst can backtrack a complex attack path in a second level while breaking through the bottleneck of large-scale graph data access performance, and the threat hunting efficiency and the high-level attack traceability are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of Internet attack detection, and in particular to a method and system for real-time monitoring of data security events. Background Art

[0002] In the current digital context of increasingly covert and complex internet attacks, traditional security monitoring technologies have a significant technological gap in multi-source heterogeneous data fusion and attack pattern recognition. Existing technology systems face three core bottlenecks in responding to internet attacks.

[0003] First, the phenomenon of multi-source data islands leads to the fragmentation of attack characteristics. The cross-dimensional entity relationships involved in Internet attacks (such as user identity, device fingerprints, and IP tracks) are difficult to effectively connect. Attackers use legitimate credentials to carry out lateral penetration. Their key digital footprints such as login authentication logs, encrypted tunnel traffic, and malicious process behavior are scattered in identity management systems, network traffic probes, and terminal protection platforms, forming an "Internet attack characteristic fog."

[0004] Secondly, the real-time correlation analysis capabilities of massive data are insufficient. Existing relational databases have weak capabilities in modeling dynamic entity relationships that frequently appear in Internet attacks. When graph databases deal with APT attack path mining at the billion-node scale, the time complexity of their graph traversal algorithms increases exponentially, resulting in inefficient cross-network attack chain tracing.

[0005] Furthermore, detection mechanisms lack the adaptability to counter the evolving nature of internet attacks. Rule-based detection solutions fail to capture new internet attack variants, such as zero-day exploits. Supervised machine learning models suffer from data lags when addressing the polymorphic nature of ransomware evasion. Unsupervised detection techniques generate high false positive rates due to the overlapping characteristics of internet attacks and normal business behavior. More critically, existing systems lack the ability to dynamically reason about the causal chain of internet attacks, making it impossible to accurately distinguish between DDoS attacks and sudden traffic spikes in complex business scenarios. This results in automated response mechanisms frequently triggering erroneous blocking operations. These technical shortcomings directly extend the window for defending against internet attacks.

[0006] Therefore, there is an urgent need for a new monitoring system that can break through data silos, realize intelligent correlation analysis, and have real-time response and self-evolution capabilities to cope with the increasingly severe data security challenges. Summary of the Invention

[0007] In order to overcome the problems raised in the above background technology, the present invention proposes a real-time monitoring method and system for data security events.

[0008] The technical solution of the present invention is: a method for real-time monitoring of data security events, comprising the following steps: S11: Data collection, which collects data in real time from multiple data sources including logs, traffic, endpoints, and threat intelligence, and initially filters out noise through edge computing; S12: Multimodal fusion, using graph neural networks to model user-device-IP associations and identify hidden attack chains across data sources; S13: Threat detection, combining rule engines, small sample learning models, and unsupervised algorithms to achieve multi-dimensional detection; S14: Causal reasoning, using the DoWhy framework to verify the causal relationship of abnormal events and locate the root cause; S15: Dynamic response, which generates response suggestions based on LLM parsing context and automatically executes response actions including blocking and isolation through SOAR; S16: Feedback optimization, using online learning and federated learning to update the model, combined with Q-Learning to optimize the SOAR strategy, forming a closed-loop iteration.

[0009] As a preferred approach, when collecting data in real time from multiple data sources including logs, traffic, endpoints, and threat intelligence, and initially filtering out noise through edge computing, specifically: S21: Multi-source data collection, real-time data collection from multiple data sources including logs, traffic, endpoints, and threat intelligence; S22: Edge lightweight processing uses TensorFlow Lite to run a lightweight anomaly detection model on the edge computing terminal, discarding more than 95% of normal data, extracting key fields from the retained data, converting it into a unified format, and then labeling the data according to its risk level, giving priority to transmitting high-priority data.

[0010] As a preferred method, when using graph neural networks to model user-device-IP associations and identify covert attack chains across data sources, the following are specifically included: S31: Graph structure modeling, receiving pre-processed structured data from the edge computing terminal and using the structured data to build a graph structure; S32: Graph neural network analysis: import the established graph structure into the GNN framework, use the GNN framework to analyze and evaluate the suspicious degree of association between entities; S33: Storage and query optimization: raw data is stored in a database that supports fast retrieval, graph data is stored in graph data, and cross-entity attack links are directly searched through a visual query language. At the same time, high-frequency query paths are pre-calculated and cached, and large-scale graph data is stored in shards.

[0011] Preferably, when receiving the pre-processed structured data from the edge computing terminal and establishing a graph structure using the structured data, the method specifically includes: S41: Entity definition and extraction, identifying core entities from multi-source data, extracting and normalizing them through rules; S42: Relationship construction and edge definition: establish relationships between entities based on behavioral logic and assign edge types and interaction weights; S43: Attribute addition and context supplementation, adding static and dynamic attributes to entities and relationships; S44: Data conversion and storage, mapping the graph structure to the graph database and creating an index.

[0012] Preferably, when importing the established graph structure into the GNN framework and using the GNN framework to analyze and evaluate the suspicious degree of the association between entities, the following steps are specifically included: S51: Node feature generation, converting entities in the graph into numerical vectors to capture their behavior patterns and associations; S52: Abnormal subgraph detection, using community partitioning algorithms to locate subnetworks with high interaction density and mark potential attack groups; S53: Association risk assessment, integrating similarity, temporal patterns, and subgraph anomalies, quantifies the suspiciousness of interactions between entities and triggers a graded response.

[0013] As a preferred method, when combining a rule engine, a small sample learning model, and an unsupervised algorithm to implement multi-dimensional detection, the following are specifically included: S61: Threat Detection, which achieves comprehensive threat detection coverage through a multi-engine parallel detection mechanism; S62: Causal root cause location: Through causal relationship analysis, distinguish between attack events and occasional anomalies and locate the root cause.

[0014] Preferably, when achieving comprehensive threat detection coverage through a multi-engine parallel detection mechanism, the detection mechanisms employed include: A11: Rule engine detection, which directly matches attack signatures based on predefined rules and generates an alert when a match is found. A12: Small-sample learning model, which extracts log semantic features through a pre-trained model and quickly adapts to new threats using a small number of samples; A13: Unsupervised anomaly detection uses statistical models to identify indicators that deviate from the baseline, calculates the degree of deviation, and issues an alarm when the threshold is exceeded.

[0015] As a priority, when distinguishing between attack events and accidental anomalies through causal analysis and locating the root cause, specifically include: S71: Hypothesis generation, based on the MITRE ATT&CK framework, lists possible attack paths; S72: Causal verification, which uses counterfactual reasoning and intervention testing to verify causality. Counterfactual reasoning simulates different scenarios to verify the cause of abnormal events, while intervention testing uses statistical methods to quantify the probability of the impact of configuration errors, malware, and other related factors. S73: Output the root cause, integrate the verification results, output the probability distribution, and guide the priority handling of high-probability threats.

[0016] As a preferred approach, when generating response suggestions based on LLM parsing context and automatically executing response actions including blocking and isolation through SOAR, the following are specifically included: S81: Context-aware decision-making, generating alert natural language summaries through LLM based on threat confidence, root causes, and business priorities; S82: Dynamic script generation, which combines the results of context-aware decision-making with the reinforcement learning strategy library and real-time scenarios to dynamically generate the optimal action chain; S83: Automated execution, calling APIs based on the generated optimal action chain to execute response actions including firewall blocking, EDR forensics, and Ansible remediation; S84: Feedback tuning: optimize the script strategy library and detection model based on the processing results and manual corrections.

[0017] A real-time data security incident monitoring system, comprising: Data acquisition module, which collects data from multiple sources in real time, filters out noise through edge computing terminals, and prioritizes high-risk data; A multimodal fusion module, which is used to model user-device-IP associations through graph neural networks and identify covert attack chains across data sources; Threat detection module, which combines rule engines, small sample learning, and unsupervised detection to cover both known attacks and unknown anomalies; The causal reasoning module is used to verify the causal relationship of abnormal events and locate the root cause using the DoWhy framework; The dynamic response module is used to generate response suggestions based on LLM and automatically perform blocking, isolation and other actions through SOAR to achieve a response within seconds; Feedback optimization module, used to update models through online learning and federated learning, and to optimize SOAR strategies through reinforcement learning, forming a closed-loop self-evolution; The storage and query module is used to store raw data and graph data in layers, supporting fast retrieval and complex associated queries.

[0018] The system management module is used to manage data source configuration, user permissions and system health monitoring to ensure stable operation.

[0019] Beneficial effects of the present invention: 1. Compared with traditional Internet attack detection technologies that rely on isolated data dimensions or simple rule associations, have difficulty capturing complex attack patterns across data sources, and face limitations such as low computational efficiency, slow link tracing, and storage access bottlenecks in massive data association analysis, this solution integrates multi-source heterogeneous data into a dynamic association network through graph structure modeling. Utilizing the GNN framework, it deeply mines implicit associations and behavioral patterns between users, devices, and IP addresses, accurately quantifying the degree of suspicion between entities and identifying hidden attack chains. Furthermore, it adopts a hybrid storage architecture and query optimization technology, using a relational database to ensure the integrity and rapid retrieval of raw data, while using a graph database to achieve visual tracking and semantic query of attack links. Leveraging pre-computed caching and sharded storage strategies, this solution overcomes the performance bottleneck of large-scale graph data access while enabling security analysts to retrace complex attack paths in seconds. This creates a synergistic and efficient system of "full data retention - semantic graph construction - intelligent path search," significantly improving threat hunting efficiency and advanced attack tracing capabilities. 2. Compared with existing Internet attack detection technologies that mostly rely on a single detection engine (such as using only a rule engine that is easily bypassed by new attacks, or using only an unsupervised learning model that lacks interpretability and poor generalization in small sample scenarios), there are shortcomings such as a single detection dimension, delayed response to new threats, high false alarm rate of abnormal events, and difficulty in locating the root cause. This solution adopts a technical architecture that integrates multi-engine collaborative detection and causal reasoning: in the threat detection stage, the rule engine is used to quickly capture known threats based on known features, the small sample learning model uses pre-trained semantic understanding capabilities combined with a small number of samples to quickly adapt to new attacks, and the unsupervised algorithm automatically mines unknown abnormal behaviors. The three engines run in parallel to achieve comprehensive coverage of known and unknown threats, significantly improving the detection breadth and new threat response speed; in the root cause location stage, based on MITRE The ATT&CK framework constructs a hypothesis space for attack paths, simulates different scenarios through counterfactual reasoning to verify anomaly causality, and uses intervention testing to quantify the probability of each factor's impact. Ultimately, it outputs a threat probability distribution map. This overcomes the technical limitations of traditional correlation analysis, which can only locate surface anomalies. It enables security teams to precisely focus on high-probability attack chains, forming an intelligent closed-loop system of "multimodal detection - causal chain tracing - risk prioritization." This reduces false alarm rates while significantly shortening the threat response chain, significantly improving security operations efficiency and defense depth. 3. Compared with existing Internet attack detection technologies, which generally directly transmit massive amounts of raw data without filtering to the central system for centralized processing, this solution has disadvantages such as high transmission bandwidth usage, high central computing pressure, low-value data interfering with detection efficiency, and delayed response to high-risk events. This solution adopts a technical path of multi-source data collection combined with edge lightweight processing. By deploying the TensorFlow Lite lightweight anomaly detection model at the edge to locally filter real-time data, it can not only use edge computing capabilities to immediately discard more than 95% of normal data to free up bandwidth resources, but also reduce transmission load by extracting key fields and standardizing formats. At the same time, the intelligent labeling mechanism based on risk level can prioritize the transmission of high-priority data. While reducing the processing pressure of the central system, it significantly improves the timeliness of high-threat event identification and system resource utilization, forming a collaborative optimization architecture of "edge filtering-intelligent classification-precise transmission". BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Shown is a schematic diagram of the workflow of the real-time monitoring method for data security incidents of the present invention; Figure 2 What is shown is a schematic diagram of the structure of the real-time monitoring system for data security incidents of the present invention. DETAILED DESCRIPTION

[0021] The present invention will be further described below with reference to the accompanying drawings and examples.

[0022] See also Figure 1-Figure 2 The present invention provides an embodiment: a method for real-time monitoring of data security events, comprising the following steps: S11: Data collection, which collects data in real time from multiple data sources including logs, traffic, endpoints, and threat intelligence, and initially filters out noise through edge computing; S12: Multimodal fusion, using graph neural networks to model user-device-IP associations and identify hidden attack chains across data sources; S13: Threat detection, combining rule engines, small sample learning models, and unsupervised algorithms to achieve multi-dimensional detection; S14: Causal reasoning, using the DoWhy framework to verify the causal relationship of abnormal events and locate the root cause; S15: Dynamic response, which generates response suggestions based on LLM parsing context and automatically executes response actions including blocking and isolation through SOAR; S16: Feedback optimization, using online learning and federated learning to update the model, combined with Q-Learning to optimize the SOAR strategy, forming a closed-loop iteration.

[0023] As described above, the present invention realizes multi-modal fusion and precise identification of hidden attack chains through real-time multi-source data collection and edge computing noise reduction, combined with graph neural networks, and uses rule engines, small sample learning and unsupervised algorithms to detect threats in multiple dimensions. It also uses the DoWhy framework to verify abnormal causality to locate the root cause, and then realizes dynamic response and automatic disposal based on LLM and SOAR. Finally, a closed-loop iteration is formed through online learning, federated learning update model and Q-Learning optimization strategy, which can significantly improve the comprehensiveness and accuracy of threat detection, quickly locate and efficiently handle security incidents, and continuously optimize monitoring and response capabilities, effectively enhancing the level of data security protection.

[0024] As a preferred approach, when collecting data in real time from multiple data sources including logs, traffic, endpoints, and threat intelligence, and initially filtering out noise through edge computing, specifically: S21: Multi-source data collection, real-time data collection from multiple data sources including logs, traffic, endpoints, and threat intelligence; S22: Edge lightweight processing uses TensorFlow Lite to run a lightweight anomaly detection model on the edge computing terminal, discarding more than 95% of normal data, extracting key fields from the retained data, converting it into a unified format, and then labeling the data according to its risk level, giving priority to transmitting high-priority data.

[0025] As described above, compared with the existing technology, which generally directly transmits massive amounts of raw data in full without screening to the central system for centralized processing, the present invention has the disadvantages of high transmission bandwidth occupancy, high central computing pressure, low-value data interfering with detection efficiency, and high-risk event response delay. This solution adopts a technical path of multi-source data collection combined with edge lightweight processing. By deploying the TensorFlow Lite lightweight anomaly detection model at the edge to perform local filtering of real-time data, it can not only use the edge computing capability to immediately discard more than 95% of normal data to release bandwidth resources, but also reduce the transmission load through key field extraction and format standardization. At the same time, the intelligent labeling mechanism based on risk level can prioritize the transmission of high-priority data, while reducing the processing pressure of the central system, significantly improving the timeliness of high-threat event identification and system resource utilization, forming a collaborative optimization architecture of "edge filtering-intelligent classification-precise transmission".

[0026] As a preferred method, when using graph neural networks to model user-device-IP associations and identify covert attack chains across data sources, the following are specifically included: S31: Graph structure modeling, receiving pre-processed structured data from the edge computing terminal and using the structured data to build a graph structure; S32: Graph neural network analysis: import the established graph structure into the GNN framework, use the GNN framework to analyze and evaluate the suspicious degree of association between entities; S33: Storage and query optimization: raw data is stored in a database that supports fast retrieval, graph data is stored in graph data, and cross-entity attack links are directly searched through a visual query language. At the same time, high-frequency query paths are pre-calculated and cached, and large-scale graph data is stored in shards.

[0027] Preferably, when receiving the pre-processed structured data from the edge computing terminal and establishing a graph structure using the structured data, the method specifically includes: S41: Entity definition and extraction, identifying core entities from multi-source data, extracting and normalizing them through rules; S42: Relationship construction and edge definition: establish relationships between entities based on behavioral logic and assign edge types and interaction weights; S43: Attribute addition and context supplementation, adding static and dynamic attributes to entities and relationships; S44: Data conversion and storage, mapping the graph structure to the graph database and creating an index.

[0028] As mentioned above, compared with traditional security analysis that relies on isolated data dimensions or simple rule associations, has difficulty capturing complex attack patterns across data sources, and faces limitations such as low computational efficiency, slow link tracing, and storage access bottlenecks in massive data association analysis, this solution integrates multi-source heterogeneous data into a dynamic association network through graph structure modeling, and uses the GNN framework to deeply mine the implicit associations and behavioral patterns between users, devices, and IP addresses. It can accurately quantify the degree of suspicion between entities and identify hidden attack chains. At the same time, it adopts a hybrid storage architecture and query optimization technology, uses a relational database to ensure the integrity and rapid retrieval of original data, and uses a graph database to achieve visual tracking and semantic query of attack links. With the help of pre-computed caching and sharding storage strategies, while breaking through the performance bottleneck of large-scale graph data access, it enables security analysts to retrace complex attack paths in seconds, forming a synergistic and efficient system of "full data retention-semantic graph construction-intelligent path search", significantly improving threat hunting efficiency and advanced attack tracing capabilities.

[0029] Preferably, when importing the established graph structure into the GNN framework and using the GNN framework to analyze and evaluate the suspicious degree of the association between entities, the following steps are specifically included: S51: Node feature generation, converting entities in the graph into numerical vectors to capture their behavior patterns and associations; S52: Abnormal subgraph detection, using community partitioning algorithms to locate subnetworks with high interaction density and mark potential attack groups; S53: Association risk assessment, integrating similarity, temporal patterns, and subgraph anomalies, quantifies the suspiciousness of interactions between entities and triggers a graded response.

[0030] As mentioned above, by constructing a three-layer progressive attack chain analysis architecture of "feature encoding-subgraph mining-risk quantification", triple technological breakthroughs are achieved in the core links of GNN graph analysis: in the node feature generation stage, multimodal behavior vectorization technology is used to encode heterogeneous features such as operation logs, communication modes, time series, etc. of entities such as users / devices / IPs into high-dimensional semantic vectors, which not only retains the spatiotemporal correlation of the original data but also realizes the computability of machine learning, providing a unified semantic basis for subsequent in-depth analysis; in the abnormal subgraph detection stage, dynamic community partitioning algorithms (such as LPA / Infomap variants) are used to automatically identify highly cohesive subnetworks, and hidden groups such as lateral mobile attack groups and botnet clusters are mined through topological structure clustering. It conceals threat carriers and breaks through the technical bottleneck that traditional rule matching cannot capture group anomalies; in the correlation risk assessment stage, it innovatively integrates three dimensions: behavioral similarity (cosine / Jaccard), time periodicity (autocorrelation / Fourier transform), and subgraph anomaly (modularity / clustering coefficient) to build a dynamic weighted risk scoring model, which can accurately identify the subtle differences between normal business collaboration and malicious collaboration, and support graded response strategies such as alarm upgrades, traffic mirroring, and terminal isolation triggered by risk levels. Ultimately, it forms an intelligent detection closed loop from "atomic behavior coding" to "group behavior modeling" to "dynamic risk grading", significantly improving the early identification capabilities and response efficiency of advanced threats such as APT attacks and supply chain penetration.

[0031] As a preferred method, when combining a rule engine, a small sample learning model, and an unsupervised algorithm to implement multi-dimensional detection, the following are specifically included: S61: Threat Detection, which achieves comprehensive threat detection coverage through a multi-engine parallel detection mechanism; S62: Causal root cause location: Through causal relationship analysis, distinguish between attack events and occasional anomalies and locate the root cause.

[0032] Preferably, when achieving comprehensive threat detection coverage through a multi-engine parallel detection mechanism, the detection mechanisms employed include: A11: Rule engine detection, which directly matches attack signatures based on predefined rules and generates an alert when a match is found. A12: Small-sample learning model, which extracts log semantic features through a pre-trained model and quickly adapts to new threats using a small number of samples; A13: Unsupervised anomaly detection uses statistical models to identify indicators that deviate from the baseline, calculates the degree of deviation, and issues an alarm when the threshold is exceeded.

[0033] As a priority, when distinguishing between attack events and accidental anomalies through causal analysis and locating the root cause, specifically include: S71: Hypothesis generation, based on the MITRE ATT&CK framework, lists possible attack paths; S72: Causal verification, which uses counterfactual reasoning and intervention testing to verify causality. Counterfactual reasoning simulates different scenarios to verify the cause of abnormal events, while intervention testing uses statistical methods to quantify the probability of the impact of configuration errors, malware, and other related factors. S73: Output the root cause, integrate the verification results, output the probability distribution, and guide the priority handling of high-probability threats.

[0034] As mentioned above, compared with existing technologies that rely on a single detection engine (such as using only a rule engine that is easily bypassed by new attacks, or using only an unsupervised learning model that lacks interpretability and has poor generalization in small sample scenarios), there are shortcomings such as a single detection dimension, delayed response to new threats, high false alarm rate of abnormal events, and difficulty in locating the root cause. This solution adopts a technical architecture that integrates multi-engine collaborative detection and causal reasoning: in the threat detection stage, the rule engine is used to quickly capture known threats based on known features, the small sample learning model uses pre-trained semantic understanding capabilities combined with a small number of samples to quickly adapt to new attacks, and the unsupervised algorithm automatically mines unknown abnormal behaviors. The three engines run in parallel to achieve comprehensive coverage of known and unknown threats, significantly improving the detection breadth and new threat response speed; in the root cause location stage, based on MITRE The ATT&CK framework constructs a hypothesis space for attack paths, simulates different scenarios through counterfactual reasoning to verify anomaly causality, and uses intervention testing to quantify the probability of influence of various factors. It ultimately outputs a threat probability distribution map, breaking through the technical limitations of traditional correlation analysis that can only locate surface anomalies. It enables security teams to accurately focus on high-probability attack chains and form an intelligent closed-loop system of "multimodal detection-causal chain tracing-risk priority sorting". While reducing the false alarm rate, it greatly shortens the threat handling response chain and significantly improves security operation efficiency and defense depth.

[0035] As a preferred approach, when generating response suggestions based on LLM parsing context and automatically executing response actions including blocking and isolation through SOAR, the following are specifically included: S81: Context-aware decision-making, generating alert natural language summaries through LLM based on threat confidence, root causes, and business priorities; S82: Dynamic script generation, which combines the results of context-aware decision-making with the reinforcement learning strategy library and real-time scenarios to dynamically generate the optimal action chain; S83: Automated execution, calling APIs based on the generated optimal action chain to execute response actions including firewall blocking, EDR forensics, and Ansible remediation; S84: Feedback tuning: optimize the script strategy library and detection model based on the processing results and manual corrections.

[0036] As described above, the present invention achieves a four-fold capability leap in the security incident handling link by constructing a closed-loop response system of "intelligent decision-making-dynamic orchestration-automatic handling-continuous optimization": the context-aware decision-making module based on LLM generates a highly readable natural language alarm summary by integrating multi-dimensional information such as threat confidence, root cause analysis results and business priority, providing the security team with an intuitive panoramic view of the event and effectively shortening the analysis time; the dynamic script generation module combines the reinforcement learning strategy library with real-time scenario characteristics (such as attack stage, asset value, business impact), and dynamically deduces the optimal response action chain through the strategy network, breaking through the traditional static script without The system can adapt to the limitations of complex and ever-changing attack scenarios; the automated execution module seamlessly integrates security tool chains such as firewalls, EDR, and Ansible to achieve full-process automated orchestration from threat blocking, host isolation to system repair, and complete attack chain cutting and damaged asset recovery within minutes; the feedback tuning mechanism captures disposal results and manually corrected data, and continuously optimizes the policy library weights and detection model parameters through online learning, forming an intelligent iterative closed loop of "practice-feedback-evolution", significantly improving the response speed, disposal accuracy and defense resilience of security operations, and effectively reducing the duration and degree of damage to key businesses caused by advanced persistent threats (APTs).

[0037] A real-time data security incident monitoring system, comprising: Data acquisition module, which collects data from multiple sources in real time, filters out noise through edge computing terminals, and prioritizes high-risk data; A multimodal fusion module, which is used to model user-device-IP associations through graph neural networks and identify covert attack chains across data sources; Threat detection module, which combines rule engines, small sample learning, and unsupervised detection to cover both known attacks and unknown anomalies; The causal reasoning module is used to verify the causal relationship of abnormal events and locate the root cause using the DoWhy framework; The dynamic response module is used to generate response suggestions based on LLM and automatically perform blocking, isolation and other actions through SOAR to achieve a response within seconds; Feedback optimization module, used to update models through online learning and federated learning, and to optimize SOAR strategies through reinforcement learning, forming a closed-loop self-evolution; The storage and query module is used to store raw data and graph data in layers, supporting fast retrieval and complex associated queries.

[0038] The system management module is used to manage data source configuration, user permissions and system health monitoring to ensure stable operation.

[0039] As described above, the present invention realizes a closed-loop capability of the entire link through a modular collaborative architecture: the data acquisition module integrates edge computing to realize noise filtering and risk-prioritized transmission of multi-source heterogeneous data, significantly reducing the central processing load and improving the timeliness of high-threat data; the multimodal fusion module uses graph neural networks to construct a user-device-IP dynamic association map, accurately capturing cross-domain hidden attack chains, breaking through the limitations of traditional isolated analysis; the threat detection module forms a dual protection system of accurate capture of known threats and active discovery of unknown anomalies through the collaboration of rule engines, small sample learning and unsupervised algorithms; the causal reasoning module realizes the cause and effect of abnormal events based on the DoWhy framework The automated traceability of the fruit chain provides an explainable chain of evidence for root cause location; the dynamic response module integrates LLM contextual understanding and SOAR automated orchestration capabilities to achieve a closed loop from alarm analysis to response and disposal in seconds, significantly shortening MTTR; the feedback optimization module uses online learning, federated learning, and reinforcement learning technologies to continuously iterate the accuracy of the detection model and the effectiveness of the response strategy, forming a self-evolving security immune system; the storage and query module uses a layered storage architecture and graph database technology to ensure the integrity of the original data and the millisecond-level traceability of the attack chain; the system management module provides basic support such as data source configuration, permission management, and health monitoring to ensure stable system operation. Through the complete capability chain of "global data perception-intelligent correlation analysis-dynamic response and disposal-continuous strategy evolution", the system significantly improves the automation, intelligence, and practical level of enterprise security operations, and effectively responds to the continuous penetration risk of advanced threats such as APT attacks and ransomware to core businesses.

[0040] Example 1: Preventing Data Leakage in High-Frequency Trading in the Financial Industry Scenario: A bank's high-frequency trading system was attacked by an APT (Internet PT) attacker. The attacker stole employee credentials via phishing emails, used legitimate identities to access the database, and stole transaction data. Traditional internet attack detection systems were unable to correlate behaviors across logs, traffic, and terminals, resulting in the attack going undetected.

[0041] System implementation steps: 1. Data collection and edge processing: The edge node (transaction server) runs the TensorFlow Lite model, filtering normal operation logs (such as compliant transaction records) in real time, retaining only abnormal behaviors (such as non-working hours logins and high-frequency queries), extracting key fields (user ID, IP address, operation type), marking the risk level (such as non-working hours logins are marked as high-risk), and transmitting them to the central node first.

[0042] 2. Multimodal fusion and threat detection: A graph structure was constructed: user node (trader A), IP node (external IP 10.0.0.1), and device node (database server DB01). GNN analysis revealed that after user A logged in from an unfamiliar IP, access frequency to DB01 increased threefold, associated with unusual SQL queries (such as SELECT * FROM trade_history). The rule engine triggered an alert (hitting the "unusually high-frequency query" rule). The small sample model identified this query pattern as 92% similar to historical internet attacks.

[0043] 3. Causal reasoning and root cause location: Hypothesis generation: The attack path is "phishing email → credential theft → data leakage." Counterfactual verification: After simulating the disabling of stolen credentials, DB01 traffic plummeted by 80%, confirming the root cause of the attack to be credential leakage (with an 88% probability). Dynamic response and feedback optimization: LLM generated a summary: "User A logged in from an unfamiliar IP address and frequently accessed transaction data, suggesting credential leakage leading to data theft."

[0044] 4. SOAR Automated Execution: Block IP 10.0.0.1, reset user A's password, isolate DB01, and start data backup.

[0045] 5. Federated Learning Update Model: By sharing attack signatures with other banks (data desensitization), the accuracy of the global model in detecting similar attacks has increased by 15%.

[0046] Implementation Results: Detection Time: Internet attacks, from initiation to blocking, take less than 3 minutes (traditional systems take several hours). False Alarm Rate: Causal reasoning reduces false alarm rates from 25% to 8%. Business Impact: Automated isolation reduces data breaches by 95%, with transaction disruptions lasting less than 10 seconds.

[0047] Example 2: Real-time blocking of ransomware attacks on medical institutions Scenario description: The PACS system (medical image storage system) of a tertiary hospital was attacked by ransomware over the Internet. The Internet attackers exploited a vulnerability to encrypt patient image data and attempted to laterally infiltrate the electronic medical record database.

[0048] System implementation steps: 1. Data collection and edge processing: The terminal device (imaging workstation) runs a lightweight model, filtering normal operations (such as doctors accessing images) and capturing abnormal behavior (such as svchost.exe encrypted files and large numbers of external connection requests). Key fields (process name, file path, external connection IP address) are extracted, marked as "high-risk," and transmitted to the central node.

[0049] 2. Multimodal fusion and threat detection: A graph structure was constructed: process nodes (malicious process P1), file nodes (image file F1), and device nodes (workstation WS01). GNN analysis revealed that process P1 was associated with multiple encrypted files, and that WS01 was linked to the malicious domain malware.cc. Unsupervised detection flagged a 200% surge in CPU usage, and the rules engine matched the malicious domain intelligence.

[0050] 3. Causal reasoning and root cause location: Hypothesis generation: The attack path is "vulnerability exploitation → file encryption → lateral movement." Intervention testing: After closing the vulnerable port, the encryption behavior stops, confirming the vulnerability as the root cause (92% probability).

[0051] 4. Dynamic response and feedback optimization: The LLM generated a summary: "The svchost.exe process on workstation WS01 encrypted image files and connected to malicious domains, suggesting ransomware." SOAR automatically executed: WS01 was isolated, the malicious domains were blocked, and the encrypted files were rolled back to the uninfected version. Online learning update: Incorporating internet attack signatures into the model reduced the detection time for similar ransomware variants from 5 minutes to 30 seconds.

[0052] Implementation effect: Response Speed: From encryption to blocking, it takes only 20 seconds (traditional EDR takes an average of 2 minutes). Data Recovery: 90% of encrypted files are automatically restored via backup scripts, with business interruption time less than 5 minutes. Compliance: Federated learning ensures that patient privacy data does not leave the hospital, complying with HIPAA requirements.

[0053] The embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the scope of knowledge of those skilled in the art without departing from the spirit of the present invention.

Claims

1. A method for real-time monitoring of data security incidents; characterized by: The following steps are involved: S11: Data collection, which collects data in real time from multiple data sources including logs, traffic, endpoints, and threat intelligence, and initially filters out noise through edge computing; S12: Multimodal fusion, using graph neural networks to model user-device-IP associations and identify hidden attack chains across data sources; S13: Threat detection, combining rule engines, small sample learning models, and unsupervised algorithms to achieve multi-dimensional detection; S14: Causal reasoning, using the DoWhy framework to verify the causal relationship of abnormal events and locate the root cause; S15: Dynamic response, which generates response suggestions based on LLM parsing context and automatically executes response actions including blocking and isolation through SOAR; S16: Feedback optimization, using online learning and federated learning to update the model, combined with Q-Learning to optimize the SOAR strategy, forming a closed-loop iteration.

2. A method for real-time monitoring of data security events according to claim 1, characterized in that: When collecting data in real time from multiple data sources including logs, traffic, endpoints, and threat intelligence, and initially filtering out noise through edge computing, the following are performed: S21: Multi-source data collection, real-time data collection from multiple data sources including logs, traffic, endpoints, and threat intelligence; S22: Edge lightweight processing uses TensorFlow Lite to run a lightweight anomaly detection model on the edge computing terminal, discarding more than 95% of normal data, extracting key fields from the retained data, converting it into a unified format, and then labeling the data according to its risk level, giving priority to transmitting high-priority data.

3. A method for real-time monitoring of data security events according to claim 2, characterized in that: When using graph neural networks to model user-device-IP associations and identify covert attack chains across data sources, the following are specifically involved: S31: Graph structure modeling, receiving pre-processed structured data from the edge computing terminal and using the structured data to build a graph structure; S32: Graph neural network analysis: import the established graph structure into the GNN framework, use the GNN framework to analyze and evaluate the suspicious degree of association between entities; S33: Storage and query optimization: raw data is stored in a database that supports fast retrieval, graph data is stored in graph data, and cross-entity attack links are directly searched through a visual query language. At the same time, high-frequency query paths are pre-calculated and cached, and large-scale graph data is stored in shards.

4. A method for real-time monitoring of data security events according to claim 3, characterized in that: When receiving pre-processed structured data from the edge computing terminal and using the structured data to build a graph structure, it specifically includes: S41: Entity definition and extraction, identifying core entities from multi-source data, extracting and normalizing them through rules; S42: Relationship construction and edge definition: establish relationships between entities based on behavioral logic and assign edge types and interaction weights; S43: Attribute addition and context supplementation, adding static and dynamic attributes to entities and relationships; S44: Data conversion and storage, mapping the graph structure to the graph database and creating an index.

5. A method for real-time monitoring of data security events according to claim 4, characterized in that: When importing the established graph structure into the GNN framework and using the GNN framework for analysis, the suspicious degree of association between entities is assessed, specifically including: S51: Node feature generation, converting entities in the graph into numerical vectors to capture their behavior patterns and associations; S52: Abnormal subgraph detection, using community partitioning algorithms to locate subnetworks with high interaction density and mark potential attack groups; S53: Association risk assessment, integrating similarity, temporal patterns, and subgraph anomalies, quantifies the suspiciousness of interactions between entities and triggers a graded response.

6. A method for real-time monitoring of data security events according to claim 5, characterized in that: When combining rule engines, small sample learning models, and unsupervised algorithms to implement multi-dimensional detection, the following are specifically included: S61: Threat Detection, which achieves comprehensive threat detection coverage through a multi-engine parallel detection mechanism; S62: Causal root cause location: Through causal relationship analysis, distinguish between attack events and occasional anomalies and locate the root cause.

7. A method for real-time monitoring of data security events according to claim 6, characterized in that: When achieving comprehensive threat detection coverage through multi-engine parallel detection, the detection mechanisms used include: A11: Rule engine detection, which directly matches attack signatures based on predefined rules and generates an alert when a match is found. A12: Small-sample learning model, which extracts log semantic features through a pre-trained model and quickly adapts to new threats using a small number of samples; A13: Unsupervised anomaly detection uses statistical models to identify indicators that deviate from the baseline, calculates the degree of deviation, and issues an alarm when the threshold is exceeded.

8. A method for real-time monitoring of data security events according to claim 7, characterized in that: When using causal analysis to distinguish between attack incidents and incidental anomalies and locate the root cause, the following are specifically included: S71: Hypothesis generation, based on the MITRE ATT&CK framework, lists possible attack paths; S72: Causal verification, which uses counterfactual reasoning and intervention testing to verify causality. Counterfactual reasoning simulates different scenarios to verify the cause of abnormal events, while intervention testing uses statistical methods to quantify the probability of the impact of configuration errors, malware, and other related factors. S73: Output the root cause, integrate the verification results, output the probability distribution, and guide the priority handling of high-probability threats.

9. A method for real-time monitoring of data security events according to claim 8, characterized in that: When generating response recommendations based on LLM parsing context and automatically executing response actions including blocking and isolation through SOAR, the following are performed: S81: Context-aware decision-making, generating alert natural language summaries through LLM based on threat confidence, root causes, and business priorities; S82: Dynamic script generation, which combines the results of context-aware decision-making with the reinforcement learning strategy library and real-time scenarios to dynamically generate the optimal action chain; S83: Automated execution, calling APIs based on the generated optimal action chain to execute response actions including firewall blocking, EDR forensics, and Ansible remediation; S84: Feedback tuning: optimize the script strategy library and detection model based on the processing results and manual corrections.

10. A real-time data security incident monitoring system, characterized by: include: Data acquisition module, which collects data from multiple sources in real time, filters out noise through edge computing terminals, and prioritizes high-risk data; A multimodal fusion module, which is used to model user-device-IP associations through graph neural networks and identify covert attack chains across data sources; Threat detection module, which combines rule engines, small sample learning, and unsupervised detection to cover both known attacks and unknown anomalies; The causal reasoning module is used to verify the causal relationship of abnormal events and locate the root cause using the DoWhy framework; The dynamic response module is used to generate response suggestions based on LLM and automatically perform blocking, isolation and other actions through SOAR to achieve a response within seconds; The feedback optimization module is used to update the model through online learning and federated learning, and to optimize the SOAR strategy through reinforcement learning to form a closed-loop self-evolution.

Images