Enterprise-side situational awareness method based on AI Agent distributed dynamic data processing

Abstract

This invention discloses an enterprise-side situational awareness method based on distributed dynamic data processing using AI agents. Distributed AI agents collect and preprocess multi-source dynamic data from the enterprise, then break down complex situational awareness tasks into executable sub-tasks and send them to the AI ​​Agent task routing module. The AI ​​Agent task routing module deeply interacts with a large language model to complete sub-tasks such as semantic understanding and risk rating, inference decision-making and rule optimization, knowledge alignment and graph generation. The inference results are returned to the distributed AI Agent and synchronized to a multi-AI Agent collaboration and situational fusion module for distributed data fusion. Finally, through multi-AI Agent collaborative situational awareness analysis, enterprise-side situational awareness results are generated and output, achieving efficient processing and accurate situational awareness of multi-source dynamic data, providing real-time support for enterprise operational decisions.

Description

Technical Field

[0001] This invention pertains to situational awareness technology, specifically an enterprise-side situational awareness method based on distributed dynamic data processing using AI Agents. Background Technology

[0002] As enterprises deepen their digital transformation and integrate with cloud services, their internal network architectures are becoming increasingly complex and highly distributed. Typical application scenarios include large enterprises' cross-regional branch offices and complex distributed interconnected network security environments. In these scenarios, the number of network nodes grows exponentially, and the data flow between nodes is not only massive but also contains a large amount of heterogeneous unstructured information. Traditional centralized situational awareness architectures rely excessively on uniformly transmitting all raw traffic and logs back to a central server for centralized processing. This data flow pattern, when faced with massive bursts of traffic, can cause severe data transmission pressure on the core backbone network. The central node is prone to becoming a performance bottleneck due to computing power overload, thus losing the ability to monitor the situation in real time.

[0003] Existing network security monitoring methods mostly rely on passive sensors, whose functions are limited to blindly collecting and simply transmitting data, lacking the ability to predict and autonomously process data value at the edge. In actual business operations, the inability to perform effective intelligent filtering and feature denoising at the data source results in enterprise networks being filled with a large number of invalid heartbeat signals, redundant normal business data, and background noise. This causes high-value threat features to be submerged in massive amounts of irrelevant information, increasing the difficulty of retrieval for the central analysis engine. Due to the severe spatial and logical disconnect between the data processing stage and the data generation site, the physical latency caused by centralized analysis makes it difficult to cope with high-frequency, instantaneous penetration attacks, failing to meet the stringent requirements of modern enterprises for timely security response.

[0004] Current situational awareness technologies still heavily rely on static signature matching or pre-defined fixed rule bases. This approach proves inadequate when facing dynamically evolving, highly stealthy, and AI-generated variant malware. Traditional methods often struggle to parse the deep semantics of business logic layers and cannot identify slow-moving penetration behaviors that span multiple time periods and lurk across different network nodes. This results in extremely high false positive and false negative rates, making it impossible to build an accurate cybersecurity risk profile from a global perspective and failing to support the precise early warning needs of enterprises for complex and covert security postures.

[0005] In distributed network environments, severe information silos commonly exist among monitoring nodes, lacking effective horizontal collaboration and knowledge alignment mechanisms. When a local node on the enterprise side is attacked or experiences abnormal fluctuations, this localized awareness information cannot be shared semantically among the distributed nodes in real time and efficiently. This prevents other nodes from simultaneously raising their alert levels or taking targeted coordinated defensive measures in a timely manner. This fragmented nature of the defense system allows attackers to exploit the blind spots and time differences between nodes to move laterally, posing a continuous threat to the enterprise's core assets. Summary of the Invention

[0006] This invention proposes an enterprise-side situational awareness method based on distributed dynamic data processing using AI agents.

[0007] The technical solution to achieve the purpose of this invention is: an enterprise-side situational awareness method based on AI Agent distributed dynamic data processing, comprising the following steps:

[0008] Deploy distributed AI agents on designated nodes within the enterprise-side network environment;

[0009] AI Agent collects and preprocesses multi-source dynamic data from the enterprise side, completes initial screening of network traffic, deep tracking of behavior, extraction of metadata features and transformation of semantic objects, and obtains standardized semantic objects and metadata features.

[0010] The pre-processed data at the edge is sent to the AI ​​Agent task routing module. The AI ​​Agent task routing module aggregates, deduplicates, and verifies the format of the data uploaded by each node. Then, it calls the central big language model orchestration engine to complete situational semantic understanding and risk rating, inference decision-making and defense strategy generation, knowledge alignment rule generation, and situation map update. The AI ​​Agent task routing module also issues dynamic task instructions to the corresponding AI Agent to achieve flexible switching of monitoring modes.

[0011] Threat intelligence is encrypted and shared and knowledge is aligned based on a peer-to-peer collaboration protocol, linking isolated alerts generated by multiple AI agents as the same attack entity.

[0012] The central side constructs a global situation map based on the collaborative results of multiple AI agents, and completes risk identification, attack tracing, early warning output and handling suggestions.

[0013] Compared with the prior art, the significant advantages of this invention are:

[0014] This invention breaks through the performance bottleneck of traditional security probes in processing data in user space. Its highlight is that it can dynamically generate new filtering strategies based on the advanced threat characteristics analyzed by eBPF and inject them into the XDP mount point.

[0015] By using XDP technology to build the first high-speed path at the network card driver layer, near-zero latency physical isolation is achieved against high-frequency attacks; simultaneously, eBPF technology is used to capture process context deep within the kernel. Leveraging eBPF's ability to span kernel and user space, high-value call and I / O characteristics are extracted directly without transmitting the original full traffic.

[0016] In this invention, each AI Agent no longer simply captures packets; instead, it uses a built-in lightweight local decision engine to directly transform messy raw traffic and logs into standard triple semantic objects containing IP, URL, and User-Agent. Through computation and packet payload distribution analysis, the AI ​​Agent uploads high-value security intelligence that has been understood and compressed, solving the pain point of centralized processing platforms being overwhelmed by massive amounts of useless intelligence.

[0017] This invention uses a large language model as the core of perception, enabling it to automatically orchestrate defense tasks based on real-time risk ratings. Instead of running fixed strategies, it features automatic switching capabilities between low-power monitoring and full-scale tracking.

[0018] This invention achieves knowledge alignment through a P2P collaboration protocol, enabling AI agents on different nodes to share attack fingerprints and using logical deduction to map scattered, isolated records to the same attack entity. This knowledge alignment technology can accurately identify the attacker's lateral movement trajectory between different nodes.

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0020] Figure 1 This is a flowchart of an enterprise-side situational awareness method based on distributed dynamic data processing using AI Agents.

[0021] Figure 2 This is a flowchart illustrating the interaction logic between the AI ​​Agent and the large model algorithm. Detailed Implementation

[0022] An enterprise-side situational awareness method based on distributed dynamic data processing using AI agents, such as Figure 1 , 2As shown, a distributed AI Agent collects and preprocesses multi-source dynamic data from the enterprise side, then breaks down the complex situational awareness task into executable sub-tasks and sends them to the AI ​​Agent task routing module. The AI ​​Agent task routing module interacts deeply with the large language model to complete sub-tasks such as semantic understanding and risk rating, inference decision-making and rule optimization, knowledge alignment and graph generation. After the inference results are returned to the distributed AI Agent, they are synchronized to the multi-AI Agent collaboration and situational fusion module for distributed data fusion. Finally, through multi-AI Agent collaborative situational awareness analysis, the enterprise-side situational awareness results are generated and output, realizing efficient processing of multi-source dynamic data and accurate situational awareness, providing real-time support for enterprise operational decisions.

[0023] Step 1: Deploy the distributed AI Agent

[0024] The implementation of this invention requires the deployment of a distributed AI Agent at key nodes in the enterprise network environment. These agents are deployed at critical locations such as the edge access layer (enterprise border router egress), core switching layer (core switch mirror ports), and application service layer (external network egress, Web application server access side).

[0025] Specifically, the AI ​​Agent is deployed on an edge acquisition node running a Linux kernel, including a user-space control program and a kernel-space data acquisition module based on XDP and eBPF. Specifically: the XDP program is mounted on the Linux network card driver layer and is used for high-speed initial screening and traffic marking of incoming data packets; the eBPF program is mounted on kernel event points and is used for dynamic tracking of calls, file input / output, and process context; XDP and eBPF share connection status and risk marking information through a kernel-space BPF Map, enabling correlation analysis between network layer traffic characteristics and host-side behavior.

[0026] Step 2: Data Acquisition and Edge Preprocessing

[0027] The AI ​​Agent acquires multi-source dynamic data through a kernel-mode data acquisition module based on XDP / eBPF. The acquired multi-source dynamic data includes: raw network traffic, logs (Syslog), web page access logs, process behavior logs, file input / output logs, call logs, etc. Edge preprocessing is performed on this data, including high-speed traffic initial screening, deep behavior tracking, data interaction and correlation analysis, metadata extraction, and semantic object transformation, resulting in standardized semantic objects and metadata features.

[0028] Specific methods for AI Agents to perform edge preprocessing on collected multi-source dynamic data include:

[0029] (1) XDP initial screening

[0030] The AI ​​Agent mounts a custom XDP program to the Linux network interface card (NIC) driver layer, performing high-speed initial screening of network traffic before packets enter the Linux network protocol stack. For known large-scale, high-frequency Distributed Denial-of-Service (DDoS) attack packets, scanner signatures, and abnormal connection patterns, it uses the Aho-Corasick multi-pattern matching algorithm to identify suspicious traffic and extract connection metadata such as source IP, destination IP, port, protocol type, and timestamp. For detected suspicious connections, the XDP program writes the corresponding connection identifier and risk label into the kernel-mode BPF Map as a basis for subsequent deep behavioral tracking.

[0031] (2) eBPF deep tracing

[0032] For service connections marked as suspicious after initial screening by XDP, the AI ​​Agent activates the eBPF tracing program. The eBPF program is attached to call and kernel event points, and by querying connection identifiers in the BPF Map, it performs in-depth tracking of behaviors associated with suspicious connections. It captures in real-time Syscall calls, file I / O operations, process contexts, and network connection behaviors triggered by the corresponding traffic, thereby enabling correlation analysis between suspicious network layer traffic and host-side behavior.

[0033] Specifically, eBPF deep tracing is based on the Bloom filter algorithm. A Bloom filter is a highly space-efficient random data structure used to check whether an element is in a set. It uses multiple hash functions to map elements to a bitmap. The eBPF program maintains a "high-risk call set" based on the Bloom filter in kernel space. When traffic triggers a Syscall, the Agent uses the Bloom filter to achieve the desired result in constant time O(log n). (K) The system determines whether the behavior matches the blacklist, thereby capturing the process context.

[0034] Among them O (K) O in (Big O Notation) : Represents the upper bound of the algorithm's time complexity, used to describe how the algorithm's running time changes with the size of the input data. K: Represents the number of hash functions. In computer science, O(n) = ... (K) This is typically considered a concrete expression of constant-time. Data source: The value of K is preset during the AI ​​Agent initialization phase and is a fixed constant. Optimal range: To balance query efficiency and false positive rate, the optimal range for K is usually 3 ≤ K ≤ 7. eBPF deep tracing utilizes the O(n) of a Bloom filter. (K)The query characteristics are defined by K, which represents the preset number of independent hash functions. In this embodiment, K is set to 5. This means that for any call's validity determination, the computational overhead involves only 5 hash operations and 5 bitmap addresses, and the query efficiency is independent of the total number of blacklist entries n. This characteristic ensures that the distributed agent can maintain a deterministic millisecond-level response latency when high-concurrency Syscalls are triggered, effectively avoiding the performance degradation problem of traditional hash tables when collisions are severe. XDP and eBPF exchange data in real time through the kernel-level BPF Map, associating network layer characteristics with host-side behavior.

[0035] (3) Metadata extraction

[0036] The AI ​​Agent incorporates a lightweight local decision-making engine that performs real-time metadata extraction and feature analysis on intercepted data. This includes statistically analyzing network traffic rates, identifying packet payload distribution characteristics, and extracting metadata features such as 5-tuples, packet arrival intervals, and transmission control protocol window scaling factors. The 5-tuple includes source IP address, destination IP address, source port, destination port, and protocol type.

[0037] Specifically, metadata extraction is implemented based on the Shannon entropy algorithm. The AI ​​Agent first performs sliding window statistics on the raw message payloads, quintuplet sequences, and connection behavior data collected by XDP and eBPF, mapping the byte value distribution, quintuplet frequency, and connection behavior sequences in the message payload to a set of probability distributions. Then, according to the Shannon entropy formula: Calculate the entropy value of the corresponding data, where P(x) i ) represents the eigenvalue x i The probability of occurrence in the current statistical window.

[0038] For message payloads, the AI ​​Agent statistically analyzes the distribution probability of different byte values ​​in the payload and calculates the payload byte entropy to quantify the randomness of traffic content; for quintuple sequences, the AI ​​Agent statistically analyzes the distribution of source IP address, destination IP address, port, and protocol combination within a unit time window and calculates connection behavior entropy to reflect the dispersion of network connection patterns.

[0039] When the entropy value is higher than the preset threshold, it is determined that the current traffic may contain encrypted tunnels, malicious obfuscated code, or abnormal communication behavior. The AI ​​Agent combines the transmission control protocol window scaling factor, message arrival interval, connection duration, and behavioral characteristics to perform high-dimensional metadata correlation analysis on the abnormal traffic and generate corresponding risk labels and metadata feature results.

[0040] in This is the information entropy value, which measures the randomness of message payload or traffic distribution. Data source and value description: Calculation results. Unit: bits. X i It is the i-th possible feature value in the feature space (e.g., byte value 0-255). The data source and value description are directly derived from the original packet payload of the XDP hijacking. P(X i ) is X i The probability of a feature value appearing within the sample window. This is calculated by the AI ​​Agent within a local sliding window. i The frequency of occurrence is divided by the total number of samples. n is the total number of feature types. Data source and value explanation: For byte entropy, n is fixed at 256. The optimal range is typically between 3.0 and 5.5 for normal business traffic; if... A value >7.0 is typically considered to indicate the presence of encrypted traffic or packed code with high randomness.

[0041] (4) Semantic object transformation

[0042] For unstructured or web page logs, the AI ​​Agent uses a named entity recognition algorithm based on a bidirectional attention mechanism to extract entities. By identifying behavioral patterns in the logs, the logs are automatically transformed into standardized semantic objects.

[0043] Specifically, for unstructured logs, the algorithm utilizes the self-attention mechanism in the Transformer architecture to assign weights to word vectors in the log text, accurately identifying semantic context. The algorithm automatically extracts key entities from the raw logs and transforms them into standardized Internet Protocol address, Uniform Resource Locator, and User Agent (IP, URL, User-Agent) triple objects.

[0044] Step 3: Situation Analysis and Task Scheduling

[0045] After each distributed AI Agent completes edge-side data collection and preprocessing, it sends the generated standardized semantic objects, connection metadata, behavioral features, risk labels, and local preliminary judgment results to the AI ​​Agent task routing module. The AI ​​Agent task routing module aggregates, deduplicates, and validates the format of the data uploaded by each node, then calls the central-side large language model orchestration engine to complete security posture semantic understanding, risk rating, inference decision-making, defense strategy generation, and knowledge alignment rule generation. Based on the output of the central-side large language model orchestration engine, the AI ​​Agent task routing module generates and issues dynamic task instructions.

[0046] (1) Semantic understanding of security situation and risk rating

[0047] The central large-scale language model orchestration engine receives standardized semantic objects such as IP, URL, and User-Agent uploaded by each distributed AI Agent, as well as metadata features such as 5-tuples, message arrival intervals, payload entropy values, calls, file I / O, and process context. It then performs semantic understanding and risk assessment on single-node abnormal events and multi-node related events. Based on threat type, abnormal behavior intensity, attack propagation path, importance of affected assets, and the degree of multi-node correlation, the model generates corresponding risk levels or risk scores. These risk levels are used to determine whether the current situation is under normal monitoring, suspected attack, attack in progress, or severe attack, and serve as the basis for subsequent task scheduling and working mode switching.

[0048] (2) Reasoning decision-making and defense strategy generation

[0049] The central large language model orchestration engine, based on risk rating results and security characteristics uploaded by each node, determines whether the current threat belongs to scanning and probing, vulnerability exploitation, malicious code execution, lateral movement, data leakage, or other attack types. It then generates defense strategies by combining historical threat intelligence and current asset status. These defense strategies include increasing the collection frequency of specific nodes, enabling eBPF deep tracking, expanding the scope of P2P attack fingerprint sharing, adjusting XDP filtering rules, or triggering global situational awareness fusion analysis.

[0050] (3) Knowledge alignment rule generation and situation map update

[0051] The central large language model orchestration engine generates or updates knowledge alignment rules based on semantic objects, risk tags, and attack fingerprints uploaded by different AI agents. This unifies field naming, entity identification, attack behavior types, and risk tags across different nodes. For events from different nodes that are suspected of pointing to the same attack entity, association and merging are performed based on attack source IP, URL pattern, User-Agent, time window, behavior sequence, and similarity fingerprint, generating knowledge alignment rules and graph update suggestions. These graph update suggestions indicate how to update elements such as attack entities, victim assets, attack paths, time sequences, and potential impact ranges in the subsequent global situational awareness graph, providing a basis for subsequent task assignment and coordinated defense.

[0052] (4) Dynamic task instruction generation and issuance

[0053] The AI ​​Agent task routing module generates differentiated dynamic task instructions for different AI Agents based on the risk rating, defense strategy, knowledge alignment rules, and situational graph results output by the central large language model orchestration engine. These dynamic task instructions include the target node, target object, monitoring mode, collection fields, execution duration, risk threshold, sharing scope, and feedback requirements. For low-risk nodes, the task instructions control the AI ​​Agent to maintain low-power heartbeat monitoring; for suspected attack nodes, the task instructions control the AI ​​Agent to increase the sampling frequency and enable enhanced collection; for high-risk or confirmed attack nodes, the task instructions control the AI ​​Agent to enable full feature capture, eBPF deep tracking, and multi-Agent collaborative linkage. After each AI Agent executes its task, it continues to send new collection results and execution feedback back to the AI ​​Agent task routing module, forming a closed loop of "edge collection—central analysis—task issuance—execution feedback."

[0054] Step 4: Multi-AI Agent Collaboration and Sharing and Global Situational Fusion

[0055] After generating dynamic task instructions in step 3 and distributing them to each distributed AI Agent, each AI Agent executes enhanced monitoring and deep tracking according to the task instructions. When any AI Agent discovers suspicious behavior or confirms a high-risk event locally, it generates an attack source fingerprint containing the attack source IP, target asset, access URL, behavior type, timestamp, risk label, and similarity fingerprint. This attack source fingerprint is then synchronously uploaded to the AI ​​Agent task routing module, which registers the task feedback and routes the data. It is also synchronized to the multi-AI Agent data fusion module for knowledge alignment and global situational awareness fusion. Simultaneously, the AI ​​Agent shares the attack source fingerprint with other AI Agents that are related to or adjacent to the network path through a point-to-point encrypted communication tunnel, triggering coordinated monitoring by other nodes.

[0056] (1) P2P intelligence sharing

[0057] When node AI Agent A detects suspicious behavior locally or receives a collaborative monitoring task from the task routing module, it generates an attack source fingerprint based on the locally collected results and shares this fingerprint with neighboring AI Agents or path-related AI Agents via an encrypted communication tunnel. The receiving AI Agent updates its risk concern list based on the shared fingerprint and enhances monitoring of identical attack sources, similar User-Agents, similar URL patterns, and related behavioral sequences. If the receiving AI Agent detects matching behavior, it sends the matching result, related evidence, and local risk label back to the AI ​​Agent task routing module.

[0058] (2) Multi-node knowledge alignment

[0059] The multi-AI Agent data fusion module receives attack source fingerprints, semantic objects, metadata features, and risk tags uploaded by each node (i.e., each AI Agent). It then performs field standardization, entity normalization, and semantic alignment on the data records generated by different nodes. For records with different field descriptions but potentially pointing to the same attack behavior, it performs association judgments based on attack source IP, target asset, URL pattern, User-Agent, time window, behavior sequence, and risk tags to confirm whether the records on different nodes belong to the continuous behavior of the same attack entity.

[0060] Specifically, the SimHash similarity alignment algorithm is used to perform dimensionality reduction matching on multi-node data records. Each AIAgent or multi-AI Agent data fusion module extracts features from the data records, such as attack source, target asset, access path, User-Agent, behavior type, time window, and risk label, and assigns weights based on feature importance to generate a fixed-length SimHash fingerprint. By calculating the Hamming distance between different fingerprints (Hamming distance refers to the number of different characters in the same position of two strings of equal length), it is determined whether the behaviors observed by different nodes belong to the same attack entity. When the Hamming distance is less than a preset threshold, the corresponding data records are merged into the same attack entity or the same attack chain segment.

[0061] (3) Global situation integration

[0062] The multi-AI Agent data fusion module generates a global situational fusion result from multi-node events that have undergone knowledge alignment. The fusion result includes the identification result of the same-source attack entity, the lateral movement path, the set of affected nodes, the attack stage judgment, and the global risk level. It serves as the input for step 5 to construct the global situational map and for the central side large language model orchestration engine to make further judgments.

[0063] Step 5: Construction of global situation map, risk identification and early warning optimization

[0064] After completing multi-AI Agent collaborative sharing, knowledge alignment, and global situational awareness fusion in step 4, the semantic objects, metadata features, behavioral features, attack source fingerprints, risk tags, and correlation results uploaded by each node are written into the global situational awareness map. This global situational awareness map is used to depict the relationships between attack entities, victim assets, attack behaviors, time series, propagation paths, and the scope of risk impact. Based on this, threat identification, attack tracing, risk warning, response suggestion generation, and detection strategy optimization are completed, providing support for enterprise-side security operation decisions.

[0065] (1) Construction of global situation map

[0066] Based on the knowledge alignment results, attack entity merging results, and multi-node association events output in step 4, a global situational awareness map is established using a graph database or relational database. The nodes in the map include attack sources, victim assets, access URLs, User-Agents, processes, files, calls, risk events, and security policies; the edge relationships include access relationships, trigger relationships, similarity relationships, temporal relationships, lateral movement relationships, and asset impact relationships. By calculating the association paths between entities, potential attack chains from external probing, vulnerability exploitation, host intrusion to lateral movement are identified, and a comprehensive global situational awareness map is generated for the enterprise.

[0067] (2) Risk identification and attack tracing

[0068] Based on the attack entities, event timelines, asset importance, behavioral severity, and multi-node correlation in the global situational awareness map, threat events are identified and attack sources are traced. For similar alerts appearing on multiple nodes, the SimHash similarity alignment results, attack source fingerprints, behavioral sequences, and time windows are combined to determine whether they belong to the same attack entity or the same attack chain, and the attack source, attack path, affected assets, attack stage, and risk level are output.

[0069] (3) Early warning generation

[0070] Based on the risk identification results, tiered early warning information is generated. This early warning information includes the early warning level, attack type, attack source, affected nodes, key evidence, attack path, cause of the risk, and recommended actions. When lateral movement, access to sensitive assets, abnormal calls, abnormal file writing, or multi-node attacks originating from the same source are identified, the early warning level for the corresponding event is increased, and a coordinated monitoring or handling task is issued to the relevant AI Agents via the AI ​​Agent task routing module. Simultaneously, early warning information is output to the central management terminal.

[0071] (4) Early warning optimization and strategy feedback

[0072] Based on historical alert results, handling feedback, false alarms, and newly emerging attack characteristics in the global situational awareness map, the early warning rules and detection strategies are optimized. Specifically, this includes: updating the malicious feature matching rules of the XDP layer based on the Aho-Corasick multi-pattern matching algorithm; updating the high-risk call set of the eBPF layer based on a Bloom filter; adjusting the abnormal payload identification threshold based on the Shannon entropy algorithm; optimizing the log semantic object extraction rules based on the named entity recognition algorithm; and optimizing the multi-node attack entity merging threshold based on the SimHash similarity alignment algorithm. The optimized rules and strategies are distributed to the corresponding distributed AI Agents through the AI ​​Agent task routing module, forming a closed loop of "data collection—edge preprocessing—centralized analysis—collaborative fusion—early warning optimization—strategy feedback".

[0073] This invention abandons the traditional mode of full traffic backhaul and instead deploys AI Agents in a distributed manner at edge collection nodes connected to key network exits, server access layers, and switch mirror ports of enterprises, thereby realizing intelligent agent deployment and high-performance data hijacking based on distributed nodes.

[0074] Implementation method: The AI ​​Agent utilizes the XDP high-speed network data processing mechanism and the eBPF programmable kernel tracing mechanism in the Linux kernel to achieve high-speed filtering, interception and behavior tracking of data packets in kernel mode.

[0075] eBPF technology is a programmable mechanism that allows restricted sandbox programs to run within the Linux kernel without modifying the Linux kernel source code or loading kernel modules. The Linux kernel provides an eBPF virtual execution environment, and the AI ​​Agent can load logic such as traffic filtering and call monitoring into kernel mode for execution. For business traffic that is initially screened as suspicious by XDP, the AI ​​Agent activates the eBPF tracing program to perform in-depth tracing of the calls, file I / O, and process context associated with the traffic.

[0076] XDP technology is a high-performance, programmable network data processing mechanism based on the eBPF framework. Its mounting point is located at the Linux network interface card (NIC) driver layer, executing processing logic before data packets enter the Linux network protocol stack. This invention leverages XDP's mounting at the NIC driver layer to enable the AI ​​Agent to build a high-speed data processing path outside the kernel protocol stack. For large-scale, high-frequency known distributed denial-of-service (DDoS) attack traffic and scanner-characteristic traffic, it performs dropping, marking, or rapid mirroring of data packets before they enter the protocol stack, preventing invalid traffic from consuming CPU resources.

[0077] The AI ​​Agent in this invention achieves joint awareness of network traffic and behavior through the collaboration of XDP and eBPF. Specifically, XDP is responsible for the initial screening and risk labeling of high-throughput network traffic, while eBPF is responsible for in-depth behavior tracking and context association. The two exchange data in real time through the kernel-level BPF Map, realizing full-stack closed-loop situational awareness from the underlying network links to host-side behavior.

[0078] Therefore, this invention solves the problem of core link bandwidth overload in traditional centralized architecture, enabling preliminary traffic profiling and risk identification at the source of data generation, and significantly reducing backend processing pressure.

[0079] This invention incorporates a lightweight local decision engine and a local feature library into each independently running AI Agent.

[0080] The AI ​​Agent performs multi-dimensional analysis on the captured raw data, calculates the traffic entropy value and packet payload feature distribution, and extracts metadata features including quintuples, packet length distribution, and packet arrival interval. It also uses a lightweight model to extract entities from unstructured Syslog logs and Web logs, transforming them into standardized semantic objects such as IP, URL, and User-Agent.

[0081] This invention solves the problem that traditional sensors only collect data but lack semantic understanding capabilities. It achieves edge pre-denoising of data, transforming the information uploaded to the central side from raw noisy data into high-value secure semantic data.

[0082] This invention introduces a situation assessment and dynamic task scheduling mechanism based on a large language model orchestration engine on the central management side, which serves as the central decision-making unit for overall situational awareness and is used to analyze complex defense intentions and generate dynamic defense strategies.

[0083] The central large language model orchestration engine adjusts the working mode of each AI Agent in real time based on the current risk rating. When it is determined that the current period is a silent period, the AI ​​Agent performs low-power heartbeat monitoring; when it is determined that the current period is a suspected attack period, the central large language model orchestration engine dynamically issues task instructions through the AI ​​Agent task routing module, so that the corresponding AI Agent automatically starts full feature capture and real-time behavior tracking.

[0084] Therefore, this invention solves the problem that traditional situational awareness cannot automatically complete dynamic monitoring and analysis under resource-constrained conditions, and realizes the flexible regulation and on-demand scaling of sensing resources.

Claims

1. An enterprise-side situational awareness method based on distributed dynamic data processing using AI agents, characterized in that, Includes the following steps: Deploy distributed AI agents on designated nodes within the enterprise-side network environment; AI Agent collects and preprocesses multi-source dynamic data from the enterprise side, completes initial screening of network traffic, deep tracking of behavior, extraction of metadata features and transformation of semantic objects, and obtains standardized semantic objects and metadata features. The pre-processed data at the edge is sent to the AI ​​Agent task routing module. The AI ​​Agent task routing module aggregates, deduplicates, and verifies the format of the data uploaded by each node. Then, it calls the central big language model orchestration engine to complete situational semantic understanding and risk rating, inference decision-making and defense strategy generation, knowledge alignment rule generation, and situation map update. The AI ​​Agent task routing module also issues dynamic task instructions to the corresponding AI Agent to achieve flexible switching of monitoring modes. Threat intelligence is encrypted and shared and knowledge is aligned based on a peer-to-peer collaboration protocol, linking isolated alerts generated by multiple AI agents as the same attack entity. The central side constructs a global situation map based on the collaborative results of multiple AI agents, and completes risk identification, attack tracing, early warning output and handling suggestions.

2. The enterprise-side situational awareness method based on AI Agent distributed dynamic data processing according to claim 1, characterized in that, The AI ​​Agent is deployed on an edge acquisition node running a Linux kernel. It includes a user-space control program and a kernel-space data acquisition module based on XDP and eBPF. The XDP program is mounted on the Linux network card driver layer and is used to perform high-speed initial screening and traffic marking of data packets entering the node. The eBPF program is mounted on the kernel event point and is used to dynamically track calls, file input / output, and process context. The XDP program and the eBPF program share connection state and risk marking information to realize the correlation analysis between network layer traffic characteristics and host-side behavior.

3. The enterprise-side situational awareness method based on AI Agent distributed dynamic data processing according to claim 1, characterized in that, The defined nodes of the enterprise-side network environment include the enterprise border router exit at the edge access layer, the switch mirror port at the core switching layer, and the external network exit and web application server access at the application service layer.

4. The enterprise-side situational awareness method based on AI Agent distributed dynamic data processing according to claim 1, characterized in that, The enterprise-side multi-source dynamic data includes raw network traffic, logs, web page access logs, process behavior logs, file input / output logs, and call logs.

5. The enterprise-side situational awareness method based on AI Agent distributed dynamic data processing according to claim 1, characterized in that, Specific methods for AI Agents to perform edge preprocessing on multi-source dynamic data from the enterprise side include: Network traffic initial screening: The AI ​​Agent uses the XDP program to perform high-speed initial screening of inbound traffic at the Linux network card driver layer, and intercepts, marks or quickly mirrors known malicious traffic and abnormal connection traffic; Deep Behavior Tracing: For traffic that passes the initial screening and is marked as suspicious, the eBPF program mounted on the kernel event point dynamically tracks the call information, file input / output information and process context information, and completes real-time data exchange between XDP and eBPF through the kernel-mode BPFMap. Metadata feature extraction: The AI ​​Agent performs multi-dimensional analysis on the captured raw data to extract metadata features; Semantic object transformation: Entity extraction is performed on unstructured logs and web page access logs, and the logs and web page access logs are transformed into standardized semantic objects.

6. The enterprise-side situational awareness method based on AI Agent distributed dynamic data processing according to claim 5, characterized in that, The specific methods by which the AI ​​Agent performs multi-dimensional analysis of the captured raw data and extracts metadata features are as follows: The Shannon entropy algorithm is used to calculate the randomness of message payload and traffic distribution. Combined with the quintuple, message arrival interval and transmission control protocol window scaling factor, abnormal encrypted traffic and malicious obfuscated code are identified.

7. The enterprise-side situational awareness method based on AI Agent distributed dynamic data processing according to claim 5, characterized in that, The specific method for extracting entities from unstructured logs and web page access logs and converting them into standardized semantic objects is as follows: A named entity recognition algorithm based on a bidirectional attention mechanism is used to perform semantic analysis on unstructured logs and web page access logs, and to extract standardized semantic objects such as Internet Protocol addresses, Uniform Resource Locators (URLs), and user agents.

8. The enterprise-side situational awareness method based on AI Agent distributed dynamic data processing according to claim 1, characterized in that, The central-side large language model orchestration engine is invoked to complete situational semantic understanding and risk assessment, inference decision-making and defense strategy generation, knowledge alignment rule generation, and situational map updates. Dynamic task instructions are then issued to the corresponding AI Agents via the AI ​​Agent task routing module, enabling flexible switching of monitoring modes, including: Security posture semantic understanding and risk rating: Based on standardized semantic objects and metadata characteristics, semantic understanding and risk assessment are performed on single-node abnormal events and multi-node related events; based on threat type, abnormal behavior intensity, attack spread path, importance of affected assets, and degree of multi-node correlation, corresponding risk level or risk score is generated; Reasoning, decision-making, and defense strategy generation: The central big language model orchestration engine determines whether the current threat belongs to scanning and probing, vulnerability exploitation, malicious code execution, lateral movement, data leakage, or other attack types based on risk rating results and security features uploaded by each node, and generates defense strategies by combining historical threat intelligence and current asset status. Knowledge alignment rule generation and situation map update: The central big language model orchestration engine generates or updates knowledge alignment rules based on semantic objects, risk tags and attack fingerprints uploaded by different AI Agents. This is used to unify field naming, entity identification, attack behavior type and risk tags between different nodes. Events from different nodes but suspected of pointing to the same attack entity are associated and merged to generate knowledge alignment rules and situation map update suggestions. Dynamic task instruction generation and issuance: The AI ​​Agent task routing module generates differentiated dynamic task instructions for different AI Agents based on the risk rating, defense strategy, knowledge alignment rules and situation map results output by the central large language model orchestration engine.

9. The enterprise-side situational awareness method based on AI Agent distributed dynamic data processing according to claim 1, characterized in that, The specific method for encrypting and sharing threat intelligence and aligning knowledge based on a peer-to-peer collaboration protocol, and for associating isolated alerts from multiple AI agents as the same attack entity, is as follows: Each AI Agent executes enhanced monitoring and deep tracking according to task instructions. When node AI Agent A discovers suspicious behavior locally or receives a collaborative monitoring task issued by the task routing module, it generates an attack source fingerprint based on the local collection results and shares the attack source fingerprint with adjacent AI Agents or path-related AI Agents through an encrypted communication tunnel. The multi-AI Agent data fusion module performs field standardization, entity normalization, and semantic alignment on alarm records generated by different AI Agents. For records with different field descriptions but that may point to the same attack behavior, it performs correlation judgment to confirm whether the records on different nodes belong to the continuous behavior of the same attack entity. The multi-AI Agent data fusion module generates a global situational fusion result from multi-AI Agent events that have undergone knowledge alignment.

10. The enterprise-side situational awareness method based on AI Agent distributed dynamic data processing according to claim 1, characterized in that, The central side constructs a global situation map based on the collaborative results of multiple AI agents, and completes the specific process of risk identification, attack tracing, early warning output, and handling suggestions as follows: The semantic objects, metadata features, behavioral features, attack source fingerprints, risk tags, and related results uploaded by each node are written into the global situation map; Based on the attack entities, event timelines, asset importance, severity of behavior, and degree of correlation among multiple nodes in the global situation map, risk identification and attack attribution are performed on threat events; Generate tiered early warning information based on the risk identification results; Based on historical alarm results, handling feedback, false alarms, and newly emerging attack characteristics in the global situation map, the early warning rules and detection strategies are optimized. The optimized rules and strategies are then distributed to the corresponding AI Agents through the AI ​​Agent task routing module.

Images