Hardware acceleration-based full-flow acquisition and intelligent analysis detection system

Abstract

The application relates to a hardware acceleration-based full-flow acquisition and intelligent analysis and detection system and belongs to the technical field of network security.The hardware acceleration-based full-flow acquisition and intelligent analysis and detection system can significantly improve the processing efficiency and accuracy of network flow data.Through the high-speed NIC, the high-performance parallel computing capability of the FPGA and the GPU, the system can realize zero packet loss and zero packet leakage in a high-bandwidth environment, ensures the efficient stability of data acquisition and preprocessing, and improves the data capture and preprocessing performance.The system can quickly screen and filter irrelevant flow, reduce system overhead, and provide high-quality data support for subsequent intelligent flow analysis.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, specifically relating to a hardware-accelerated full traffic acquisition and intelligent analysis detection system. Background Technology

[0002] Against the backdrop of rapid development in information technology, the continuous advancement of digital transformation has made data a core production factor in the digital economy era, with its strategic position and economic value becoming increasingly prominent. However, the accompanying data security issues are also becoming increasingly severe. With the deepening of the Internet of Things, network application scenarios are becoming more diverse, and the network security environment is becoming more complex. In particular, abnormal network traffic, as a major manifestation of network attacks, not only seriously threatens network stability but also poses a huge risk to user privacy and the security of critical information infrastructure. Although various abnormal traffic detection systems have been deployed in the field of network security, traditional detection methods generally suffer from low computational efficiency, insufficient real-time performance, and poor adaptability when facing massive, high-dimensional, and dynamically changing modern network traffic. Especially when dealing with new types of network attacks, existing systems often fail to meet actual protection needs, which has become a bottleneck in improving the effectiveness of the overall network security protection system. Summary of the Invention

[0003] (a) Technical problems to be solved

[0004] The technical problem to be solved by this invention is to propose an efficient, real-time, and adaptive network anomaly traffic analysis and detection system to achieve high-speed, low-latency capture and preprocessing of network traffic, improve detection accuracy, and ensure the efficiency of network security protection.

[0005] (II) Technical Solution

[0006] To address the aforementioned technical problems, this invention provides a hardware-accelerated full-traffic acquisition and intelligent analysis detection system, comprising a data acquisition and preprocessing module, an adaptive intelligent traffic profiling module, and a deep source tracing analysis module.

[0007] The data acquisition and preprocessing module is used to acquire traffic with zero packet loss using network interface card and DPDK technology, and to perform protocol classification, noise filtering, duplicate data detection and feature extraction through FPGA and GPU. The acquired data is then transferred to host memory and converted into a format suitable for analysis.

[0008] The adaptive intelligent traffic profiling module is used to label suspicious traffic in the data output by the data acquisition and preprocessing module using dynamically adjusted autoencoders and GANs, and to identify attack behaviors by combining multimodal methods such as deep learning, machine learning and rule matching. Then, the abnormal traffic where the attack behavior is located is sent to the deep source tracing analysis module.

[0009] The deep source tracing analysis module is used to build network topology based on GNN. For the attack behavior analyzed by the adaptive intelligent traffic profiling module, it analyzes the propagation path of the abnormal traffic and high-risk nodes where the attack behavior is located, and combines threat intelligence to trace the attack source and generate a source tracing report.

[0010] The present invention also provides a method for full traffic acquisition and intelligent analysis and detection based on the system described above.

[0011] (III) Beneficial Effects

[0012] The hardware-accelerated full-traffic acquisition and intelligent analysis detection system of this invention can significantly improve the processing efficiency and accuracy of network traffic data. Utilizing the high-performance parallel computing capabilities of high-speed NICs, FPGAs, and GPUs, the system can achieve zero packet loss and zero packet leakage in high-bandwidth environments, ensuring efficient and stable data acquisition and preprocessing, and improving data capture and preprocessing performance. By quickly filtering irrelevant traffic, system overhead is reduced, and high-quality data support is provided for subsequent intelligent traffic analysis.

[0013] The system's adaptive intelligent traffic profiling module uses a dynamically adjustable autoencoder and GAN to label suspicious traffic in real time. Combined with multimodal detection methods such as deep learning, machine learning, and rule matching, it accurately analyzes attack behavior from suspicious traffic. This adaptive intelligent traffic profiling module can automatically optimize model parameters based on real-time changes in traffic patterns, continuously adapting to new network environments and attack types. This improves the detection and response capabilities to complex attacks and new threats, ensuring the system maintains highly efficient and accurate identification capabilities in the face of constantly changing network traffic.

[0014] The system can also perform in-depth source tracing analysis of traffic propagation paths, helping users locate the source of attacks and the affected scope. The visualization module provides users with intuitive data analysis and reporting functions, helping cybersecurity teams monitor and make decisions more efficiently, and enhancing the convenience and practicality of cybersecurity management. Attached Figure Description

[0015] Figure 1 A system architecture diagram of a hardware-accelerated full-traffic acquisition and intelligent analysis detection system provided in an embodiment of the present invention;

[0016] Figure 2 A flowchart illustrating the abnormal traffic detection process of a hardware-accelerated full traffic acquisition and intelligent analysis detection system provided in this embodiment of the invention.

[0017] Figure 3 A flowchart of the data acquisition and preprocessing module of the hardware-accelerated full-traffic acquisition and intelligent analysis detection system provided in this embodiment of the invention;

[0018] Figure 4 A flowchart of the adaptive intelligent traffic profiling module of the hardware-accelerated full traffic acquisition and intelligent analysis detection system provided in the embodiments of this invention application;

[0019] Figure 5 The flowchart of the deep source tracing analysis module of the hardware-accelerated full traffic acquisition and intelligent analysis detection system provided in the embodiments of this invention application is shown. Detailed Implementation

[0020] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0021] This invention innovatively proposes a high-efficiency, real-time, and adaptive network anomaly traffic analysis and detection system. Through hardware acceleration and parallel data processing, the system achieves high-speed, low-latency capture and preprocessing of network traffic; it utilizes autoencoders, generative adversarial networks, and multimodal detection methods to construct a dynamic traffic model, significantly improving detection accuracy; and it integrates deep learning and rule-based matching to accurately identify complex anomalies and perform source tracing analysis, comprehensively ensuring the high efficiency of network security protection.

[0022] The hardware-accelerated full-traffic acquisition and intelligent analysis detection system includes a data acquisition and preprocessing module, an adaptive intelligent traffic profiling module, an in-depth source tracing analysis module, and a visualization module. The specific components are as follows:

[0023] The data acquisition and preprocessing module utilizes hardware acceleration to achieve high-speed traffic capture and efficient preprocessing, ensuring zero packet loss and zero packet omissions in high-bandwidth networks. This module employs a four-layer architecture: the data acquisition layer uses high-performance network interface cards (NICs) and DPDK technology to capture data packets with zero copy and perform preliminary protocol classification; the hardware acceleration layer combines programmable FPGAs and graphics cards (GPUs) to perform functions such as packet decoding, traffic protocol classification, noise filtering, duplicate data detection, and feature extraction; the data transmission layer efficiently transmits the data processed by the hardware acceleration layer to the host memory via the PCIe interface and converts it to JSON format; the control and management layer is responsible for module configuration, real-time monitoring, and performance optimization, ensuring system efficiency and resource utilization.

[0024] The adaptive intelligent traffic profiling module achieves high-precision identification and real-time response to abnormal network traffic through the collaborative work of an abnormal traffic labeling layer and a multimodal anomaly detection layer. The module first performs unsupervised modeling of historical benign traffic, using an autoencoder to learn the low-dimensional features and inherent patterns of the traffic, and combining this with a generative adversarial network (GAN) to generate pseudo-samples and slightly anomalous samples, achieving dynamic evolution and adversarial enhancement, and improving the model's sensitivity to anomaly patterns. Subsequently, the module labels real-time traffic as anomalies, marking traffic with a high comprehensive anomaly score based on autoencoder reconstruction error and GAN discriminator feedback as suspicious traffic. Suspicious traffic then enters the multimodal anomaly detection layer, which integrates deep learning, machine learning, and rule matching algorithms, comprehensively considering statistical features, time series data, and behavioral patterns to determine abnormal traffic and extract key features to support subsequent source tracing analysis. Simultaneously, suspicious traffic with scores below a threshold is associated with specific data packet content, providing manual verification to confirm potential attack types.

[0025] The deep source tracing analysis module, based on Generative Neural Networks (GNNs), captures the propagation patterns of abnormal traffic by constructing a network topology graph and performing node embedding and graph convolution, identifying high-risk paths and potentially affected nodes. The module combines a threat intelligence database and time-series analysis to trace the attack source and affected area, generating a source tracing report, providing risk mitigation recommendations, and displaying propagation paths and risks. Furthermore, the GNN model is continuously updated to adapt to new attack patterns, and the dynamic updating of the threat intelligence database enhances source tracing analysis capabilities and detection accuracy.

[0026] The visualization module provides an interactive interface to present users with the detection results and source analysis reports of abnormal traffic in an intuitive and easy-to-understand way. Through this module, users can view the trend changes and type distribution information of abnormal traffic using dynamic charts. Furthermore, users can explore the details of abnormal traffic through zooming, filtering, and querying functions. This visualization module supports exporting detection results and reports to multiple formats, facilitating user archiving and further analysis, making the monitoring and analysis of abnormal network traffic more efficient and convenient.

[0027] See system framework Figure 1 The overall architecture of this system consists of four core modules: data acquisition and preprocessing module, adaptive intelligent traffic analysis module, in-depth source tracing analysis module, and visualization display module.

[0028] The data acquisition and preprocessing module achieves efficient capture and preprocessing of the entire traffic through hardware acceleration and multi-layer optimization, ensuring zero packet loss and zero packet omission in data acquisition under high-bandwidth networks;

[0029] The adaptive intelligent traffic profiling module combines multimodal feature modeling, deep learning, and a rule engine to achieve accurate detection and real-time response to abnormal traffic.

[0030] The deep source tracing analysis module provides correlation analysis and attack chain tracing based on network graphs and threat intelligence, supporting in-depth mining of complex threats;

[0031] The visualization module presents the analysis results in dynamic charts and an interactive interface, supporting report generation and export, helping users intuitively understand the network situation and assisting in decision-making. Through hierarchical connection and collaboration between modules, the system achieves a complete closed loop of "full traffic collection - intelligent analysis - in-depth source tracing - visualization display," ensuring both the efficiency and accuracy of detection, and improving the user's operating experience and management efficiency.

[0032] The overall detection process of the hardware-accelerated full-traffic acquisition and intelligent analysis detection system can be found in [link to system details]. Figure 2 First, the data acquisition and preprocessing module utilizes a high-performance network card combined with DPDK technology to collect traffic with zero packet loss. It then uses FPGA and GPU to perform protocol classification, noise filtering, duplicate data detection, and feature extraction. The processed data is rapidly transferred to host memory via the PCIe interface and converted into a format suitable for analysis. Next, the adaptive intelligent traffic profiling module employs a dynamically adjusted autoencoder and GAN to label suspicious traffic in the data processed by the preprocessing module. It then uses multimodal methods such as deep learning, machine learning, and rule matching to accurately identify attack behaviors. Subsequently, abnormal traffic enters the deep source tracing analysis module. Based on GNN, it constructs a network topology and analyzes the propagation path and high-risk nodes of the abnormal traffic identified by the adaptive intelligent traffic profiling module. Combined with threat intelligence, it traces the attack source, generates a source tracing report, and provides risk mitigation suggestions. Finally, all analysis results are presented through a visualization module. Users can interactively analyze traffic trends and detailed information through dynamic charts to help make more accurate decisions. The specific functions of each module are described below.

[0033] See the flowchart for the data acquisition and preprocessing module. Figure 3 The data acquisition and preprocessing module adopts a four-layer architecture: data acquisition layer, hardware acceleration layer, data transmission layer, and control and management layer. This architecture ensures the module's efficiency and scalability, meeting the high requirements for traffic capture and preprocessing in high-bandwidth network environments. The specific steps for implementing the data acquisition layer function are as follows:

[0034] Step one involves deploying a high-performance network interface card (NIC) at the gateway sending end to handle network traffic in real time. The NIC performs fragmentation on the incoming traffic. Based on a preset maximum segment size, it automatically divides large blocks of traffic data into multiple TCP segments, dynamically generating and filling the TCP / IP header information for each segment, and calculating checksums. This process offloads the protocol encapsulation, fragmentation, and checksum calculation tasks, which would normally be performed multiple times by the CPU, to dedicated hardware, significantly reducing CPU processing frequency and thus greatly improving network transmission efficiency. At the host receiving end, multiple consecutively arriving TCP packets are aggregated. The high-performance NIC performs sequence checks on the packets at the underlying level to ensure there are no packet losses or out-of-order issues. It then merges multiple small packets belonging to the same data stream into a complete, larger data segment, adds a precise hardware timestamp, and submits it to the operating system protocol stack. By reducing the number of interrupt triggers and the overhead of packet-by-packet processing, the CPU load on the receiving path is effectively reduced, improving the system's ability to handle high-throughput network traffic.

[0035] Step two utilizes the multi-queue receive-side scaling (RSS) function of the network interface card to distribute traffic to different receive queues based on the five-tuple characteristics of the traffic (source IP, destination IP, source port, destination port, and protocol type). The traffic is then distributed to different CPU cores for parallel processing. This achieves large-scale network traffic distribution and parallel computing, significantly improving real-time processing capabilities.

[0036] Step three: After queuing, the network interface card (NIC) is accessed directly via DPDK technology, bypassing the traditional operating system protocol stack and avoiding multiple data copies between the kernel and the user. This significantly reduces latency and CPU usage, ensuring stable data acquisition in high-speed network environments. Simultaneously, the acquisition layer performs basic protocol classification (such as TCP, UDP, HTTP, DNS) on the traffic during data capture and appends the results as metadata. This allows subsequent analysis to directly target traffic from different protocols, thereby improving analysis efficiency.

[0037] The hardware acceleration layer is internally divided into several sub-functions, including traffic classification, duplicate data detection, preliminary rule filtering, noise filtering, and feature extraction. This module fully leverages the advantages of programmable hardware and general-purpose parallel processors to build an efficient, low-latency traffic preprocessing pipeline. The specific steps of the hardware acceleration layer are as follows:

[0038] Step 1: Traffic Classification and Rule Filtering. Utilizing the lookup table (LUT) and multi-stage pipeline structure within the FPGA, key information from the network and transport layers is parsed and matched in real time. During parsing, predefined blacklist and whitelist rules are used to determine the legality of data packets and perform rapid filtering. Because this mechanism is implemented entirely in FPGA hardware without CPU intervention, it maintains zero packet loss and nanosecond-level response even in high-bandwidth environments, greatly alleviating the processing pressure on subsequent analysis modules.

[0039] Step two, duplicate data detection. This layer employs a two-layer deduplication mechanism. First, on the FPGA side, a hash function based on the five-tuple (source / destination IP, port, protocol) and packet digest information is used to quickly identify and filter historical duplicate traffic. Then, on the GPU side, a sliding time window combined with a hash table is introduced to perform deep comparison and removal of duplicate packets within a short period. This two-level deduplication mechanism effectively avoids data redundancy in high-concurrency scenarios.

[0040] Step three, noise filtering, leverages the powerful parallel computing capabilities of GPU's CUDA to construct a dynamic identification method based on a probabilistic model. This layer collects samples of normal business traffic, constructs a Gaussian mixture model with multi-dimensional features, and uses CUDA to quickly calculate the probability of newly arriving traffic. This identifies abnormal behaviors that significantly deviate from the feature distribution and removes them as noise. This mechanism possesses continuous learning and adaptive updating capabilities, constantly optimizing as network traffic characteristics change, thus improving the accuracy and robustness of noise filtering.

[0041] Step four, feature extraction: After traffic cleaning, the system fully leverages the parallel computing advantages of GPUs to extract multi-dimensional structural and behavioral features from the data traffic. The extracted features cover key indicators such as packet time, average flow rate, connection duration, and request-to-response ratio. Rule matching and information gain-based feature importance evaluation are combined to improve the effectiveness and discriminative power of feature representation, providing high-quality input for the subsequent adaptive intelligent traffic profiling module.

[0042] The data transmission layer transmits the data processed by the hardware acceleration layer to the host memory at high speed, ensuring low latency and high bandwidth. The specific steps of the data transmission layer function are as follows:

[0043] Step 1: The data preprocessed by the hardware acceleration layer is transmitted to the host memory in a low-latency, high-bandwidth manner via the PCIe high-speed bus.

[0044] Step two: After the data enters the host memory, it is stored in multi-level data buffers to coordinate the difference between the high-speed transmission of the hardware layer and the processing capability of the host side, so as to avoid data loss or packet loss due to instantaneous congestion.

[0045] Step three: All cached and scheduled data will be uniformly converted into a standardized JSON format. This provides structured and standardized input support for subsequent intelligent analysis.

[0046] The control and management layer supports users in configuring data acquisition rules, preprocessing parameters, and hardware acceleration strategies, and monitors the operational status of the data acquisition module (such as packet capture rate, preprocessing efficiency, and hardware resource utilization), while also providing alarm functions. The specific steps of the control and management layer functions are as follows:

[0047] Step one: Users set data collection rules, preprocessing parameters, and hardware acceleration strategies according to their actual needs. For example (key protocol types to be collected, traffic range, filtering and cleaning strategies).

[0048] Step two: During module operation, the control and management layer continuously monitors key status indicators, including packet capture rate, preprocessing efficiency, and hardware resource utilization. These real-time indicators help users promptly grasp system performance and identify potential bottlenecks or hidden problems.

[0049] Step three: When the monitoring module detects an anomaly (such as a sudden drop in capture rate, hardware resource overload, or a significant decrease in processing efficiency), the control and management layer will trigger an alarm. The system will then provide clear feedback to the user through logs, a visual interface, or a notification mechanism.

[0050] The adaptive intelligent traffic profiling module consists of an anomaly traffic labeling layer and a multimodal anomaly detection layer, used for efficient identification and analysis of abnormal network traffic. The anomaly traffic labeling layer uses autoencoders and generative adversarial networks (GANs) to perform unsupervised learning on historical benign traffic, thereby capturing its complex patterns and feature distributions. During runtime, this layer compares real-time traffic with the model output, labels suspicious traffic, and sends it to the multimodal anomaly detection layer. The multimodal anomaly detection layer integrates deep learning, machine learning, and rule matching algorithms to analyze traffic features from multiple dimensions, achieving comprehensive anomaly detection. The algorithms collaborate to leverage their strengths, improving the accuracy of identifying complex anomaly traffic. Detailed information is extracted from traffic deemed anomaly, providing crucial information for subsequent source tracing analysis. The workflow of the adaptive intelligent traffic profiling module is described in [link to documentation]. Figure 4 .

[0051] Step 1: Unsupervised modeling of historical positive traffic.

[0052] 1. Data Collection and Cleaning: The system collects a large amount of historical, healthy traffic data. The data undergoes noise reduction and preprocessing to remove obviously abnormal or missing samples.

[0053] 2. Autoencoder Training: The cleaned traffic data is input into the autoencoder to learn the low-dimensional feature representation and inherent patterns of the traffic. The original traffic is then reconstructed using the decoder. The autoencoder optimizes the reconstruction error, learning the inherent patterns and low-dimensional feature distribution of the traffic, providing a baseline feature space for anomaly detection.

[0054] 3. GAN Generation and Optimization: The low-dimensional features extracted by the encoder are input into the generator to produce pseudo-samples that resemble the distribution of real benign traffic features. The discriminator distinguishes between real and generated samples and continuously optimizes its discrimination ability. The generator, using the discriminator's feedback as an optimization target, adjusts its generation strategy to make the generated samples closer to the real benign traffic distribution. Through iterative training of the generator and discriminator, the system gradually improves the realism and discrimination accuracy of the generated samples.

[0055] 4. Dynamic Evolution and Adversarial Enhancement: The autoencoder provides the latest traffic feature representation, supplying input for the generator to generate samples. The samples generated by the GAN include normal, benign samples and slightly anomalous samples near the normal boundary; the latter is used to enhance the autoencoder's sensitivity to anomalous patterns. The generated samples are continuously added to the training set for incremental training of the autoencoder and GAN, thereby achieving dynamic updates of training data and model parameters.

[0056] Step 2: Real-time traffic anomaly labeling and adaptive updates

[0057] 1. Reconstruction error verification: If the difference between the input traffic and the reconstruction result of the autoencoder (reconstruction error) exceeds the suspicious threshold, the traffic is suspected of being abnormal.

[0058] 2. Discriminator Verification: The traffic is then input into the GAN's discriminator. If the discriminator also gives a low confidence score, its anomaly is further confirmed.

[0059] 3. Comprehensive judgment and labeling of suspicious traffic: The reconstruction error and the discriminator feedback results are fused. If both point to anomalies, the traffic is directly labeled as suspicious; if only one is abnormal, a weighted judgment is made based on a threshold; if both are normal, the traffic is considered benign.

[0060] 4. Adaptive Model Update: A sliding window mechanism is used for incremental training of the model, allowing it to continuously adapt to changes in the traffic environment. An adaptive learning rate strategy is introduced: when the model error is large, the learning rate is increased to accelerate convergence; when the error tends to stabilize, the learning rate is decreased to improve accuracy. This ensures that the detection model is always in optimal condition.

[0061] Step 3: Multimodal anomaly detection and feature extraction

[0062] 1. Suspicious traffic transmission: Transmit traffic marked as suspicious in step 2 to the multimodal anomaly detection layer.

[0063] 2. Multi-algorithm Fusion Detection: First, key information such as statistical features, time-series features, and behavioral patterns are extracted from traffic data, and a deep learning model is used to automatically learn high-dimensional features. For time-series features, LSTM is selected to capture time dependencies and calculate the anomaly probability of suspicious traffic. Simultaneously, a random forest method is used to calculate the anomaly probability of the statistical features of suspicious traffic. Furthermore, based on an efficient rule matching engine, suspicious traffic is compared with an attack rule base to calculate the probability of suspicion and anomaly. Finally, the anomaly score is calculated by weighting and summing the above three methods. Traffic with a score exceeding a set anomaly traffic threshold is identified as abnormal traffic.

[0064] 3. Anomaly Feature Extraction: Extract key features from traffic identified as abnormal, including source IP, destination IP, port number, and protocol type. This ensures the completeness of the source tracing analysis and provides comprehensive data support for subsequent in-depth source tracing analysis.

[0065] 4. Manual judgment of suspicious abnormal traffic: For traffic with a score lower than the set abnormal traffic threshold, the specific traffic data packet content is associated with the data for further manual analysis and determination of the attack type.

[0066] The deep source tracing analysis module constructs a network topology map based on devices, IPs, and ports. It utilizes graph embedding and graph convolutional neural networks to mine the interactions between nodes, identifying the propagation paths and potential spread trends of abnormal traffic. Combined with comparison and similarity calculations against an external threat intelligence database, this module can accurately locate the attack source, affected nodes, and their threat levels, generating a structured source tracing report to support visualization and emergency decision-making. The workflow of the deep source tracing analysis module is described below. Figure 5 The specific steps for implementing the deep source tracing analysis module are as follows:

[0067] Step one: The deep source tracing analysis module first constructs a heterogeneous network topology graph based on the collected network device logs, boundary traffic records, and IP communication relationships. Nodes represent IP addresses, host devices, ports, and asset types, while edges represent communication relationships or potential attack paths. To better reflect the contextual structure information in the topology graph, the system uses the Node2Vec algorithm for graph embedding. By simulating different random walk strategies, it generates low-dimensional vector representations, providing more discriminative feature input data for subsequent graph neural network analysis.

[0068] Step two, in identifying abnormal propagation paths, the module utilizes a graph neural network (GNN) to perform multi-layer feature aggregation on the topology graph. This involves fusing historical behavioral information such as node communication frequency, traffic direction changes, and neighbor node information to generate a feature representation of the fused structural context. It automatically learns attack propagation patterns, captures complex interactions between nodes, identifies the propagation trend of abnormal traffic in the topology graph, potential diffusion paths, potential high-risk jump points, and victim assets, thus constructing a complete attack chain.

[0069] Step three involves the module comparing the IP addresses and domain names appearing in the abnormal traffic with the local threat intelligence database. This process quickly identifies known attack behaviors and improves detection coverage. The intelligence database is continuously updated online, integrating external threat intelligence from multiple sources to ensure the system's detection capabilities continue to evolve.

[0070] Step 4, reconstructing the attack trajectory over time:

[0071] 1. Behavioral Sequence Extraction: First, historical traffic data is grouped according to the source IP, and behavioral sequences of the same source IP in different time periods are extracted. Each behavioral sequence includes information such as access target (destination IP, port), access timestamp, protocol type, session duration, data volume, and connection frequency. An ordered behavioral chain is constructed by sorting by time, providing a basic data structure for subsequent trajectory analysis.

[0072] 2. Spatiotemporal Feature Fusion: The behavioral sequences are combined with the analysis results of the topology graph generated by GNN. Each node not only retains the network topology embedding vector but also incorporates time-series features. By using spatiotemporal features to model the dynamic interaction relationships between nodes, the system can identify the propagation trend of attacks in the network structure over time.

[0073] 3. Attack Path Construction: For each source IP behavior sequence, a preliminary attack path graph is generated based on the access order and GNN neighbor relationships. Connections between nodes are weighted with attributes such as time interval weight, protocol type weight, and traffic volume to reflect the strength and directionality of the attack propagation.

[0074] 4. Causal Relationship Chain Inference: Analyze the interaction order and dependencies of each node in the path graph to construct a causal chain. By inferring the event triggering order, traffic correlation, and key node indicators (such as traffic volume, node centrality, and anomaly score), identify the attack starting point, key propagation nodes, and the final victim.

[0075] Step 5: When a new attack pattern is detected or the latest threat sample is received, the GNN model will be incrementally trained through an online fine-tuning mechanism without having to completely reconstruct the model structure.

[0076] Step six: Simultaneously, the threat intelligence matching strategy will also be dynamically adjusted. The system will optimize feature weights and matching thresholds based on the expansion of the intelligence database, ensuring that the intelligence matching always remains consistent with the latest intelligence feature distribution, thereby further improving the timeliness and accuracy of the results.

[0077] Step seven generates a structured source tracing report, which includes the source node of the abnormal traffic, propagation path, potential victims, event timeline, threat level, and handling recommendations. The results are then simultaneously transmitted to the visualization module for operations and maintenance personnel or security analysts to quickly review and respond, significantly improving the interpretability and response efficiency of the event.

[0078] The visualization module presents the results of abnormal traffic detection and source analysis to users in an intuitive, interactive, and scalable manner. This helps users efficiently grasp the trends and distribution of abnormal network traffic, and also improves the accuracy and convenience of analysis and decision-making.

[0079] Step one: The visualization module first receives multi-source data from the deep source analysis module, and performs data cleaning and format conversion during the access phase to ensure that the data meets the structured requirements for subsequent visualization presentation.

[0080] Step two: After the data preparation is completed, the module presents the data intuitively through dynamic charts. For example, a line chart can be used to show the trend of abnormal traffic over time, and a bar chart or pie chart can be used to show the distribution of different types of abnormalities.

[0081] Step three: Users can use the interactive functions provided by the system, such as zooming, filtering, and querying, to explore the visualization results from multiple dimensions, and flexibly customize the view according to business needs, so as to conduct more accurate analysis and comparison of specific abnormal behaviors.

[0082] Step four: The module supports generating detailed analysis reports, which include abnormal traffic detection results, source tracing analysis paths, affected assets and disposal recommendations, and provides export functions in multiple formats to facilitate users' archiving, reporting and subsequent in-depth analysis.

[0083] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A hardware-accelerated full-flow acquisition and intelligent analysis detection system, characterized in that, It includes a data acquisition and preprocessing module, an adaptive intelligent traffic profiling module, and an in-depth source tracing analysis module; The data acquisition and preprocessing module is used to acquire traffic with zero packet loss using network interface card and DPDK technology, and to perform protocol classification, noise filtering, duplicate data detection and feature extraction through FPGA and GPU. The acquired data is then transferred to host memory and converted into a format suitable for analysis. The adaptive intelligent traffic profiling module is used to label suspicious traffic in the data output by the data acquisition and preprocessing module using dynamically adjusted autoencoders and GANs, and to identify attack behaviors by combining multimodal methods such as deep learning, machine learning and rule matching. Then, the abnormal traffic where the attack behavior is located is sent to the deep source tracing analysis module. The deep source tracing analysis module is used to build network topology based on GNN. For the attack behavior analyzed by the adaptive intelligent traffic profiling module, it analyzes the propagation path and high-risk nodes of the abnormal traffic where the attack behavior is located, and combines threat intelligence to trace the attack source and generate a source tracing report. The data acquisition and preprocessing module includes a data acquisition layer, a hardware acceleration layer, and a data transmission layer; The specific implementation steps of the data acquisition layer function are as follows: Step 1: Deploy a network interface card at the gateway sending end to access network traffic in real time. The network interface card performs fragmentation on the access traffic, automatically dividing the traffic data block into multiple TCP segments according to the preset maximum segment size. At the same time, it dynamically generates and fills the TCP / IP header information of each segment and calculates the checksum. At the host receiving end, it aggregates multiple TCP data packets that arrive continuously. The network interface card performs sequence checks on data packets at the lower level to ensure that there are no packet losses or out-of-order packets. Then, multiple small packets belonging to the same data stream are merged into a complete, larger data segment, and a hardware timestamp is added before being handed over to the operating system protocol stack. Step 2: Utilize the multi-queue receiving-side scaling function of the network interface card to allocate traffic to different receiving queues based on the five-tuple characteristics of the traffic, and then distribute the traffic to different CPU cores for parallel processing. Step 3: After completing the queuing process, the network interface card is directly accessed through DPDK technology. At the same time, the traffic is classified according to basic protocols during the data capture process, and the classification results are attached as metadata. The hardware acceleration layer is internally divided into several sub-functions, including traffic classification, duplicate data detection, preliminary rule filtering, noise filtering, and feature extraction. The specific implementation steps of the hardware acceleration layer are as follows: Step 1, Traffic Classification and Rule Filtering: Relying on the lookup table and multi-level pipeline structure inside the FPGA, key information of the network layer and transport layer is parsed and matched in real time; during the parsing process, the legality of data packets is judged and filtered in combination with predefined blacklist and whitelist rules. Step 2, Duplicate Data Detection: First, on the FPGA side, the data packets are identified and historical duplicate traffic is filtered by comparing the hash function calculated based on the five-tuple and the message digest information; then, on the GPU side, a sliding time window combined with a hash table is introduced to perform in-depth comparison and elimination of duplicate packets within a short period of time. Step 3, Noise Filtering: Utilize the parallel computing capabilities of GPUs to achieve dynamic identification based on probabilistic models: By collecting samples of normal business traffic, construct a Gaussian mixture model with multi-dimensional features, and use CUDA to calculate the probability of newly entering traffic, thereby identifying abnormal behaviors that deviate significantly from the feature distribution and removing them as noise. Step 4, Feature Extraction: After completing the traffic cleaning, the parallel computing capabilities of the GPU are used to extract multi-dimensional structural and behavioral features from the data traffic. The extracted features include packet time, average flow rate, connection duration, request-response ratio, and rule matching and feature importance evaluation based on information gain are combined to improve the effectiveness and discriminativeness of feature representation. The data transfer layer transmits the data processed by the hardware acceleration layer to the host memory. The specific implementation steps of the data transfer layer are as follows: Step 1: The data preprocessed by the hardware acceleration layer is transmitted to the host memory in a low-latency, high-bandwidth manner via the PCIe high-speed bus. Step two: After the data enters the host memory, it is stored in multi-level data buffers to coordinate the difference between the transmission at the hardware layer and the processing capabilities on the host side. Step 3: Convert all cached and scheduled data into a standardized JSON format. The data acquisition and preprocessing module also includes a control and management layer. This layer supports user configuration of data acquisition rules, preprocessing parameters, and hardware acceleration strategies, monitors the operational status of the data acquisition module, and provides alarm functions. The specific implementation steps of the control and management layer are as follows: Continuously monitor key status indicators of the data acquisition and preprocessing module, including packet capture rate, preprocessing efficiency, and hardware resource utilization; trigger alarm functions when anomalies are detected, and intuitively feed back the anomalies to the user through logs, visual interfaces, or notification mechanisms. The adaptive intelligent traffic profiling module can identify and analyze abnormal network traffic, including an abnormal traffic labeling layer and a multimodal anomaly detection layer. The abnormal traffic labeling layer is used to perform unsupervised learning on historical benign traffic through autoencoders and generative adversarial networks, thereby capturing its complex patterns and feature distributions. During operation, the abnormal traffic labeling layer compares real-time traffic with the model output, marks suspicious traffic, and sends it to the multimodal anomaly detection layer. The multimodal anomaly detection layer is used to integrate deep learning, machine learning, and rule matching algorithms to analyze traffic features from multiple dimensions and achieve comprehensive anomaly detection. The implementation steps of the adaptive intelligent traffic profiling module are as follows: Step 1: Unsupervised modeling of historical positive traffic 11. Data collection and cleaning: Collect historical positive traffic data, perform noise reduction and preprocessing on the data, and remove abnormal or missing samples; 12. Autoencoder training: The cleaned traffic data is input into the autoencoder to learn the low-dimensional feature representation and intrinsic pattern of the traffic, and the original traffic is reconstructed using the decoder. The autoencoder uses the reconstruction error as the optimization target to learn the intrinsic pattern and low-dimensional feature distribution of the traffic, providing a benchmark feature space for anomaly detection.

13. GAN generation and optimization: The low-dimensional features extracted by the encoder are input into the generator to generate pseudo-samples that are similar to the distribution of real benign traffic features; The discriminator distinguishes between real samples and generated samples and continuously optimizes its discrimination ability; the generator, on the other hand, uses the feedback from the discriminator as the optimization target and adjusts its generation strategy to make the generated samples closer to the real benign traffic distribution. The generator and discriminator are trained alternately and iteratively to gradually improve the authenticity of the generated samples and the accuracy of the discrimination.

14. Dynamic Evolution and Adversarial Enhancement: The autoencoder provides the latest traffic feature representation, providing input for the generator to generate samples; the samples generated by GAN include normal benign samples as well as anomalous samples close to the normal boundary, the latter used to enhance the autoencoder's sensitivity to anomalous patterns. The generated samples are continuously added to the training set for incremental training of the autoencoder and GAN. Step 2: Real-time traffic anomaly labeling and adaptive updates 21. Reconstruction error verification: If the difference between the input traffic and the reconstruction result of the autoencoder, i.e., the reconstruction error, exceeds the suspicious threshold, then the input traffic is suspected of being abnormal.

22. Discriminator verification: The input traffic is then fed into the discriminator of the GAN. If the discriminator also gives a low confidence score, its anomaly is further confirmed.

23. Comprehensive judgment and marking of suspicious traffic: The reconstruction error and the discriminator feedback results are fused. If both point to anomalies, the traffic is directly marked as suspicious. If only one is abnormal, a weighted judgment is made based on the threshold. If both are normal, the traffic is considered benign.

24. Adaptive Model Update: The sliding window mechanism is used to incrementally train the model, enabling it to continuously adapt to changes in the traffic environment; an adaptive learning rate strategy is introduced, which increases the learning rate to accelerate convergence when the model error is large, and decreases the learning rate to improve accuracy when the error tends to stabilize, ensuring that the detection model is always in the best state. Step 3: Multimodal anomaly detection and feature extraction 31. Suspicious Traffic Transmission: Transmit traffic marked as suspicious in step two to the multimodal anomaly detection layer; 22. Multi-algorithm fusion detection: First, key information such as statistical features, time series features, and behavioral patterns are extracted from traffic data, and a deep learning model is used to automatically learn high-dimensional features. For time series features, LSTM is selected to capture time dependencies and calculate the anomaly probability of suspicious traffic. At the same time, the random forest method is used to calculate the anomaly probability of the statistical features of suspicious traffic. In addition, based on a rule matching engine, suspicious traffic is compared with the attack rule base to calculate the suspicious anomaly probability. Finally, the three calculation results are weighted and summed to obtain an anomaly score. Traffic with a score higher than the set anomaly traffic threshold is identified as abnormal traffic.

23. Anomaly Feature Extraction: Extract key features from traffic identified as abnormal, including source IP, destination IP, port number, and protocol type, to ensure the completeness of the source tracing analysis; 24. Manual Judgment of Suspicious and Abnormal Traffic: For traffic with a score lower than the set abnormal traffic threshold, the specific traffic data packet content is associated with the data for further manual analysis and determination of the attack type; The specific implementation steps of the deep source tracing analysis module are as follows: Step 1: First, based on the collected network device logs, boundary traffic records, and IP communication relationships, a heterogeneous network topology graph is constructed. Nodes represent IP addresses, host devices, ports, and asset types, while edges represent communication relationships or potential attack paths. To better reflect the contextual structure information of the nodes in the heterogeneous network topology graph, the Node2Vec algorithm is used for graph embedding. By simulating different random walk strategies, low-dimensional vector representations are generated, providing more discriminative feature input data for subsequent graph neural network analysis. Step 2: In the identification of abnormal propagation paths, a graph neural network is used to perform multi-layer feature aggregation on the topology graph, fusing historical behavioral information such as the communication frequency of nodes, changes in traffic direction, and information of neighboring nodes to generate a feature representation of the fused structural context. Automatically learn attack propagation patterns, capture complex interactions between nodes, identify the propagation trend of abnormal traffic in the topology graph, potential diffusion paths, potential high-risk jump points and victim assets, and construct a complete attack chain. Step 3: Compare the IP addresses and domain names appearing in the abnormal traffic with the local threat intelligence database to identify known attack behaviors and improve detection coverage; Step 4, reconstructing the attack trajectory over time:

41. Behavior Sequence Extraction: First, historical traffic data is grouped according to source IP, and behavior sequences of the same source IP in different time periods are extracted; each behavior sequence includes access target, access timestamp, protocol type, session duration, data volume, and connection frequency; an ordered behavior chain is constructed by sorting by time to provide a basic data structure for subsequent trajectory analysis; 42. Spatiotemporal Feature Fusion: The behavioral sequence is combined with the analysis results of the topology graph generated by GNN. Each node not only retains the network topology embedding vector, but also introduces time series features. The dynamic interaction relationship between nodes is modeled using spatiotemporal features, enabling the system to identify the propagation trend of attacks in the network structure over time.

43. Attack Path Construction: For each source IP behavior sequence, generate a preliminary attack path graph based on the access order and the relationship with GNN neighbors; The connections between nodes are weighted by attributes such as time interval, protocol type, and traffic volume to reflect the strength and directionality of the attack propagation.

44. Causal Relationship Chain Inference: Analyze the interaction order and dependencies of each node in the path graph to construct causal chains; By inferring the sequence of event triggers, traffic correlations, and key node indicators, the attack origin, key propagation nodes, and final victims can be identified. Step 5: When a new attack pattern is detected or the latest threat sample is received, the GNN model is incrementally trained through an online fine-tuning mechanism without having to completely reconstruct the model structure. Step 6: Dynamically adjust the threat intelligence matching strategy: Optimize feature weights and matching thresholds based on the expansion of the local threat intelligence database, so that the intelligence matching always remains consistent with the latest intelligence feature distribution; Step 7: Finally, a structured source tracing report is generated, which includes the source node of the abnormal traffic, the propagation path, potential victims, the timeline of the event, the threat level, and the handling recommendations. It also includes a visualization module, which presents the analysis results of the in-depth source tracing analysis module in the form of dynamic charts, allowing users to interactively analyze traffic trends and information through dynamic charts; The visualization module can present the results of abnormal traffic detection and source tracing analysis to users in an intuitive, interactive, and scalable manner; The specific implementation steps of the visualization module are as follows: Step 1: First, receive multi-source data from the deep source tracing analysis module, and perform data cleaning and format conversion during the access phase to ensure that the data meets the structured requirements for subsequent visualization presentation. Step 2: After the data preparation is completed, present it intuitively through dynamic charts. Line charts are used to show the trend of abnormal traffic over time, and bar charts or pie charts are used to show the distribution of different types of abnormalities. Step 3: Generate an analysis report, which includes abnormal traffic detection results, source tracing analysis path, affected assets, and disposal recommendations.

Images