Communication network security situation prediction method and system based on big data

Abstract

The present invention relates to the field of digital information transmission technology and proposes a communication network security situation prediction method and system based on big data. In terms of data fusion, it breaks through the limitations of single data and integrates multi-source heterogeneous data of 4 major categories and 12 subcategories. Combined with a dynamic weighting mechanism, feature extraction is more comprehensive and accurate. In terms of model architecture, CNN-Bi-LSTM-Attention three-level fusion and PSO optimization are adopted to effectively capture spatiotemporal features, improve convergence speed, achieve high prediction accuracy and low false alarm rate. In terms of risk assessment, a hierarchical assessment system is constructed, combining dynamic thresholds with Monte Carlo simulation to achieve early warning. The model training module innovatively adopts a federated learning model to ensure data privacy while improving the model's generalization ability. The predictive analysis module deploys an optimized hybrid model to support high-concurrency predictions with short response time. The visual decision module provides three-dimensional visualization and geographic drill-down functions, and the output data can be seamlessly connected to third-party platforms.

Description

Technical Field

[0001] The present invention relates to the technical field of digital information transmission, and in particular to a communication network security situation prediction method and system based on big data. Background Art

[0002] In the field of communication networks, with the rapid development of information technology, network scale continues to expand, the architecture becomes increasingly complex, and security situation prediction faces unprecedented challenges.

[0003] Traditional security situation prediction methods often rely on limited data sources, making it difficult to capture the full network picture. For example, some solutions focus solely on network traffic data, ignoring device logs, external threat intelligence, and user behavior. This results in a lack of detection for emerging threats like zero-day attacks, with detection accuracy rates as low as 58.3%. Furthermore, these solutions often use static weighting for data fusion, making them unable to adapt to dynamically changing threat scenarios.

[0004] In terms of time series modeling, commonly used fixed time window extraction methods, such as 24-hour windows, are unable to effectively capture long-term abnormal behavior patterns, resulting in a false alarm rate as high as 18.7% during sudden DDoS attacks. Furthermore, model parameter adjustment relies on manually set thresholds, resulting in response delays exceeding 30 minutes for new protocol attacks, making it difficult to cope with complex and dynamic network attacks. Furthermore, existing technologies for multi-source data fusion remain at the simple splicing level, with insufficient feature correlation mining. Time series modeling lacks the ability to capture bidirectional temporal dependencies, and the model exhibits poor adaptability, preventing real-time optimization based on network changes.

[0005] Therefore, a communication network security situation prediction method and system based on big data is proposed to build a comprehensive, intelligent and adaptive communication network security situation prediction system to fill the gap in existing technologies and meet the growing demand for network security protection. Summary of the Invention

[0006] Technical problems solved

[0007] When fusing multi-source data, existing technologies only stay at the simple splicing level, and feature correlation mining is insufficient; time series modeling lacks the ability to capture bidirectional time dependencies; the model has poor adaptability and cannot be optimized in real time according to network changes.

[0008] Technical Solution

[0009] To achieve the above-mentioned solution, the present invention provides the following technical solution: a communication network security situation prediction method based on big data, which is applied to real-time security risk assessment and accurate early warning of communication networks, comprising the following steps:

[0010] Step 1: Multi-source heterogeneous data collection and standardization: Collect data from 4 categories and 12 subcategories across the network layer, device layer, intelligence layer, and behavior layer. Use Apache Avro to define a unified data format, achieve end-to-end transmission delay of ≤50ms through the Kafka message queue, and use DS evidence theory to fuse conflicting data.

[0011] Step 2: Dynamic feature engineering and weight assignment: Generate 86-dimensional raw features, including traffic entropy, TCP connection reset rate, and vulnerability CVSS score. Use a bidirectional long short-term memory network to model temporal dependencies. Input the feature sequence of the previous 72 hours with a time step of 15 minutes, and output a 64-dimensional time series vector. Based on the Transformer multi-head attention mechanism, dynamically assign feature weights to highlight key indicators such as abnormal traffic growth rate and the probability of high-risk vulnerability exploitation.

[0012] Step 3: CNN-Bi-LSTM-Attention hybrid model prediction: Build a three-level fusion architecture; use the particle swarm algorithm to dynamically optimize hyperparameters, with a learning rate range of 0.0005-0.002, the number of attention heads of 4-16, and the LSTM hidden layer dimension of 128-512. The validation set AUC value reaches 0.953.

[0013] Step 4: Hierarchical risk assessment and intelligent early warning: A three-level assessment system is constructed using the analytic hierarchy process combined with the entropy weight method, with asset value weighting 0.25, threat level weighting 0.40, and vulnerability weighting 0.35. Real-time anomaly detection dynamically adjusts the threshold confidence level to 95% based on the 3σ principle combined with Bayesian optimization. The trend prediction module uses Monte Carlo simulation to generate the risk probability distribution for the next 72 hours, providing a four-hour advance warning of high-risk attacks.

[0014] Preferably, the multi-source heterogeneous data collection also includes: Modbus / TCP, OPCUA, and S7 communication protocol analysis in industrial Internet scenarios; MFA verification results and login IP geolocation information of user authentication logs; API call frequency and sensitive data access path records of privileged account operation traces.

[0015] Preferably, in the dynamic feature engineering, the flow entropy value calculation The probability of occurrence of a protocol type within a specific time window; the temporal feature enhancement uses a 256-dimensional hidden layer to capture bidirectional behavior patterns; the Transformer mechanism uses a feature mapping function 、 Calculate correlation, key vector dimensions is 64.

[0016] Preferably, the fusion architecture includes 8 5×5 convolution kernels to extract the spatial features of traffic packet payloads, the middle bidirectional LSTM layer outputs 128-dimensional temporal features, and the top attention enhancement layer combines contextual information such as asset vulnerability scores to output the risk level for the next 24 hours in the range of 1-10 with a resolution of 0.5.

[0017] Preferably, in the hybrid model prediction: the convolution layer generates a 200-dimensional feature map through the ReLU activation function to identify the patterned features of the HTTP request header anomaly; the attention enhancement layer combines the geographical entropy value of the attack source to reflect the degree of geographical dispersion of the attack IP and optimize the weight distribution.

[0018] Preferably, in the hierarchical risk assessment: the asset value weight considers the business impact and data sensitivity level through the fuzzy comprehensive evaluation method; the threat level weight is based on the Bayesian probability calculation of the attack frequency and vulnerability exploitation maturity in the attack chain stage; the vulnerability weight is determined by the baseline check tool score and the vulnerability scanning results to determine the patch missing rate and configuration compliance.

[0019] The communication network security situation prediction system based on big data is applied to the communication network security situation prediction method based on big data as described above. The system includes the following modules:

[0020] Data acquisition module: Deploys Flume distributed acquisition nodes, with a single node supporting 10Gbps line-speed processing; integrates an industrial protocol parsing submodule to support Modbus / TCP, OPCUA, and S7 communication protocols.

[0021] Data processing module: This module implements millisecond-level feature extraction based on the Flink stream processing framework, including data cleaning, format conversion, and protocol parsing functions. It uses the Apache Avro format to store standardized data and transmits it through the Kafka message queue.

[0022] Model training module: supports federated learning mode, local processing of raw data, and parameter aggregation through secure multi-party computing; compatible with TensorFlow / PyTorch framework, and supports GPU cluster accelerated training;

[0023] Prediction and Analysis Module: Deploys a CNN-Bi-LSTM-Attention hybrid model to provide real-time risk scoring, attack chain tracing, and trend prediction; supports 5,000+ concurrent prediction requests with a response time of ≤100ms.

[0024] Visualization decision module: Develop a three-dimensional visualization interface, including a network asset risk heat map, a real-time attack traffic dynamic map, and a historical situation comparison analysis module; provide a RESTful API to output risk data to a third-party security management platform.

[0025] Preferably, the data collection module also includes: a mainstream device log analysis module; a CNVD vulnerability patch status real-time synchronization interface, which includes vulnerability level and affected asset range information.

[0026] Preferably, in the model training module: the federated learning mode ensures data privacy through secure multi-party computing; the PSO algorithm dynamically adjusts hyperparameters, and the model convergence speed is increased by 70% compared with traditional grid search.

[0027] Preferably, in the visualization decision module: the risk heat map uses color gradient to intuitively display the asset risk distribution; the attack traffic dynamic map marks the characteristics of abnormal port access and TCP flag combination attack in real time.

[0028] Beneficial effects

[0029] Compared with the existing technology, the present invention provides a communication network security situation prediction method and system based on big data, which has the following beneficial effects:

[0030] 1. This big data-based communication network security situation prediction method breaks through the limitations of single data in terms of data fusion, integrating multi-source heterogeneous data from four categories and 12 subcategories, and combining it with a dynamic weighting mechanism to make feature extraction more comprehensive and accurate. In terms of model architecture, it adopts a three-level CNN-Bi-LSTM-Attention fusion and PSO optimization to effectively capture spatiotemporal features, improve convergence speed, and achieve high prediction accuracy and low false alarm rate. In terms of risk assessment, it constructs a hierarchical assessment system, combining dynamic thresholds and Monte Carlo simulation to achieve early warning, providing an efficient, intelligent, and accurate solution for communication network security situation prediction.

[0031] 2. This big data-based communication network security situation prediction system has a data acquisition module that supports industrial protocol parsing and high-speed multi-source data collection, breaking through the limitations of a single data source in traditional systems. The data processing module uses Flink to implement millisecond-level feature extraction, significantly improving timeliness compared to traditional batch processing. The model training module innovatively adopts a federated learning model to ensure data privacy while improving model generalization capabilities. The predictive analysis module deploys an optimized hybrid model to support high-concurrency predictions with short response times. The visual decision-making module provides three-dimensional visualization and geographic drill-down capabilities, and the output data can be seamlessly connected to third-party platforms, building an efficient, secure, and intelligent network security situation prediction system. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of the method flow of the present invention;

[0033] Figure 2 Schematic diagram of the system framework of the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0035] See also Figures 1 and 2 The present invention proposes a communication network security situation prediction method and system based on big data, including the following contents:

[0036] 1. A communication network security situation prediction method based on big data includes the following steps:

[0037] 1. Multi-source heterogeneous data collection and standardization

[0038] 1.1 Data Source Coverage: A data source system has been established, covering the network layer, device layer, intelligence layer, and behavior layer, encompassing four major categories and 12 subcategories of data. At the network layer, the collection of NetFlowv9 flow data is crucial. It contains 32 basic attributes, such as the five-tuple (source IP, destination IP, protocol, source port, destination port), byte rate, TCP flags, and IP fragmentation rate. These attributes provide essential data for network behavior analysis. Furthermore, SDN controller configuration logs, synchronized every five minutes, provide timely information on network topology changes. At the device layer, firewall session logs are processed at a rate of ≥2 million records per second and support log parsing for a variety of mainstream devices, including Huawei USG6000 and Sangfor AF. This provides a wealth of information related to device operation and security. Furthermore, IPS intrusion event records are included, including attack signatures, CVE numbers, and attack chain stages, to help accurately identify attack behavior. The intelligence layer synchronizes VirusTotal hash reputation data in real time via an API, with an update frequency of 10 seconds. This platform covers 98% of global malware sample libraries and provides rapid feedback on a file's security reputation. It also obtains CNVD vulnerability patch status, including vulnerability level, scope of affected assets, and remediation recommendations, which are crucial for assessing network vulnerabilities. The behavioral layer collects user authentication logs, including MFA (multi-factor authentication) verification results and login IP geolocation information, which can be used to analyze user login behavior for anomalies. Privileged account operation traces record API call frequency and sensitive data access paths, providing a basis for detecting potential risks from privileged accounts.

[0039] 1.2 Data Standardization: To ensure effective fusion and analysis of multi-source data, Apache Avro was adopted to define a unified data format. Kafka message queues were used to enable low-latency access to multi-source data, with end-to-end transmission latency ≤ 50ms, ensuring timely data transmission. Conflicting data was fused using the DS evidence theory, which effectively resolves data conflicts by comprehensively processing evidence provided by different data sources. The data cleansing accuracy reached 98.9%, laying a solid foundation for subsequent precise analysis.

[0040] 2. Dynamic Feature Engineering and Weight Allocation

[0041] 2.1 Basic feature extraction: This step generates 86-dimensional original features, which reflect the network security situation from multiple angles. In the traffic feature, the traffic entropy value is calculated by the formula Calculated, where It represents the probability of occurrence of each protocol type within a specific time window. This formula is used to measure the degree of disorder in the distribution of protocol types in traffic. The higher the entropy value, the more complex the protocol types in the traffic, and the greater the potential risk. In addition, attention is also paid to features such as the TCP connection reset rate and the frequency of abnormal port access. Device characteristics include the number of firewall policy hits, which reflects the execution of firewall rules; the IPS rule matching delay, which can be used to evaluate the response speed of the intrusion detection system; the device CPU utilization abnormal fluctuation coefficient, which can help determine whether the device has performance fluctuations due to attacks or abnormal loads. In terms of intelligence features, the vulnerability CVSS (Common Vulnerability Scoring System) score directly reflects the severity of the vulnerability; the threat intelligence confidence is based on the cross-validation results of multi-source intelligence, providing a quantitative indicator for the reliability of the intelligence; the attack source IP reputation level (level 1-10, data from the MISP threat sharing platform) can quickly determine the danger level of the attack source.

[0042] 2.2 Temporal Feature Enhancement: To better capture the temporal characteristics of the data, a bidirectional long short-term memory (Bi-LSTM) network is used to model time series dependencies. The feature sequence of the previous 72 hours is input, with a time step set to 15 minutes. A 256-dimensional hidden layer is used to capture both forward (past to present) and reverse (present to past) behavior patterns. Unlike traditional LSTMs, which only capture unidirectional temporal dependencies, Bi-LSTMs can more comprehensively mine temporal information from the data, outputting a 64-dimensional time series vector containing temporal context. This effectively addresses the information loss problem caused by unidirectional dependencies in traditional LSTMs and provides richer temporal dimension features for subsequent analysis.

[0043] 2.3 Dynamic Weight Allocation: Feature weights are calculated based on the Transformer multi-head attention mechanism (8 attention heads). The formula is: .in, For the dimensional features, 、 are feature mapping functions, which map input features to different vector spaces and are used to calculate the correlation between features. The key vector dimension influences the scale of attention calculations. In this solution, the Transformer mechanism dynamically weights different feature dimensions, highlighting the impact of key indicators such as abnormal traffic growth rate (weight can be dynamically increased to 0.35) and the probability of high-risk vulnerability exploitation (weight 0.28). Compared to traditional fixed-weight methods, the Transformer can adaptively adjust the weight of each feature based on the real-time data situation, more accurately reflecting the contribution of different features to the network security situation at different times.

[0044] 3. CNN-Bi-LSTM-Attention Hybrid Model Prediction

[0045] 3.1 Model Architecture Design: This solution adopts an innovative three-level fusion architecture. The bottom convolutional layer uses 128 5×5 convolution kernels to extract spatial features from the traffic packet payload and generates a 200-dimensional feature map through the ReLU activation function. The convolution kernel can automatically learn local patterns in traffic packets, such as HTTP request header anomalies, TCP flag combination attacks, and other patterned features. These features are crucial for identifying common types of network attacks. The middle-level bidirectional LSTM layer receives the feature sequence output by the convolutional layer and simultaneously learns the dependencies between past and future time steps through a bidirectional recursive structure, outputting 128-dimensional time series features. This structure can effectively identify periodic attacks, such as brute force cracking attempts at dawn, as well as long-term latent threats, such as vulnerability scanning activities that last for a week. The top-level attention enhancement layer dynamically weights the temporal features output by the Bi-LSTM, combining them with contextual information such as the asset vulnerability score (derived from vulnerability scanners) and the attack source geographic entropy (reflecting the geographical dispersion of attacking IP addresses). It ultimately outputs a 24-hour risk level on a scale of 1-10 with a resolution of 0.5, enabling quantitative prediction of security trends. This layered fusion architecture allows the model to fully leverage features at different levels, improving prediction accuracy and reliability.

[0046] 3.2 Adaptive Optimization Mechanism: To improve model performance, a particle swarm optimization (PSO) algorithm was introduced to dynamically adjust model hyperparameters. The learning rate search range was set to 0.0005-0.002. The learning rate determines the step size of parameter updates during model training. An appropriate learning rate enables the model to converge to the optimal solution more quickly. The number of attention heads was dynamically optimized between 4 and 16. This number affects the model's focus on different feature dimensions, and dynamic adjustment allows for better adaptation to diverse data characteristics. The LSTM hidden layer dimension ranged from 128 to 512, which determines the LSTM model's memory and expressive power. These ranges were determined based on extensive experiments and analysis of various cybersecurity scenarios. Using the PSO algorithm to search for the optimal parameter combination within these ranges, the model converged 70% faster than traditional grid search, achieving an AUC of 0.953 on the validation set, significantly improving model training efficiency and prediction accuracy.

[0047] 4. Layered risk assessment and intelligent early warning

[0048] 4.1 Assessment System Construction: A three-tiered assessment system was constructed using the Analytic Hierarchy Process (AHP) combined with the entropy weighting method. The weights of the first-tier indicators were dynamically calibrated using expert scoring and data entropy. Asset value was weighted at 0.25, and business impact and data sensitivity were considered using a fuzzy comprehensive evaluation method. Data was sourced from the asset CMDB system. Business impact assessed the impact of an attack on business operations, while data sensitivity reflected the sensitivity of the data held by the asset. Threat level was weighted at 0.40, and attack frequency and vulnerability exploitation maturity were calculated based on Bayesian probabilities at each stage of the attack chain. Data was sourced from intrusion detection systems and Common Vulnerability Databases. Analysis of attack chain stages facilitated a more accurate assessment of threat development. Bayesian probabilities combined prior knowledge with new evidence to update estimates of event probability. Vulnerability was weighted at 0.35, and patch missing rates and configuration compliance were determined by combining baseline check tool scores with vulnerability scan results. Data was sourced from vulnerability scanners and configuration audit systems. This comprehensive assessment system, which considers multiple factors, provides a more comprehensive and accurate assessment of network security posture.

[0049] 4.2 Early Warning Strategy Design: Real-time anomaly detection dynamically adjusts thresholds based on the 3σ principle combined with Bayesian optimization. The 3σ principle is a statistically based method used to determine whether data is abnormal. When the real-time risk score exceeds the dynamic threshold (95% confidence level), a multi-level early warning is triggered, including email, SMS, and API callback, with a false alarm rate controlled below 2.5%. The trend prediction module uses Monte Carlo simulation to generate the risk probability distribution for the next 72 hours, providing a 4-hour advance warning of high-risk attacks with an accuracy rate of ≥90%. Monte Carlo simulation simulates possible future risk scenarios through multiple random samplings, providing attack path tracing with accuracy down to the specific IP address and port, providing detailed and timely support for security decision-making.

[0050] 2. A communication network security situation prediction system based on big data applied to the above method includes the following modules:

[0051] 1. Data Acquisition Module: This module deploys Flume distributed acquisition nodes, with a single node supporting 10Gbps line rate processing and providing efficient data acquisition capabilities. It integrates an industrial protocol parsing submodule, supporting Modbus / TCP, OPC UA, and S7 communication protocols, specifically addressing the specific needs of industrial internet scenarios. These protocols are widely used for inter-device communication within the industrial internet. By parsing them, critical information such as the operating status and control instructions of industrial equipment can be obtained, providing important data support for security analysis of the industrial internet.

[0052] 2. Data Processing Module: This module implements millisecond-level feature extraction based on the Flink stream processing framework. It includes functional units such as data cleaning, format conversion, and protocol parsing. The Flink framework's efficient stream processing capabilities enable rapid processing of massive amounts of collected data. Data cleaning removes noisy and duplicate data. Format conversion unifies data from different sources into a format suitable for subsequent analysis. Protocol parsing further analyzes the data content, extracting valuable features and providing high-quality data for model training.

[0053] 3. Model Training Module: Supports federated learning, localizes raw data processing, and aggregates parameters through secure multi-party computation to ensure data privacy. In scenarios where data privacy is crucial, federated learning allows all participants to jointly train models without sharing raw data. Compatible with TensorFlow / PyTorch frameworks and supporting GPU cluster-accelerated training, it leverages the strengths of existing mainstream deep learning frameworks and improves model training efficiency through GPU clusters, enabling rapid iterative optimization of models.

[0054] 4. Predictive Analysis Module: This module deploys an optimized hybrid predictive model, providing core functions such as real-time risk scoring, attack chain tracing, and trend prediction, supporting over 5,000 concurrent prediction requests. Real-time risk scoring provides rapid feedback on the current network security status, attack chain tracing helps analyze the path and source of attacks, and trend prediction provides forward-looking information on future security trends. Its high concurrent processing capability meets the real-time requirements of large-scale network environments.

[0055] 5. Visual Decision-Making Module: Develop a three-dimensional visualization interface, including a network asset risk heat map and support for geographic information drill-down, which can intuitively display the distribution of asset risks. Different colors and heat levels are used to indicate the degree of risk, and users can use the geographic information drill-down function to gain in-depth understanding of the asset risk situation in a specific area. A real-time attack traffic dynamic graph, accurate to the port level, clearly displays the attack traffic situation, helping users to promptly detect abnormal traffic. A historical situation comparison and analysis module facilitates users to understand the changing trends of the security situation. It also supports RESTful API to output risk data to third-party security management platforms, facilitating integration with other systems and providing comprehensive data support and intuitive display methods for security decision-making.

[0056] System operation process: First, the data acquisition module collects multi-source heterogeneous data from the network layer, device layer, intelligence layer, and behavior layer through Flume distributed collection nodes, and transmits it to the data processing layer via Kafka message queues. In the data processing module, the Flink stream processing framework is used for data cleansing, format conversion, and protocol parsing to generate standardized data. Next, the processed data is input into the model training module. In federated learning mode, the PSO algorithm dynamically adjusts the hyperparameters of the CNN-Bi-LSTM-Attention hybrid model for training. The trained model is deployed in the predictive analysis module, which processes real-time data and outputs risk scores, attack chain tracing, and trend prediction results. Finally, the visualization decision module displays these results in an intuitive 3D visualization interface and can export data to third-party platforms via a RESTful API to support security decision-making.

[0057] 3. Industry Implementation Cases

[0058] Example 1: Telecom 5G Core Network Security Situation Prediction

[0059] Deployment Environment: This implementation utilizes NetFlow data from 200 5G base stations, processing an average daily volume of 120 TB. With a 10-second time window, it accurately captures real-time traffic changes within the 5G network. It also integrates logs from Huawei NE5000E routers, processing 500,000 records per second and providing ample data for analyzing network device operational status. The model training cluster utilizes eight NVIDIA A100 servers, leveraging their powerful computing power to accelerate model training. Deployed using TensorFlow Serving, it supports incremental training with minute-by-minute model updates, ensuring the model can quickly adapt to changes in the network environment.

[0060] Technical Implementation: Utilizing the documented multi-source, heterogeneous data collection solution, we comprehensively collect NetFlow data at the network layer and router logs at the device layer. During data standardization, we use Apache Avro to define a unified format, enabling low-latency transmission through Kafka message queues to ensure timely data processing. Dynamic feature engineering involves calculating 86-dimensional raw features, such as traffic entropy. A Bi-LSTM network is applied to the previous 72 hours of feature sequences (with a 15-minute time step). This network outputs a 64-dimensional time series vector after passing through a 256-dimensional hidden layer, capturing bidirectional temporal dependencies. Feature weights are calculated using a Transformer multi-head attention mechanism (8 attention heads) to highlight key indicators such as abnormal traffic growth rates. A CNN-Bi-LSTM-Attention hybrid model uses 128 5×5 convolution kernels at the bottom layer to extract spatial features of traffic packets. A bidirectional LSTM at the middle layer learns temporal features. The top attention enhancement layer combines information such as asset vulnerability scores to output risk ratings. A particle swarm algorithm is introduced to dynamically adjust hyperparameters such as the learning rate (0.0005-0.002) and the number of attention heads (4-16).

[0061] Application Results: This system achieves 4.2 hours of lead time for DDoS attack precursor detection on 5G slicing networks, an 8.4-fold improvement compared to existing solutions, significantly increasing attack response preparation time. On the CICIDS 2017 dataset, the F1 score for unknown attack detection reached 0.89, a 22% improvement over traditional LSTM models, effectively enhancing its ability to identify unknown threats. The system has passed ETSI NFVISG security certification, meeting the stringent 3GPP TS 33.501 security requirements for 5G networks, demonstrating its safety and reliability in real-world applications.

[0062] Example 2: Application of Federated Learning in Financial Transaction Networks

[0063] Deployment Environment: A federated learning network spanning 15 branches was built to ensure that raw transaction data remained local, using secure multi-party computing technology to protect data privacy. This distributed architecture integrated data resources from various branches, providing a rich data source for model training.

[0064] Technical Implementation: Multi-source heterogeneous data collection covers behavioral data such as user authentication logs and privileged account operation traces, as well as external threat intelligence data. During data processing, Apache Avro and Kafka are used for standardization and transmission. Dynamic feature engineering adds new financial-specific features, such as the coefficient of variation of single transaction amounts, which reflects abnormal fluctuations in transaction amounts, and the IP-region jump frequency, which reflects the number of IP logins across regions per unit time, to construct a 142-dimensional financial industry feature space. The Transformer multi-head attention mechanism is used to calculate feature weights and highlight key financial risk characteristics. Prediction is performed using a CNN-Bi-LSTM-Attention hybrid model, combined with a particle swarm algorithm to optimize hyperparameters. Furthermore, the system supports a federated learning model, with each branch's raw data processed locally and parameters aggregated through secure multi-party computing.

[0065] Application Results: Model parameter update latency is ≤50ms, meeting GDPR data compliance requirements and ensuring data security and privacy. During trading hours (9:00-17:00), real-time prediction accuracy reached 94.8%, successfully intercepting a new phishing attack chain. The complete attack path, from phishing email to database scraping, was identified in less than 150ms, effectively protecting financial transaction security. Accurate situation predictions reduced risk reserves by 45%, significantly optimizing risk control strategies and reducing operating costs.

[0066] Example 3: Industrial Internet Production Line Security Protection

[0067] Deployment Environment: Deploy specialized data collection nodes and data processing equipment for industrial Internet production lines. At the data collection layer, integrate industrial protocol parsing submodules to support industrial protocols such as Modbus / TCP and OPCUA, meeting the communication data collection needs of industrial equipment.

[0068] Technical Implementation: Collect operational data from industrial equipment, such as Modbus / TCP register read and write data, and device communication cycles. The parser developed using JNetPcap supports 10Gbps line rate processing, with a parsing delay of ≤10μs, and can identify over 80 network protocols, including industry-specific protocols. A three-level feature processing architecture is constructed to calculate basic features, such as the coefficient of abnormal fluctuation in device CPU utilization. Time series features are enhanced using a Bi-LSTM network, and weights are dynamically assigned using the Transformer multi-head attention mechanism. An industrial protocol deep parsing module is developed to support Modbus / TCP register read and write anomaly detection, with a parsing delay of <1ms, capable of quickly identifying unauthorized access to the PLC programming port. A device fingerprint library is constructed, containing fingerprints of over 2,000 industrial controllers. Feature dimensions include firmware hash value, communication cycle jitter rate, and instruction set checksum, for detecting abnormal device behavior.

[0069] Application Results: The system achieved a 92.1% accuracy rate in detecting abnormal device behavior, effectively identifying abnormal device operations. The system significantly reduced the response time for production line cybersecurity incidents from 58 minutes to 3.5 minutes, shifting from post-event response to pre-event prevention and ensuring the continuity and stability of industrial production. The system has also achieved IIC Industrial Internet Consortium security certification, meeting the ISO / IEC 62443 requirements for industrial control system security, providing strong assurance for the security of industrial Internet production lines.

[0070] 4. Experimental Data

[0071] 1. Benchmark Dataset Comparison Experiment (Test Set: CSE-CIC-IDS2018)

[0072]

[0073] 2. Comparison of parameter optimization effects

[0074]

[0075] 3. Prediction delay test in different scenarios

[0076]

[0077] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A communication network security situation prediction method based on big data, applied to real-time security risk assessment and accurate early warning of communication networks, characterized by: The following steps are involved: Step 1: Multi-source heterogeneous data collection and standardization: Collect data from four categories and 12 subcategories across the network layer, device layer, intelligence layer, and behavior layer. Use Apache Avro to define a unified data format, achieve end-to-end transmission latency of ≤50ms through Kafka message queues, and fuse conflicting data using DS evidence theory. Step 2: Dynamic Feature Engineering and Weight Assignment: Generate 86-dimensional raw features, including traffic entropy, TCP connection reset rate, and vulnerability CVSS score. Use a bidirectional long short-term memory network to model temporal dependencies. Input the feature sequence of the previous 72 hours with a time step of 15 minutes, and output a 64-dimensional time series vector. Based on the Transformer multi-head attention mechanism, dynamically assign feature weights, highlighting key indicators such as abnormal traffic growth rate and the probability of high-risk vulnerability exploitation. Step 3: CNN-Bi-LSTM-Attention hybrid model prediction: A three-level fusion architecture is constructed. The bottom convolutional layer uses eight 5×5 convolution kernels to extract the spatial features of the traffic packet payload and generates a 200-dimensional feature map through the ReLU activation function. The middle bidirectional LSTM layer outputs 128-dimensional time series features. The top-level attention enhancement layer combines contextual information such as asset vulnerability scores and the geographic entropy of the attack source to output a risk level for the next 24 hours, ranging from 1 to 10 with a resolution of 0.

5. The particle swarm algorithm dynamically optimizes hyperparameters, with a learning rate range of 0.0005-0.002, the number of attention heads from 4 to 16, and the LSTM hidden layer dimension from 128 to 512. The validation set AUC value reached 0.

953. Step 4. Hierarchical risk assessment and intelligent early warning: A three-level assessment system is constructed using the hierarchical analysis method combined with the entropy weight method. The asset value weight is 0.25, considering the business impact and data sensitivity level through the fuzzy comprehensive evaluation method. The threat level weight is 0.40, based on the Bayesian probability calculation of the attack frequency and vulnerability exploitation maturity at the attack chain stage. The vulnerability weight is 0.35, determining the patch missing rate and configuration compliance based on the baseline inspection tool score and vulnerability scanning results. Real-time anomaly detection dynamically adjusts the threshold confidence level to 95% based on the 3σ principle combined with Bayesian optimization. The trend prediction module uses Monte Carlo simulation to generate the risk probability distribution for the next 72 hours, providing an early warning of high-risk attacks 4 hours in advance.

2. The communication network security situation prediction method based on big data according to claim 1 is characterized by: The multi-source heterogeneous data collection also includes: Modbus / TCP, OPCUA, and S7 communication protocol analysis in industrial Internet scenarios; MFA verification results and login IP geolocation information in user authentication logs; API call frequency and sensitive data access path records of privileged account operation traces.

3. The communication network security situation prediction method based on big data according to claim 1 is characterized by: In the dynamic feature engineering, the traffic entropy value calculation The probability of occurrence of a protocol type within a specific time window; the temporal feature enhancement uses a 256-dimensional hidden layer to capture bidirectional behavior patterns; the Transformer mechanism uses a feature mapping function 、 Calculate correlation, key vector dimensions is 64.

4. The communication network security situation prediction method based on big data according to claim 1 is characterized by: The fusion architecture includes 8 5×5 convolution kernels to extract the spatial features of traffic packet payloads, the middle bidirectional LSTM layer outputs 128-dimensional temporal features, and the top attention enhancement layer combines contextual information such as asset vulnerability scores to output the risk level for the next 24 hours in the range of 1-10 with a resolution of 0.

5.

5. The method for predicting communication network security situation based on big data according to claim 1, characterized in that: In the hybrid model prediction, the convolutional layer generates a 200-dimensional feature map through the ReLU activation function to identify the pattern features of HTTP request header anomalies; The attention enhancement layer combines the geographical entropy of the attack source to reflect the geographical dispersion of the attacking IP and optimize the weight distribution.

6. The method for predicting communication network security situation based on big data according to claim 1, characterized in that: In the hierarchical risk assessment, asset value weights are evaluated using a fuzzy comprehensive evaluation method to consider business impact and data sensitivity. Threat level weights are calculated based on the Bayesian probability of the attack chain stage to calculate attack frequency and vulnerability exploitation maturity. Vulnerability weights are determined by baseline check tool scores and vulnerability scan results to determine patch missing rates and configuration compliance.

7. A communication network security situation prediction system based on big data, characterized by: The system is applied to the communication network security situation prediction method based on big data as described in any one of claims 1 to 6 above, and the system includes the following modules: Data acquisition module: Deploy Flume distributed acquisition nodes, with a single node supporting 10Gbps line-speed processing; integrate industrial protocol analysis submodules, supporting Modbus / TCP, OPCUA, and S7 communication protocols; Data processing module: This module implements millisecond-level feature extraction based on the Flink stream processing framework, including data cleaning, format conversion, and protocol parsing. It uses the Apache Avro format to store standardized data and transmits it through the Kafka message queue. Model training module: supports federated learning mode, local processing of raw data, and aggregation of parameters through secure multi-party computing; Compatible with TensorFlow / PyTorch framework and supports GPU cluster accelerated training; Prediction and Analysis Module: Deploys a CNN-Bi-LSTM-Attention hybrid model to provide real-time risk scoring, attack chain tracing, and trend prediction; supports 5,000+ concurrent prediction requests with a response time of ≤100ms; Visualization decision module: Develop a three-dimensional visualization interface, including a network asset risk heat map, a real-time attack traffic dynamic map, and a historical situation comparison and analysis module; Provide RESTful API to output risk data to third-party security management platforms.

8. The communication network security situation prediction system based on big data according to claim 7 is characterized in that: The data collection module also includes: a mainstream device log analysis module; a CNVD vulnerability patch status real-time synchronization interface, which includes vulnerability level and affected asset range information.

9. The communication network security situation prediction system based on big data according to claim 7, characterized in that: In the model training module: the federated learning mode ensures data privacy through secure multi-party computing; the PSO algorithm dynamically adjusts hyperparameters, and the model convergence speed is 70% faster than traditional grid search.

10. The communication network security situation prediction system based on big data according to claim 7, characterized in that: In the visualization decision module: the risk heat map uses color gradients to intuitively display the asset risk distribution; the attack traffic dynamic map marks the characteristics of abnormal port access and TCP flag combination attacks in real time.

Images