Communication network security situation prediction method and system based on big data

Abstract

The invention relates to the technical field of digital information transmission, and provides a communication network security situation prediction method and system based on big data, which break through the limitation of single data in the aspect of data fusion, integrate four large classes and 12 subclasses of multi-source heterogeneous data, and combine a dynamic weighting mechanism to make feature extraction more comprehensive and accurate; on the aspect of model architecture, CNN-Bi-LSTM-Attention three-level fusion and PSO optimization are adopted, spatial and temporal features are effectively captured, the convergence speed is increased, the prediction accuracy is high, and the false alarm rate is low; in the risk assessment aspect, a layered assessment system is constructed, and early warning is realized in combination with a dynamic threshold value and Monte Carlo simulation; the model training module innovatively adopts a federated learning mode, and the model generalization ability is improved while data privacy is guaranteed; the prediction analysis module deploys an optimization hybrid model, supports high-concurrency prediction and is short in response time; the visual decision-making module provides three-dimensional visualization and geographical drilling functions, and output data can be seamlessly connected with a third-party platform.

Description

Technical Field

[0001] The present invention relates to the technical field of digital information transmission, and in particular to a communication network security situation prediction method and system based on big data. Background Art

[0002] In the field of communication networks, with the rapid development of information technology, the scale of networks continues to expand, the architecture becomes increasingly complex, and security situation prediction faces unprecedented challenges.

[0003] Traditional security situation prediction methods mostly rely on limited data sources and are unable to capture the full picture of the network. For example, some solutions only focus on network traffic data, ignoring device logs, external threat intelligence, and user behavior characteristics, resulting in weak detection of new threats such as 0day attacks, with a detection accuracy rate as low as 58.3%. In addition, when fusing data, they mostly use static weight aggregation, which cannot adapt to dynamically changing threat scenarios.

[0004] In terms of time series modeling, the commonly used fixed time window extraction method, such as a 24-hour window, cannot effectively capture long-term abnormal behavior patterns, and the false alarm rate is as high as 18.7% during sudden DDoS attacks. At the same time, the adjustment of model parameters relies on manually set thresholds, and the response delay to new protocol attacks exceeds 30 minutes, making it difficult to cope with complex and changeable network attacks. In addition, when fusing multi-source data, the existing technology only stays at the simple splicing level, and the feature correlation mining is insufficient; time series modeling lacks the ability to capture bidirectional time dependencies; the model has poor adaptability and cannot be optimized in real time according to network changes.

[0005] Therefore, a communication network security situation prediction method and system based on big data is proposed to build a comprehensive, intelligent and adaptive communication network security situation prediction system to fill the gap in existing technologies and meet the growing needs of network security protection. Summary of the invention

[0006] Technical issues solved When fusing multi-source data, existing technologies only stay at the simple splicing level, and feature correlation mining is insufficient; time series modeling lacks the ability to capture bidirectional time dependencies; the model has poor adaptability and cannot be optimized in real time according to network changes.

[0007] Technical Solution To achieve the above-mentioned solution, the present invention provides the following technical solution: a communication network security situation prediction method based on big data, which is applied to real-time security risk assessment and accurate early warning of communication networks, comprising the following steps: Step 1. Multi-source heterogeneous data collection and standardization: Collect 12 subcategories of data in 4 major categories, namely the network layer, device layer, intelligence layer, and behavior layer. Use Apache Avro to define a unified data format, achieve end-to-end transmission latency ≤ 50ms through the Kafka message queue, and use the D-S evidence theory to fuse conflicting data.

[0008] Step 2. Dynamic feature engineering and weight assignment: Generate 86-dimensional original features, including traffic entropy value, TCP connection reset rate, and vulnerability CVSS score. Use a bidirectional long short-term memory network to model temporal dependencies, input the feature sequence of the previous 72 hours, with a time step of 15 minutes, and output a 64-dimensional temporal vector. Based on the Transformer multi-head attention mechanism, dynamically assign feature weights to highlight key indicators such as the abnormal traffic growth rate and the probability of exploiting high-risk vulnerabilities.

[0009] Step 3. CNN-Bi-LSTM-Attention hybrid model prediction: Build a three-level fusion architecture; use the particle swarm algorithm to dynamically optimize hyperparameters, with the learning rate range of 0.0005 - 0.002, the number of attention heads of 4 - 16, and the LSTM hidden layer dimension of 128 - 512. The AUC value of the validation set reaches 0.953.

[0010] Step 4. Hierarchical risk assessment and intelligent early warning: Use the analytic hierarchy process combined with the entropy weight method to construct a three-level assessment system, with the asset value weight of 0.25, the threat level weight of 0.40, and the vulnerability weight of 0.35; real-time anomaly detection is based on the 3σ principle combined with Bayesian optimization to dynamically adjust the threshold with a confidence level of 95%; the trend prediction module uses Monte Carlo simulation to generate the risk probability distribution for the next 72 hours and issues a 4-hour early warning for high-risk attacks.

[0011] Preferably, the multi-source heterogeneous data collection further includes: parsing Modbus / TCP, OPC UA, and S7 communication protocols in the industrial Internet scenario; MFA verification results and login IP geolocation information of user authentication logs; API call frequencies and sensitive data access path records of privileged account operation trajectories.

[0012] Preferably, in the dynamic feature engineering, in the calculation of the traffic entropy value, is the probability of the protocol type appearing within a specific time window; temporal feature enhancement uses a 256-dimensional hidden layer to capture bidirectional behavior patterns; the Transformer mechanism calculates the correlation through the feature mapping function and The key vector dimension is 64.

[0013] Preferably, the fusion architecture includes 8 5×5 convolutional kernels to extract the spatial features of the traffic packet payload, the middle-layer bidirectional LSTM layer outputs 128-dimensional temporal features, and the top-layer attention enhancement layer combines context information such as asset vulnerability scores to output a risk level range of 1-10 for the next 24 hours, with a resolution of 0.5 levels.

[0014] Preferably, in the hybrid model prediction: the convolutional layer generates a 200-dimensional feature map through the ReLU activation function to identify the patterned features of HTTP request header anomalies; the attention enhancement layer combines the geographical entropy value of the attack source to reflect the geographical dispersion degree of the attack IP and optimize the weight allocation.

[0015] Preferably, in the hierarchical risk assessment: the asset value weight considers the business impact degree and data sensitivity level through the fuzzy comprehensive evaluation method; the threat level weight calculates the attack frequency and vulnerability exploitation maturity based on the Bayesian probability in the attack chain stage; the vulnerability weight determines the patch missing rate and configuration compliance degree from the scores of the baseline inspection tool and the vulnerability scan results.

[0016] A communication network security situation prediction system based on big data is applied to the communication network security situation prediction method as described above. The system includes the following modules: Data collection module: Deploy Flume distributed collection nodes, and a single node supports 10Gbps line speed processing; integrate an industrial protocol parsing sub-module, which supports Modbus / TCP, OPCUA, and S7 communication protocols.

[0017] Data processing module: Realize millisecond-level feature extraction based on the Flink stream processing framework, including data cleaning, format conversion, and protocol parsing functions; store the standardized data in the Apache Avro format and transmit it through the Kafka message queue.

[0018] Model training module: Support the federated learning mode, perform local processing on the original data, and aggregate the parameters through secure multi-party computing; be compatible with the TensorFlow / PyTorch framework and support GPU cluster accelerated training; Prediction and analysis module: Deploy a CNN-Bi-LSTM-Attention hybrid model to provide real-time risk scoring, attack chain traceability, and trend prediction; support more than 5000 concurrent prediction requests, and the response time ≤ 100ms.

[0019] Visualization and decision-making module: Develop a three-dimensional visualization interface, including a network asset risk heat map, a real-time attack traffic dynamic map, and a historical situation comparison and analysis module; provide a RESTful API to output risk data to a third-party security management platform.

[0020] Preferably, the data acquisition module further includes: a mainstream device log parsing module; a real-time synchronization interface for CNVD vulnerability patch status, including vulnerability levels and information on the scope of affected assets.

[0021] Preferably, in the model training module: the federated learning mode ensures data privacy through secure multi-party computation; the PSO algorithm dynamically adjusts hyperparameters, and the model convergence speed is increased by 70% compared to traditional grid search.

[0022] Preferably, in the visualization and decision-making module: the risk heat map intuitively shows the asset risk distribution using color gradients; the attack traffic dynamic graph real-time annotates the characteristics of abnormal port access and TCP flag combination attacks.

[0023] Advantageous Effects Compared with the prior art, the present invention provides a method and system for predicting the security situation of a communication network based on big data, having the following advantageous effects: 1. For the method for predicting the security situation of a communication network based on big data, in terms of data fusion, it breaks through the limitation of single data, integrates multi-source heterogeneous data of 4 major categories and 12 sub-categories, and combines a dynamic weighting mechanism to make feature extraction more comprehensive and accurate; in terms of the model architecture, it adopts a three-level fusion of CNN-Bi-LSTM-Attention and PSO optimization, effectively captures spatio-temporal features, improves the convergence speed, has a high prediction accuracy, and a low false alarm rate; in terms of risk assessment, it constructs a hierarchical assessment system, combines dynamic thresholds with Monte Carlo simulation to achieve early warning, and provides an efficient, intelligent, and accurate solution for predicting the security situation of a communication network.

[0024] 2. For the system for predicting the security situation of a communication network based on big data, the data acquisition module supports industrial protocol parsing and high-speed acquisition of multi-source data, breaking through the limitation of a single data source in traditional systems; the data processing module realizes millisecond-level feature extraction based on Flink, greatly improving the timeliness compared to traditional batch processing; the model training module innovatively adopts the federated learning mode to improve the generalization ability of the model while ensuring data privacy; the prediction and analysis module deploys an optimized hybrid model, supports high-concurrency prediction and has a short response time; the visualization and decision-making module provides three-dimensional visualization and geographical drilling-down functions, and the output data can be seamlessly docked with third-party platforms, building an efficient, secure, and intelligent network security situation prediction system. Brief Description of the Drawings

[0025] Figure 1 It is a schematic flowchart of the method of the present invention; Figure 2 It is a schematic framework diagram of the system of the present invention. Detailed Embodiments

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] See also Figures 1 to 2 The present invention proposes a communication network security situation prediction method and system based on big data, including the following contents: 1. A communication network security situation prediction method based on big data includes the following steps: 1. Multi-source heterogeneous data collection and standardization 1.1 Data source coverage: A data source system has been built, covering the network layer, device layer, intelligence layer, and behavior layer, with a total of 4 categories and 12 subcategories of data. At the network layer, the collection of NetFlowv9 traffic data is very critical. It contains 32 basic attributes such as five-tuple (source IP, destination IP, protocol, source port, destination port), byte rate, TCP flag, IP fragmentation rate, etc. These attributes provide basic data for network behavior analysis. At the same time, the SDN controller configuration log synchronized every 5 minutes can timely reflect the changes in network topology. At the device layer, the access firewall session log has a processing capacity of ≥2 million records / second, and supports log analysis of various mainstream devices such as Huawei USG6000 and Sangfor AF, from which a wealth of equipment operation and security related information can be obtained. In addition, IPS intrusion event records are also included, including attack signature codes, CVE numbers, attack chain stages and other information, which helps to accurately identify attack behaviors. The intelligence layer synchronizes VirusTotal hash reputation data in real time through API, with an update frequency of 10 seconds / time. The platform covers 98% of the global malware sample library and can quickly feedback the security reputation of the file. At the same time, the CNVD vulnerability patch status is obtained, among which the vulnerability level, the scope of affected assets, repair suggestions and other information are crucial to assessing network vulnerability. The behavior layer collects user authentication logs, including MFA (multi-factor authentication) verification results and login IP geolocation information, which can be used to analyze whether there are abnormalities in user login behavior. The privileged account operation track records the frequency of API calls and sensitive data access paths, providing a basis for detecting potential risks of privileged accounts.

[0028] 1.2 Data Standardization: To ensure the effective integration and analysis of multi-source data, Apache Avro is used to define a unified data format. With the help of the Kafka message queue, low-latency access to multi-source data is achieved, and the end-to-end transmission latency is ≤50ms, ensuring the timeliness of data transmission. The D-S evidence theory is used to fuse conflicting data. By comprehensively processing the evidence provided by different data sources, this theory can effectively solve the data conflict problem, and the data cleaning accuracy rate reaches 98.9%, laying a solid foundation for subsequent accurate analysis.

[0029] 2. Dynamic Feature Engineering and Weight Allocation 2.1 Basic Feature Extraction: In this step, 86-dimensional original features are generated, which reflect the network security situation from multiple perspectives. Among the traffic features, the traffic entropy value is calculated by the formula where represents the occurrence probability of each protocol type within a specific time window. This formula is used to measure the distribution chaos degree of protocol types in traffic. The higher the entropy value, the more complex the protocol types in the traffic, and the greater the potential risk may be. In addition, features such as the TCP connection reset rate and the abnormal port access frequency are also concerned. The device features include the number of firewall policy hits, which reflects the execution situation of firewall rules; the IPS rule matching delay, which can be used to evaluate the response speed of the intrusion detection system; and the abnormal fluctuation coefficient of device CPU utilization, which can help determine whether the device has performance fluctuations due to being attacked or having abnormal loads. In terms of intelligence features, the vulnerability CVSS (Common Vulnerability Scoring System) score directly reflects the severity of the vulnerability; the threat intelligence confidence level is based on the cross-validation results of multi-source intelligence, providing a quantitative indicator for the reliability of intelligence; the reputation level of the attack source IP (1-10 levels, data from the MISP threat sharing platform) can quickly judge the danger degree of the attack source.

[0030] 2.2 Temporal Feature Enhancement: To better capture the time features of data, a bidirectional long short-term memory network (Bi-LSTM) is used to model the time series dependence relationship. The feature sequence of the previous 72 hours is input, and the time step is set to 15 minutes. The forward (from the past to the present) and backward (from the present to the past) behavior patterns are captured through a 256-dimensional hidden layer. Different from traditional LSTM that can only capture unidirectional time dependence, Bi-LSTM can more comprehensively mine the time information in the data, output a 64-dimensional temporal vector containing time context, effectively solving the problem of information loss caused by the unidirectional dependence of traditional LSTM, and providing richer time dimension features for subsequent analysis.

[0031] 2.3 Dynamic Weight Allocation: Based on the Transformer multi-head attention mechanism (8 attention heads), the feature weights are calculated by the formula . Among them, is the -dimensional feature, , are feature mapping functions that map input features to different vector spaces for calculating the correlation between features. is the key vector dimension, which affects the scale of attention calculation. In this solution, the Transformer mechanism can highlight the impact of key indicators such as the abnormal traffic growth rate (the weight can be dynamically increased to 0.35) and the probability of high-risk vulnerability exploitation (weight 0.28) by dynamically weighting different feature dimensions. Compared with the traditional fixed-weight method, the Transformer can adaptively adjust the weights of each feature according to the real-time situation of the data, more accurately reflecting the contribution of different features to the network security situation at different times.

[0032] 3. CNN-Bi-LSTM-Attention Hybrid Model Prediction 3.1 Model Architecture Design: This solution adopts an innovative three-level fusion architecture. The underlying convolutional layer uses 128 5×5 convolutional kernels to extract spatial features from the traffic packet payload, generating a 200-dimensional feature map through the ReLU activation function. The convolutional kernels can automatically learn local patterns in the traffic packets, such as patterns of abnormal HTTP request headers and TCP flag combination attacks, which are crucial for identifying common network attack types. The middle-layer bidirectional LSTM layer receives the feature sequence output by the convolutional layer and learns the dependencies of past and future time steps through a bidirectional recursive structure, outputting 128-dimensional temporal features. This structure can effectively identify periodic attacks, such as brute-force cracking attempts in the early morning every day, and long-term latent threats, such as vulnerability scanning behaviors lasting for a week. The top-layer attention enhancement layer dynamically weights the temporal features output by the Bi-LSTM, combines context information such as asset vulnerability scores (from vulnerability scanners) and the geographical entropy value of the attack source (reflecting the geographical dispersion of attack IPs), and finally outputs the risk level for the next 24 hours, ranging from 1 to 10 levels with a resolution of 0.5 levels, realizing the quantitative prediction of the security situation. Through this hierarchical fusion architecture, the model can make full use of features at different levels to improve the accuracy and reliability of prediction.

[0033] 3.2 Adaptive Optimization Mechanism: To improve the model performance, the Particle Swarm Optimization (PSO) algorithm is introduced to dynamically adjust the model hyperparameters. The search range of the learning rate is set to 0.0005 - 0.002. The learning rate determines the step size of parameter updates during model training, and an appropriate learning rate can enable the model to converge to the optimal solution faster. The number of attention heads is dynamically optimized between 4 and 16. The number of attention heads affects the model's attention to different feature dimensions, and dynamic adjustment can better adapt to different data features. The dimension of the LSTM hidden layer is 128 - 512, and the hidden layer dimension determines the memory and expression capabilities of the LSTM model. The determination of these ranges is based on a large number of experiments and the analysis of different network security scenarios. By searching for the optimal parameter combination within these ranges using the PSO algorithm, the model convergence speed is increased by 70% compared to the traditional grid search, and the AUC value of the validation set reaches 0.953, significantly improving the model training efficiency and prediction accuracy.

[0034] 4. Hierarchical Risk Assessment and Intelligent Early Warning 4.1 Construction of the Assessment System: The Analytic Hierarchy Process (AHP) combined with the entropy weight method is used to construct a three - level assessment system. The weights of the first - level indicators are dynamically calibrated through expert scoring and data entropy values. The weight of asset value is set to 0.25. The business impact degree and data sensitivity level are considered through the fuzzy comprehensive evaluation method, and the data comes from the asset CMDB system. The business impact degree evaluates the impact of an asset being attacked on business operations, and the data sensitivity level reflects the sensitivity of the data carried by the asset. The weight of threat level is 0.40. Based on the Bayesian probability at each stage of the attack chain, the attack frequency and vulnerability exploitation maturity are calculated, and the data comes from the intrusion detection system and the CVE database. The analysis of each stage of the attack chain helps to more accurately assess the development trend of threats, and the Bayesian probability can combine prior knowledge and new evidence to update the estimated probability of an event occurring. The weight of vulnerability is 0.35, which is determined by the score of the baseline inspection tool plus the vulnerability scanning results to calculate the patch missing rate and configuration compliance degree, and the data comes from the vulnerability scanner and the configuration audit system. This assessment system that comprehensively considers various factors can more comprehensively and accurately assess the network security situation.

[0035] 4.2 Design of the Early Warning Strategy: Real - time anomaly detection dynamically adjusts the threshold based on the 3σ principle combined with Bayesian optimization. The 3σ principle is a statistics - based method used to determine whether data is abnormal. When the real - time risk score exceeds the dynamic threshold (confidence level 95%), multi - level early warnings are triggered, including emails, text messages, and API callbacks, and the false alarm rate is controlled below 2.5%. The trend prediction module uses Monte Carlo simulation to generate the risk probability distribution for the next 72 hours, and gives an early warning of high - risk attacks 4 hours in advance, with an accuracy rate ≥90%. Monte Carlo simulation simulates possible future risk situations through multiple random samplings, provides the traceability of the attack path, accurate to the specific IP address and port, and provides detailed and timely support for security decision - making.

[0036] II. A big data-based communication network security situation prediction system applied to the above method, comprising the following modules: 1. Data acquisition module: In this module, Flume distributed acquisition nodes are deployed. Each single node supports 10Gbps wire speed processing and has efficient data acquisition capabilities. An industrial protocol parsing sub-module is integrated, supporting Modbus / TCP, OPCUA, and S7 communication protocols, specifically meeting the special requirements of the industrial Internet scenario. In the industrial Internet, these protocols are widely used in communication between devices. By parsing them, key information such as the operating status and control instructions of industrial devices can be obtained, providing important data support for the security situation analysis of the industrial Internet.

[0037] 2. Data processing module: Based on the Flink stream processing framework, millisecond-level feature extraction is achieved, including functional units such as data cleaning, format conversion, and protocol parsing. The efficient stream processing ability of the Flink framework can quickly process the massive amounts of data collected. In the data cleaning link, noise data and duplicate data are removed. Format conversion unifies the data from different data sources into a format suitable for subsequent analysis. Protocol parsing further deeply analyzes the data content to extract valuable features, providing high-quality data for model training.

[0038] 3. Model training module: Supports the federated learning mode. The original data is processed locally, and the parameters are aggregated through secure multi-party computing to ensure data privacy and security. In some scenarios, data privacy is crucial. The federated learning mode allows all participating parties to jointly train the model without sharing the original data. It is compatible with the TensorFlow / PyTorch framework and supports GPU cluster accelerated training, which can make full use of the advantages of existing mainstream deep learning frameworks and improve the model training efficiency through the GPU cluster, enabling the model to be quickly iteratively optimized.

[0039] 4. Prediction and analysis module: An optimized hybrid prediction model is deployed, providing core functions such as real-time risk scoring, attack chain tracing, and trend prediction, supporting more than 5000 concurrent prediction requests. Real-time risk scoring can quickly feedback the current security state of the network. Attack chain tracing helps analyze the path and source of the attack. Trend prediction provides forward-looking information for the future security situation. The high concurrent processing ability meets the real-time requirements in a large-scale network environment.

[0040] 5. Visualization Decision Module: Develop a 3D visualization interface that includes a network asset risk heat map, supports geographical information drilling down, and can intuitively display the asset risk distribution. The risk level is represented by different colors and heat levels. Users can use the geographical information drilling down function to deeply understand the asset risk situation in a specific area. A real-time attack traffic dynamic graph, accurate to the port level, clearly presents the attack traffic situation, helping users to detect abnormal traffic in a timely manner. A historical situation comparison and analysis module facilitates users to understand the change trend of the security situation. At the same time, it supports RESTful API to output risk data to a third-party security management platform, facilitating integration with other systems and providing comprehensive data support and an intuitive display method for security decision-making.

[0041] System Operation Process: First, the data collection module uses Flume distributed collection nodes to collect multi-source heterogeneous data from the network layer, device layer, intelligence layer, and behavior layer, and transmits it to the data processing layer through the Kafka message queue. In the data processing module, the Flink stream processing framework is used for data cleaning, format conversion, and protocol parsing to generate standardized data. Then, the processed data is input into the model training module. In the federated learning mode, the hyperparameters of the CNN-Bi-LSTM-Attention hybrid model are dynamically adjusted through the PSO algorithm for training. The trained model is deployed in the prediction and analysis module to process real-time data and output risk scores, attack chain tracing, and trend prediction results. Finally, the visualization decision module displays these results in an intuitive 3D visualization interface and can output data to a third-party platform through RESTful API to provide support for security decision-making.

[0042] III. Industry Implementation Cases Example 1: Security Situation Prediction of Telecom 5G Core Network Deployment Environment: In this example, the NetFlow data of 200 5G base stations is accessed, with a daily processing volume of up to 120TB. The time window is set to 10 seconds, which can accurately capture the real-time traffic changes of the 5G network. At the same time, the logs of Huawei NE5000E routers are accessed, with 500,000 logs processed per second, providing sufficient data for the analysis of the operating status of network devices. The model training cluster is configured with 8 NVIDIA A100 servers, which accelerate model training with powerful computing capabilities and adopt TensorFlow Serving for deployment, supporting the incremental training mode of minute-level model updates to ensure that the model can quickly adapt to changes in the network environment.

[0043] Technical implementation: The multi-source heterogeneous data collection scheme in the document is adopted to comprehensively collect NetFlow data at the network layer, router logs at the device layer, etc. In the data standardization stage, Apache Avro is used to define a unified format, and low-latency transmission is achieved through the Kafka message queue to ensure timely data processing. In terms of dynamic feature engineering, 86-dimensional original features such as traffic entropy values are calculated. The Bi-LSTM network is used to input the feature sequence of the previous 72 hours (time step: 15 minutes), and a 64-dimensional time series vector is output through a 256-dimensional hidden layer to capture bidirectional time dependencies. Based on the Transformer multi-head attention mechanism (8 attention heads), the feature weights are calculated to highlight key indicators such as the abnormal traffic growth rate. Through the CNN-Bi-LSTM-Attention hybrid model, 128 5×5 convolutional kernels at the bottom layer extract the spatial features of traffic packets, the middle layer of bidirectional LSTM learns the time series features, and the top layer attention enhancement layer combines information such as asset vulnerability scores to output the risk level. The particle swarm algorithm is introduced to dynamically adjust hyperparameters such as the learning rate (0.0005 - 0.002) and the number of attention heads (4 - 16).

[0044] Application effect: In the detection of the precursor of DDoS attacks in the 5G slice network, the lead time reaches 4.2 hours, which is 8.4 times higher than the existing scheme, greatly increasing the preparation time for dealing with attacks. On the CICIDS2017 dataset, the F1 value of unknown attack detection reaches 0.89, which is 22% higher than that of the traditional LSTM model, effectively improving the ability to identify unknown threats. The system has passed the ETSINFVISG security certification and meets the strict requirements of 3GPP TS33.501 for 5G network security, proving its security and reliability in practical applications.

[0045] Example 2: Federal learning application in financial transaction networks Deployment environment: A federated learning network across 15 branches is constructed to ensure that the original transaction data does not leave the local area. The Secure Multi-Party Computation technology is used to ensure data privacy. Through a distributed architecture, the data resources of each branch are integrated to provide a rich data source for model training.

[0046] Technical implementation: The multi-source heterogeneous data collection covers behavioral layer data such as user authentication logs and privileged account operation traces, as well as external threat intelligence data. During the data processing, Apache Avro and Kafka are used for standardization and transmission. The dynamic feature engineering adds financial-specific features. For example, the coefficient of variation of a single transaction amount reflects the abnormal fluctuation of the transaction amount, and the IP geographical jump frequency reflects the number of times the logged-in IP crosses regions within a unit time, constructing a 142-dimensional financial industry feature space. The Transformer multi-head attention mechanism is used to calculate the feature weights to highlight the key financial risk features. The CNN-Bi-LSTM-Attention hybrid model is used for prediction, and the particle swarm algorithm is combined to optimize the hyperparameters. At the same time, the federated learning mode is supported, and the original data of each branch is processed locally, and the parameters are aggregated through secure multi-party computing.

[0047] Application effect: The model parameter update delay ≤ 50ms, meeting the GDPR data compliance requirements and ensuring data security and privacy. During the trading period (9:00 - 17:00), the real-time prediction accuracy reaches 94.8%. A new type of phishing attack chain was successfully intercepted, and the recognition time of the complete attack path from the phishing email to the database dragnet < 150ms, effectively protecting the financial transaction security. Based on the accurate situation prediction, the risk reserve fund provision is reduced by 45%, significantly optimizing the risk control strategy and reducing the operation cost.

[0048] Example 3: Industrial Internet production line security protection Deployment environment: For the industrial Internet production line, dedicated collection nodes and data processing devices are deployed. In the data collection layer, an industrial protocol parsing sub-module is integrated, supporting industrial protocols such as Modbus / TCP and OPCUA to meet the data collection requirements of industrial equipment communication.

[0049] Technical implementation: The operation data of industrial equipment is collected, such as Modbus / TCP register read and write data, device communication cycle, etc. The parser developed by JNetPcap supports 10Gbps line speed processing, with a parsing delay ≤ 10μs, and can identify more than 80 network protocols, including industry-specific protocols. A three-level feature processing architecture is constructed to calculate basic features, such as the abnormal fluctuation coefficient of the device CPU utilization rate. The Bi-LSTM network is used to enhance the time series features, and the Transformer multi-head attention mechanism is used to dynamically allocate weights. An industrial protocol deep parsing module is developed to support the detection of Modbus / TCP register read and write anomalies, with a parsing delay < 1ms, and can quickly identify unauthorized access to the PLC programming port. A device fingerprint library is constructed, containing more than 2000 industrial controller fingerprints, and the feature dimensions include firmware hash value, communication cycle jitter rate, instruction set checksum, etc., for detecting abnormal device behaviors.

[0050] Application effect: The accuracy rate of detecting abnormal behaviors of devices reaches 92.1%, effectively identifying abnormal operations of devices. The response time of production line network security incidents is greatly shortened from 58 minutes to 3.5 minutes, realizing the transformation from "post-event response" to "pre-event prevention", and ensuring the continuity and stability of industrial production. It has passed the security certification of the IIC Industrial Internet Consortium, meets the requirements of ISO / IEC 62443 for the security of industrial control systems, and provides a strong guarantee for the security of industrial Internet production lines.

[0051] IV. Experimental data 1. Comparative experiment on benchmark datasets (test set: CSE-CIC-IDS2018)

[0052] 2. Comparison of parameter optimization effects

[0053] 3. Prediction latency test in different scenarios

[0054] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A communication network security situation prediction method based on big data, which is applied to the real-time security risk assessment and precise early warning of communication networks, and is characterized in that: It includes the following steps: Step 1, Multi-source heterogeneous data collection and standardization: Collect 12 sub-categories of data in 4 major categories including the network layer, device layer, intelligence layer, and behavior layer. Use Apache Avro to define a unified data format, achieve end-to-end transmission latency ≤ 50ms through the Kafka message queue, and use the D-S evidence theory to fuse conflicting data; Step 2, Dynamic feature engineering and weight assignment: Generate 86-dimensional original features, including traffic entropy value, TCP connection reset rate, and vulnerability CVSS score. Use a bidirectional long short-term memory network to model temporal dependencies, input the feature sequence of the previous 72 hours, with a time step of 15 minutes, and output a 64-dimensional temporal vector. Based on the Transformer multi-head attention mechanism, dynamically assign feature weights to highlight key indicators such as the abnormal traffic growth rate and the probability of exploiting high-risk vulnerabilities; Step 3, CNN-Bi-LSTM-Attention hybrid model prediction: Build a three-level fusion architecture; Dynamically optimize hyperparameters using the particle swarm algorithm, with the learning rate range of 0.0005 - 0.002, the number of attention heads of 4 - 16, and the dimension of the LSTM hidden layer of 128 - 512. The AUC value of the validation set reaches 0.953; Step 4, Hierarchical risk assessment and intelligent early warning: Use the analytic hierarchy process combined with the entropy weight method to build a three-level assessment system, with the asset value weight of 0.25, the threat level weight of 0.40, and the vulnerability weight of 0.35; Real-time anomaly detection is based on the 3σ principle combined with Bayesian optimization to dynamically adjust the threshold with a confidence level of 95%; The trend prediction module uses Monte Carlo simulation to generate the risk probability distribution for the next 72 hours and gives an early warning of high-risk attacks 4 hours in advance.

2. The method for predicting the security situation of a communication network based on big data according to claim 1, wherein: The multi-source heterogeneous data collection also includes: Parsing of Modbus / TCP, OPCUA, and S7 communication protocols in the industrial Internet scenario; MFA verification results and login IP geolocation information of user authentication logs; API call frequencies and sensitive data access path records of privileged account operation tracks.

3. The method for predicting the security situation of a communication network based on big data according to claim 1, wherein: In the dynamic feature engineering, in the calculation of traffic entropy value, is the occurrence probability of protocol types within a specific time window; the time series feature enhancement uses a 256-dimensional hidden layer to capture bidirectional behavior patterns; the Transformer mechanism calculates the correlation through the feature mapping functions and The key vector dimension is 64.

4. The communication network security situation prediction method based on big data according to claim 1, characterized in that: The fusion architecture includes 8 5×5 convolutional kernels to extract spatial features of traffic packet payloads, the middle-layer bidirectional LSTM layer outputs 128-dimensional temporal features, and the top-layer attention enhancement layer combines context information such as asset vulnerability scores to output a risk level range of 1 - 10 levels for the next 24 hours, with a resolution of 0.5 levels.

5. The method for predicting the security situation of a communication network based on big data according to claim 1, characterized in that: In the hybrid model prediction: The convolutional layer generates a 200-dimensional feature map through the ReLU activation function to identify the patterned features of abnormal HTTP request headers; The attention enhancement layer combines the geographical entropy value of the attack source to reflect the degree of geographical dispersion of the attack IP and optimize the weight assignment.

6. The method for predicting the security situation of a communication network based on big data according to claim 1, wherein: In the hierarchical risk assessment: The asset value weight considers the business impact degree and data sensitivity level through the fuzzy comprehensive evaluation method; The threat level weight calculates the attack frequency and vulnerability exploitation maturity based on the Bayesian probability at the attack chain stage; The vulnerability weight determines the patch missing rate and configuration compliance based on the scores of the baseline inspection tool and the vulnerability scan results.

7. A communication network security situation prediction system based on big data, characterized in that: The system is applied to the big data-based communication network security situation prediction method as described in any one of claims 1 to 6 above. The system includes the following modules: Data Acquisition Module: Deploy Flume distributed acquisition nodes, with a single node supporting 10Gbps line speed processing; integrate an industrial protocol parsing sub-module, supporting Modbus / TCP, OPCUA, and S7 communication protocols; Data Processing Module: Achieve millisecond-level feature extraction based on the Flink stream processing framework, including data cleaning, format conversion, and protocol parsing functions; store standardized data in the Apache Avro format and transmit it through the Kafka message queue; Model Training Module: Support the federated learning mode, perform local processing of raw data, and aggregate parameters through secure multi-party computing; Be compatible with the TensorFlow / PyTorch framework and support GPU cluster acceleration for training; Prediction and Analysis Module: Deploy a CNN-Bi-LSTM-Attention hybrid model to provide real-time risk scoring, attack chain tracing, and trend prediction; support more than 5000 concurrent prediction requests, with a response time ≤ 100ms; Visualization and Decision-making Module: Develop a three-dimensional visualization interface, including a network asset risk heat map, a real-time attack traffic dynamic map, and a historical situation comparison and analysis module; Provide a RESTful API to output risk data to a third-party security management platform.

8. The communication network security situation prediction system based on big data according to claim 7, characterized in that: The data acquisition module further includes: a mainstream device log parsing module; a real-time synchronization interface for CNVD vulnerability patch status, including vulnerability levels and information on the scope of affected assets.

9. The communication network security situation prediction system based on big data according to claim 7, characterized in that: In the model training module: The federated learning mode ensures data privacy through secure multi-party computing; the PSO algorithm dynamically adjusts hyperparameters, and the model convergence speed is increased by 70% compared to the traditional grid search.

10. The communication network security situation prediction system based on big data according to claim 7, characterized in that: In the visualization and decision-making module: The risk heat map intuitively displays the asset risk distribution using color gradients; the attack traffic dynamic map real-time annotates the characteristics of abnormal port access and TCP flag bit combination attacks.

Images