Network data security operation integration method based on big data and AI

Abstract

The application provides a network data security operation integration method based on big data and AI, and has the beneficial effects of significantly improved threat detection accuracy, enhanced automatic response capability, optimized visual effect, and self-adaptive learning capability.

Description

Technical Field

[0001] This invention relates to the field of network security technology, specifically to an integrated method for network data security operation based on big data and AI. This method achieves intelligent and automated operation of network data security through technologies such as dynamic threat index, online learning, and partition visualization. Background Technology

[0002] With the rapid development of information technology, cybersecurity threats are becoming increasingly complex and diverse. Traditional cybersecurity protection methods mainly rely on static rule bases and fixed security policies, which have the following technical shortcomings: (1) Low threat identification accuracy Traditional security systems employ signature-based detection methods, which are ineffective in identifying unknown threats and variant attacks; the false alarm rate remains high, typically between 8% and 15%, leaving security operations personnel overwhelmed with dealing with invalid alerts; and the lack of refined classification capabilities for endpoint data makes it difficult to develop differentiated security strategies based on different data types.

[0003] (2) Delayed response mechanism Security incident response relies on manual operation, with an average response time of over 30 minutes from threat discovery to action; the lack of automated response mechanisms makes it impossible to complete a series of response actions such as isolation, evidence collection, and recovery in a short time; and the invocation of the SOAR (Security Orchestration, Automation, and Response) system requires manual judgment and triggering, which is inefficient.

[0004] (3) Weak visualization ability Traditional Security Operations Centers (SOCs) use static display screens, resulting in untimely information updates; they lack dynamic visualization capabilities based on "zone-time" divisions, making it impossible to intuitively display the security posture in different areas and at different times; and the display method of threat indices is simplistic, making it difficult to reflect the dynamic trends of threat changes.

[0005] (4) Poor learning adaptability The existing threat index calculation formula is fixed and cannot be adaptively adjusted according to changes in the network environment; it lacks an online learning mechanism, so the system cannot learn and optimize from historical security incidents; when faced with new attack methods, the system has poor adaptability and the protection effect decreases over time.

[0006] In summary, existing technologies suffer from technical problems such as low threat identification accuracy, lagging response mechanisms, weak visualization capabilities, and poor learning and adaptation capabilities. There is an urgent need for an integrated network data security operation method that can achieve high-precision threat detection, automated response, intelligent visualization, and adaptive learning. Summary of the Invention

[0007] The purpose of this invention is to provide an integrated network data security operation method based on big data and AI, which significantly improves threat detection accuracy, enhances automated response capabilities, optimizes visualization effects, and provides adaptive learning capabilities.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solution: The integrated approach to network data security operations based on big data and AI includes the following steps: Step S1) Collect multi-dimensional data such as network traffic, system logs, user behavior data, and file operation records in real time through a data collection agent deployed on the terminal device; Step S2) Use a classification algorithm based on an improved BERT-CNN hybrid model to perform fine classification of the collected data and establish a three-layer classification label system including data type, sensitivity, and risk level; Step S3) Construct a multi-dimensional dynamic threat index calculation model, which integrates four dimensions: basic threat value, environmental risk coefficient, behavioral anomaly degree, and historical attack correlation degree, and calculates the threat index in real time. The calculation formula is: TI = (BT × W_BT + ER × W_ER + BA × W_BA + HA × W_HA) × AF × TF, where TI is the comprehensive threat index, BT, ER, BA, and HA are the threat values ​​of the four dimensions, W_BT, W_ER, W_BA, and W_HA are dynamic weight coefficients, AF is the asset importance coefficient, and TF is the time factor; Step S4) The improved online gradient descent algorithm (AOGD) is used to continuously optimize the weight coefficients in the threat index calculation formula. By collecting the results of security incident handling and false alarm information, the model parameters are adaptively updated. Step S5) Design a “partition-time” visualization dashboard to divide the network topology into multiple security zones. Each zone independently displays a threat index heatmap, attack type distribution map, threat trend line chart, and asset risk radar chart, supporting four display modes: real-time mode, replay mode, comparison mode, and prediction mode. Step S6) Set up a multi-level threat threshold system including attention threshold (3 points), warning threshold (5 points), emergency threshold (7 points), and danger threshold (9 points). When the threat index exceeds the preset threshold, the corresponding SOAR script will be automatically triggered. Step S7) The SOAR script includes three standardized response processes: isolation, forensics, and recovery. The isolation script performs network isolation, host isolation, user isolation, and access isolation operations. The forensics script performs memory imaging, disk imaging, log collection, network packet capture, and file extraction operations. The recovery script performs threat removal, configuration repair, patch installation, policy hardening, and monitoring enhancement operations. The average response time is controlled within 5 minutes.

[0009] Furthermore, the refined data classification adopts a three-layer classification system: the first layer is the data type classification, including four categories: network data, system data, user data, and application data; the second layer is the sensitivity classification, including four categories: public data, internal data, sensitive data, and core data; and the third layer is the risk level classification, including four categories: low risk, medium risk, high risk, and extremely high risk.

[0010] Furthermore, the improved BERT-CNN hybrid model includes an input layer, a BERT encoding layer, a CNN convolutional layer, and a classification layer. The input layer receives the original data text and metadata, the BERT encoding layer extracts the semantic features of the data, the CNN convolutional layer captures the local features and patterns of the data, and the classification layer outputs the three-level classification labels of the data.

[0011] Furthermore, the weight coefficients in the dynamic threat index calculation model are adaptively adjusted according to changes in the network environment: the weight coefficients for normal periods are W_BT=0.3, W_ER=0.2, W_BA=0.3, and W_HA=0.2; the weight coefficients for periods with high attack incidence are W_BT=0.2, W_ER=0.3, W_BA=0.2, and W_HA=0.3; and the weight coefficients for sensitive areas are W_BT=0.2, W_ER=0.4, W_BA=0.2, and W_HA=0.2.

[0012] Furthermore, the improved online gradient descent algorithm (AOGD) employs an adaptive learning rate, calculated as follows: ,in, , Let be the model parameters for the t-th iteration. Let be the adaptive learning rate for the t-th iteration. For the loss function in gradient at, β is the initial learning rate, and β is the decay coefficient. This is the cumulative sum of squares of the historical gradient.

[0013] Furthermore, the partitioning method of the "partition-time" visualization screen includes physical layer partitioning and logical layer partitioning. The physical layer partitioning divides the network into office area, server area, DMZ area, and management area, while the logical layer partitioning divides the network into core business area, general business area, test and development area, and external access area.

[0014] Furthermore, each display mode of the visualization screen has a different refresh rate and functional characteristics. The real-time mode refreshes the data every 5 seconds, the playback mode supports selecting historical time points to view the threat situation, the comparison mode supports comparing the threat situation at different time periods, and the prediction mode predicts future threat trends based on historical data.

[0015] Furthermore, the execution flow of the SOAR script includes: when the threat index exceeds the threshold, firstly, the isolation script is executed, completing network isolation, host isolation, user isolation, and access isolation operations within 45 seconds; then, the forensics script is executed, completing memory imaging, disk imaging, log collection, network packet capture, and file extraction operations within 2 minutes; finally, the recovery script is executed, completing threat removal, configuration repair, patch installation, policy hardening, and monitoring enhancement operations within 5 minutes.

[0016] Furthermore, the SOAR script library supports expansion and customization, allowing for the design of specialized scripts for different industries and scenarios, including scripts for automatically generating regulatory reports for the financial industry, scripts for cross-departmental collaborative responses for government agencies, and scripts for business continuity assurance for enterprises.

[0017] Furthermore, the method supports hierarchical protection deployment, with classified networks using physical isolation deployment and unclassified networks using logical isolation deployment, establishing a secure cross-network data exchange channel, and accessing national-level threat intelligence platforms and industry threat intelligence sharing mechanisms.

[0018] Compared with the prior art, the present invention has the following advantages: This invention presents an integrated network data security operation method based on big data and AI, which has the beneficial effects of significantly improving threat detection accuracy, enhancing automated response capabilities, optimizing visualization effects, and improving adaptive learning capabilities. Detailed Implementation

[0019] The embodiments of the present invention will now be described in further detail. Example 1

[0020] A large enterprise has 5,000 terminal devices distributed in multiple areas such as the office area, server area, and DMZ area, facing various security threats such as viruses, Trojans, and phishing emails.

[0021] The network data security operation architecture of this embodiment includes a data acquisition layer, a data processing layer, an analysis and decision-making layer, and a visualization layer. The data acquisition layer deploys lightweight data acquisition agents on all terminals, traffic mirroring devices at the network egress, and NetFlow data export configured on the core switches. The data processing layer deploys a data processing cluster consisting of three high-performance servers, configured with a Kafka message queue supporting 100,000 data entries per second, and uses Elasticsearch to store indexed data. The analysis and decision-making layer deploys an AI analysis engine supporting real-time threat index calculation, configures the SOAR platform, integrates various security device APIs, establishes a threat intelligence center, and accesses external threat intelligence sources. The visualization layer deploys a 55-inch visualization screen supporting 4K resolution, designed with a multi-view display interface, including an overview chart, a partitioned chart, and a trend chart, supporting access from both PC and mobile devices.

[0022] By deploying the aforementioned security network, the company's network data security operation capabilities have been significantly improved. Specifically, detection capabilities have been enhanced: 1256 attacks of various threats were successfully detected, achieving a detection rate of 99.6%; false alarm control resulted in only 12 false alarms, a false alarm rate of 0.95%; response speed improved to an average response time of 3 minutes and 45 seconds, an 88% improvement; and operational efficiency reduced the number of security operations personnel from 8 to 3, resulting in a 62.5% increase in efficiency.

[0023] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the concept of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for integrated network data security operation based on big data and AI, characterized in that... Includes the following steps: Step S1) Collect multi-dimensional data such as network traffic, system logs, user behavior data, and file operation records in real time through a data collection agent deployed on the terminal device; Step S2) Use a classification algorithm based on an improved BERT-CNN hybrid model to perform fine classification of the collected data and establish a three-layer classification label system including data type, sensitivity, and risk level; Step S3) Construct a multi-dimensional dynamic threat index calculation model, which integrates four dimensions: basic threat value, environmental risk coefficient, behavioral anomaly degree, and historical attack correlation degree, and calculates the threat index in real time. The calculation formula is: TI = (BT × W_BT + ER × W_ER + BA × W_BA + HA × W_HA) × AF × TF, where TI is the comprehensive threat index, BT, ER, BA, and HA are the threat values ​​of the four dimensions, W_BT, W_ER, W_BA, and W_HA are dynamic weight coefficients, AF is the asset importance coefficient, and TF is the time factor; Step S4) The improved online gradient descent algorithm (AOGD) is used to continuously optimize the weight coefficients in the threat index calculation formula. By collecting the results of security incident handling and false alarm information, the model parameters are adaptively updated. Step S5) Design a “partition-time” visualization dashboard to divide the network topology into multiple security zones. Each zone independently displays a threat index heatmap, attack type distribution map, threat trend line chart, and asset risk radar chart, supporting four display modes: real-time mode, replay mode, comparison mode, and prediction mode. Step S6) Set up a multi-level threat threshold system including attention threshold (3 points), warning threshold (5 points), emergency threshold (7 points), and danger threshold (9 points). When the threat index exceeds the preset threshold, the corresponding SOAR script will be automatically triggered. Step S7) The SOAR script includes three standardized response processes: isolation, forensics, and recovery. The isolation script performs network isolation, host isolation, user isolation, and access isolation operations. The forensics script performs memory imaging, disk imaging, log collection, network packet capture, and file extraction operations. The recovery script performs threat removal, configuration repair, patch installation, policy hardening, and monitoring enhancement operations. The average response time is controlled within 5 minutes.

2. The integrated network data security operation method based on big data and AI according to claim 1, characterized in that: The refined data classification adopts a three-layer classification system: the first layer is the data type classification, which includes four categories: network data, system data, user data, and application data; The second layer is a classification based on sensitivity, including four categories: public data, internal data, sensitive data, and core data. The third layer is the risk level classification, which includes four categories: low risk, medium risk, high risk, and extremely high risk.

3. The integrated network data security operation method based on big data and AI according to claim 1 or 2, characterized in that: The improved BERT-CNN hybrid model includes an input layer, a BERT encoding layer, a CNN convolutional layer, and a classification layer. The input layer receives raw data text and metadata, the BERT encoding layer extracts semantic features of the data, the CNN convolutional layer captures local features and patterns of the data, and the classification layer outputs three-level classification labels for the data.

4. The integrated network data security operation method based on big data and AI according to claim 1, characterized in that: The weighting coefficients in the dynamic threat index calculation model are adaptively adjusted according to changes in the network environment: the weighting coefficients for normal periods are W_BT=0.3, W_ER=0.2, W_BA=0.3, and W_HA=0.2; the weighting coefficients for periods with high attack incidence are W_BT=0.2, W_ER=0.3, W_BA=0.2, and W_HA=0.3; and the weighting coefficients for sensitive areas are W_BT=0.2, W_ER=0.4, W_BA=0.2, and W_HA=0.

2.

5. The integrated network data security operation method based on big data and AI according to claim 1 or 4, characterized in that: The improved online gradient descent algorithm (AOGD) employs an adaptive learning rate, calculated using the following formula: ,in, , Let be the model parameters for the t-th iteration. Let be the adaptive learning rate for the t-th iteration. For the loss function in gradient at, β is the initial learning rate, and β is the decay coefficient. This is the cumulative sum of squares of the historical gradient.

6. The integrated network data security operation method based on big data and AI according to claim 1, characterized in that: The "partition-time" visualization screen uses physical layer partitioning and logical layer partitioning. Physical layer partitioning divides the network into office area, server area, DMZ area, and management area. Logical layer partitioning divides the network into core business area, general business area, test and development area, and external access area.

7. The integrated network data security operation method based on big data and AI according to claim 1 or 6, characterized in that: Each display mode of the visualization screen has a different refresh rate and functional characteristics. The real-time mode refreshes the data every 5 seconds, the playback mode allows you to select historical time points to view the threat situation, the comparison mode allows you to compare the threat situation at different time periods, and the prediction mode predicts future threat trends based on historical data.

8. The integrated network data security operation method based on big data and AI according to claim 1, characterized in that: The execution flow of the SOAR script includes: when the threat index exceeds the threshold, the isolation script is executed first, and network isolation, host isolation, user isolation and access isolation operations are completed within 45 seconds; then the forensics script is executed, and memory imaging, disk imaging, log collection, network packet capture and file extraction operations are completed within 2 minutes; finally, the recovery script is executed, and threat removal, configuration repair, patch installation, policy hardening and monitoring enhancement operations are completed within 5 minutes.

9. The integrated network data security operation method based on big data and AI according to claim 1 or 8, characterized in that: The SOAR script library supports expansion and customization, and can design dedicated scripts for different industries and scenarios, including scripts for automatic generation of regulatory reports for the financial industry, scripts for cross-departmental collaborative response for government agencies, and scripts for business continuity assurance for enterprises.

10. The integrated network data security operation method based on big data and AI according to claim 1, characterized in that: The method supports hierarchical protection deployment, with classified networks using physical isolation deployment and unclassified networks using logical isolation deployment, establishing a secure cross-network data exchange channel and connecting to the national-level threat intelligence platform and industry threat intelligence sharing mechanism.