Network security intelligent management and control system based on big data

Abstract

The invention discloses a network security intelligent management and control system based on big data, and relates to the field of network security management. Comprising a data acquisition processing module, an intelligence fusion analysis module, a threat dynamic detection module, an attack deduction prediction module, a virtual mapping simulation module, an edge collaborative defense module, a defense strategy optimization module, an automatic decision execution module and a threat intelligence sharing module. According to the method, the detection capability of complex network attacks is improved, the attack path in the network environment can be visually presented, the security operation and maintenance efficiency is improved, the attack path is accurately blocked, and the attack success rate is reduced; comprehensive security event priority ranking can be realized, the scientificity of defense decision is improved, meanwhile, the system can adapt to different attack scenes, the defense efficiency is improved, it is ensured that a defense strategy always adapts to the current security situation, resource waste is avoided, and the defense cost-benefit ratio is improved.

Description

Technical Field

[0001] The present invention relates to the field of network security management, and in particular to a network security intelligent management and control system based on big data. Background Art

[0002] With the rapid development of cloud computing, the Internet of Things (IoT) and the Industrial Internet, the complexity of the global network environment has increased significantly, and the methods of network attacks have evolved from traditional single-point attacks to multi-stage, cross-domain coordinated complex attacks, such as APT attacks and supply chain attacks. Traditional security defense systems mainly rely on static rule matching and intrusion detection systems (IDS), but these methods are difficult to cope with the ever-changing attack strategies, resulting in lagging security protection and lack of active defense capabilities. In recent years, big data analysis and artificial intelligence technologies have been widely used in the field of network security. Through intelligent analysis of a large amount of multimodal data such as security logs, network traffic, and social media intelligence, the accuracy of threat detection and response can be improved. However, the current security system still faces many challenges in data integration, cross-modal analysis, and automated defense.

[0003] The existing network security intelligent management and control system has poor detection capabilities for complex network attacks and cannot intuitively present the attack path in the network environment, reducing the efficiency of security operation and maintenance and increasing the success rate of attacks. In addition, the existing network security intelligent management and control system's defense decisions are less scientific and cannot adapt to different attack scenarios. It cannot ensure that the defense strategy always adapts to the current security situation, which easily leads to resource waste and reduces the defense cost-effectiveness ratio. To this end, we propose a network security intelligent management and control system based on big data. Summary of the invention

[0004] The purpose of the present invention is to solve the defects in the prior art and provide a network security intelligent management and control system based on big data.

[0005] The present invention proposes a network security intelligent management and control system based on big data, the system comprising: a data acquisition and processing module, an intelligence fusion analysis module, a threat dynamic detection module, an attack deduction and prediction module, a virtual mapping simulation module, an edge collaborative defense module, a defense strategy optimization module, a decision automatic execution module and a threat intelligence sharing module; The data collection and processing module is used to collect security information from various data sources such as network traffic, system logs, social media texts, and dark web information, and pre-process the collected security information; The intelligence fusion analysis module is used to perform correlation analysis on security information from different data sources, mine potential attack patterns, and update the dynamic threat intelligence library in real time; The threat dynamic detection module dynamically monitors network security risks and analyzes the threat impact range based on the dynamic threat intelligence library; The attack deduction and prediction module performs phase deduction and path prediction on the monitored network security risks and formulates security defense strategies in advance; The virtual mapping simulation module is used to dynamically construct a three-dimensional network topology environment and simulate different attack methods to verify the effectiveness of the security defense strategy; The edge collaborative defense module deploys security monitoring and protection functions in edge devices and analyzes global threats; The defense strategy optimization module constructs an impact assessment matrix and dynamically optimizes the security defense strategy according to the verification results of the virtual mapping simulation module; The automatic decision-making execution module automatically adjusts the defense measures according to the severity of the real-time network security risk monitoring results; The threat intelligence sharing module is based on blockchain technology to carry out cross-organizational threat intelligence sharing and coordinated scheduling of defense resources.

[0006] It should be further explained that the specific steps of the data collection and processing module to collect and pre-process security event information from various data sources such as network traffic, system logs, social media texts, and dark web information are as follows: S1.1: The data collection and processing module obtains network traffic data through traffic mirroring, DPI or network security equipment, obtains system log data from the host operating system and application logs, uses social media APIs to capture relevant security discussions and suspicious information, and then crawls hacker forums, underground market transaction records and malicious activity discussion content through the Tor network; S1.2: Delete duplicate data from each security information from different data sources through hash verification or text similarity calculation, remove special symbols, garbled characters, and HTML tags, and then use the mean filling method to fill the missing values ​​of numerical structured data, use the mode filling method to fill the missing values ​​of numerical structured data, delete invalid entries in unstructured text, and use regular expressions to extract the timestamp, IP address, and error code fields in log data of different formats to parse log entries of different formats into a unified format; S1.3: Convert security information from different sources into a unified vector representation, then extract network traffic features, log features, and text features, and store each set of processed security information in a security data warehouse.

[0007] It should be further explained that the specific steps of the intelligence fusion analysis module to mine potential attack patterns are as follows: S2.1: Convert the pre-processed security information of each data source into a unified multi-dimensional vector representation, extract different security events from each security information, and use the extracted different security events as nodes. Each node contains feature information such as event type, timestamp, and source IP. Then, based on the correlation between each security event, establish edges to represent the causal relationship between events. S2.2: Based on similar IP and port numbers, time proximity, content similarity, and attack pattern matching, the correlation between different security events is calculated, and corresponding edge weights are added to the corresponding edges to establish a complete security event graph. After that, isolated nodes in the security event graph are removed, and the Louvain algorithm is used to cluster each security event. S2.3: Perform random walks on the security event graph to generate event sequences, extract feature information of each node, and construct a feature matrix of the corresponding node. Then, the security event graph is input into the graph neural network GNN, and GNN aggregates the feature matrix of each node in the security event graph with its adjacency matrix respectively; S2.4: After multiple rounds of information aggregation through GNN, the final security event features of different modal security information are output, position codes are added to the extracted security event features, and the query Q, key K and value V matrices of each security event feature are calculated respectively. Then, based on the multi-head attention mechanism, the security event features of each modality are fused with multi-head attention and the final features are output; S2.5: Based on the final features generated, the event status at different times is modeled in time series, and based on the modeling results, the next attack event category distribution is predicted, and the predicted attack event category is compared with the attack events in the current dynamic threat intelligence library. If the dynamic threat intelligence library does not record the attack event category, the predicted attack event category is added to the dynamic threat intelligence library.

[0008] It should be further explained that the specific steps of the attack deduction and prediction module for performing stage deduction and path prediction on the monitored network security risks are as follows: S3.1: Extract the timestamp, IP, port, event type, and user identity information of each network security risk, and mark the attack behavior in stages according to the APT knowledge base, then integrate the network security risks in chronological order, and calculate the time interval and spatial correlation between each network security risk; S3.2: Establish an attack stage set based on each network security risk, where each individual in the set represents an attack stage. Calculate the probability of the occurrence of the remaining attack stages when any attack stage in the attack stage set occurs. Based on the calculated occurrence probability, establish the conditional dependency between the attack stages. S3.3: Take each attack stage as a node and the conditional dependencies between attack stages as edges to build a corresponding attack graph. Based on the attack graph, calculate the joint probability distribution of all attack stages and use it as the prior probability of each attack node. Then set the minimum time delay and maximum time window of each attack stage, and calculate the spatial similarity of the attack source IP to set the spatial constraints of each attack stage. S3.4: According to the currently monitored attack events, based on the prior probability of the corresponding attack stage, calculate the distribution of each attack stage, and gradually deduce each attack stage through recursive calculation, and update the attack path of each node based on the deduction results of the attack stage; S3.5: Based on the attack stage deduction results, the Viterbi algorithm is used to predict the optimal probability of each attack stage as the future state. According to the predicted probability of each attack stage, the success probability of each attack path is obtained through path backtracking. If the attack success probability of the path exceeds the preset warning threshold, an early warning is triggered, and defensive measures such as blocking IP and adjusting access policies are taken.

[0009] It should be further explained that the virtual mapping simulation module dynamically constructs a three-dimensional network topology environment and simulates the specific steps of different attack methods as follows: S4.1: Through active scanning and passive monitoring methods, collect the information of the hosts, servers, switches and routers in the current network, and establish a network device set based on the collected device information, then monitor the traffic data between the network devices, and extract the communication mode, port usage and protocol type information in the traffic data, and then extract the access control list, firewall rules and intrusion detection system log of each network device to analyze the current security protection strategy; S4.2: Build a traffic matrix based on the collected traffic data information, analyze the security vulnerabilities of each network device through vulnerability scanning tools, calculate the attack probability of each network device, predict each attack path based on the attack deduction module, calculate the shortest attack path between each network device to build an attack surface diagram, and then map the attack surface diagram to a three-dimensional coordinate space; S4.3: Calculate the repulsive and attractive forces between each network device, and obtain the position coordinates of each network device in the three-dimensional coordinate space based on the calculation results. Then monitor the access of new devices, policy adjustments, and traffic changes in real time, and update the attack surface diagram based on the changes. If a new attack path is found, recalculate the optimal attack path and topology mapping, and adjust the network connection weight through elastic network modeling.

[0010] It should be further explained that the specific steps of the defense strategy optimization module to dynamically optimize the security defense strategy are as follows: S5.1: Collect factors that affect security defense strategies, such as business continuity, data value, repair cost, attack success rate, defense effectiveness, threat propagation, and compliance requirements, and then quantify and model each influencing factor. Then, based on the quantified influencing factors, establish an impact assessment matrix, where each row in the matrix represents a security incident and each column represents an influencing factor. S5.2: Set the weight of each influencing factor through the expert scoring method, and calculate the comprehensive impact score through weighted summation. Use Max-Min normalization to calculate the impact score of each security event, and sort the security events from high to low priority according to the normalized impact score. Then, establish the corresponding state space according to the number of security events in the current network system, the number of triggered defense strategies, the remaining available resources of the system, and the current threat level status information; S5.3: Establish an action space based on the defense strategies that can be executed by adding traffic monitoring rules, adjusting firewall policies, deploying new security patches, and isolating infected hosts. Then, set corresponding reward functions based on successfully blocking attacks, misjudging normal requests as attacks, and successfully breaking through defenses. Calculate the transition probability of the state changing to any other state in the state space after selecting any action under different network system states; S5.4: Establish and initialize a set of Q value tables for each network system state, select actions through the ε-greedy strategy, and calculate the reward value of executing the selected action under each network system state based on the reward function, and update the network system state at the same time, and update the corresponding Q value in the Q value table, and then repeat the action selection, reward value calculation, state update and Q value table update. If the Q value update amplitude is lower than the preset threshold, stop training and store the optimal strategy; S5.5: Collect the current network system status. If the impact score under the current network system status exceeds the preset threshold, the defense strategy optimization is triggered. Based on the Q value table generated by training, the defense action with the highest current Q value is selected, and the current security policy is adjusted. Then, the attack behavior is monitored. If the attack mode changes, the latest attack sample is recorded and the Q value is updated to generate a new defense strategy.

[0011] Beneficial effects of the present invention: The invention extracts key information such as timestamp, IP, port, event type and so on of network security risks, marks the attack behavior in stages in combination with the APT knowledge base, calculates the time interval and spatial correlation between the attack stages, establishes an attack stage set, then constructs an attack graph based on the conditional dependency of the attack stages, calculates the joint probability distribution, sets the prior probability, deduces the attack stage by recursive calculation, and predicts the optimal attack path in the future by using the Viterbi algorithm. If the probability of successful attack exceeds the threshold, an early warning is triggered and defensive measures are taken. At the same time, network device information is collected by active scanning and passive monitoring, traffic patterns, security policies and device vulnerabilities are analyzed, a traffic matrix is ​​constructed, and the shortest attack path is calculated in combination with the attack deduction, an attack surface diagram is constructed, and the attack surface diagram is mapped to a three-dimensional coordinate space. The device position is calculated by repulsion and gravity, and the attack surface diagram and the optimal attack path are updated in real time to optimize the defense strategy and dynamically adjust the network connection weight, thereby improving the detection capability of complex network attacks, being able to intuitively present the attack path in the network environment, improving the efficiency of security operation and maintenance, realizing accurate blocking of the attack path, and reducing the success rate of the attack.

[0012] The present invention collects factors affecting security defense strategies, such as business continuity, data value, and repair cost, and performs quantitative modeling to establish an impact assessment matrix. The weight is determined by an expert scoring method, and the impact score of the security event is calculated by weighted summation. The Max-Min normalization is used for sorting. Subsequently, a state space is constructed based on state information such as the number of security events and system resources, and defense actions such as adding monitoring rules and adjusting firewalls are defined to establish an action space. The Q-learning algorithm is then used to calculate the benefits of state transfer based on a reward function, and the ε-greedy strategy is used to optimize action selection. At the same time, the Q value table is updated and the optimal strategy is trained. After that, the system monitors the network status in real time. If the impact score exceeds a threshold, defense optimization is triggered, and the optimal defense action is selected to adjust the security strategy. The Q value is dynamically updated according to the latest attack sample, and the defense strategy is continuously optimized to improve the system's ability to cope with unknown attacks. Comprehensive security event priority sorting can be achieved, and the scientific nature of defense decision-making can be improved. At the same time, the system can adapt to different attack scenarios, improve defense efficiency, ensure that the defense strategy always adapts to the current security situation, avoid resource waste, and improve the defense cost-effectiveness ratio. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present invention will be further described below in conjunction with the accompanying drawings.

[0014] Figure 1 This is a framework diagram of a network security intelligent management and control system based on big data. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0017] Embodiment 1 The embodiment of the present invention provides a network security intelligent management and control system based on big data. Figure 1 , Figure 1 A framework diagram of a network security intelligent management and control system based on big data provided by an embodiment of the present invention. The system includes: a data acquisition and processing module, an intelligence fusion analysis module, a threat dynamic detection module, an attack deduction and prediction module, a virtual mapping simulation module, an edge collaborative defense module, a defense strategy optimization module, a decision automatic execution module, and a threat intelligence sharing module.

[0018] The data collection and processing module is used to collect security information from various data sources such as network traffic, system logs, social media texts, and dark web information, and pre-process the collected security information.

[0019] Specifically, the data acquisition and processing module obtains network traffic data through traffic mirroring, DPI or network security equipment, obtains system log data from the host operating system and application log, uses social media API to capture relevant security discussions and suspicious information, and then crawls transaction records and malicious activity discussions in hacker forums and underground markets through the Tor network. Through hash verification or text similarity calculation, it deletes duplicate data from various security information from different data sources, and then removes special symbols, garbled characters and HTML tags. Then, the mean filling method is used to fill the missing values ​​of numerical structured data, and the majority filling method is used to fill the missing values ​​of numerical structured data, and invalid entries in unstructured text are deleted. The timestamp, IP address and error code fields in log data of different formats are extracted through regular expressions to parse log entries of different formats into a unified format, and the security information from different sources is converted into a unified vector representation. Then, network traffic features, log features and text features are extracted, and the processed groups of security information are stored in the security data warehouse.

[0020] The intelligence fusion analysis module is used to perform correlation analysis on security information from different data sources, explore potential attack patterns, and update the dynamic threat intelligence library in real time.

[0021] Specifically, the preprocessed security information of each data source is converted into a unified multi-dimensional vector representation, and different security events are extracted from each security information. The extracted different security events are used as nodes, and each node contains the event type, timestamp and source IP feature information. Then, based on the correlation between each security event, edges are established to represent the causal relationship between events. According to similar IP and port, time proximity, content similarity and attack pattern matching, the correlation between different security events is calculated, and corresponding edge weights are added to the corresponding edges to establish a complete security event graph. Then, isolated nodes in the security event graph are removed, and the Louvain algorithm is used to cluster each security event. Random walks are performed on the security event graph to generate event sequences, and the feature information of each node is extracted to construct the feature matrix of the corresponding node. Then, the security event graph is input into the In the graph neural network GNN, GNN aggregates the feature matrix of each node in the security event graph with its adjacency matrix respectively. After multiple rounds of information aggregation through GNN, the final security event features of different modal security information are output, and position codes are added to the extracted security event features. The query Q, key K and value V matrices of each security event feature are calculated respectively. Then, based on the multi-head attention mechanism, the security event features of each modality are multi-headed fused and the final features are output. Based on the generated final features, the event status at different times is modeled in time series, and based on the modeling results, the next attack event category distribution is predicted, and the predicted attack event category is compared with the attack events in the current dynamic threat intelligence library. If the dynamic threat intelligence library does not record the attack event category, the predicted attack event category is added to the dynamic threat intelligence library.

[0022] The threat dynamic detection module dynamically monitors network security risks and analyzes the scope of threat impact based on the dynamic threat intelligence library; the attack deduction and prediction module performs stage deduction and path prediction on the monitored network security risks and formulates security defense strategies in advance.

[0023] Specifically, extract the timestamp, IP, port, event type, and user identity field information of each network security risk, and mark the attack behavior in stages according to the APT knowledge base, then integrate the network security risks in chronological order, and calculate the time interval and spatial correlation between each network security risk. Establish an attack stage set based on each network security risk, where each individual in the set represents an attack stage. Calculate the probability of the occurrence of the remaining attack stages when any attack stage in the attack stage set occurs. Based on the calculated occurrence probability, establish the conditional dependency between the attack stages, take each attack stage as a node, and the conditional dependency between the attack stages as an edge, establish the corresponding attack graph, and calculate the joint probability distribution of all attack stages based on the attack graph. The prior probability of each attack node is calculated, and then the minimum time delay and maximum time window of each attack stage are set, and the spatial similarity of the attack source IP is calculated to set the spatial constraints of each attack stage. According to the currently monitored attack events, based on the prior probability of the corresponding attack stage, the distribution of each attack stage is calculated, and each attack stage is gradually deduced through recursive calculation. Based on the deduction results of the attack stage, the attack path of each node is updated. Based on the deduction results of the attack stage, the Viterbi algorithm is used to predict the optimal probability of each attack stage as the future state. According to the predicted probability of each attack stage, the success probability of each attack path is obtained through path backtracking. If the attack success probability of the path exceeds the preset warning threshold, the warning is triggered, and the defense measures such as blocking the IP and adjusting the access strategy are taken.

[0024] Embodiment 2 The embodiment of the present invention provides a network security intelligent management and control system based on big data. Figure 1 , Figure 1 A framework diagram of a network security intelligent management and control system based on big data provided by an embodiment of the present invention. The system includes: a data acquisition and processing module, an intelligence fusion analysis module, a threat dynamic detection module, an attack deduction and prediction module, a virtual mapping simulation module, an edge collaborative defense module, a defense strategy optimization module, a decision automatic execution module, and a threat intelligence sharing module.

[0025] The virtual mapping simulation module is used to dynamically build a three-dimensional network topology environment and simulate different attack methods to verify the effectiveness of security defense strategies.

[0026] Specifically, through active scanning and passive monitoring methods, the information of each network device such as hosts, servers, switches and routers in the current network is collected, and a network device set is established based on the collected device information, and then the traffic data between each network device is monitored, and the communication mode, port usage and protocol type information in the traffic data are extracted, and then the access control list, firewall rules and intrusion detection system log of each network device are extracted, and the current security protection strategy is analyzed. A traffic matrix is ​​constructed based on the collected traffic data information, and the security vulnerabilities of each network device are analyzed through vulnerability scanning tools. The attack probability of each network device is calculated, and each attack path is predicted based on the attack deduction module. The shortest attack path between each network device is calculated to construct an attack surface diagram, and then the attack surface diagram is mapped to a three-dimensional coordinate space, and the repulsion and attraction between each network device are calculated. The position coordinates of each network device in the three-dimensional coordinate space are obtained based on the calculation results. Then, the access of new devices, policy adjustments and traffic changes are monitored in real time, and the attack surface diagram is updated according to the change amount. If a new attack path is found, the optimal attack path and topology mapping are recalculated, and the network connection weight is adjusted through elastic network modeling.

[0027] The edge collaborative defense module deploys security monitoring and protection functions in edge devices and analyzes global threats; the defense strategy optimization module builds an impact assessment matrix and dynamically optimizes the security defense strategy based on the verification results of the virtual mapping simulation module.

[0028] Specifically, we collect factors that affect security defense strategies, such as business continuity, data value, repair cost, attack success rate, defense effectiveness, threat propagation, and compliance requirements, and then quantify and model each influencing factor. Then, based on the quantified influencing factors, we establish an impact assessment matrix, where each row in the matrix represents a security incident, and each column represents an influencing factor. We set the weight value of each influencing factor through the expert scoring method, and calculate the comprehensive impact score through weighted summation. We use Max-Min normalization to calculate the impact score of each security incident, and according to the normalized impact score, we sort each security incident in order of priority from large to small. Then, we establish a corresponding state space based on the number of security incidents in the current network system, the number of triggered defense strategies, the remaining available resources of the system, and the current threat level. We establish an action space based on the executable defense strategies of adding traffic monitoring rules, adjusting firewall strategies, deploying new security patches, and isolating infected hosts. Then, we successfully block attacks based on the action space. , misjudging normal requests as attacks and attacks that successfully break through defenses, setting corresponding reward functions, and calculating the transition probability of the state changing to any other state in the state space after selecting any action under different network system states, establishing and initializing a set of Q-value tables for each network system state, selecting actions through the ε-greedy strategy, and calculating the reward value for executing the selected action under each network system state based on the reward function, while updating the network system state and the corresponding Q-value in the Q-value table, then repeating action selection, reward value calculation, state update, and Q-value table update. If the Q-value update amplitude is lower than the preset threshold, stop training, store the optimal strategy, and collect the current network system state. If the impact score under the current network system state exceeds the preset threshold, trigger defense strategy optimization, and select the defense action with the highest current Q-value based on the Q-value table generated by training, and adjust the current security strategy. Then monitor the attack behavior. If the attack mode changes, record the latest attack sample and update the Q-value to generate a new defense strategy.

[0029] The automatic decision-making execution module automatically adjusts defense measures according to the severity of real-time network security risk monitoring results; the threat intelligence sharing module uses blockchain technology to share threat intelligence across organizations and coordinate the scheduling of defense resources.

[0030] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A network security intelligent management and control system based on big data, characterized in that: include: Data collection and processing module, intelligence fusion analysis module, threat dynamic detection module, attack deduction and prediction module, virtual mapping simulation module, edge collaborative defense module, defense strategy optimization module, decision automatic execution module and threat intelligence sharing module; The data collection and processing module is used to collect and pre-process security information from various data sources such as network traffic, system logs, social media texts, and dark web information; The intelligence fusion analysis module is used to perform correlation analysis on security information from different data sources, mine potential attack patterns, and update the dynamic threat intelligence library in real time; The threat dynamic detection module dynamically monitors network security risks and analyzes the threat impact range based on the dynamic threat intelligence library; The attack deduction and prediction module performs phase deduction and path prediction on the monitored network security risks and formulates security defense strategies in advance; The virtual mapping simulation module is used to dynamically construct a three-dimensional network topology environment and simulate different attack methods to verify the effectiveness of the security defense strategy; The edge collaborative defense module deploys security monitoring and protection functions in edge devices and analyzes global threats; The defense strategy optimization module constructs an impact assessment matrix and dynamically optimizes the security defense strategy according to the verification results of the virtual mapping simulation module; The automatic decision-making execution module automatically adjusts the defense measures according to the severity of the real-time network security risk monitoring results; The threat intelligence sharing module is based on blockchain technology to carry out cross-organizational threat intelligence sharing and coordinated scheduling of defense resources.

2. According to the big data-based network security intelligent management and control system of claim 1, it is characterized in that: The specific steps of the data acquisition and processing module to collect and pre-process security event information from network traffic, system logs, social media texts, and dark web information data sources are as follows: S1.1: The data collection and processing module obtains network traffic data through traffic mirroring, DPI or network security equipment, obtains system log data from the host operating system and application logs, uses social media APIs to capture relevant security discussions and suspicious information, and then crawls hacker forums, underground market transaction records and malicious activity discussion content through the Tor network; S1.2: Delete duplicate data from each security information from different data sources through hash verification or text similarity calculation, remove special symbols, garbled characters, and HTML tags, and then use the mean filling method to fill the missing values ​​of numerical structured data, use the mode filling method to fill the missing values ​​of numerical structured data, delete invalid entries in unstructured text, and use regular expressions to extract the timestamp, IP address, and error code fields in log data of different formats to parse log entries of different formats into a unified format; S1.3: Convert security information from different sources into a unified vector representation, then extract network traffic features, log features, and text features, and store each set of processed security information in a security data warehouse.

3. According to claim 2, a network security intelligent management and control system based on big data is characterized in that: The specific steps of the intelligence fusion analysis module to mine potential attack patterns are as follows: S2.1: Convert the pre-processed security information of each data source into a unified multi-dimensional vector representation, extract different security events from each security information, and use the extracted different security events as nodes. Each node contains feature information such as event type, timestamp, and source IP. Then, based on the correlation between each security event, establish edges to represent the causal relationship between events. S2.2: Based on similar IP and port numbers, time proximity, content similarity, and attack pattern matching, the correlation between different security events is calculated, and corresponding edge weights are added to the corresponding edges to establish a complete security event graph. After that, isolated nodes in the security event graph are removed, and the Louvain algorithm is used to cluster each security event. S2.3: Perform random walks on the security event graph to generate event sequences, extract feature information of each node, and construct a feature matrix of the corresponding node. Then, the security event graph is input into the graph neural network GNN, and GNN aggregates the feature matrix of each node in the security event graph with its adjacency matrix respectively; S2.4: After multiple rounds of information aggregation through GNN, the final security event features of different modal security information are output, position codes are added to the extracted security event features, and the query Q, key K and value V matrices of each security event feature are calculated respectively. Then, based on the multi-head attention mechanism, the security event features of each modality are fused with multi-head attention and the final features are output; S2.5: Based on the final features generated, the event status at different times is modeled in time series, and based on the modeling results, the next attack event category distribution is predicted, and the predicted attack event category is compared with the attack events in the current dynamic threat intelligence library. If the dynamic threat intelligence library does not record the attack event category, the predicted attack event category is added to the dynamic threat intelligence library.

4. According to the big data-based network security intelligent management and control system of claim 3, it is characterized in that: The specific steps of the attack deduction and prediction module for performing stage deduction and path prediction on the monitored network security risks are as follows: S3.1: Extract the timestamp, IP, port, event type, and user identity information of each network security risk, and mark the attack behavior in stages according to the APT knowledge base, then integrate the network security risks in chronological order, and calculate the time interval and spatial correlation between each network security risk; S3.2: Establish an attack stage set based on each network security risk, where each individual in the set represents an attack stage. Calculate the probability of the occurrence of the remaining attack stages when any attack stage in the attack stage set occurs. Based on the calculated occurrence probability, establish the conditional dependency between the attack stages. S3.3: Take each attack stage as a node and the conditional dependencies between attack stages as edges to build a corresponding attack graph. Based on the attack graph, calculate the joint probability distribution of all attack stages and use it as the prior probability of each attack node. Then set the minimum time delay and maximum time window of each attack stage, and calculate the spatial similarity of the attack source IP to set the spatial constraints of each attack stage. S3.4: According to the currently monitored attack events, based on the prior probability of the corresponding attack stage, calculate the distribution of each attack stage, and gradually deduce each attack stage through recursive calculation, and dynamically update the subsequent attack path based on the deduction results; S3.5: Based on the attack stage deduction results, the Viterbi algorithm is used to predict the optimal probability of each attack stage as the future state. According to the predicted probability of each attack stage, the success probability of each attack path is obtained through path backtracking. If the attack success probability of the path exceeds the preset warning threshold, an early warning is triggered, and defensive measures such as blocking IP and adjusting access policies are taken.

5. According to claim 4, a network security intelligent management and control system based on big data is characterized in that: The virtual mapping simulation module dynamically constructs a three-dimensional network topology environment and simulates different attack methods in the following specific steps: S4.1: Through active scanning and passive monitoring methods, collect the information of the hosts, servers, switches and routers in the current network, and establish a network device set based on the collected device information, then monitor the traffic data between the network devices, and extract the communication mode, port usage and protocol type information in the traffic data, and then extract the access control list, firewall rules and intrusion detection system log of each network device to analyze the current security protection strategy; S4.2: Build a traffic matrix based on the collected traffic data information, analyze the security vulnerabilities of each network device through vulnerability scanning tools, calculate the attack probability of each network device, predict each attack path based on the attack deduction module, calculate the shortest attack path between each network device to build an attack surface diagram, and then map the attack surface diagram to a three-dimensional coordinate space; S4.3: Calculate the repulsive and attractive forces between each network device, and obtain the position coordinates of each network device in the three-dimensional coordinate space based on the calculation results. Then monitor the access of new devices, policy adjustments, and traffic changes in real time, and update the attack surface diagram based on the changes. If a new attack path is found, recalculate the optimal attack path and topology mapping, and adjust the network connection weight through elastic network modeling.

6. A network security intelligent management and control system based on big data according to claim 5, characterized in that: The specific steps of dynamically optimizing the security defense strategy by the defense strategy optimization module are as follows: S5.1: Collect factors that affect security defense strategies, such as business continuity, data value, repair cost, attack success rate, defense effectiveness, threat propagation, and compliance requirements, and then quantify and model each influencing factor. Then, based on the quantified influencing factors, establish an impact assessment matrix, where each row in the matrix represents a security incident and each column represents an influencing factor. S5.2: Set the weight of each influencing factor through the expert scoring method, and calculate the comprehensive impact score through weighted summation. Use Max-Min normalization to calculate the impact score of each security event, and sort the security events from high to low priority according to the normalized impact score. Then, establish the corresponding state space according to the number of security events in the current network system, the number of triggered defense strategies, the remaining available resources of the system, and the current threat level status information; S5.3: Establish an action space based on the defense strategies that can be executed by adding traffic monitoring rules, adjusting firewall policies, deploying new security patches, and isolating infected hosts. Then, set corresponding reward functions based on successfully blocking attacks, misjudging normal requests as attacks, and successfully breaking through defenses. Calculate the transition probability of the state changing to any other state in the state space after selecting any action under different network system states; S5.4: Establish and initialize a set of Q value tables for each network system state, select actions through the ε-greedy strategy, and calculate the reward value of executing the selected action under each network system state based on the reward function, and update the network system state at the same time, and update the corresponding Q value in the Q value table, and then repeat the action selection, reward value calculation, state update and Q value table update. If the Q value update amplitude is lower than the preset threshold, stop training and store the optimal strategy; S5.5: Collect the current network system status. If the impact score under the current network system status exceeds the preset threshold, the defense strategy optimization is triggered. Based on the Q value table generated by training, the defense action with the highest current Q value is selected, and the current security policy is adjusted. Then, the attack behavior is monitored. If the attack mode changes, the latest attack sample is recorded and the Q value is updated to generate a new defense strategy.

Images