A network security event correlation detection method based on big data analysis

Abstract

This invention relates to the field of information security technology, specifically to a method for detecting network security event correlations based on big data analysis. It includes the following steps: data acquisition; feature extraction and fusion; correlation detection: using an improved Apriori-Bayesian fusion algorithm to discretize the event feature vectors; mining frequent itemsets using the improved Apriori algorithm; risk assessment and result output. This invention employs an improved Apriori-Bayesian fusion algorithm to discretize event feature vectors by type, while introducing a security event weight factor to calculate the weighted support of itemsets and dynamically adjusting the minimum support threshold to mine frequent itemsets. It combines this with a Bayesian network to calculate correlation confidence and corrects the confidence through spatiotemporal correlation coefficients, enabling scientific determination of correlation relationships between network security events. This solves the problems of limited accuracy and lack of quantitative correction in traditional correlation determination techniques, reducing the probability of false positives and false negatives.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and more specifically, to a method for detecting network security incident correlations based on big data analysis. Background Technology

[0002] As cyberattacks evolve into complex, multi-step, cross-device attack chains, security data from a single source is no longer sufficient to support the complete identification and risk assessment of attack behaviors. Relying solely on fragmented data can easily lead to incomplete attack scenario reconstruction and significant biases in correlation determination, failing to meet enterprises' needs for accurate perception of cybersecurity posture. Therefore, how to effectively integrate multi-source security data, extract deep features, and efficiently detect correlations, while simultaneously building a comprehensive risk assessment system, has become a critical issue that urgently needs to be addressed in the field of cybersecurity incident analysis.

[0003] In existing technologies, relevant patents have explored the field of network security event correlation analysis. For example, Chinese patent CN202111038025.7 discloses a rule-based network security event correlation analysis method and system. The core of this method is to use a rule-based correlation analysis algorithm to integrate traffic and multi-source log information. This involves filtering logs and traffic records, extracting fields, merging and removing redundancy, aggregating similar events to form an event queue, defining attack scenario correlation relationships using tree-like rules, and then matching rules to map multi-source records to reconstruct the attack process. Another Chinese patent CN202110457126.1 discloses a network security event correlation analysis method and system. This method aggregates security events based on session-related information defined in the reporting message format. Event information is stored in a backend database in different ways, and the frontend presents nodes containing source / destination IPs in a specific arrangement. The associated IPs are connected according to the events to form a security event triggering relationship topology, achieving universal adaptability and intuitive display.

[0004] Despite the design advantages of the above technical solutions, they also have the following technical shortcomings: First, the data dimensions are not fully covered and lack feature guidance: CN202111038025.7 only focuses on the fusion analysis of traffic information and multi-source log information, without supplementing asset attribute information closely related to security events and dynamically updated threat reference data, resulting in the data foundation being unable to fully support the multi-dimensional characterization of attack scenarios; CN202110457126.1 focuses on session-related information to achieve event aggregation, but also does not include key information such as vulnerability details and asset dependencies, making it difficult to accurately locate core features that are valuable for correlation detection due to the lack of guidance from effective external information in the feature extraction process. Secondly, the feature mining depth is insufficient and the accuracy of association judgment is limited: CN202111038025.7 relies on preset tree rules to define association relationships, without deeply mining and integrating the spatial distribution features and time series features hidden in the data, and without quantitatively correcting the reliability of the association results. It is easy to misjudge associations due to changes in the scenario by relying solely on rule matching; CN202110457126.1 only completes the initial aggregation of events through session information, without standardizing and transforming features for the heterogeneity of multi-source data. The association judgment relies solely on the topological connection display of IP nodes, lacks a scientific definition of the association strength, and is difficult to distinguish the actual significance of different association relationships. Third, the risk assessment dimensions are too narrow and cannot support comprehensive decision-making: CN202111038025.7 only reconstructs the attack process, without quantitatively assessing the threat level and impact scope of related events, and cannot reflect the actual harm of the events to business systems; CN202110457126.1 only visualizes the triggering relationship of security events, without combining historical data to analyze the spread trend of events, nor classifying the impact levels according to asset importance, and cannot form a comprehensive risk conclusion covering threat, impact, and spread, making it difficult to provide a comprehensive basis for security protection decisions. In view of this, we propose a network security event correlation detection method based on big data analysis. Summary of the Invention

[0005] The purpose of this invention is to provide a network security incident correlation detection method based on big data analysis, in order to solve the problems mentioned in the background technology, such as incomplete data dimension coverage and lack of feature guidance, insufficient feature mining depth and limited accuracy of correlation judgment, and single risk assessment dimension that cannot support comprehensive decision-making.

[0006] To address the aforementioned technical problems, this invention provides a network security event correlation detection method based on big data analysis, comprising the following steps:

[0007] S100. Data Acquisition: Collect raw data of multi-source network security events, including network device logs, terminal behavior logs, network traffic data, and security device alarm data; and collect asset information associated with the raw data of the multi-source network security events (the asset information includes the identifiers of terminal devices, servers, and business systems, their respective business modules, and their importance level in the business architecture); simultaneously collect threat intelligence data, including real-time updated threat intelligence data such as known attack signature databases, vulnerability information databases, and attack chain graphs.

[0008] S200, Feature Extraction and Fusion: The collected raw data of multi-source network security events and threat intelligence data are formatted, outliers are removed, duplicate data is deduplicated, noise is reduced, and standardization is performed. A dual-attention CNN-LSTM fusion model with threat intelligence guidance is used to extract spatial and temporal features from the preprocessed data, and the extracted spatial and temporal features are fused to obtain the event feature vector.

[0009] S300. Association Detection: An improved Apriori-Bayesian fusion algorithm is used to discretize the event feature vectors (continuous features are divided into discrete intervals using K-means clustering, while discrete features retain their original categories); frequent itemsets are mined using the improved Apriori algorithm, association confidence is calculated using a Bayesian network, and a spatiotemporal association coefficient, including spatial and temporal association coefficients, is introduced to correct the confidence (spatial association coefficient similarity is calculated using the geocoding of IP addresses, and temporal association coefficient is calculated by taking the reciprocal of the event time interval); the association relationship between network security events is determined based on the corrected confidence.

[0010] S400 Risk Assessment and Results Output: For the correlation of cybersecurity incidents determined by correlation detection, conduct risk assessment in terms of threat level, scope of impact, and probability of spread, and generate and output an assessment report including correlation details, risk level, and response recommendations.

[0011] As a further improvement to this technical solution, in S100, data acquisition includes the acquisition of raw data from multiple network security incidents, asset information acquisition, and threat intelligence data acquisition, wherein:

[0012] The collection of raw data from multi-source cybersecurity incidents includes the following steps:

[0013] S110.1 Network Device Log Collection: Core device logs are collected via SNMPv3 protocol, and access layer device logs are received via Syslog protocol; before collection, the logs are filtered by the device's built-in rules, retaining only logs containing keywords such as "abnormal traffic," "port abnormality," and "authentication failure," and extracting device identifier, event timestamp, and event level fields; and associating the core device and access layer device with the assets to which they belong (such as core business server clusters, access layer terminal areas, etc.) and the asset importance level;

[0014] S110.2 Terminal Behavior Log Collection: A collection agent is deployed on the terminal to capture events through operating system kernel hooks; process creation events are collected in real time, and file operation events are collected in batches at fixed intervals to extract the terminal MAC address, process path, file hash value, and operation result fields; and the terminal asset information (such as department, asset type, and whether it is a critical business terminal) is associated with the terminal MAC address.

[0015] S110.3 Network Traffic Data Collection: Deploy traffic probes at core nodes to analyze traffic based on DPI technology; collect and record the connection state sequence of TCP sessions in full; sample UDP traffic by "source IP + destination port" to extract session duration, packet size distribution, retransmission rate, and application layer protocol identifier; and associate the source and destination IPs involved in the traffic with asset information (such as the business system to which the server IP belongs, the user group to which the terminal IP belongs, etc.).

[0016] S110.4 Security device alarm data collection: collected via RESTful API interface; IDSIPS devices pull data at fixed intervals, and antivirus devices use an instant push mode after alarm generation, extracting attack signature ID, source and destination IP ports, alarm confidence level, and handling suggestions; and associating the asset information corresponding to the source and destination IP ports involved in the alarm (such as the asset category and protection level of the attacked IP).

[0017] Asset information collection includes the following steps:

[0018] S120.1 Asset Basic Information Collection: Synchronize asset data from the asset ledger system, including the unique identifier of terminal equipment, server, and business system, hardware configuration, business module to which it belongs, role and importance level in the business architecture; adopt a combination of scheduled synchronization and change-triggered synchronization to ensure the real-time nature of asset information;

[0019] S120.2 Asset Association Collection: Through network topology discovery tools and business configuration management systems, collect network connection relationships and business dependencies between assets; generate asset association maps to provide a basis for subsequent risk diffusion analysis;

[0020] Threat intelligence data collection includes the following steps:

[0021] S130.1, Collection of known attack signature database: Interact with multiple heterogeneous intelligence sources and normalize them through an aggregation engine; the malicious IP database is stored in the structure of "IP + threat type + first appearance time", and the full synchronization cycle and incremental synchronization trigger conditions are configured. The AC automaton algorithm is used for signature matching, and invalid signatures are cleaned up periodically.

[0022] S130.2 Vulnerability Information Database Collection: Based on data synchronization from the official CVE interface, the synchronization frequency is consistent with the official updates. Extract CVE-ID, release date, affected versions, CVSS score and sub-items, and exploit public status fields; trigger endpoint scanning tasks for high-risk vulnerabilities;

[0023] S130.3 Attack Chain Graph Acquisition: Graph extraction algorithms are used to extract nodes from intelligence reports and construct a graph according to the attack stage model; node association weights are calculated based on the frequency of historical attack events, the graph topology update cycle is configured, and an emergency update is triggered when the number of newly disclosed events reaches a set number.

[0024] As a further improvement to this technical solution, in step S200, the collected raw data of multi-source network security events and threat intelligence data are subjected to format unification, outlier removal, duplicate data deduplication, and noise reduction, including the following steps:

[0025] S210.1, Format Unification: Regular expressions are used to match IP address, port number, timestamp, and attack type fields in unstructured logs and convert them into key-value pairs; all types of timestamps are uniformly converted to UTC format, accurate to milliseconds; lexical analysis tools are used to extract core semantic keywords from text description fields; and the signature format in threat intelligence is uniformly converted to hexadecimal strings.

[0026] S210.2 Outlier Removal: For numerical features, the interquartile range method is used to calculate the upper quartile Q3 and lower quartile Q1 of the data, and values ​​outside the range determined by Q1, Q3 and the set coefficient are removed; for categorical features, rare categories with a frequency below the set threshold are removed.

[0027] S210.3 Deduplication of Duplicate Data: Using "source IP + destination IP + event type + timestamp" as the composite primary key, retain the first occurrence of the record and delete subsequent duplicate entries; for log data, additionally check the consistency of process path and operation result fields, and if they are completely consistent, they are judged as duplicate data and removed.

[0028] S210.4 Noise Reduction: For network traffic data packet content, a sliding window averaging method is used to smooth burst noise, with the window size set to 5-10 consecutive data packets; for text logs, meaningless words are filtered out through a preset stop word list, while core feature words (such as attack behavior, device identification, etc.) are retained; for fuzzy feature descriptions in threat intelligence, they are standardized through a thesaurus mapping table.

[0029] As a further improvement to this technical solution, in step S200, the collected raw data of multi-source network security incidents and threat intelligence data undergoes standardized preprocessing, including the following steps:

[0030] S220.1. Z-score standardization is applied to continuous numerical features;

[0031] S220.2. For features with a fixed value range, min-max standardization is applied to map them to the [0,1] interval;

[0032] S220.3 For skewed distribution characteristics, perform logarithmic transformation first and then standardize;

[0033] S220.4. Categorical features are converted into binary vectors using one-hot encoding; high cardinality categories are mapped into low-dimensional dense vectors using embedding techniques.

[0034] As a further improvement to this technical solution, in S200, the threat intelligence-guided dual-attention CNN-LSTM fusion model includes a spatial feature extraction branch (CNN), a temporal feature extraction branch (LSTM), and a feature fusion layer, wherein:

[0035] The spatial feature extraction branch of CNN includes an input layer, convolutional layers, a threat intelligence-spatial attention layer, and pooling layers, among which:

[0036] The input layer receives preprocessed spatial feature data, including IP address embedding vectors, port feature encoding, protocol type identifiers, and device type features;

[0037] A convolutional layer consists of a preset number of sub-layers, each configured with a convolutional kernel of a specified size and stride, and each layer is followed by batch normalization and a ReLU activation function.

[0038] The threat intelligence-spatial attention layer adjusts feature weights based on threat intelligence database data. If the IP address corresponding to a feature matches a malicious IP database, the port belongs to a high-risk port list, or the protocol type is a known commonly used attack protocol, the weight of the corresponding feature channel is increased by a preset ratio.

[0039] The pooling layer uses global max pooling to reduce the dimensionality of the convolution output features, generating spatial feature vectors of fixed dimensions;

[0040] The temporal feature extraction branch of LSTM includes an input layer, an LSTM layer, a temporal attention layer, and a fully connected layer; among which:

[0041] The input layer receives time-series data sorted by timestamp, including event occurrence sequences, traffic change sequences, and alarm frequency sequences, with the sequence length being a preset value;

[0042] An LSTM layer consists of a preset number of LSTM units and is used for temporal feature extraction. Each layer retains the output of all time steps or the output of the last time step.

[0043] The time attention layer assigns weights based on event time intervals and attack timing features in threat intelligence. The weight of adjacent events with time intervals less than a set threshold is increased, and the weight of event sequences that match the timing of known attack chains is strengthened.

[0044] The fully connected layer adjusts the dimension of the LSTM output features to generate a temporal feature vector with the same dimension as the spatial feature vector.

[0045] The feature fusion layer performs element-wise weighted fusion of spatial and temporal feature vectors. The weights are set based on the contribution of spatial and temporal feature vectors to historical correlation detection. After nonlinear transformation by the activation function, the fused comprehensive feature vector is output for subsequent event feature vector generation.

[0046] As a further improvement to this technical solution, in step S200, the generation of the event feature vector includes the following steps:

[0047] S230.1 Feature concatenation: The spatial feature vector output by the CNN branch of spatial feature extraction and the temporal feature vector output by the LSTM branch of temporal feature extraction are concatenated sequentially according to their dimensions to form an initial fusion vector whose dimension is the sum of the dimensions of the spatial feature vector and the temporal feature vector.

[0048] S230.2 Feature Enhancement: The initial fusion vector from S230.1 is input into a fully connected layer for nonlinear transformation. The number of neurons in the fully connected layer is the same as the dimension of the initial fusion vector. The activation function is LeakyReLU to enhance the correlation between features of different dimensions.

[0049] S230.3 Feature Filtering: Apply L1 regularization constraints to the enhanced feature vectors. The regularization coefficient is set according to the cross-validation results of historical detection data. By suppressing weights with smaller absolute values, the top 80% of feature dimensions that contribute to the correlation detection are retained.

[0050] S230.4 Normalization: Perform L2 normalization on the filtered feature vectors to make the L2 norm of the feature vectors 1;

[0051] S230.5 Output Event Feature Vector: Output a fixed-dimensional event feature vector obtained after processing, which is used for subsequent association rule mining and detection of network security events.

[0052] As a further improvement to this technical solution, in step S300, the discretization processing of the event feature vector includes the following steps:

[0053] S310.1 Discretization of Continuous Features: For continuous features (such as flow rate change rate, alarm frequency fluctuation value, etc.), K-means clustering algorithm is used to divide them into discrete intervals; number of clusters. The initial cluster centers are determined by the elbow method, and the farthest priority strategy is adopted for selection. The iteration termination condition is that the change in the cluster centers is less than the set threshold or the preset number of iterations is reached.

[0054] S310.2, Discrete Feature Processing: For discrete features (such as protocol type, attack type identifier, etc.), retain the original category label; for rare categories with a frequency below a set threshold, merge them into the "Other" category;

[0055] S310.3 Feature encoding conversion: Convert all discretized features into integer codes and establish a mapping table between the codes and the original feature values.

[0056] As a further improvement to this technical solution, in step S300, the improved Apriori algorithm is used to mine frequent itemsets, including the following steps:

[0057] S320.1 Introduce a security event weighting factor, and assign corresponding weight coefficients to different events according to their risk levels. Combined with itemsets Number of transactions and total number of transactions The weighted support of the itemset is calculated by multiplying the weight coefficient by the proportion of transactions. ;

[0058] S320.2 Setting the initial threshold for minimum support based on the average frequency of historical event associations. If the number of frequently generated itemsets is lower than the preset value, then the initial threshold of the current minimum support will be used. With attenuation coefficient The product update threshold is repeatedly adjusted until the itemset quantity requirement is met.

[0059] S320.3 Setting the threshold for the proportion of rare events Statistical analysis of the frequency of rare events , for satisfying Less than the proportional threshold The itemset containing rare events multiplied by the total number of transactions is filtered in advance to reduce the size of the candidate set;

[0060] S320.4, Weight the support of itemsets With the adjusted minimum support threshold By comparison, itemsets with support levels not lower than a threshold are selected as frequent itemsets for subsequent association rule mining.

[0061] As a further improvement to this technical solution, in step S300, a confidence level correction based on the spatiotemporal correlation coefficient and a correlation determination are introduced, including the following steps:

[0062] S330.1 Calculate the spatial correlation coefficient : Obtain the latitude and longitude coordinates of the IP address of the event to be associated and Combined with Earth's radius Calculate the spherical distance between two points Then based on the preset distance threshold By distance With threshold The mapping relationship determines the spatial correlation coefficient. ;

[0063] S330.2 Calculate the time correlation coefficient Extract the timestamps of the two network security events to be correlated and calculate the time interval. , the time interval Compared with the preset time threshold, if If it does not exceed the preset time threshold, then based on Calculate the time correlation coefficient If it exceeds this threshold, the time correlation coefficient will... The value is 0;

[0064] S330.3, Correcting the correlation confidence level : Obtain the raw confidence score of the Bayesian network output Using weighting coefficients (satisfy ), through the original confidence level Spatial correlation coefficient Time correlation coefficient The weighted summation is used to calculate the corrected confidence level. ;

[0065] S330.4 Determining Association Relationship: Preset strong association threshold and weak association threshold (where the strong association threshold is higher than the weak association threshold), and adjust the confidence level. Compare with two thresholds; when A strong association is defined as a condition that is not lower than the strong association threshold. A relationship is considered weak if it is not lower than the weak association threshold but lower than the strong association threshold. If the value is below the weak association threshold, it is considered as having no association.

[0066] As a further improvement to this technical solution, the risk assessment and result output in S400 includes the following steps:

[0067] S410.1 Threat Level Assessment: Determine the types of network security events identified by the associated detection, and determine the basic degree of harm by combining the actual harm of this type of network security event to the business system in historical detection; adjust the basic degree of harm according to the strength of event association (strong association or weak association), increase the degree of harm for strongly associated events, and decrease the degree of harm for weakly associated events, thus forming a threat level assessment conclusion;

[0068] S410.2 Impact Scope Assessment: Statistically identify the asset objects involved in the related events (terminal devices, servers, business systems, corresponding to multi-source data related asset information collected in S100); classify the importance of assets according to their actual role in the business architecture; and combine the number of affected assets of each importance level to form an impact scope assessment conclusion.

[0069] S410.3, Diffusion Probability Assessment: Retrieve past event records from the historical database that are consistent with the current related event type, analyze the diffusion path and diffusion rate of similar events; adjust the diffusion rate based on the occurrence time and degree of correlation of the current related event to form a diffusion probability assessment conclusion;

[0070] S410.4 Determination of Comprehensive Risk Level: Based on the assessment conclusions of comprehensive threat level, scope of impact, and probability of spread, weights are allocated according to the priority of business impact (threat level and scope of impact have higher weights than probability of spread); the comprehensive results are divided into four levels of comprehensive risk: "extremely high", "high", "medium" and "low".

[0071] S410.5 Assessment Report Generation and Output: Generate an assessment report that includes association details (unique event identifier, association path, association strength, corresponding S300 association detection conclusion), risk level (comprehensive risk level and assessment conclusions of each dimension), and response recommendations (matching disposal measures according to the comprehensive risk level); the assessment report is accompanied by an association topology diagram and a risk heat map, supports export to common document formats, and provides an API interface for security management systems to call.

[0072] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0073] 1. This invention collects raw data of network security events from multiple sources, including network device logs, terminal behavior logs, network traffic data, and security device alarm data. It also collects related asset information and real-time updated threat intelligence data. Furthermore, it performs standardized preprocessing on the collected data, including format unification, outlier removal, duplicate data deduplication, noise reduction, and standardization. This solves the problem of incomplete data dimension coverage in traditional technologies and provides a comprehensive and standardized data foundation for subsequent feature extraction and correlation detection of network security events.

[0074] 2. This invention employs a dual-attention CNN-LSTM fusion model guided by threat intelligence. This model extracts spatial features such as IP address embedding vectors, port feature encoding, protocol type identifiers, and device type features through CNN branches and adjusts feature weights in conjunction with threat intelligence. It extracts temporal features such as event occurrence sequences, traffic change sequences, and alarm frequency sequences through LSTM branches and assigns weights based on event time intervals and attack timing features. Then, a feature fusion layer completes the element-wise weighted fusion of spatial and temporal features. Subsequent features are also processed by feature concatenation, feature enhancement, feature filtering, and normalization to generate event feature vectors. This model can accurately extract and fuse multi-dimensional event features, solving the problems of insufficient feature mining depth and lack of effective guidance in traditional technologies, and improving the support capability of features for correlation detection.

[0075] 3. This invention employs an improved Apriori-Bayes fusion algorithm to discretize event feature vectors by type. Continuous features are divided into intervals using K-means clustering, while discrete features retain their original categories. Simultaneously, a security event weight factor is introduced to calculate the weighted support of itemsets and dynamically adjust the minimum support threshold to uncover frequent itemsets. A Bayesian network is combined to calculate the association confidence, which is then corrected using spatiotemporal correlation coefficients. The spatial correlation coefficient is calculated based on the latitude and longitude of the IP address, while the temporal correlation coefficient is calculated based on the event time interval. This approach can scientifically determine the association relationships between network security events, solving the problems of limited accuracy and lack of quantitative correction in traditional association determination techniques, and reducing the probability of false positives and false negatives.

[0076] 4. This invention assesses the risk of network security incident correlations from three dimensions: threat level, scope of impact, and probability of spread. The threat level is adjusted by combining the historical harm of the incident type with the strength of the correlation. The scope of impact is determined by statistically analyzing the assets involved and classifying the importance of assets according to business roles. The probability of spread is adjusted by referring to the historical spread paths and occurrence rates of similar incidents and combining them with the current situation. Finally, an assessment report is generated that includes correlation details, risk level, and response suggestions. This report can comprehensively reflect the actual harm of network security incidents and solves the problem of the single dimension of risk assessment in traditional technologies, providing an intuitive and practical basis for security protection decisions. Attached Figure Description

[0077] Figure 1 This is a schematic diagram illustrating the steps of the network security event correlation detection method in this invention;

[0078] Figure 2 This is a schematic diagram illustrating the steps involved in generating event feature vectors in this invention;

[0079] Figure 3 This is a schematic diagram illustrating the steps of discretizing event feature vectors in this invention. Detailed Implementation

[0080] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0081] like Figures 1-3 As shown in the figure, this embodiment provides a network security event correlation detection method based on big data analysis, including the following steps:

[0082] S100. Data Acquisition: Collect raw data of multi-source network security events, including network device logs, terminal behavior logs, network traffic data, and security device alarm data; and collect asset information associated with the raw data of the multi-source network security events (the asset information includes the identifiers of terminal devices, servers, and business systems, their respective business modules, and their importance level in the business architecture); simultaneously collect threat intelligence data, including real-time updated threat intelligence data such as known attack signature databases, vulnerability information databases, and attack chain graphs.

[0083] In this step, in S100, data collection includes the collection of raw data on multi-source cybersecurity incidents, asset information collection, and threat intelligence data collection, wherein:

[0084] The collection of raw data from multi-source cybersecurity incidents includes the following steps:

[0085] S110.1 Network Device Log Collection: Core device logs are collected via SNMPv3 protocol, and access layer device logs are received via Syslog protocol; before collection, the logs are filtered by the device's built-in rules, retaining only logs containing keywords such as "abnormal traffic," "port abnormality," and "authentication failure," and extracting device identifier, event timestamp, and event level fields; and associating the core device and access layer device with the assets to which they belong (such as core business server clusters, access layer terminal areas, etc.) and the asset importance level;

[0086] Specifically, core devices use the SNMPv3 protocol to collect logs, while access layer devices receive logs via the Syslog protocol. Before collection, built-in filtering rules are configured on the device side to retain only logs containing keywords such as "abnormal traffic," "port abnormality," and "authentication failure." Device identifiers, event timestamps, and event level fields are extracted from the filtered logs. Through a pre-established "device-asset association table," core devices and access layer devices are associated with corresponding assets (such as core business server clusters and access layer terminal areas), and the identifiers and asset importance levels of the associated assets are obtained synchronously.

[0087] S110.2 Terminal Behavior Log Collection: A collection agent is deployed on the terminal to capture events through operating system kernel hooks; process creation events are collected in real time, and file operation events are collected in batches at fixed intervals to extract the terminal MAC address, process path, file hash value, and operation result fields; and the terminal asset information (such as department, asset type, and whether it is a critical business terminal) is associated with the terminal MAC address.

[0088] Specifically, a data collection agent is deployed on the terminal device. The agent captures terminal events through operating system kernel hook technology. Real-time collection mode is used for process creation events, and file operation events are collected in batches at fixed time intervals. The terminal MAC address, process path, file hash value, and operation result fields are extracted from the captured events. The "Terminal-Asset Information Mapping Table" is queried based on the terminal MAC address to obtain asset information such as the department to which the terminal belongs, asset type, and whether it is a critical business terminal.

[0089] S110.3 Network Traffic Data Collection: Deploy traffic probes at core nodes to analyze traffic based on DPI technology; collect and record the connection state sequence of TCP sessions in full; sample UDP traffic by "source IP + destination port" to extract session duration, packet size distribution, retransmission rate, and application layer protocol identifier; and associate the source and destination IPs involved in the traffic with asset information (such as the business system to which the server IP belongs, the user group to which the terminal IP belongs, etc.).

[0090] Specifically, traffic probe devices are deployed at core network nodes. The probes analyze the network traffic flowing through them using DPI technology. TCP sessions are collected in full and the complete connection state sequence is recorded. UDP traffic is sampled based on the combination of "source IP + destination port". Session duration, packet size distribution, retransmission rate, and application layer protocol identifier fields are extracted from the parsed traffic data. Through the "IP-asset mapping table", the source IP and destination IP involved in the traffic are associated with the corresponding business systems (server IP association) and user groups (terminal IP association) to obtain relevant asset information.

[0091] S110.4 Security device alarm data collection: collected via RESTful API interface; IDSIPS devices pull data at fixed intervals, and antivirus devices use an instant push mode after alarm generation, extracting attack signature ID, source and destination IP ports, alarm confidence level, and handling suggestions; and associating the asset information corresponding to the source and destination IP ports involved in the alarm (such as the asset category and protection level of the attacked IP).

[0092] Specifically, a data interaction channel is established with security devices through a RESTful API interface. The IDS / IPS device pulls alarm data at a fixed set time interval, and the antivirus device is configured to push data to the collection end immediately after the alarm is generated. The attack feature ID, source and destination IP ports, alarm confidence, and handling suggestions are extracted from the acquired alarm data. Based on the source and destination IP ports in the alarm data, the "IP Port-Asset Mapping Table" is queried to obtain asset information such as the asset category and protection level of the attacked IP.

[0093] Asset information collection includes the following steps:

[0094] S120.1 Asset Basic Information Collection: Synchronize asset data from the asset ledger system, including the unique identifier of terminal equipment, server, and business system, hardware configuration, business module to which it belongs, role and importance level in the business architecture; adopt a combination of scheduled synchronization and change-triggered synchronization to ensure the real-time nature of asset information;

[0095] Specifically, a data synchronization channel is established with the company's existing asset ledger system to synchronize asset data from terminal devices, servers, and business systems. The synchronized data includes the asset's unique identifier, hardware configuration, business module, role in the business architecture, and asset importance level. The data synchronization mechanism is configured as "scheduled synchronization + change-triggered synchronization." Scheduled synchronization is executed according to a set period, and when asset information in the asset ledger system changes, immediate synchronization is triggered to ensure that the collected basic asset information is consistent with the actual asset status.

[0096] S120.2 Asset Association Collection: Through network topology discovery tools and business configuration management systems, collect network connection relationships and business dependencies between assets; generate asset association maps to provide a basis for subsequent risk diffusion analysis;

[0097] Specifically, deploy network topology discovery tools to scan the enterprise network and obtain network connectivity relationships between assets; connect to the business configuration management system to extract business dependencies between assets from the system; integrate the collected network connectivity relationships and business dependencies, and generate an asset association graph according to the structure of "node (asset) - edge (association type)". This graph is used for subsequent risk diffusion analysis of network security incidents.

[0098] Threat intelligence data collection includes the following steps:

[0099] S130.1, Collection of known attack signature database: Interact with multiple heterogeneous intelligence sources and normalize them through an aggregation engine; the malicious IP database is stored in the structure of "IP + threat type + first appearance time", and the full synchronization cycle and incremental synchronization trigger conditions are configured. The AC automaton algorithm is used for signature matching, and invalid signatures are cleaned up periodically.

[0100] Specifically, it connects to multiple heterogeneous threat intelligence sources (such as public threat intelligence platforms and commercial intelligence services) and obtains attack signature data through the interfaces provided by the intelligence sources; it deploys an aggregation engine to normalize data from different intelligence sources, unifying data formats and field definitions; it stores malicious IP data in a structure of "IP + threat type + first appearance time", configures a full synchronization cycle (such as once a week) and incremental synchronization trigger conditions (such as when a set number of new signature data is added in a single instance); it uses the AC automaton algorithm to match the collected attack signature codes, and regularly filters and cleans up invalid signatures (such as signatures of long-term unrelated attack events).

[0101] S130.2 Vulnerability Information Database Collection: Based on data synchronization from the official CVE interface, the synchronization frequency is consistent with the official updates. Extract CVE-ID, release date, affected versions, CVSS score and sub-items, and exploit public status fields; trigger endpoint scanning tasks for high-risk vulnerabilities;

[0102] Specifically, vulnerability data is synchronized through the interface provided by the official CVE website, with the synchronization frequency consistent with the official CVE update frequency; CVE-ID, release date, affected versions, CVSS score and sub-items, and exploit public status fields are extracted from the synchronized vulnerability data; a high-risk vulnerability triggering mechanism is set up so that when a vulnerability with a CVSS score reaching the high-risk standard is collected, an endpoint scanning task is automatically triggered to check whether the vulnerability exists on the enterprise's internal endpoint devices.

[0103] S130.3 Attack Chain Graph Acquisition: Graph extraction algorithms are used to extract nodes from intelligence reports and construct a graph according to the attack stage model; node association weights are calculated based on the frequency of historical attack events, the graph topology update cycle is configured, and an emergency update is triggered when the number of newly disclosed events reaches a set number.

[0104] Specifically, a graph extraction algorithm is used to analyze threat intelligence reports and extract attack-related nodes (such as attack phases, attack behaviors, and attack vectors). Based on the attack phase model (such as by "initial access-execution-persistence"), the extracted nodes are constructed into an attack chain graph. The weight of the relationship between nodes is calculated based on the frequency of the relationship in historical attack events. A regular update cycle for the attack chain graph is configured (such as once every two weeks). When the number of newly disclosed attack events reaches a set threshold, an emergency update of the graph is triggered to ensure the timeliness of the graph.

[0105] S200, Feature Extraction and Fusion: The collected raw data of multi-source network security events and threat intelligence data are formatted, outliers are removed, duplicate data is deduplicated, noise is reduced, and standardization is performed. A dual-attention CNN-LSTM fusion model with threat intelligence guidance is used to extract spatial and temporal features from the preprocessed data, and the extracted spatial and temporal features are fused to obtain the event feature vector.

[0106] In this step, S200, the collected multi-source network security event raw data and threat intelligence data are subjected to format unification, outlier removal, duplicate data deduplication, and noise reduction, including the following steps:

[0107] S210.1, Format Unification: Regular expressions are used to match IP address, port number, timestamp, and attack type fields in unstructured logs and convert them into key-value pairs; all types of timestamps are uniformly converted to UTC format, accurate to milliseconds; lexical analysis tools are used to extract core semantic keywords from text description fields; and the signature format in threat intelligence is uniformly converted to hexadecimal strings.

[0108] Specifically, this embodiment uses regular expressions to write targeted matching rules for IP address, port number, timestamp, and attack type fields, converting these fields in unstructured logs into key-value pairs. A time format conversion tool is used to uniformly convert various timestamps to UTC format (accurate to milliseconds), resolving cross-data source time alignment issues. A lexical analysis tool is used to extract core semantic keywords from text-based description fields, filtering out meaningless words. An encoding conversion tool is used to uniformly convert the signature codes in threat intelligence into hexadecimal strings, ensuring signature code format consistency and laying the foundation for subsequent feature matching. This design logic focuses on customizing rules for core dimensions of network security data, avoiding redundant interference from general tools, and achieving automated processing of multi-source data from heterogeneous to unified.

[0109] S210.2 Outlier Removal: For numerical features, the interquartile range method is used to calculate the upper quartile Q3 and lower quartile Q1 of the data, and values ​​outside the range determined by Q1, Q3 and the set coefficient are removed; for categorical features, rare categories with a frequency below the set threshold are removed.

[0110] Specifically, addressing the issues of high misjudgment rates for numerical feature outliers and interference from niche categories in the model caused by the traditional fixed threshold method, this design employs the interquartile range (IMR) method for numerical features (such as session duration and packet size). It calculates the upper quartile (Q3) and lower quartile (Q1) using statistical tools, sets coefficients (based on the distribution patterns of security data), and eliminates values ​​outside the range determined by Q1, Q3, and the set coefficients. For categorical features (such as protocol type and attack type), it statistically analyzes the frequency of each type, sets thresholds (based on the proportion of high-frequency network security threats), and eliminates rare categories with frequencies below the set thresholds. This design logic abandons the "one-size-fits-all" fixed threshold model, adopts an objective statistical method based on data distribution, and adapts to different security data characteristics. This improves the accuracy of anomaly identification while simplifying data dimensions to avoid interference from niche features in the model.

[0111] S210.3 Deduplication of Duplicate Data: Using "source IP + destination IP + event type + timestamp" as the composite primary key, retain the first occurrence of the record and delete subsequent duplicate entries; for log data, additionally check the consistency of process path and operation result fields, and if they are completely consistent, they are judged as duplicate data and removed.

[0112] Specifically, this embodiment uses "source IP + destination IP + event type + timestamp" as the composite primary key. A data deduplication tool retains the first occurrence of the record and deletes subsequent duplicate entries, covering the core identification dimensions of network security events. For log-type data (such as network device logs and terminal behavior logs), the process path and operation result fields are additionally verified. Only when the composite primary key is completely consistent with the above fields is it judged as duplicate data and removed, avoiding the omission of non-duplicate events that are "of the same subject and in the same sequence but with different operations". This design logic achieves accurate identification of duplicate data through multi-dimensional joint judgment, which saves storage resources and avoids duplicate data interfering with subsequent correlation detection.

[0113] S210.4 Noise Reduction: For network traffic data packet content, a sliding window averaging method is used to smooth burst noise, with the window size set to 5-10 consecutive data packets; for text logs, meaningless words are filtered out through a preset stop word list, while core feature words (such as attack behavior, device identification, etc.) are retained; for fuzzy feature descriptions in threat intelligence, they are standardized through a thesaurus mapping table.

[0114] Specifically, this embodiment uses a sliding window averaging method to smooth burst noise in network traffic data packets, with the window size set to 5-10 consecutive data packets (balancing noise smoothing effect and real-time performance); for text logs, redundant information is filtered through a preset stop word list (focusing on security scenarios and filtering meaningless and non-security-related words), retaining core feature words such as attack behavior and device identification; for fuzzy feature descriptions in threat intelligence, a synonym mapping table is established (customized based on the "multiple descriptions of the same threat" problem in the threat intelligence database) to achieve standardized processing. This design logic is tailored to the noise characteristics of different data types, improving data purity and providing high-quality input for subsequent preprocessing and feature extraction.

[0115] Furthermore, the preset stop word list includes, but is not limited to, the following core meaningless words: "normal", "regular", "no abnormality", "system default", and "temporary". The mapping rule of the synonym mapping table is to perform bidirectional mapping between the abbreviations and full names of technical terms. For example, "SQL injection" is mapped to "structured query language injection", "DDoS attack" is mapped to "distributed denial-of-service attack", and "XSS vulnerability" is mapped to "cross-site scripting vulnerability", etc., to ensure that the characteristics of the same security event with different expressions can be uniformly identified.

[0116] In this step, S200, the collected raw data of multi-source network security incidents and threat intelligence data undergoes standardized preprocessing, including the following steps:

[0117] S220.1. Z-score standardization is adopted for continuous numerical features. The mean and standard deviation of the features are calculated by using data standardization tools. The feature values ​​are converted into a distribution with a mean of 0 and a standard deviation of 1 according to the standardization formula, eliminating the interference of features of different magnitudes on the subsequent model. This design logic is adapted to the unbounded characteristics of continuous features, ensuring that features participate in model calculations at the same magnitude.

[0118] S220.2. For features with fixed value ranges, min-max standardization is used to map them to the [0,1] interval. For features with fixed value ranges (such as CVSS score, retransmission rate, and asset importance level), min-max standardization is used. The feature values ​​are mapped to the [0,1] interval by formula using data standardization tools to unify the feature value range and facilitate the model's unified calculation of feature weights.

[0119] S220.3 For skewed distribution features, perform logarithmic transformation first and then standardize. For features with skewed distribution (such as network traffic packet size and terminal file operation frequency), perform logarithmic transformation first using data transformation tools to alleviate data skewness, and then perform standardization to make the transformed data closer to a normal distribution and improve the representativeness of the features.

[0120] S220.4. One-hot encoding is used to convert categorical features into binary vectors; embedding techniques are used to map high-cardinality categories into low-dimensional dense vectors. Low-cardinality categorical features (such as protocol type, attack type, with a small number of categories) are converted into binary vectors using one-hot encoding; high-cardinality categorical features (such as business system name, terminal department, with a large number of categories) are mapped into low-dimensional dense vectors using embedding techniques, which reduces feature dimensionality while preserving semantic relationships between categories.

[0121] In this step, in S200, the threat intelligence-guided dual-attention CNN-LSTM fusion model includes a spatial feature extraction branch (CNN), a temporal feature extraction branch (LSTM), and a feature fusion layer, wherein:

[0122] The spatial feature extraction branch of CNN includes an input layer, convolutional layers, a threat intelligence-spatial attention layer, and pooling layers, among which:

[0123] The input layer receives preprocessed spatial feature data, including IP address embedding vectors, port feature encoding, protocol type identifiers, and device type features;

[0124] A convolutional layer consists of a preset number of sub-layers, each configured with a convolutional kernel of a specified size and stride, and each layer is followed by batch normalization and a ReLU activation function.

[0125] The threat intelligence-spatial attention layer adjusts feature weights based on threat intelligence database data. If the IP address corresponding to a feature matches a malicious IP database, the port belongs to a high-risk port list, or the protocol type is a known commonly used attack protocol, the weight of the corresponding feature channel is increased by a preset ratio.

[0126] The pooling layer uses global max pooling to reduce the dimensionality of the convolution output features, generating spatial feature vectors of fixed dimensions;

[0127] Specifically, addressing the lack of threat intelligence guidance in traditional spatial feature extraction models, the spatial feature extraction branch includes an input layer, convolutional layers, a threat intelligence-spatial attention layer, and pooling layers. The input layer receives preprocessed spatial feature data. The convolutional layer consists of a preset number of sub-layers, each configured with a specified kernel size and stride. Each layer is followed by batch normalization (to avoid gradient vanishing) and a ReLU activation function (to introduce non-linear features). The threat intelligence-spatial attention layer adjusts feature weights based on threat intelligence database data (malicious IP database, high-risk port list, and known commonly used attack protocols). If the IP address corresponding to a feature matches a malicious IP database, the port belongs to the high-risk port list, or the protocol type is a known commonly used attack protocol, the weight of the corresponding feature channel is increased by a preset ratio to strengthen key spatial features. The pooling layer uses global max pooling to reduce the dimensionality of the convolutional output features, generating a fixed-dimensional spatial feature vector. This design logic combines threat intelligence with spatial feature extraction, making the model more focused on high-risk spatial features and improving the targeting of feature extraction.

[0128] Furthermore, in constructing the threat intelligence-guided dual-attention CNN-LSTM fusion model, the convolutional layer contains three sub-layers, each using a 3×3 kernel size and a stride of 1; the LSTM layer consists of two layers, with the sequence length determined based on the temporal feature dimension of the input security events. Specifically, the feature sequence of 12 consecutive security event logs is used as input (i.e., the sequence length is 12). These parameters are determined based on the average length of the network security event data, the feature dimension (statistically, the feature dimension of a single event is 64 dimensions), and the temporal correlation of historical attack behaviors (the time span of a typical attack chain is concentrated in 1-2 hours, and sampling is done at 5 minutes per log entry, hence 12 entries are selected), which can effectively extract the spatial features and long-term dependency features of events.

[0129] The temporal feature extraction branch of LSTM includes an input layer, an LSTM layer, a temporal attention layer, and a fully connected layer; among which:

[0130] The input layer receives time-series data sorted by timestamp, including event occurrence sequences, traffic change sequences, and alarm frequency sequences, with the sequence length being a preset value;

[0131] An LSTM layer consists of a preset number of LSTM units and is used for temporal feature extraction. Each layer retains the output of all time steps or the output of the last time step.

[0132] The time attention layer assigns weights based on event time intervals and attack timing features in threat intelligence. The weight of adjacent events with time intervals less than a set threshold is increased, and the weight of event sequences that match the timing of known attack chains is strengthened.

[0133] The fully connected layer adjusts the dimension of the LSTM output features to generate a temporal feature vector with the same dimension as the spatial feature vector.

[0134] Specifically, addressing the pain point of traditional models' temporal feature extraction not incorporating attack timing patterns, the temporal feature extraction branch LSTM includes an input layer, an LSTM layer, a temporal attention layer, and a fully connected layer. The input layer receives temporal data sorted by timestamp. The LSTM layer consists of a preset number of LSTM units for temporal feature extraction, with each layer retaining the output of all time steps or the output of the last time step. The temporal attention layer assigns weights based on event time intervals and attack timing features in threat intelligence (such as the temporal correlation of each stage of the attack chain). The weight of adjacent events with time intervals less than a set threshold is increased, and the weight of event sequences matching the known attack chain timing is strengthened. The fully connected layer adjusts the dimension of the LSTM output features to generate a temporal feature vector with the same dimension as the spatial feature vector. This design logic combines event timing characteristics with attack chain patterns, enabling the model to more accurately capture temporal correlation features.

[0135] The feature fusion layer performs element-wise weighted fusion of spatial and temporal feature vectors. The weights are set based on the contribution of spatial and temporal feature vectors to historical correlation detection. After nonlinear transformation by the activation function, the fused comprehensive feature vector is output for subsequent event feature vector generation.

[0136] Specifically, to address the issue that traditional model feature fusion does not consider historical contributions, the feature fusion layer performs element-wise weighted fusion of spatial and temporal feature vectors. The weights are set based on the contributions of spatial and temporal feature vectors in historical association detection. After nonlinear transformation by the activation function, the fused comprehensive feature vector is output for subsequent event feature vector generation. This design logic dynamically allocates fusion weights based on historical detection data to ensure the optimal combination of spatial and temporal features and improve the expressive power of the comprehensive features.

[0137] like Figure 2 As shown, in this step, the generation of the event feature vector in S200 includes the following steps:

[0138] S230.1 Feature concatenation: The spatial feature vector output by the CNN branch of spatial feature extraction and the temporal feature vector output by the LSTM branch of temporal feature extraction are concatenated sequentially according to their dimensions to form an initial fusion vector whose dimension is the sum of the dimensions of the spatial feature vector and the temporal feature vector.

[0139] Specifically, the spatial feature vector output by the CNN branch (spatial feature extraction branch) and the temporal feature vector output by the LSTM branch (temporal feature extraction branch) are concatenated dimensionally. For example, if both the spatial and temporal feature vectors are 256-dimensional, the concatenation results in an initial fusion vector with a dimension of 512. This step improves upon the limitations of single-dimensional features by integrating core information from both spatial (e.g., IP address, port attributes) and temporal (e.g., event sequences, alarm frequency changes) dimensions. This ensures the initial fusion vector fully covers the key features in the preprocessed data, providing a comprehensive information foundation for subsequent feature optimization and preventing incomplete characterization of complex attack scenarios in subsequent correlation detection due to missing feature dimensions.

[0140] S230.2 Feature Enhancement: The initial fusion vector from S230.1 is input into a fully connected layer for nonlinear transformation. The number of neurons in the fully connected layer is the same as the dimension of the initial fusion vector. The activation function is LeakyReLU to enhance the correlation between features of different dimensions.

[0141] Specifically, the initial fusion vector obtained in S230.1 is input into a fully connected layer for nonlinear transformation. The number of neurons in the fully connected layer is consistent with the dimension of the initial fusion vector (e.g., if the initial fusion vector is 512-dimensional, the number of neurons is set to 512). The activation function is LeakyReLU. The improvement of this step lies in strengthening the intrinsic correlation between features of different dimensions through nonlinear transformation. Compared with traditional linear feature processing methods, the LeakyReLU activation function can effectively retain more feature details, avoid the loss of feature information during transformation, and ultimately improve the nonlinear expressive power of features. This makes the features more consistent with the actual characteristics of "multi-factor correlation" in cybersecurity incidents, providing more representative feature support for subsequent accurate correlation detection.

[0142] S230.3 Feature Filtering: Apply L1 regularization constraints to the enhanced feature vectors. The regularization coefficient is set according to the cross-validation results of historical detection data. By suppressing weights with smaller absolute values, the top 80% of feature dimensions that contribute to the correlation detection are retained.

[0143] Specifically, L1 regularization is applied to the enhanced feature vector, where the regularization coefficient is determined through cross-validation of historical correlation detection data (e.g., based on security event data from the past 3 months, a coefficient of 0.01 is selected after 5-fold cross-validation). L1 regularization suppresses dimensions with smaller absolute weights in the feature vector, ultimately retaining the top 80% of feature dimensions contributing to correlation detection. The improvement in this step lies in the targeted removal of redundant features. Compared to traditional feature processing without filtering, it effectively reduces the interference of irrelevant or low-contribution features (such as the "device model minor parameter" feature, which has minimal impact on correlation determination) on subsequent calculations. While simplifying feature dimensions, it ensures that core features are not lost, significantly improving the computational efficiency of subsequent correlation detection algorithms.

[0144] S230.4 Normalization: Perform L2 normalization on the filtered feature vectors to make the L2 norm of the feature vectors 1;

[0145] Specifically, L2 normalization is performed on the feature vectors after feature filtering. By calculating the squares and square roots of each element in the feature vector, the vector is adjusted to a standard form with an L2 norm of 1. The improvement of this step lies in unifying the numerical scale of the feature vectors. This avoids the bias in weight calculation for features with larger numerical scales during subsequent association rule mining due to differences in the numerical magnitudes of different feature dimensions (such as "session duration" ranging from 1 to 1000 seconds and "protocol type encoding" ranging from 0 to 10). This ensures that each feature has a fair weight ratio in association detection, improving the stability and accuracy of association confidence calculation.

[0146] S230.5 Output Event Feature Vector: Output a fixed-dimensional event feature vector obtained after processing, which is used for subsequent association rule mining and detection of network security events.

[0147] Specifically, the feature vectors processed by S230.1-S230.4 are output as standardized vectors with fixed dimensions (e.g., the final output dimension is uniformly 400 dimensions, determined after filtering in S230.3). The improvement in this step lies in ensuring the uniformity of the format and dimensions of the output vectors. Compared to the traditional case where the output vector dimensions are not fixed, fixed-dimensional vectors can directly adapt to the input requirements of the improved Apriori-Bayes fusion algorithm in the subsequent S300 association detection without additional dimension adjustment. This ensures the smoothness of the association rule mining and event association determination process, while providing a standardized vector benchmark for feature comparison and association analysis of different batches of security events.

[0148] S300. Association Detection: An improved Apriori-Bayesian fusion algorithm is used to discretize the event feature vectors (continuous features are divided into discrete intervals using K-means clustering, while discrete features retain their original categories); frequent itemsets are mined using the improved Apriori algorithm, association confidence is calculated using a Bayesian network, and a spatiotemporal association coefficient, including spatial and temporal association coefficients, is introduced to correct the confidence (spatial association coefficient similarity is calculated using the geocoding of IP addresses, and temporal association coefficient is calculated by taking the reciprocal of the event time interval); the association relationship between network security events is determined based on the corrected confidence.

[0149] Understandably, while the traditional Apriori algorithm can uncover frequent itemsets in large-scale transaction data, it has limitations in enterprise-level cybersecurity event correlation detection: First, it uses indiscriminate support calculation, failing to differentiate event risk levels, and high-risk but low-frequency key event itemsets are easily missed due to insufficient support; second, the fixed minimum support threshold cannot adapt to the dynamic distribution of security events as "high-risk low-frequency, low-risk high-frequency," and it does not filter redundant itemsets containing rare events, leading to an expansion of the candidate set size, reduced algorithm efficiency, and increased risk of false positives in subsequent correlation determinations.

[0150] To address the aforementioned issues, this embodiment employs an improved Apriori algorithm, achieving optimization through three core improvements: First, it introduces a security event weighting factor. The weighted support formula is used. First, it calculates the support of itemsets to specifically reflect the value of high-risk event itemsets; second, it dynamically adjusts the minimum support threshold based on historical data, using a decay coefficient. Third, set a threshold for the proportion of rare events to achieve adaptive matching between the threshold and changes in security event data. Pre-filter the candidate set containing rare events to reduce the size of the candidate set;

[0151] Through the above improvements, the improved Apriori algorithm effectively solves the limitations of traditional algorithms, can specifically reflect the value of high-risk event itemsets, achieve adaptive matching between thresholds and changes in security event data, and can also compress the size of candidate sets, providing a more efficient and accurate frequent itemset foundation for the correlation detection of network security events, and ensuring the quality and efficiency of subsequent correlation analysis.

[0152] like Figure 3 As shown, in this step, the discretization process of the event feature vector in S300 includes the following steps:

[0153] S310.1 Discretization of Continuous Features: For continuous features (such as flow rate change rate, alarm frequency fluctuation value, etc.), K-means clustering algorithm is used to divide them into discrete intervals; number of clusters. The initial cluster centers are determined by the elbow method, and the farthest priority strategy is adopted for selection. The iteration termination condition is that the change in the cluster centers is less than the set threshold or the preset number of iterations is reached.

[0154] Specifically, for continuous features such as traffic change rate and alarm frequency fluctuations, the K-means clustering algorithm is used to divide discrete intervals, including:

[0155] First, determine the number of clusters. Calculate different using the elbow method (Values ​​2-10) corresponding to the sum of squared clustering errors The formula is: ,in For the sum of squared clustering errors, For the first The number of samples in each cluster. For continuous features, For the first In the cluster, the first The first sample 3D eigenvalues For the first In the cluster, the first Cluster centers with dimensional features, The inflection point where the rate of descent suddenly decreases is considered optimal. ;

[0156] Second, initial cluster centers are selected using a farthest-first strategy, employing the Euclidean distance formula. The sample with the largest distance from the already selected center is selected as the new center; where As a sample, Selected cluster centers;

[0157] Third, set an iteration termination condition, which is when the cluster center change... Alternatively, iteration may stop when the number of iterations reaches a preset maximum of 50; where... For the first The iteration of the ... The center Dimensional value;

[0158] Fourth, divide the data into discrete intervals, sort them by cluster center values ​​from smallest to largest, and assign discrete labels to each interval.

[0159] S310.2, Discrete Feature Processing: For discrete features (such as protocol type, attack type identifier, etc.), retain the original category label; for rare categories with a frequency below a set threshold, merge them into the "Other" category;

[0160] Specifically, for discrete features such as protocol type and attack type identifier, the original category labels are directly retained; the frequency of each discrete category in the event feature vector dataset is counted, a frequency threshold is set, and rare categories with a frequency lower than the set threshold are merged into the "other" category to avoid rare categories interfering with subsequent frequent itemset mining.

[0161] S310.3 Feature encoding conversion: Convert all discretized features into integer codes and establish a mapping table between the codes and the original feature values.

[0162] Specifically, all discretized features are uniformly converted into integer codes, and a unique integer is assigned to each discrete label (e.g., protocol type "TCP" is encoded as 1, "UDP" is encoded as 2, and "other" is encoded as 3). At the same time, an "encoding-original feature value" mapping table is established to clarify the correspondence between integer codes and original feature values, which facilitates the reverse lookup of original feature information in subsequent association results.

[0163] In this step, in S300, frequent itemsets are mined using the improved Apriori algorithm, including the following steps:

[0164] S320.1 Introduce a security event weighting factor, and assign corresponding weight coefficients to different events according to their risk levels. Combined with itemsets Number of transactions and total number of transactions The weighted support of the itemset is calculated by multiplying the weight coefficient by the proportion of transactions. ;

[0165] Specifically, a security event weighting factor is introduced. The weighted support of an itemset is calculated using the following formula: ,in For itemsets Weighted support Weighting coefficients assigned based on the risk level of an event (e.g., high-risk events) Medium-risk events Low-risk events ), For containing itemsets Number of transactions (datasets containing itemsets) (number of events) The total number of transactions (the total number of all events in the dataset); this is determined first based on the event risk level. , then statistics and Substituting these values ​​into the formula yields the weighted support of the itemset, ensuring that itemsets corresponding to high-risk events receive more attention in frequent itemset mining.

[0166] S320.2 Setting the initial threshold for minimum support based on the average frequency of historical event associations. If the number of frequently generated itemsets is lower than the preset value, then the initial threshold of the current minimum support will be used. With attenuation coefficient The product update threshold is repeatedly adjusted until the itemset quantity requirement is met.

[0167] Specifically, the initial threshold for minimum support is set based on the average frequency of historical event associations. If according to the current If the number of generated frequent itemsets is lower than a preset value, the threshold is updated according to the following formula: ,in This is the updated minimum support threshold. The decay coefficient is used as the adjustment factor; this is repeated until the number of frequent itemsets generated meets the requirements for association rule mining. The possible value is 0.8, which is suitable for the typical distribution characteristics of network security data.

[0168] S320.3 Setting the threshold for the proportion of rare events Statistical analysis of the frequency of rare events , for satisfying Less than the proportional threshold The itemset containing rare events multiplied by the total number of transactions is filtered in advance to reduce the size of the candidate set;

[0169] Specifically, set a threshold for the proportion of rare events. Count the frequency of each event in the dataset. If the frequency of the event satisfy If the event is rare, then the event is determined to be rare. Traverse all initial candidate itemsets. If an itemset contains a rare event, then directly remove the candidate itemset to compress the size of the candidate set and improve the efficiency of the algorithm.

[0170] S320.4, Weight the support of itemsets With the adjusted minimum support threshold By comparison, itemsets with support levels not lower than a threshold are selected as frequent itemsets for subsequent association rule mining.

[0171] Specifically, iterate through the candidate itemset filtered by rare events and calculate the weighted support for each itemset. ;Will With the adjusted minimum support threshold If a comparison is made, If the candidate itemset is selected, it is determined to be a frequent itemset; the selected frequent itemsets are stored in the database for subsequent calculation of Bayesian network association confidence.

[0172] In step S300, the spatiotemporal correlation coefficient is introduced to correct the confidence level and determine the correlation relationship, including the following steps:

[0173] S330.1 Calculate the spatial correlation coefficient : Obtain the latitude and longitude coordinates of the IP address of the event to be associated and Combined with Earth's radius Calculate the spherical distance between two points Then based on the preset distance threshold By distance With threshold The mapping relationship determines the spatial correlation coefficient. ;

[0174] Specifically, first obtain the latitude and longitude coordinates of the IP address of the event to be associated. and Combined with Earth's radius (Using a value of 6371 km, the internationally accepted average radius of the Earth), the spherical distance between two points is calculated using the Haversine formula. The formula is: ;

[0175] Then set the preset distance threshold. (Using a value of 100km to suit the coverage of enterprise intranets or regional networks), the spatial correlation coefficient is determined according to the following formula. The formula is: ; The value ranges from 0 to 1; a larger value indicates a stronger spatial correlation between the events to be correlated. When (i.e., two network entities belong to the same IP address location), The value of is 1.

[0176] S330.2 Calculate the time correlation coefficient Extract the timestamps of the two network security events to be correlated and calculate the time interval. , the time interval Compared with the preset time threshold, if If it does not exceed the preset time threshold, then based on Calculate the time correlation coefficient If it exceeds this threshold, the time correlation coefficient will... The value is 0;

[0177] Specifically, extract the timestamps of the two network security events to be correlated and calculate the time interval between them. ( The unit is minutes); set a preset time threshold. (A value of 30 minutes is available, which is based on the normal time interval setting for multi-step operations in a network attack chain.)

[0178] If time interval No more than The time correlation coefficient is calculated using the following formula. : ;

[0179] like Exceed Then the time correlation coefficient The value is 0. The value ranges from 0 to 1. The larger the value, the stronger the correlation between the events to be associated in the time dimension.

[0180] S330.3, Correcting the correlation confidence level : Obtain the raw confidence score of the Bayesian network output Using weighting coefficients (satisfy ), through the original confidence level Spatial correlation coefficient Time correlation coefficient The weighted summation is used to calculate the corrected confidence level. ;

[0181] Specifically, first obtain the raw confidence scores of the Bayesian network output. Set weighting coefficients (satisfy ,in The weights of the original confidence levels. The weights of the spatial correlation coefficients, As the weight of the time correlation coefficient, in this embodiment The corrected association confidence level is calculated using the following formula. : ;in, Spatial correlation coefficient; This is the time correlation coefficient; The value ranges from 0 to 1, and the larger the value, the stronger the reliability of the association of the events to be associated.

[0182] S330.4 Determining Association Relationship: Preset strong association threshold and weak association threshold (where the strong association threshold is higher than the weak association threshold), and adjust the confidence level. Compare with two thresholds; when A strong association is defined as a condition that is not lower than the strong association threshold. A relationship is considered weak if it is not lower than the weak association threshold but lower than the strong association threshold. If the value is below the weak association threshold, it is considered as having no association.

[0183] Specifically, a strong correlation threshold and a weak correlation threshold are preset (where the strong correlation threshold is higher than the weak correlation threshold; in this embodiment, the strong correlation threshold can be 0.7, and the weak correlation threshold can be 0.4); the corrected confidence level is then... Comparison with strong correlation threshold and weak correlation threshold:

[0184] like If so, the events to be associated are determined to be strongly associated;

[0185] like If so, it is determined to be a weak association;

[0186] like If so, it is determined to be unrelated;

[0187] The association determination results (including association type and adjusted confidence level) will be used to determine the association results. The IP address (latitude and longitude, event time interval) is bound to the corresponding frequent itemsets and stored in the associated detection result database to provide data support for subsequent risk assessment of network security incidents.

[0188] S400 Risk Assessment and Results Output: For the correlation of cybersecurity incidents determined by correlation detection, conduct risk assessment in terms of threat level, scope of impact, and probability of spread, and generate and output an assessment report including correlation details, risk level, and response recommendations.

[0189] In this step, the risk assessment and result output in S400 includes the following steps:

[0190] S410.1 Threat Level Assessment: Determine the types of network security events identified by the associated detection, and determine the basic degree of harm by combining the actual harm of this type of network security event to the business system in historical detection; adjust the basic degree of harm according to the strength of event association (strong association or weak association), increase the degree of harm for strongly associated events, and decrease the degree of harm for weakly associated events, thus forming a threat level assessment conclusion;

[0191] Specifically, based on the network security incident logs of the past three years, the system statistically analyzes indicators such as the duration of business interruption and the amount of data leakage for different incident types, and establishes a mapping table of "incident type - basic severity level" (which is stored in the risk assessment rule base). After obtaining the incident type determined by S300, the system retrieves the corresponding basic severity level. Combining this with historical related event data (strong correlation amplifies the severity, weak correlation reduces it), the system adjusts the correlation strength according to S300 to form four levels of conclusion: "extremely high", "high", "medium", and "low".

[0192] S410.2 Impact Scope Assessment: Statistically identify the asset objects involved in the related events (terminal devices, servers, business systems, corresponding to multi-source data related asset information collected in S100); classify the importance of assets according to their actual role in the business architecture; and combine the number of affected assets of each importance level to form an impact scope assessment conclusion.

[0193] Specifically, the number of terminals, servers, and business systems involved in the related events is counted from the S100 asset information database; the assets are classified as "critical," "important," and "general" according to their business functions (whether they carry core transactions or store sensitive data) and network topology location; and combined with the business coverage of the affected assets, an assessment conclusion of "broad," "relatively broad," "general," and "small" is formed.

[0194] S410.3, Diffusion Probability Assessment: Retrieve past event records from the historical database that are consistent with the current related event type, analyze the diffusion path and diffusion rate of similar events; adjust the diffusion rate based on the occurrence time and degree of correlation of the current related event to form a diffusion probability assessment conclusion;

[0195] Specifically, retrieve records of similar related events from the historical database, and statistically analyze the basic diffusion rate and typical paths; adjust the incidence rate based on the network load at the current event time (higher working time load indicates a higher diffusion probability) and the degree of S300 correlation (stronger correlation indicates a higher probability) to form a three-level conclusion of "high", "medium" and "low".

[0196] S410.4 Determination of Comprehensive Risk Level: Based on the assessment conclusions of comprehensive threat level, scope of impact, and probability of spread, weights are allocated according to the priority of business impact (threat level and scope of impact have higher weights than probability of spread); the comprehensive results are divided into four levels of comprehensive risk: "extremely high", "high", "medium" and "low".

[0197] Specifically, based on historical risk event loss data, and with the following weights determined: threat level 40%, impact range 40%, and spread probability 20%;

[0198] The conclusions of each dimension are quantified and assigned values: if the dimension conclusion is "extremely high (threat) / extremely wide (scope of influence) / extremely high probability (spread)," it is scored as 4 points; if it is "high / wide / high probability," it is scored as 3 points; if it is "medium / medium range / medium probability," it is scored as 2 points; if it is "low / small range / low probability," it is scored as 1 point.

[0199] And calculate according to the following formula : and based on Divided into four levels:

[0200] A value of ≥3.5 indicates "extremely high risk";

[0201] This is classified as "high risk";

[0202] It is classified as "medium risk";

[0203] It is classified as "low risk".

[0204] S410.5 Assessment Report Generation and Output: Generate an assessment report that includes association details (unique event identifier, association path, association strength, corresponding S300 association detection conclusion), risk level (comprehensive risk level and assessment conclusions of each dimension), and response recommendations (matching disposal measures according to the comprehensive risk level); the assessment report is accompanied by an association topology diagram and a risk heat map, supports export to common document formats, and provides an API interface for security management systems to call.

[0205] Specifically, it extracts event identifiers, associated paths, and associated strengths from S300; integrates assessment conclusions from various dimensions and comprehensive risk levels, and matches historically verified response recommendations according to the level; generates an associated topology map (node ​​table assets, edge table associations) and a risk heat map; the report supports exporting to PDF and DOC formats and provides a RESTful API for security management systems to call.

[0206] Those skilled in the art will understand that the process of implementing all or part of the steps of the above embodiments can be carried out by hardware or by a program instructing the relevant hardware.

[0207] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting network security event correlation based on big data analysis, characterized in that, Includes the following steps: S100. Data Acquisition: Acquire raw data of multi-source network security events, including network device logs, terminal behavior logs, network traffic data, and security device alarm data, and acquire asset information associated with the raw data of the multi-source network security events; at the same time, acquire threat intelligence data including known attack signature databases, vulnerability information databases, and attack chain graphs that are updated in real time. S200, Feature Extraction and Fusion: The collected multi-source network security event raw data and threat intelligence data undergo format unification, outlier removal, duplicate data deduplication, noise reduction, and standardization preprocessing. A threat intelligence-guided dual-attention CNN-LSTM fusion model is used to extract spatial and temporal features from the preprocessed data, and these extracted features are fused to obtain an event feature vector. In S200, the threat intelligence-guided dual-attention CNN-LSTM fusion model includes a spatial feature extraction branch (CNN), a temporal feature extraction branch (LSTM), and a feature fusion layer. in: The spatial feature extraction branch of CNN includes an input layer, convolutional layers, a threat intelligence-spatial attention layer, and pooling layers, among which: The input layer receives preprocessed spatial feature data, including IP address embedding vectors, port feature encoding, protocol type identifiers, and device type features; The threat intelligence-spatial attention layer adjusts feature weights based on threat intelligence database data. If the IP address corresponding to a feature matches a malicious IP database, the port belongs to a high-risk port list, or the protocol type is a known commonly used attack protocol, the weight of the corresponding feature channel is increased by a preset ratio. The temporal feature extraction branch of LSTM includes an input layer, an LSTM layer, a temporal attention layer, and a fully connected layer; among which: The input layer receives time-series data sorted by timestamp, including event occurrence sequences, traffic change sequences, and alarm frequency sequences, with the sequence length being a preset value; An LSTM layer consists of a preset number of LSTM units and is used for temporal feature extraction. Each layer retains the output of all time steps or the output of the last time step. The time attention layer assigns weights based on event time intervals and attack timing features in threat intelligence. The weight of adjacent events with time intervals less than a set threshold is increased, and the weight of event sequences that match the timing of known attack chains is strengthened. The fully connected layer adjusts the dimension of the LSTM output features to generate a temporal feature vector with the same dimension as the spatial feature vector. The feature fusion layer performs element-wise weighted fusion of spatial and temporal feature vectors. The weights are set based on the contribution of spatial and temporal feature vectors to historical association detection. After nonlinear transformation by the activation function, the fused comprehensive feature vector is output for subsequent event feature vector generation. S300, Association Detection: An improved Apriori-Bayesian fusion algorithm is used to discretize the event feature vector; frequent itemsets are mined using the improved Apriori algorithm; association confidence is calculated using a Bayesian network; and a spatiotemporal association coefficient, including spatial and temporal association coefficients, is introduced to correct the confidence; the association between network security events is determined based on the corrected confidence; in S300, the spatiotemporal association coefficient is introduced to correct the confidence and determine the association, including the following steps: S330.1 Calculate the spatial correlation coefficient : Obtain the latitude and longitude coordinates of the IP address of the event to be associated and Combined with Earth's radius Calculate the spherical distance between two points Then based on the preset distance threshold By distance With threshold The mapping relationship determines the spatial correlation coefficient. ; S330.2 Calculate the time correlation coefficient Extract the timestamps of the two network security events to be correlated and calculate the time interval. , the time interval Compared with the preset time threshold, if If it does not exceed the preset time threshold, then based on Calculate the time correlation coefficient If it exceeds this threshold, the time correlation coefficient will... The value is 0; S330.3, Correcting the correlation confidence level : Obtain the raw confidence score of the Bayesian network output Using weighting coefficients Through the original confidence level Spatial correlation coefficient Time correlation coefficient The weighted summation is used to calculate the corrected confidence level. ; S330.4 Determining Association Relationships: Preset strong association thresholds and weak association thresholds, and adjust the confidence level accordingly. Compare with two thresholds; when A strong association is defined as a condition that is not lower than the strong association threshold. A relationship is considered weak if it is not lower than the weak association threshold but lower than the strong association threshold. If the value is below the weak association threshold, it is considered as having no association. S400 Risk Assessment and Results Output: For the correlation of cybersecurity incidents determined by correlation detection, conduct risk assessment in terms of threat level, scope of impact, and probability of spread, and generate and output an assessment report including correlation details, risk level, and response recommendations.

2. The network security event correlation detection method based on big data analysis according to claim 1, characterized in that, In S100, data collection includes the collection of raw data on multi-source cybersecurity incidents, asset information collection, and threat intelligence data collection, wherein: The collection of raw data from multi-source cybersecurity incidents includes the following steps: S110.1 Network Device Log Collection: Collect core device logs via SNMPv3 protocol and receive access layer device logs via Syslog protocol; before collection, the logs are filtered by the device's built-in rules, retaining only logs containing keywords such as "abnormal traffic", "port abnormality", and "authentication failure", and extracting device identifier, event timestamp, and event level fields; and associating the identifiers and asset importance levels of the core device and access layer device. S110.2 Terminal Behavior Log Collection: A collection agent is deployed on the terminal to capture events through operating system kernel hooks; process creation events are collected in real time, and file operation events are collected in batches at fixed intervals to extract the terminal MAC address, process path, file hash value, and operation result fields; and the corresponding terminal asset information is associated with the terminal MAC address. S110.3 Network Traffic Data Collection: Deploy traffic probes on core nodes to analyze traffic based on DPI technology; collect and record the connection state sequence of TCP sessions in full; sample UDP traffic by "source IP + destination port" dimension to extract session duration, packet size distribution, retransmission rate, and application layer protocol identifier; and associate the source and destination IPs involved in the traffic with the asset information corresponding to them. S110.4 Security device alarm data collection: collected via RESTful API interface; IDSIPS devices pull data at fixed intervals, and antivirus devices use an instant push mode after alarm generation, extracting attack feature ID, source and destination IP ports, alarm confidence level, and handling suggestion fields; and associating the asset information corresponding to the source and destination IP ports involved in the alarm; Asset information collection includes the following steps: S120.1 Asset Basic Information Collection: Synchronize asset data from the asset ledger system, including the unique identifier of terminal equipment, server, and business system, hardware configuration, business module to which it belongs, role and importance level in the business architecture; adopt a combination of scheduled synchronization and change-triggered synchronization to ensure the real-time nature of asset information; S120.2 Asset Association Collection: Through network topology discovery tools and business configuration management systems, collect network connection relationships and business dependencies between assets; generate asset association maps to provide a basis for subsequent risk diffusion analysis; Threat intelligence data collection includes the following steps: S130.1, Collection of known attack signature database: Interact with multiple heterogeneous intelligence sources and normalize them through an aggregation engine; the malicious IP database is stored in the structure of "IP + threat type + first appearance time", and the full synchronization cycle and incremental synchronization trigger conditions are configured. The AC automaton algorithm is used for signature matching, and invalid signatures are cleaned up periodically. S130.2 Vulnerability Information Database Collection: Based on data synchronization from the official CVE interface, the synchronization frequency is consistent with the official updates. Extract CVE-ID, release date, affected versions, CVSS score and sub-items, and exploit public status fields; trigger endpoint scanning tasks for high-risk vulnerabilities; S130.3 Attack Chain Graph Acquisition: Graph extraction algorithms are used to extract nodes from intelligence reports and construct a graph according to the attack stage model; node association weights are calculated based on the frequency of historical attack events, the graph topology update cycle is configured, and an emergency update is triggered when the number of newly disclosed events reaches a set number.

3. The network security event correlation detection method based on big data analysis according to claim 2, characterized in that, In step S200, the collected raw data of multi-source network security incidents and threat intelligence data are subjected to format unification, outlier removal, duplicate data deduplication, and noise reduction, including the following steps: S210.1, Format Unification: Regular expressions are used to match IP address, port number, timestamp, and attack type fields in unstructured logs and convert them into key-value pairs; all types of timestamps are uniformly converted to UTC format; lexical analysis tools are used to extract core semantic keywords from text description fields; and the signature format in threat intelligence is uniformly converted to hexadecimal strings. S210.2 Outlier Removal: For numerical features, the interquartile range method is used to calculate the upper quartile Q3 and lower quartile Q1 of the data, and values ​​outside the range determined by Q1, Q3 and the set coefficient are removed; for categorical features, rare categories with a frequency below the set threshold are removed. S210.3 Deduplication of Duplicate Data: Using "source IP + destination IP + event type + timestamp" as the composite primary key, retain the first occurrence of the record and delete subsequent duplicate entries; for log data, additionally check the consistency of process path and operation result fields, and if they are completely consistent, they are judged as duplicate data and removed. S210.4 Noise Reduction: For network traffic data packet content, a sliding window averaging method is used to smooth burst noise, with the window size set to 5-10 consecutive data packets; for text logs, meaningless words are filtered out through a preset stop word list, while core feature words are retained; for fuzzy feature descriptions in threat intelligence, they are standardized through a thesaurus mapping table.

4. The network security event correlation detection method based on big data analysis according to claim 3, characterized in that, In step S200, the collected raw data of multi-source network security incidents and threat intelligence data undergoes standardized preprocessing, including the following steps: S220.

1. Z-score standardization is applied to continuous numerical features; S220.

2. For features with a fixed value range, min-max standardization is applied to map them to the [0,1] interval; S220.3 For skewed distribution characteristics, perform logarithmic transformation first and then standardize; S220.

4. Categorical features are converted into binary vectors using one-hot encoding; high cardinality categories are mapped into low-dimensional dense vectors using embedding techniques.

5. The network security event correlation detection method based on big data analysis according to claim 4, characterized in that, In step S200, the generation of the event feature vector includes the following steps: S230.1 Feature concatenation: The spatial feature vector output by the CNN branch of spatial feature extraction and the temporal feature vector output by the LSTM branch of temporal feature extraction are concatenated sequentially according to their dimensions to form an initial fusion vector whose dimension is the sum of the dimensions of the spatial feature vector and the temporal feature vector. S230.2 Feature Enhancement: The initial fusion vector from S230.1 is input into a fully connected layer for nonlinear transformation. The number of neurons in the fully connected layer is the same as the dimension of the initial fusion vector. The activation function is LeakyReLU to enhance the correlation between features of different dimensions. S230.3 Feature Filtering: Apply L1 regularization constraints to the enhanced feature vectors. The regularization coefficient is set according to the cross-validation results of historical detection data. By suppressing weights with smaller absolute values, the top 80% of feature dimensions that contribute to the correlation detection are retained. S230.4 Normalization: Perform L2 normalization on the filtered feature vectors to make the L2 norm of the feature vectors 1; S230.5 Output Event Feature Vector: Output a fixed-dimensional event feature vector obtained after processing, which is used for subsequent association rule mining and detection of network security events.

6. The network security event correlation detection method based on big data analysis according to claim 5, characterized in that, In step S300, the discretization of the event feature vector includes the following steps: S310.1 Discretization of Continuous Features: For continuous features, K-means clustering algorithm is used to divide them into discrete intervals; number of clusters... The initial cluster centers are determined by the elbow method, and the farthest priority strategy is adopted for selection. The iteration termination condition is that the change in the cluster centers is less than the set threshold or the preset number of iterations is reached. S310.2, Discrete Feature Processing: For discrete features, retain the original category label; for rare categories whose frequency is below a set threshold, merge them into the "Other" category; S310.3 Feature encoding conversion: Convert all discretized features into integer codes and establish a mapping table between the codes and the original feature values.

7. The network security event correlation detection method based on big data analysis according to claim 6, characterized in that, In S300, frequent itemsets are mined using the improved Apriori algorithm, including the following steps: S320.1 Introduce a security event weighting factor, and assign corresponding weight coefficients to different events according to their risk levels. Combined with itemsets Number of transactions and total number of transactions The weighted support of the itemset is calculated by multiplying the weight coefficient by the proportion of transactions. ; S320.2 Setting the initial threshold for minimum support based on the average frequency of historical event associations. If the number of frequently generated itemsets is lower than the preset value, then the initial threshold of the current minimum support will be used. With attenuation coefficient The product update threshold is repeatedly adjusted until the itemset quantity requirement is met. S320.3 Setting the threshold for the proportion of rare events Statistical analysis of the frequency of rare events , for satisfying Less than the proportional threshold The itemset containing rare events multiplied by the total number of transactions is filtered in advance to reduce the size of the candidate set; S320.4, Weight the support of itemsets With the adjusted minimum support threshold By comparison, itemsets with support levels not lower than a threshold are selected as frequent itemsets for subsequent association rule mining.

8. The network security event correlation detection method based on big data analysis according to claim 7, characterized in that, The risk assessment and result output in S400 includes the following steps: S410.1 Threat Level Assessment: Determine the type of network security incident associated with the detection and judgment, and determine the basic degree of harm by combining the actual harm of this type of network security incident to the business system in historical detection. The basic hazard level is adjusted according to the strength of the correlation between events; the hazard level is increased for strongly correlated events and decreased for weakly correlated events, thus forming a threat level assessment conclusion. S410.2 Impact Scope Assessment: Analyze the assets involved in the related events; classify the importance of assets according to their actual role in the business architecture; and combine the number of assets affected by each level of importance to form an impact scope assessment conclusion. S410.3, Diffusion Probability Assessment: Retrieve past event records from the historical database that are consistent with the current related event type, analyze the diffusion path and diffusion rate of similar events; adjust the diffusion rate based on the occurrence time and degree of correlation of the current related event to form a diffusion probability assessment conclusion; S410.4 Determination of Comprehensive Risk Level: Based on the assessment conclusions of comprehensive threat level, scope of impact, and probability of spread, weights are allocated according to the priority of business impact; The overall results are divided into four levels of overall risk: "extremely high", "high", "medium" and "low". S410.5 Assessment Report Generation and Output: Generate an assessment report that includes related details, risk level, and response recommendations.

Images