Network security data dynamic identification method and system based on big data

Abstract

The invention discloses a network security data dynamic identification method and system based on big data, and relates to the technical field of network security data identifier.The method comprises the steps that historical network security data are obtained, and then a data feature set of the network security data is obtained through analysis; according to the method, a network security data identification model among platforms is obtained through training, target network security data are identified through the network security data identification model when data transmission is carried out among the platforms, and influence feature values of the platforms corresponding to the target network security data are obtained through feature analysis of the platforms; and finally, according to the influence feature value of each target network security data corresponding to each platform, judging the threat level of each target network security data, and formulating a backup strategy. According to the method, threat feature omission caused by a single data type is avoided, a complete threat portrait of the network security data among the platforms is formed, and the data traceability requirement and the quick response of the high threat data are realized.

Description

Technical Field

[0001] This invention relates to the field of network security data identification technology, specifically to a method and system for dynamic identification of network security data based on big data. Background Technology

[0002] Network security data refers to various types of data information generated during network activities that are directly related to the security status. It directly reflects the security status of networks, devices, and data. Identifying network security data can provide a major basis for threat identification and protection strategies. Therefore, this application proposes a dynamic identification method and system for network security data based on big data.

[0003] Existing technologies for cybersecurity data identification still have some shortcomings, specifically in the following aspects: Existing technologies for identifying cybersecurity data mostly rely on single-type data analysis, making it difficult to capture complex attack chain characteristics. At the same time, existing technologies store static and dynamic data separately, which is not conducive to the analysis of the correlation between dynamic and static data, resulting in insufficient ability to integrate multi-source heterogeneous data. Furthermore, existing technologies have a relatively single dimension for threat assessment, only providing defined rules to judge the threat situation, and cannot achieve differentiated threat assessment for different platforms. Summary of the Invention

[0004] In view of the above-mentioned technical deficiencies, the purpose of this invention is to provide a method and system for dynamic identification of network security data based on big data.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: In the first aspect, the present invention provides a method for dynamic identification of network security data based on big data, including: Step 1, information acquisition: acquiring network security data transmitted between various platforms in history, and then analyzing the data feature set of each network security data to train a network security data identification model between various platforms.

[0006] Step 2, Data Identification: When transmitting data between platforms, the target network security data is identified using a network security data identification model.

[0007] Step 3, Threat Analysis: Based on the transmission path of each target network security data, obtain the corresponding platforms for each target network security data, and analyze the characteristics of each platform to obtain the impact characteristic value of each target network security data on each platform.

[0008] Step 4: Data Backup: Determine the threat level of each target network security data based on the impact characteristic values ​​of each platform corresponding to each target network security data, and formulate a backup strategy.

[0009] In a second aspect, the present invention provides a network security data dynamic identification system based on big data, comprising: an information acquisition module for acquiring network security data transmitted between various platforms in history, and then analyzing the data feature set of each network security data to train a network security data identification model between various platforms.

[0010] Data identification module: Used to identify target network security data through a network security data identification model when transmitting data between different platforms.

[0011] Threat Analysis Module: This module is used to obtain the platforms corresponding to the network security data of each target based on the transmission path of the target network security data, and to obtain the impact characteristic values ​​of each platform corresponding to each target network security data based on the characteristics of each platform.

[0012] Data backup module: Used to determine the threat level of each target network security data based on the impact characteristic values ​​of each platform corresponding to each target network security data, and to formulate backup strategies.

[0013] The beneficial effects of this invention are as follows: 1. The identification of network security data in this invention covers three types of data: dynamic, static and behavioral, avoiding the omission of threat features caused by a single data type, and forming a complete threat profile of network security data across platforms.

[0014] 2. This invention designs a two-dimensional assessment system based on threat signature values ​​and platform compatibility coefficients. It quantifies the threat posed by different threats to different platforms through five dimensions. Through refined threat assessment, it avoids the limitations and identification errors caused by a one-size-fits-all assessment method. At the same time, this invention achieves rapid response to data traceability needs and high-threat data by performing graded backup of data at different risk levels.

[0015] 3. This invention achieves fully automated identification of network security data from historical data collection and model training to real-time data identification, threat assessment, and backup execution, significantly reducing manual costs. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the implementation steps of the method of the present invention.

[0018] Figure 2 This is a schematic diagram of the system structure connection of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Reference Figure 1 As shown, the first aspect of the present invention provides a method for dynamic identification of network security data based on big data, including: Step 1, information acquisition: acquiring network security data transmitted between various platforms in history, and then analyzing the data feature set of each network security data to train a network security data identification model between various platforms.

[0021] In a specific example, the process of acquiring network security data transmitted between various platforms in history is as follows: By using traffic acquisition devices deployed at the boundaries of each platform, network data packets within a preset historical time period are captured. The network data packets are parsed to obtain network security data. The selected network security data is deduplicated, and duplicate data content is deleted, retaining unique data samples. The deduplicated network security data is classified and stored according to the source platform and the target platform of transmission to form a historical data sample set for each platform. The network security data includes static data, dynamic data, and behavioral data.

[0022] It should be noted that the security event identifiers include virus signatures, intrusion detection rule matching identifiers, and abnormal login record identifiers, etc.

[0023] It should also be noted that static data includes basic network asset information, general security rules, and asset associations. Basic asset information includes server and terminal IP addresses, hardware models, operating system versions, and deployment locations. General security rules include malicious code signatures, cross-platform blacklists and whitelists, general firewall policies, and security compliance baseline standards. Asset associations include network topology, the binding relationship between business systems and servers, and basic cross-platform user account information. Dynamic data includes real-time network traffic data, log data, and threat intelligence data. Network traffic data includes source / destination IPs, ports, protocol types, traffic volume, and transmission duration of data packets. Log data includes system operation logs, application access logs, and device status logs. Threat intelligence data includes third-party synchronized malicious IP databases, domain name databases, attack methods, and vulnerability information. Behavioral data includes behavioral audit data and security event alert data. Behavioral audit data includes cross-platform file transfer records. Security event alert data includes the threat type of the alerted event, the event that occurred, and the assets affected.

[0024] In a specific example, the analysis yields a set of data features for each network security data point. The specific analysis process is as follows: After preprocessing the network security data points in the historical data sample sets of each platform, features are extracted from three dimensions—static features, dynamic features, and correlation features—based on the attributes and cross-platform universality of the network security data. The extracted features of each network security data point are then normalized, and the normalized data features are integrated into a set of data recipe rules.

[0025] It should be noted that preprocessing includes data cleaning, data anonymization, and data standardization. Data cleaning, data anonymization, and data standardization are all existing technologies, so they will not be described in detail here.

[0026] It should be noted that the characteristics of each network security data are processed through Min-Max normalization or Z-score standardization. Min-Max normalization and Z-score standardization are existing technologies, so they will not be elaborated on further.

[0027] It should be noted that the data recipe rule set includes recipe rules such as marking the source IP as a threat in the known malicious IP database and triggering interception if the HTTP request contains an EXP fragment with a Log4j vulnerability.

[0028] In a specific example, the training process for obtaining a cybersecurity data recognition model across platforms is as follows: The data features in the data rule formula set are preprocessed. A CNN-LSTM-Attention fusion model is used as the core framework of the cybersecurity data recognition model. The input layer of the CNN-LSTM-Attention fusion model receives the preprocessed data rule formula set. The CNN model uses a two-layer convolutional neural network to extract the spatial correlation of static features and output a 64-dimensional static feature vector. The LSTM model uses a bidirectional LSTM to capture the temporal dependence of dynamic features and output a dynamic feature vector. Finally, a multi-head self-attention mechanism and domain adaptation technology are introduced through the Attention layer to weightedly fuse static and dynamic features. The output layer uses a fully connected layer and a sigmoid activation function to output threat feature values ​​of the cybersecurity data, where threat feature values ​​are data values ​​from 0 to 1. The preprocessed data rule formula set is divided into a training set and a validation set according to a preset ratio. The core framework of the cybersecurity data recognition model is then trained and validated to obtain the cybersecurity data recognition model across platforms.

[0029] After pruning and INT8 quantization, the network security data identification model is generated into a multi-platform deployment version and deployed to various platforms.

[0030] It should be noted that preprocessing includes sample labeling and cleaning, cross-platform sample partitioning and balancing, and feature dimension optimization. Among these, sample labeling and cleaning, cross-platform sample partitioning and balancing, and feature dimension optimization are all existing technologies, so they will not be described in detail here.

[0031] It should be noted that the spatial correlation of static features, such as the combined features of dangerous ports and encryption protocols, and the temporal dependence of dynamic features, such as abnormal changes from low frequency to sudden high frequency, are addressed by introducing a multi-head self-attention mechanism through the Attention layer to perform weighted fusion of static and dynamic features. For example, the abnormal port feature has a weight of 0.8, and the normal traffic fluctuation has a weight of 0.2. The above examples are only illustrative and are not the only instances.

[0032] It should be noted that the domain adaptation module is used for adversarial training to allow the model to learn platform-independent features. Through the domain classifier, it competes with the main classifier: the main classifier tries to minimize the threat identification loss, while the domain classifier tries to maximize the platform judgment accuracy. The adversarial process is achieved through the gradient reversal layer, which ultimately makes the fused features insensitive to the platform but sensitive to the threat.

[0033] It should be noted that model pruning and INT8 quantization are existing technologies, so they will not be elaborated on further.

[0034] Step 2, Data Identification: When transmitting data between platforms, the target network security data is identified using a network security data identification model.

[0035] In a specific example, when data is transmitted between platforms, the target network security data is identified through a network security data identification model. The specific process is as follows: the original transmission data is captured by traffic acquisition devices deployed at the boundaries of each platform, and the original transmission data is converted into data of the same format through protocol parsing. After preprocessing the original transmission data, it is input into the network security data identification model, and the threat feature value of each original transmission data is output through the network security data identification model.

[0036] The threat characteristic values ​​of each original transmitted data are compared with the set threat characteristic threshold. Each original transmitted data that is greater than or equal to the threat characteristic threshold is recorded as target network security data, and the rest is recorded as non-target network security data. Network security data that cannot be identified by the network security data identification model is recorded as unknown network security data.

[0037] It should be noted that the threat signature threshold is a critical value for determining whether the original transmitted data poses a threat to each platform, and its specific value is set by the relevant personnel.

[0038] Step 3, Threat Analysis: Based on the transmission path of each target network security data, obtain the corresponding platforms for each target network security data, and analyze the characteristics of each platform to obtain the impact characteristic value of each target network security data on each platform.

[0039] It should be noted that each platform includes source nodes, relay nodes, and receiving nodes.

[0040] In a specific example, the process of obtaining the receiving platforms corresponding to the target network security data based on the transmission path of each target network security data is as follows: after any target network security data is identified, the header information of the target network security data is extracted, the transmission trajectory of the target network security data is restored through data packet tracing technology, and the source node, each relay node and each receiving node of the target network security data are obtained through the transmission trajectory.

[0041] In a specific example, the impact characteristic values ​​of each target network security data corresponding to each platform are obtained based on the characteristic analysis of each platform. The specific analysis process is as follows: The impact of each platform on the network security data is divided into five dimensions: threat class adaptation, platform vulnerability matching, platform protection capability, platform business sensitivity, and platform resource adaptation. Threat class adaptation, platform vulnerability matching, platform protection capability, platform business sensitivity, and platform resource adaptation are assigned weights of 0.35, 0.25, 0.20, 0.15, and 0.05 respectively, corresponding to score ranges of 0-0.35, 0-0.25, and 0-0.25. Scores of 0, 0-0.15, and 0-0.05 were assigned by several staff members to score the impact of each target cybersecurity data on each platform across each dimension. The average of the scores assigned by the staff members across the five dimensions for each platform was recorded as the final score for each dimension of the corresponding platform. The weighted sum of the weights of each dimension of each platform and the final score was then calculated to obtain the adaptation coefficient of each platform to each target cybersecurity data. Finally, the adaptation coefficient of each platform to each target cybersecurity data was multiplied by the threat characteristic value of each target cybersecurity data to obtain the impact characteristic value of each target cybersecurity data on each platform.

[0042] For example, if the threat characteristic value of a target network security data is 0.8, and the scores of a platform for the threat class adaptation, platform vulnerability matching, platform protection capability, platform business sensitivity, and platform resource adaptation of the target network security data are 0.15, 0.10, 0.05, 0.05, and 0.05 respectively, then the adaptation coefficient of the platform for the target network security data is calculated to be 0.4. Based on the threat characteristic value and adaptation coefficient of the target network security data, the impact characteristic value of the target network security data on the platform is calculated to be 0.32.

[0043] Step 4: Data Backup: Determine the threat level of each target network security data based on the impact characteristic values ​​of each platform corresponding to each target network security data, and formulate a backup strategy.

[0044] In a specific example, the process of determining the threat level of each target network security data based on the impact characteristic value of each target network security data corresponding to each platform is as follows: the impact characteristic value of each target network security data corresponding to each platform is compared with the set impact characteristic threshold. When the impact characteristic value of a target network security data corresponding to a certain platform is greater than or equal to the set impact characteristic threshold, the target network security data is recorded as a level 1 threat data of that platform; otherwise, it is recorded as a level 2 threat data. Based on this, the security protection level of each target network security data for each platform is obtained.

[0045] It should be noted that the impact characteristic threshold is a critical value used to determine the level of impact of network security data on the platform. Its specific value is set by the relevant staff and is not specifically limited here.

[0046] In a specific example, the process of formulating a backup strategy is as follows: for Level 1 threat data, a high-priority backup strategy is initiated; for Level 2 threat data, a low-priority backup strategy is initiated; for unknown network security data, a customized task application is made, and the data recipe rule set is updated according to the unknown network security data, and the network security data identification model is optimized.

[0047] It should be noted that the high-priority backup strategy includes the original data packets, identification process logs, and threat analysis reports, and performs cross-platform, off-site real-time backups with a real-time backup frequency and a retention period of more than 90 days. The priority backup strategy includes standardized feature data and identification results, and performs daily incremental backups and weekly full backups with a retention period of 30-60 days.

[0048] Reference Figure 2 As shown, the second aspect of the present invention provides a network security data dynamic identification system based on big data, comprising: an information acquisition module: used to acquire network security data transmitted between various platforms in history, and then analyze the data feature set of each network security data to train a network security data identification model between various platforms.

[0049] Data identification module: Used to identify target network security data through a network security data identification model when transmitting data between different platforms.

[0050] Threat Analysis Module: This module is used to obtain the platforms corresponding to the network security data of each target based on the transmission path of the target network security data, and to obtain the impact characteristic values ​​of each platform corresponding to each target network security data based on the characteristics of each platform.

[0051] Data backup module: Used to determine the threat level of each target network security data based on the impact characteristic values ​​of each platform corresponding to each target network security data, and to formulate backup strategies.

[0052] The examples described in this invention are not limited to the specific embodiments listed above. The examples are merely illustrative to facilitate understanding of the invention and do not constitute a limitation on the scope of protection of this invention. Any modifications, equivalent substitutions, etc., made within the spirit and principles of this invention should be included within the scope of protection.

[0053] The above description is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined in this specification, they should all fall within the protection scope of the present invention.

Claims

1. A method for dynamic identification of network security data based on big data, characterized in that, Includes the following modules and steps: Step 1: Information Acquisition: Acquire network security data transmitted between various platforms in history, and then analyze the data feature set of each network security data to train a network security data recognition model between platforms. Step 2, Data Identification: When transmitting data between platforms, the target network security data is identified using a network security data identification model; Step 3, Threat Analysis: Based on the transmission path of each target network security data, obtain the corresponding platforms for each target network security data, and analyze the characteristics of each platform to obtain the impact characteristic value of each target network security data on each platform. Step 4: Data Backup: Determine the threat level of each target network security data based on the impact characteristic values ​​of each platform corresponding to each target network security data, and formulate a backup strategy.

2. The method for dynamic identification of network security data based on big data according to claim 1, characterized in that, The specific process for obtaining the network security data transmitted between various platforms in history is as follows: By deploying traffic acquisition devices at the boundaries of each platform, network data packets within a preset historical time period are captured. The network data packets are parsed to obtain network security data. The selected network security data is deduplicated, and duplicate data content is deleted, retaining unique data samples. The deduplicated network security data is classified and stored according to the source platform and the target platform, forming a historical data sample set for each platform. The network security data includes static data, dynamic data, and behavioral data.

3. The method for dynamic identification of network security data based on big data according to claim 2, characterized in that, The analysis yields a set of data features for each network security data point. The specific analysis process is as follows: After preprocessing the network security data from the historical data sample sets of various platforms, features are extracted from three dimensions—static features, dynamic features, and correlation features—based on the attributes and cross-platform universality of the network security data. The extracted features of each network security data are then normalized, and the normalized data features are integrated into a set of data recipe rules.

4. The method for dynamic identification of network security data based on big data according to claim 3, characterized in that, The training process yields network security data identification models across various platforms. The specific training process is as follows: The data features in the data rule formula set are preprocessed. A CNN-LSTM-Attention fusion model is used as the core framework of the network security data recognition model. The input layer of the CNN-LSTM-Attention fusion model receives the preprocessed data rule formula set. The CNN model uses a two-layer convolutional neural network to extract the spatial correlation of static features and output a 64-dimensional static feature vector. The LSTM model uses a bidirectional LSTM to capture the temporal dependence of dynamic features and output a dynamic feature vector. Finally, a multi-head self-attention mechanism and domain adaptation technology are introduced through the Attention layer to perform weighted fusion of static and dynamic features. The output layer uses a fully connected layer and a sigmoid activation function to output the threat feature value of the network security data, where the threat feature value is a data value between 0 and 1. The preprocessed data rule formula set is divided into training set and validation set according to a preset ratio. The core framework of the network security data identification model is trained and validated to obtain the network security data identification model between various platforms. After pruning and INT8 quantization, the network security data identification model is generated into a multi-platform deployment version and deployed to various platforms.

5. The method for dynamic identification of network security data based on big data according to claim 4, characterized in that, When transmitting data between platforms, the target network security data is identified through a network security data identification model. The specific process is as follows: By deploying traffic acquisition devices at the boundaries of each platform, the raw transmission data is captured, and the raw transmission data is converted into data of the same format through protocol parsing. After preprocessing the raw transmission data, it is input into the network security data identification model, and the network security data identification model outputs the threat characteristic values ​​of each raw transmission data. The threat characteristic values ​​of each original transmitted data are compared with the set threat characteristic threshold. Each original transmitted data that is greater than or equal to the threat characteristic threshold is recorded as target network security data, and the rest is recorded as non-target network security data. Network security data that cannot be identified by the network security data identification model is recorded as unknown network security data.

6. The method for dynamic identification of network security data based on big data according to claim 5, characterized in that, The specific process of obtaining the receiving platforms corresponding to the target network security data based on the transmission path of each target network security data is as follows: Once any target network security data is identified, the header information of the target network security data is extracted, and the transmission trajectory of the target network security data is reconstructed through packet tracing technology. The source node, each relay node, and each receiving node of the target network security data are obtained through the transmission trajectory.

7. The method for dynamic identification of network security data based on big data according to claim 6, characterized in that, The impact characteristic values ​​of each target network security data on each platform are obtained based on the characteristic analysis of each platform. The specific analysis process is as follows: The impact of network security data on each platform is categorized into five dimensions: threat adaptation, platform vulnerability matching, platform protection capabilities, platform business sensitivity, and platform resource adaptation. Each dimension is assigned a weight of 0.35, 0.25, 0.20, 0.15, and 0.05, respectively, corresponding to score ranges of 0-0.35, 0-0.25, 0-0.20, 0-0.15, and 0-0.

05. Several staff members assign scores to the impact of each target network security data on each platform across each dimension. The average score from these staff members across the five dimensions is recorded as the final score for each dimension on each platform. The weighted sum of the weights for each dimension and the final score yields the adaptation coefficient for each platform to the target network security data. Finally, the adaptation coefficient is multiplied by the threat characteristic value of the target network security data to obtain the impact characteristic value of each target network security data on each platform.

8. The method for dynamic identification of network security data based on big data according to claim 7, characterized in that, The specific process for determining the threat level of each target network security data based on the impact characteristic values ​​of each platform corresponding to each target network security data is as follows: The impact characteristic value of each target network security data corresponding to each platform is compared with the set impact characteristic threshold. When the impact characteristic value of a target network security data corresponding to a platform is greater than or equal to the set impact characteristic threshold, the target network security data is recorded as the first-level threat data of that platform; otherwise, it is recorded as the second-level threat data. Based on this, the security protection level of each target network security data for each platform is obtained.

9. The method for dynamic identification of network security data based on big data according to claim 8, characterized in that, The specific process for formulating the backup strategy is as follows: For Level 1 threat data, a high-priority backup strategy is initiated; for Level 2 threat data, a low-priority backup strategy is initiated; for unknown network security data, a customized task application is made, and the data recipe rule set is updated based on the unknown network security data, and the network security data identification model is optimized.

10. A system executed using the big data-based network security data dynamic identification method according to any one of claims 1-9, characterized in that, include: Information acquisition module: used to acquire network security data transmitted between various platforms in history, and then analyze the data feature set of each network security data to train a network security data recognition model between platforms; Data recognition module: used to identify target network security data through a network security data recognition model when transmitting data between different platforms; Threat Analysis Module: Used to obtain the platforms corresponding to the network security data of each target based on the transmission path of each target network security data, and to obtain the impact characteristic value of each platform corresponding to each target network security data based on the characteristics of each platform; Data backup module: Used to determine the threat level of each target network security data based on the impact characteristic values ​​of each platform corresponding to each target network security data, and to formulate backup strategies.

Images