Network security situation awareness method and system based on multi-dimensional data fusion

Abstract

The invention relates to the technical field of network security, and discloses a network security situation awareness method and system based on multidimensional data fusion, and the method comprises the steps: 1, collecting network behavior data from at least two different data sources; 2, cleaning and formatting the network behavior data; 3, a security situation score is calculated by adopting a formula # imgabs0 #, S is the security situation score, wi is the weight of the ith dimension, and Di is the data value of the network behavior data of the corresponding dimension; and 4, predicting potential security threats according to the security situation scores. The system corresponds to the method. According to the invention, comprehensive monitoring and accurate early warning of the network security state can be realized.

Description

Technical Field

[0001] This application relates to the field of network security technology, and specifically to a network security situation awareness method and system based on multi-dimensional data fusion. Background Art

[0002] With the rapid development of information technology, the forms and means of network attacks are becoming increasingly complex and changeable. Traditional network security defense measures are facing unprecedented challenges. Existing network security solutions often focus on single-dimensional data analysis or rely on static rule sets to identify potential threats, which leads to being unable to cope when facing new and complex attack patterns. In addition, traditional methods usually lack the ability to adapt to the dynamic changes of the network environment and cannot adjust security policies in time to cope with the evolving threat situation.

[0003] 1. Problem of data source singularity

[0004] Many existing network security systems rely only on limited data sources for threat detection, such as firewall logs, intrusion detection system (IDS) alerts, etc. However, in a modern network environment, security threats may come from multiple different channels, including but not limited to abnormal user behavior, application vulnerability exploitation, external threat intelligence, etc. The limitation of a single data source restricts the system's ability to comprehensively perceive the network security situation.

[0005] 2. Lack of multi-dimensional data analysis ability

[0006] Most current technologies fail to fully utilize the internal relationships between data from different dimensions for comprehensive analysis. For example, although some systems can collect network traffic data and host activity records, they rarely combine this information for a comprehensive assessment. This separated data processing method is likely to overlook some hidden security threats because real attacks often span multiple levels and need to be discovered through cross-dimensional correlation.

[0007] 3. Static threshold setting and high false alarm rate

[0008] Most existing systems use fixed thresholds to determine whether there is a security threat. However, the network environment is dynamically changing, and fixed thresholds are difficult to adapt to all situations, which may lead to excessive false alarms or missed alarms. Especially during peak periods or specific time periods, normal network activities may also trigger the alarm mechanism, increasing the unnecessary management burden.

[0009] 4. Lack of flexibility in response strategies

[0010] Traditional response strategies are usually executed based on preset rules, and corresponding measures are initiated once a threat is detected. However, this approach ignores the specific characteristics of each threat and its impact on business operations. An ideal response strategy should be flexible enough to automatically adjust the response level according to the actual situation, thereby minimizing interference with normal business processes.

[0011] Based on the above background and the deficiencies of the existing technology, there is an urgent need for a more intelligent, efficient, and adaptable network security solution. Summary of the Invention

[0012] The purpose of this application is to provide a network security situation awareness method and system based on multi-dimensional data fusion, aiming to overcome the defects in the existing technology and achieve comprehensive monitoring and accurate early warning of network security status.

[0013] To achieve the above purpose, this application discloses the following technical solutions:

[0014] In the first aspect, this application discloses a network security situation awareness method based on multi-dimensional data fusion, and the method includes the following steps:

[0015] Step 1: Collect network behavior data from at least two different data sources;

[0016] Step 2: Clean and format the network behavior data;

[0017] Step 3: Use the formula to calculate the security situation score, where S is the security situation score, w i is the weight of the i-th dimension, and D i is the data value of the network behavior data corresponding to the dimension;

[0018] Step 4: Predict potential security threats based on the security situation score.

[0019] Preferably, in Step 1, it further includes: identifying and marking abnormal data points in the collected network behavior data.

[0020] Preferably, the identifying and marking abnormal data points in the collected network behavior data specifically includes:

[0021] Using the anomaly metric function A(x i ) to analyze the deviation degree of the data point x i in the corresponding dimension; the specific formula of the anomaly metric function A(x i ) is:

[0022]

[0023] where xij The value of the data point x i at the j-th dimension, E j is the average value of all data points on the j-th dimension, σ j is the standard deviation of all data points on the j-th dimension, Corr(x i , X) is the correlation score of the data point x i and all other data points X in all dimensions, and γ is a preset weight coefficient for adjusting the importance of the deviation degree.

[0024] Preferably, in the third step, the weight w of each dimension is optimized by constructing an association graph, which includes a number of nodes and a number of edges. Among them, each node represents a different data point, and the weight of each edge is used to represent the strength of the correlation between two data points.

[0025] Preferably, in the third step, the optimized weight w i ' of the i-th dimension is obtained according to the following rule:

[0026] w i ' = w i + α * Σ j≠i (E ij - w j )

[0027] where α is a preset learning rate, and E ij is the edge weight between dimension i and dimension j obtained from the association graph.

[0028] Preferably, in the fourth step, potential security threats of different network environments are predicted based on a dynamically updated evaluation threshold. When the security situation score of a network environment is lower than the corresponding evaluation threshold, it indicates that there are potential security threats in this network environment.

[0029] Preferably, the dynamic update of the evaluation threshold is obtained through the following formula:

[0030] T' = T base * (1 + k)

[0031] where T base is a preset basic threshold, and k is a preset adjustment coefficient.

[0032] Preferably, the method further includes:

[0033] Step Five: Display the network security status in a visual way.

[0034] Preferably, the fifth step specifically includes:

[0035] The security status is displayed by converting the security situation score into color intensity, wherein the color intensity conversion is realized by a color mapping function, and the formula of the color mapping function is:

[0036]

[0037] Among them, S max and S min are the maximum and minimum security posture scores in all dimensions,

[0038] Express The result is rounded down.

[0039] In a second aspect, the present application discloses a network security situation awareness system based on multi-dimensional data fusion, which is applicable to the network security situation awareness method based on multi-dimensional data fusion as described above, including:

[0040] A data collection module, for collecting network behavior data from at least two different data sources;

[0041] A data preprocessing module, for cleaning and formatting the collected data;

[0042] Data analysis module, using formula Calculate the security situation score, where S is the security situation score, w i is the weight of the i-th dimension, D i is the data value of the corresponding dimension;

[0043] The risk assessment module is used to predict potential security threats based on the security situation score.

[0044] Compared with the prior art, the network security situation awareness method and system based on multi-dimensional data fusion of the present application has the following beneficial effects:

[0045] 1. This application overcomes the defects in the prior art and provides a more intelligent, efficient and adaptable network security solution. By integrating multiple data sources, using advanced algorithm models and implementing dynamic adjustment mechanisms, it can achieve comprehensive monitoring and accurate early warning of network security status, and can automatically generate effective response strategies according to actual needs;

[0046] 2. This application is based on the degree of deviation of a single data point in its dimension and in-depth analysis of the relationship between the data point and other dimensional data points to more accurately identify potential abnormal data points, reduce false positives and false negatives, and improve the overall accuracy and reliability of anomaly detection;

[0047] 3. By dynamically adjusting the evaluation threshold, this application can better adapt to changes during network traffic peaks or specific time periods, avoiding excessive alarms or missed alarms caused by fixed thresholds, thereby improving the accuracy of the early warning system.

[0048] 4. By converting the security posture score into an easily understandable color intensity for display, this application helps administrators identify potential security threats faster and more accurately. Especially in a complex network environment, it can provide multi-level information display and improve decision-making efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of this application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0050] Figure 1 It is a schematic flowchart of the network security posture awareness method based on multi-dimensional data fusion provided in this embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] The following will clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, rather than all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0052] In this article, the term "including" is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article, or device. Without more limitations, the elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article, or device including the elements.

[0053] This embodiment provides a network security posture awareness method based on multi-dimensional data fusion as shown in Figure 1 The method includes the following steps:

[0054] Step 1: Collect network behavior data from at least two different data sources;

[0055] Step 2: Clean and format the network behavior data;

[0056] Step 3: Use the formula Calculate the security posture score, where S is the security posture score, and w i is the weight of the i-th dimension, and D i is the data value of the network behavior data corresponding to the dimension;

[0057] Step Four: Predict potential security threats based on the security posture score.

[0058] Next, each step will be introduced in detail.

[0059] In Step One, the data sources include firewall logs, intrusion detection system (IDS) alerts, external threat intelligence, etc. The collected network behavior data includes: firewall logs - recording all traffic information entering and leaving the network; IDS alerts - monitoring and reporting any suspicious activities; external threat intelligence - the latest threat information from third-party service providers.

[0060] Furthermore, Step One also includes: identifying and marking abnormal data points from the collected network behavior data, specifically including:

[0061] Using the anomaly metric function A(x i ) to analyze the degree of deviation of the data point x i in the corresponding dimension; the specific formula of the anomaly metric function A(x i ) is:

[0062]

[0063] where x ij is the value of the data point x i in the j-th dimension, E j is the average value of all data points in the j-th dimension, σ j is the standard deviation of all data points in the j-th dimension, Corr(x i , X) is the correlation score of the data point x i with all other data points X in all dimensions, and γ is a preset weight coefficient for adjusting the importance of the degree of deviation. It should be noted that the correlation score can be obtained by using any one of the calculation methods in the prior art, and can be but is not limited to the Euclidean distance method, the cosine similarity method, etc. In this way, those data points that may pose a threat to network security can be captured more accurately, even if these data points do not seem prominent in a single dimension. In addition, this method based on multi-dimensional data fusion enhances the ability to understand complex network environments and helps to early warn of potential security threats. Therefore, by combining multi-dimensional information and emphasizing the importance of the relationship between data points, a more comprehensive and in-depth data analysis perspective is provided, which goes beyond the traditional method based on the statistical characteristics of a single dimension.

[0064] In step two, all the collected data need to be standardized and normalized so that they can be compared on a unified scale during subsequent analysis. For example, for the traffic data in the firewall logs, we can convert it to the average traffic per second; for IDS alerts, we can calculate the alert frequency per hour, etc. In addition, it is necessary to remove noise data and incomplete records to ensure data quality.

[0065] In step three, the weight w of each dimension is optimized through the constructed association graph. The association graph is a structure used to represent the relationships between data points. It describes the interactions between data points through nodes and edges. Among them, each node represents a different data point, and the weight of each edge is used to represent the strength of the correlation between two data points. The steps for constructing the association graph can be the following techniques:

[0066] D1: Prepare the data:

[0067] First, ensure that all the data to be analyzed has been collected and preprocessed. This includes but is not limited to network traffic data, log files, user behavior data, etc. Standardize or normalize the data for each dimension to ensure that data of different magnitudes can be compared on the same scale.

[0068] D2 Calculate the similarity or distance matrix:

[0069] Select a suitable similarity metric method (such as cosine similarity, Euclidean distance, etc.), and choose the most suitable method according to your application scenario. Suppose we choose cosine similarity as an example. Calculate the similarity scores between each pair of data points to form a similarity matrix S. If distance is used, the distance can be converted into a similarity score (for example, take the reciprocal).

[0070] D3: Determine the threshold or K-Nearest Neighbors (KNN):

[0071] Based on the similarity matrix, you need to decide how to connect these points. A common way is to set a threshold for the similarity score. Only when the similarity between two points exceeds this threshold will an edge be established between them. Another way is to adopt the K-Nearest Neighbors (KNN) strategy, that is, for each node, only keep the edges between it and its top K most similar neighbors.

[0072] D4: Construct the graph structure:

[0073] Using the selected method (threshold method or KNN method), create the nodes and edges of the graph. Each data point corresponds to a node in the graph, and based on the similarity score or distance, edges are established between the points that meet the conditions, and corresponding weights are assigned (i.e., the inverse of the similarity score or distance). In actual operation, this step can be implemented programmatically, such as using the NetworkX library in Python or other graph processing tools.

[0074] D5: Attributes and Analysis of the Graph:

[0075] After the construction of the graph is completed, the attributes of the graph can be further analyzed as needed, such as calculating the centrality of nodes, community detection, etc., which can help better understand the relationships between data points and potential abnormal patterns.

[0076] Given this, assume there are n data points, and the similarity matrix S between them has been calculated, where S ij represents the similarity score between data points i and j. If the threshold method is used, let the threshold be θ, then for all S ij > θ, an edge is established between data points i and j, and the weight of the edge is S ij . If the KNN method is used, for each data point i, find its top k neighbors with the highest similarity, and establish edges between these k neighbors and i. In this way, an association graph reflecting the complex relationships between data points can be constructed. This graph not only helps to identify abnormal data points but can also be used for various applications such as subsequent weight adjustment and clustering analysis.

[0077] Furthermore, in step three, the rule for obtaining the optimized weight w i ' for the i-th dimension is:

[0078] w i ' = w i + α * ∑ j≠i (E ij - w j )

[0079] where α is the preset learning rate, and E ij is the edge weight between dimension i and dimension j obtained from the association graph.

[0080] Specifically, in the weight optimization process, it includes the following steps:

[0081] S1: Initialization Phase:

[0082] For each dimension i = 1, 2,..., n, initialize the weight wi = 1 / n, indicating that each dimension has the same contribution at the beginning. Collect historical data over a period of time as the training sample set, including data in normal states and known attack events.

[0083] S2: Feature extraction and transformation:

[0084] For the data of each dimension, apply a specific transformation function to map it into a common space for comparison. For example, for network traffic data, a packet length distribution histogram can be used; for log files, it can be operation frequency statistics, etc. Input the transformed feature vectors into the DMDAL model.

[0085] S3: Construct an association graph based on the similarity between feature vectors.

[0086] S4: Use the information in the association graph to update the weights of each dimension.

[0087] Through the above method for weight optimization based on dynamic multi-dimensional association learning, the network security status can be evaluated more precisely, showing innovation and uniqueness in improving system performance.

[0088] In step four, based on the dynamically updated evaluation threshold, predict potential security threats in different network environments. When the security posture score of a network environment is lower than the corresponding evaluation threshold, it indicates that there are potential security threats in this network environment. The relationship between the security posture score and the threshold directly determines whether to trigger an alarm or other response measures. If the security posture score is lower than the set threshold, it indicates that there may be security risks in the network and immediate action is required; otherwise, the network is considered to be in a normal state. Adopting a dynamic adjustment mechanism can make the threshold change with the change of the network environment, so as to more accurately reflect the current security status. For example, during the peak period of network traffic, appropriately increasing the threshold can reduce unnecessary alarms; while when an increase in suspected attack activities is detected, lowering the threshold can capture potential threats faster.

[0089] Furthermore, the dynamic update of the evaluation threshold is obtained through the following formula:

[0090] T' = T base *(1 + k)

[0091] where, T base is the preset basic threshold, and k is the preset adjustment coefficient. It should be noted that k can be, but is not limited to, a value determined by using the expert experience in the field of network security to set the relative importance of each influencing factor.

[0092] As a preferred implementation manner of this embodiment, this network security situation awareness method based on multi-dimensional data fusion further includes: Step five: Display the network security status in a visual way. Specifically including:

[0093] The security status is displayed by converting the security situation score into color intensity. The color intensity conversion is realized by the color mapping function. The visualization uses the heat map technology to intuitively display the network security status. Assuming that the system monitors m key network nodes, the security situation score of each node is S j ,j=1,2,...,m. The color intensity of the heat map is proportional to the security situation score of the corresponding node. The formula of the color mapping function is:

[0094]

[0095] Among them, S max and S min are the maximum and minimum security posture scores in all dimensions,

[0096] Express The result is rounded down.

[0097] This allows administrators to quickly identify areas of the network that require special attention.

[0098] In summary, the network security situation awareness method based on multi-dimensional data fusion in this embodiment has the following technical advances:

[0099] 1. This application overcomes the defects in the prior art and provides a more intelligent, efficient and adaptable network security solution. By integrating multiple data sources, using advanced algorithm models and implementing dynamic adjustment mechanisms, it can achieve comprehensive monitoring and accurate early warning of network security status, and can automatically generate effective response strategies according to actual needs;

[0100] 2. This application is based on the degree of deviation of a single data point in its dimension and in-depth analysis of the relationship between the data point and other dimensional data points to more accurately identify potential abnormal data points, reduce false positives and false negatives, and improve the overall accuracy and reliability of anomaly detection;

[0101] 3. This application can better adapt to changes in network traffic peaks or specific time periods by dynamically adjusting the evaluation threshold, avoiding excessive alarms or missed reports caused by fixed thresholds, thereby improving the accuracy of the early warning system;

[0102] 4. This application helps administrators identify potential security threats faster and more accurately by converting security situation scores into easy-to-understand color intensities for display, especially in complex network environments, and can provide multi-level information display to improve decision-making efficiency.

[0103] In the second aspect, this embodiment provides a network security situation awareness system based on multi-dimensional data fusion, which is applicable to the network security situation awareness method based on multi-dimensional data fusion as described above, including:

[0104] A data collection module for collecting network behavior data from at least two different data sources;

[0105] A data preprocessing module for cleaning and formatting the collected data;

[0106] A data analysis module, using the formula to calculate the security situation score, where S is the security situation score, w i is the weight of the i-th dimension, and D i is the data value of the corresponding dimension;

[0107] A risk assessment module for predicting potential security threats based on the security situation score.

[0108] It should be noted that the network security situation awareness system based on multi-dimensional data fusion in this embodiment corresponds to the aforementioned network security situation awareness method based on multi-dimensional data fusion. Therefore, for the content that specifically describes the network security situation awareness system based on multi-dimensional data fusion in this embodiment (including but not limited to technical effects, specific technical steps, etc.), reference can be made to the description in the aforementioned network security situation awareness method based on multi-dimensional data fusion, and this text will not elaborate here.

[0109] Finally, it should be noted that the above are only the preferred embodiments of the present application and are not used to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A network security situation awareness method based on multi-dimensional data fusion, characterized in that: The method comprises the following steps: Step 1: Collect network behavior data from at least two different data sources; Step 2: Cleaning and formatting the network behavior data; Step 3: Use the formula Calculate the security situation score, where S is the security situation score, w i is the weight of the i-th dimension, D i is the data value of the network behavior data of the corresponding dimension; Step 4: Predict potential security threats based on the security situation score.

2. The network security situation awareness method based on multi-dimensional data fusion according to claim 1 is characterized in that: The step 1 also includes: identifying the collected network behavior data and marking abnormal data points.

3. The network security situation awareness method based on multi-dimensional data fusion according to claim 2 is characterized in that: The identification of the collected network behavior data and marking of abnormal data points specifically includes: Using the anomaly measurement function A(x i )Analyze data point x i The degree of deviation in the dimension to which it belongs; the abnormal measurement function A(x i ) is as follows: Among them, x ij For data point x i The value of the jth dimension, E j is the average value of all data points in the jth dimension, σ j is the standard deviation of all data points in the jth dimension, Corr(x i ,X) is the data point x i The correlation score with all other data points X in all dimensions, γ is the preset weight coefficient used to adjust the importance of the deviation degree.

4. The network security situation awareness method based on multi-dimensional data fusion according to claim 1 is characterized in that: In the step three, the weight w of each dimension is optimized by constructing an association graph, wherein the association graph includes a plurality of nodes and a plurality of edges, wherein each node represents a different data point, and the weight of each edge is used to express the strength of the correlation between two data points.

5. The network security situation awareness method based on multi-dimensional data fusion according to claim 4 is characterized in that: In step 3, the optimized weight w of the i-th dimension i The acquisition rules are: w i '=w i +a*S j≠i (E ij -w j ) Among them, α is the preset learning rate, E ij is the edge weight between dimension i and dimension j obtained based on the association graph.

6. The network security situation awareness method based on multi-dimensional data fusion according to claim 1 is characterized in that: In the step 4, potential security threats are predicted for different network environments based on the dynamically updated evaluation threshold. When the security situation score of a network environment is lower than the corresponding evaluation threshold, it indicates that there is a potential security threat in the network environment.

7. The network security situation awareness method based on multi-dimensional data fusion according to claim 6 is characterized in that: The dynamic update of the evaluation threshold is obtained by the following formula: T'=T base *(1+k) Among them, T base is the preset basic threshold, and k is the preset adjustment coefficient.

8. The network security situation awareness method based on multi-dimensional data fusion according to claim 1 is characterized in that: The method further includes: Step 5: Display the network security status in a visual way.

9. The network security situation awareness method based on multi-dimensional data fusion according to claim 8 is characterized in that: The step five specifically includes: The security status is displayed by converting the security situation score into color intensity, wherein the color intensity conversion is realized by a color mapping function, and the formula of the color mapping function is: Among them, S max and S min are the maximum and minimum security posture scores in all dimensions, Express The result is rounded down.

10. A network security situation awareness system based on multi-dimensional data fusion, applicable to the network security situation awareness method based on multi-dimensional data fusion as described in any one of claims 1 to 9, characterized in that: include: A data collection module, for collecting network behavior data from at least two different data sources; A data preprocessing module, for cleaning and formatting the collected data; Data analysis module, using formula Calculate the security situation score, where S is the security situation score, w i is the weight of the i-th dimension, D i is the data value of the corresponding dimension; The risk assessment module is used to predict potential security threats based on the security situation score.

Images