Network security situation awareness method based on multi-source heterogeneous data

Abstract

The invention discloses a network security situation awareness method based on multi-source heterogeneous data, and belongs to the technical field of network security, and the method comprises the steps: configuring a plurality of groups of independent data sources, so as to collect multi-source heterogeneous monitoring data; carrying out range transformation dimensionless preprocessing on the acquired data according to the reference sequence, and calculating a grey correlation degree by adopting an adaptive weight mechanism to obtain a quality evaluation reference value; establishing an evidence weight distribution mechanism based on the quality evaluation reference value, and constructing a recognition framework to calculate a basic probability assignment so as to fuse and / or redistribute the data; and performing stratified sampling on the data source according to the quality score to obtain a situation evolution trend by constructing regression prediction of the feature vector. Through the scheme of the invention, the accuracy and stability of network security situation prediction can be improved.

Description

Technical Field

[0001] The present application relates to the field of network security, and in particular to a network security situation awareness method based on multi-source heterogeneous data. Background Art

[0002] Network security situation awareness technology refers to the technology that collects, processes, analyzes and integrates various security-related information in the network to conduct a comprehensive, accurate and real-time assessment and prediction of the network security status and provide support for security decision-making. The main uses of network security situation awareness technology include: real-time monitoring of network security events, identification of potential security threats, assessment of network security risks, prediction of future security situation, assistance in security decision-making, and improvement of network security protection capabilities. Its application areas include network security operation center (SOC), security information and event management (SIEM) system, threat intelligence platform, intrusion detection system (IDS), intrusion prevention system (IPS), etc. In the existing network security situation awareness technology system, the processing and integration of multi-source heterogeneous data is the key link to achieve accurate situation assessment. At present, the industry mainly uses data preprocessing, feature extraction, association analysis, evidence fusion and other technical means to process data from different sources, different formats and different semantics. In actual applications, the characteristics of different data sequences are quite different. The fixed resolution coefficient cannot be dynamically adjusted according to the data characteristics, resulting in the deviation of the correlation calculation results from the actual situation when processing data sequences with periodic fluctuations or obvious trend changes, which reduces the accuracy of situation awareness.

[0003] However, when dealing with highly conflicting evidence, the existing technology simply uses normalization processing, ignoring the credibility differences of different evidence sources, which may weaken important evidence, thereby affecting the accuracy of the fusion results and failing to effectively handle conflicting information in a complex and changeable network security environment. The lack of a special processing mechanism for the peak and trough positions of periodic fluctuation data makes it impossible for the correlation calculation results to highlight the importance of key time points, resulting in the inability to detect network attack events in a timely manner and failing to meet the needs of real-time situation awareness. There is no weighted processing for the large interval of trend change data change rate, resulting in the weakening of important trend turning information, and the inability to take timely countermeasures, which cannot meet the needs of dynamic situation awareness. In the process of dynamic selection of data sources, a simple uniform sampling or random sampling method is used, which fails to fully utilize the hierarchical characteristics of the data source, resulting in insufficient representativeness of the selected data source, affecting the accuracy of situation awareness, and failing to meet the needs of comprehensive situation awareness. In summary, the existing network security situation awareness technology has many limitations when processing multi-source heterogeneous data, especially when the data shows strong volatility, trend or conflict, and cannot accurately reflect the true correlation and credibility of the data, and it is difficult to meet the requirements of situation awareness in the increasingly complex network security environment.

[0004] Therefore, there is an urgent need for a technical solution to improve the accuracy and stability of network security situation prediction. Summary of the invention

[0005] In order to solve the deficiencies of the prior art, the embodiment of the present application discloses a network security situation awareness method based on multi-source heterogeneous data. The present application solves the technical problems of the prior art such as the time series correlation between different data sources and the large difference in data quality.

[0006] The embodiment of the present application discloses a network security situation awareness method based on multi-source heterogeneous data, including: configuring multiple groups of independent data sources to collect multi-source heterogeneous monitoring data; performing dimensionless preprocessing of the collected data by range transformation according to a reference sequence, and calculating the grey correlation degree using an adaptive weight mechanism to obtain a quality assessment benchmark value; establishing an evidence weight allocation mechanism based on the quality assessment benchmark value, and constructing a recognition framework to calculate the basic probability assignment to fuse and / or redistribute the data; and stratifying and sampling the data sources according to the quality scores to obtain the situation evolution trend by constructing a regression prediction of a feature vector.

[0007] In a possible implementation, multiple groups of independent data sources are configured to collect multi-source heterogeneous monitoring data, including: setting up multiple groups of independent data sources, and respectively configuring accuracy indicator weights, integrity indicator weights and consistency indicator weights; hierarchically numbering data sources based on monitoring areas, and establishing a time synchronization mechanism to ensure synchronization between data sources; setting a monitoring time window, and collecting monitoring data including object identification, timestamp and equipment status within the window according to the sampling interval.

[0008] In a possible implementation, the collected data is dimensionally preprocessed by range transformation based on a reference sequence, and the grey correlation degree is calculated using an adaptive weight mechanism to obtain a quality assessment benchmark value, including: comparing the data sequence with the reference sequence and calculating the absolute value of the corresponding element difference; preprocessing the sequence data by range transformation method; calculating the grey correlation degree based on the absolute difference sequence to obtain a quality assessment benchmark value; and assigning weights to the correlation degrees of peak and trough positions and large change rate intervals through an adaptive weight adjustment mechanism.

[0009] In a possible implementation, an evidence weight allocation mechanism is established based on a quality assessment benchmark value, and an identification framework is constructed to calculate basic probability assignments to fuse and / or redistribute data, including: establishing an evidence weight allocation mechanism based on data quality, using quality assessment values ​​as evidence weights; constructing an identification framework to calculate basic probability assignments of evidence to different results; using nonlinear mapping functions for probability allocation, and synthesizing evidence through an adaptive weight adjustment factor based on quality assessment; and attenuating and adjusting the weights of conflicting evidence sources through a distance-based evidence conflict detection mechanism.

[0010] In a possible implementation, data sources are stratified and sampled according to quality scores to obtain the situation evolution trend by constructing a regression prediction of a feature vector, including: calculating the comprehensive quality index of the data source and setting a quality assessment cycle; dividing the data source into different levels according to the comprehensive quality score; using a stratified sampling strategy to determine the number of samples for each level; calculating the dynamic weight of the selected data source and using an exponential smoothing transition strategy to switch the data source; constructing a feature vector and using fuzzy normalization to calculate the situation value; performing exponential sliding average smoothing on the situation value and using a non-parametric regression method to predict the situation evolution trend.

[0011] In a possible implementation, the grey correlation degree is calculated based on the absolute difference sequence to obtain a quality assessment benchmark value, including: obtaining a reference sequence and a comparison sequence based on dimensionless preprocessing; calculating the absolute difference between the reference sequence and the comparison sequence at each sampling point to obtain an absolute difference sequence; determining the two-level minimum difference and the two-level maximum difference in the absolute difference sequence, and dividing the sum of the maximum difference between the two-level minimum difference and the product of the resolution coefficient by the sum of the maximum difference between the absolute difference and the product of the resolution coefficient; setting a first weighting coefficient for the correlation of the peak and trough positions for the periodic fluctuation of the data sequence; setting a second weighting coefficient for the correlation in a larger change rate interval for the trend change of the data sequence; and calculating the average correlation between the data source and the reference sequence within the entire time window to obtain a quality assessment benchmark value.

[0012] In a possible implementation, a nonlinear mapping function is used for probability allocation, and evidence synthesis is performed through an adaptive weight adjustment factor based on quality assessment, including: using quality assessment values ​​and characteristic coefficients to calculate the support of the data source for each subset through an exponential function; forming a basic probability allocation matrix, and calculating the degree of conflict between two evidence sources; when the degree of conflict exceeds a preset threshold, calculating the cube root of the quality assessment values ​​of the two evidence sources as an adaptive weight adjustment factor; multiplying the ratio of the adaptive weight adjustment factor to the evidence combination result by the intersection combination of the basic probabilities; calculating the distance matrix between the evidence sources, and performing an exponential decay adjustment on the weights of the evidence sources that exceed the distance threshold.

[0013] In a possible implementation, a feature vector is constructed and a situation value is calculated using fuzzy standardization processing, including: constructing a feature vector including the proportion of abnormal traffic, the number of alarm events, the proportion of attacked assets, the vulnerability index and the degree of compliance with the security policy; performing fuzzy standardization processing on the feature vector to obtain a standardized matrix; constructing a feature judgment matrix, and obtaining a dynamic weight vector by calculating the maximum eigenvalue and its eigenvector; determining a positive ideal solution and a negative ideal solution, and calculating the Euclidean distance from the current state to the positive and negative ideal solutions; calculating the ratio of the distance to the negative ideal solution to the sum of the distances to the positive and negative ideal solutions to obtain a situation value; performing exponential sliding average processing on the situation value, and calculating the rate of change within a continuous time window.

[0014] In a network security situation awareness method based on multi-source heterogeneous data as disclosed above, the embodiment of the present application can more accurately process multi-source heterogeneous data with volatility and trend characteristics by introducing an adaptive weight adjustment mechanism, a probability distribution function based on nonlinear mapping, an evidence conflict detection mechanism based on Mahalanobis distance, and a stratified sampling and exponential smoothing transition strategy. Furthermore, in some embodiments, the dynamic characteristics of the data sequence are considered in the calculation of the correlation degree, and the accuracy of the association analysis can be improved by differentially weighting the positions of the feature points; in the process of evidence fusion, evidence conflicts are more reasonably handled through nonlinear mapping and adaptive weight adjustment; when selecting data sources, a stratified sampling strategy based on entropy weight is adopted to improve the pertinence of data source selection, and smooth switching of data sources is achieved through exponential smoothing. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 A schematic diagram of a process of a network security situation awareness method based on multi-source heterogeneous data disclosed in an embodiment of the present application;

[0017] Figure 2 A scatter plot of the fusion effect of DS evidence theory disclosed in an embodiment of the present application;

[0018] Figure 3 A situation prediction error box plot disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0019] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0020] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor represent the necessary logical order between them. It should also be understood that in the embodiments of the present disclosure, "multiple" can refer to two or more, and "at least one" can refer to one, two or more. It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, in the absence of explicit limitation or contrary revelation given in the context, it can generally be understood as one or more. In addition, the term "and / or" in the present disclosure is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are an "or" relationship. It should also be understood that the description of each embodiment in the present disclosure emphasizes the differences between the embodiments, and the same or similar parts can refer to each other. For the sake of brevity, they will not be repeated one by one.

[0021] At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. The techniques, methods and devices known to ordinary technicians in the relevant fields may not be discussed in detail, but where appropriate, the techniques, methods and devices should be considered part of the specification. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] Figure 1A flow chart of a network security situation awareness method based on multi-source heterogeneous data disclosed in an embodiment of the present application. It should be understood that conventional network security situation awareness technologies mainly include vulnerability assessment technology based on active scanning, security log analysis technology based on SIEM system, anomaly detection technology based on flow analysis, intrusion detection technology based on NIDS / HIDS, etc. Among them, the vulnerability assessment technology based on active scanning uses scanning tools such as Nessus and Nmap to regularly scan network assets, and obtains potential threats by analyzing the returned information; the log analysis technology based on SIEM uses ELK, Splunk and other platforms to uniformly collect and analyze the logs of firewalls, IDS and other devices; anomaly detection based on flow analysis uses tools such as Wireshark and Snort to monitor network traffic in real time; intrusion detection based on NIDS / HIDS identifies suspicious behaviors by deploying sensors at key nodes of the network. In terms of data processing, these technologies mainly use rule-based matching to extract features, use threshold-based alarm triggering mechanisms, and multi-source data fusion methods based on simple weighting. However, this processing method has obvious technical limitations when facing multi-source heterogeneous data: the rule matching method relies too much on the preset feature library and cannot effectively handle unknown threats such as zero-day attacks; the simple threshold trigger mechanism is prone to false positives and false negatives, especially when facing volatile network traffic data; the traditional weighted fusion method does not consider the correlation and conflict between data sources, which affects the reliability of the fusion results. In addition, when there are multiple monitoring areas and a large number of device nodes in the network environment, the time series correlation and data quality differences between different data sources make it difficult for traditional methods to accurately reflect the overall security situation of the network.

[0024] In view of the above technical problems, this technical solution adopts a differentiated processing strategy for data sequences with different characteristics by introducing an adaptive analysis mechanism based on gray correlation in the data acquisition link. When processing periodic network traffic data, by applying weights to the peak and trough positions, the sensitivity to traffic mutation points is improved; for attack behavior sequences with obvious trends, by assigning weights to intervals with larger change rates, the grasp of the evolution of attack trends is enhanced. In the evidence fusion stage, the evidence conflict detection mechanism based on Mahalanobis distance is used to replace the traditional similarity calculation method, and the probability distribution function of nonlinear mapping is introduced to replace simple weighting, so that the fusion results can more accurately reflect the credibility differences of different data sources. In addition, when dynamically selecting data sources, a stratified sampling strategy based on entropy weight is adopted, and the sampling ratios of different quality levels are determined by the Neyman optimal allocation method, which overcomes the problem that the traditional uniform sampling method does not consider the quality differences of data sources enough. At the same time, the smooth switching of data sources is achieved through the exponential smoothing transition strategy, avoiding the jump of monitoring results caused by direct switching in traditional methods. These technical improvement measures specifically solve the specific problems existing in conventional situation awareness technology in multi-source heterogeneous data processing, and disclose a more accurate and stable data processing solution.

[0025] like Figure 1 As shown, in step S101, multiple groups of independent data sources are configured to collect multi-source heterogeneous monitoring data. This includes: setting multiple groups of independent data sources, respectively configuring the accuracy index weight, integrity index weight and consistency index weight; hierarchically numbering the data sources based on the monitoring area, establishing a time synchronization mechanism to ensure synchronization between data sources; setting a monitoring time window, and collecting monitoring data including object identification, timestamp and device status according to the sampling interval within the window.

[0026] Specifically, first, for the devices in the network environment, multi-source heterogeneous data based on different heterogeneous data sources are obtained to measure the multi-dimensional attributes of the monitored objects. The specific operation is to obtain the operating data from the device nodes in each monitoring area, set multiple independent data sources in each area, and set the weight indicators of accuracy, completeness and consistency for each data source to obtain the evaluation data of the monitored object from each independent data source in each time segment. When acquiring data, each independent data source is numbered according to its physical location, and a unified time synchronization benchmark is set so that the data sources are synchronized within the system unit time. For each data source, the monitoring time window length is set to 30-35 seconds, and a set of monitoring data is collected at intervals of 0.5-1.5 seconds in each time window. Each set of monitoring data contains basic information such as the object's identity, timestamp, and device status. In the process of data acquisition, the original data is preliminarily screened by setting a gray correlation threshold with a demarcation coefficient of 0.5.

[0027] In an embodiment of the present application, a distributed multi-source heterogeneous data acquisition system is first constructed to obtain multi-dimensional attribute data of each monitored object in a network environment. Specifically, a number of data acquisition devices are configured in each monitoring area, and these devices continuously observe the monitored objects based on different data sources. The system sets corresponding evaluation index weights for each independent data source, including an accuracy index weight between 0.31 and 0.42, an integrity index weight between 0.28 and 0.37, and a consistency index weight between 0.25 and 0.35. These weight indicators are used to quantitatively evaluate the observation quality of each data source on the monitored object within a unit time segment.

[0028] In a preferred embodiment, the system assigns a unique physical location number to each independent data source. The numbering rule adopts a hierarchical structure, with the first layer representing the monitoring area number, the second layer representing the device group number in the area, and the third layer representing the specific device number. For example, the number "2-03-047" represents the 47th data source in the 3rd group of devices in the 2nd monitoring area. At the same time, the system establishes a unified time synchronization mechanism, and by deploying a high-precision time server, it ensures that the time synchronization error between data sources does not exceed 0.02 seconds.

[0029] Specifically for data collection, the system sets a fixed monitoring time window for each data source, with a window length of about 30 seconds. In each monitoring window, a set of monitoring data is obtained with a sampling interval of 0.5-1.5 seconds. Each set of monitoring data contains multiple key fields: unique object identifier, timestamp information accurate to milliseconds, current operating status parameters of the device, network connection status information, resource occupancy and other basic data items. The specific attributes and value ranges of these data items are configured according to actual monitoring needs.

[0030] In step S102, the collected data is preprocessed by range transformation dimensionless according to the reference sequence, and the grey correlation degree is calculated by adaptive weight mechanism to obtain the quality assessment reference value. This includes: comparing the data sequence with the reference sequence, calculating the absolute value of the corresponding element difference; preprocessing the sequence data dimensionless by range transformation method; calculating the grey correlation degree according to the absolute difference sequence to obtain the quality assessment reference value; assigning weights to the correlation degree of the peak and trough positions and the interval with large change rate through adaptive weight adjustment mechanism.

[0031] In one implementation scenario, the quality assessment and preprocessing of the acquired multi-source heterogeneous data are carried out. The correlation between different data sequences is quantitatively analyzed by establishing a grey correlation analysis method, and the data quality value generated by each data source is calculated. The specific steps are: first, the data sequence obtained by each data source is compared with the standard reference sequence, and the absolute value of the difference between the corresponding elements in the two sequences is calculated to obtain the minimum difference and the maximum difference. Then, the correlation coefficient is calculated based on the minimum difference and the maximum difference, and the quality assessment value of the data source is obtained by averaging the correlation coefficients between each indicator and the corresponding elements of the reference sequence. In this process, the resolution coefficient of the correlation calculation is set to 0.37 to enhance the distinguishing ability of the evaluation results. For each data source, the data integrity rate in a complete time window is calculated. When the integrity rate is lower than 75%, the data in the time window is marked as low-quality data.

[0032] Among them, the grey correlation degree is calculated according to the absolute difference sequence to obtain the quality assessment benchmark value, including: obtaining a reference sequence and a comparison sequence based on dimensionless preprocessing; calculating the absolute difference between the reference sequence and the comparison sequence at each sampling point to obtain an absolute difference sequence; determining the two-level minimum difference and the two-level maximum difference in the absolute difference sequence, and dividing the sum of the maximum difference between the two-level minimum difference and the product of the resolution coefficient by the sum of the maximum difference between the absolute difference and the product of the resolution coefficient; setting a first weighting coefficient for the correlation of the peak and trough positions in view of the periodic fluctuation of the data sequence; setting a second weighting coefficient for the correlation of a larger change rate interval in view of the trend change of the data sequence; and calculating the average correlation between the data source and the reference sequence in the entire time window to obtain the quality assessment benchmark value.

[0033] In one embodiment, when performing grey correlation calculation, the sequence data collected from each data source is first dimensionlessly preprocessed. Set the initial reference sequence X0 = {x0(1), x0(2), ..., x0(n)}, where n is the number of sampling points in the time window, and the comparison sequence is X i ={x i (1), x i (2), ..., x i (n)}. The dimensionless processing is performed through the range transformation method. For each sequence, the value of the k-th sampling point is divided by the initial value of the sequence to obtain the dimensionless sequence, the reference sequence: Comparing sequences: Then calculate the absolute difference Δ between the reference sequence and the comparison sequence at the kth sampling point i (k)=|x′0(k)-x′ i (k)|, where X′0 and X′ iare the dimensionless reference sequence and comparison sequence respectively. Determine the minimum difference and maximum difference between the two sequences, i.e., minminΔ i (k) and maxmaxΔ i (k). Based on the absolute difference sequence, the improved Deng's grey relational degree calculation formula is adopted: Among them, 0.37 is the resolution coefficient. For each data source, calculate its gray correlation degree with the reference sequence in the entire time window In the calculation process, an adaptive weight adjustment mechanism is introduced for different types of data features: when the data sequence shows obvious periodic fluctuations, the correlation between the peak and trough positions is given a weight of 1.27 times; when the sequence shows obvious trend changes, the correlation between the intervals with larger change rates is given a weight of 1.18 times. The final gray correlation degree is γ i is the quality assessment benchmark value of the data source.

[0034] At step S103, an evidence weight allocation mechanism is established based on the quality assessment benchmark value, and a recognition framework is constructed to calculate the basic probability assignment to fuse and / or redistribute the data. This includes: establishing an evidence weight allocation mechanism based on data quality, using the quality assessment value as the evidence weight; constructing a recognition framework to calculate the basic probability assignment of evidence to different results; using a nonlinear mapping function to allocate probabilities, and synthesizing evidence through an adaptive weight adjustment factor based on quality assessment; and attenuating and adjusting the weights of conflicting evidence sources through a distance-based evidence conflict detection mechanism.

[0035] Specifically, multi-source heterogeneous data are fused based on the DS evidence theory. First, an evidence weight allocation mechanism based on data quality is established, and the quality assessment value of each data source is used as its evidence weight. For each monitored object, the evaluation results from different data sources are regarded as independent evidence, and an identification framework containing all possible results is established. The initial body of evidence is obtained by calculating the basic probability assignment of each evidence to different results. When fusing evidence, the improved DS combination rule is adopted, and the trust threshold is set to 0.83. When the credibility of a result exceeds the threshold, it is used as the fusion result. For conflicting evidence, a conflict redistribution strategy based on data quality is introduced to redistribute the conflicting information according to the quality weight ratio of each data source.

[0036] Among them, probability allocation is performed using a nonlinear mapping function, and evidence synthesis is performed through an adaptive weight adjustment factor based on quality assessment, including: using quality assessment values ​​and characteristic coefficients to calculate the support of the data source for each subset through an exponential function; forming a basic probability allocation matrix, and calculating the degree of conflict between two evidence sources; when the degree of conflict exceeds a preset threshold, calculating the cube root of the quality assessment values ​​of the two evidence sources as an adaptive weight adjustment factor; multiplying the ratio of the adaptive weight adjustment factor to the evidence combination result by the intersection combination of the basic probabilities; calculating the distance matrix between the evidence sources, and performing an exponential decay adjustment on the weights of the evidence sources that exceed the distance threshold.

[0037] In one embodiment, based on the DS evidence theory, when multi-source heterogeneous data are fused, the recognition framework Θ = {θ1, θ2, .., θn} is first constructed, where Θi represents possible state results. For the evidence generated by each data source Di, based on its quality evaluation value γ i Construct the basic probability distribution function m i When assigning the base probability, a nonlinear mapping function based on quality assessment is adopted: Where A is a subset of the recognition frame Θ, s i (A) is the support of data source Di for subset A, β i is the characteristic coefficient of the data source, which is determined according to the historical performance of the data source and has a value range of [1.6, 2.8]. For each data source, its support for different state combinations is calculated to form a basic probability distribution matrix. When performing evidence synthesis, the improved Dempster combination rule is adopted: when the conflict degree between two evidence sources exceeds 0.83, an adaptive weight adjustment factor based on quality assessment is introduced The corrected combination formula is: Where B∩C=A. For multiple evidence sources, they are combined in a pairwise progressive manner, and the weight coefficient is updated after each combination. When highly conflicting evidence appears, an evidence conflict detection mechanism based on Mahalanobis distance is introduced. By calculating the distance matrix D=[d ij ], where d ij Represents the distance between evidence sources i and j. When the distance exceeds the set threshold of 1.73, the weight of the corresponding evidence source is attenuated: Where λ is the attenuation coefficient, which is 0.62.

[0038] At step S104, the data sources are stratified and sampled according to the quality scores, so as to obtain the situation evolution trend by constructing the regression prediction of the feature vector. This includes: calculating the comprehensive quality index of the data source and setting the quality evaluation cycle; dividing the data source into different levels according to the comprehensive quality score; using the stratified sampling strategy to determine the number of samples at each level; calculating the dynamic weight of the selected data source and using the exponential smoothing transition strategy to switch the data source; constructing the feature vector and using the fuzzy normalization to calculate the situation value; performing the exponential sliding average smoothing on the situation value and using the non-parametric regression method to predict the situation evolution trend.

[0039] Among them, constructing a feature vector and using fuzzy standardization to calculate the situation value include: constructing a feature vector including the proportion of abnormal traffic, the number of alarm events, the proportion of attacked assets, the vulnerability index and the degree of compliance with the security policy; performing fuzzy standardization on the feature vector to obtain a standardized matrix; constructing a feature judgment matrix, and obtaining a dynamic weight vector by calculating the maximum eigenvalue and its eigenvector; determining the positive ideal solution and the negative ideal solution, and calculating the Euclidean distance from the current state to the positive and negative ideal solutions; calculating the ratio of the distance to the negative ideal solution to the sum of the distances to the positive and negative ideal solutions to obtain the situation value; performing exponential sliding average processing on the situation value, and calculating the rate of change within a continuous time window.

[0040] In one embodiment, based on the results of the fusion processing, the network security situation is continuously perceived. First, the performance of each data source in the evaluation cycle is quantitatively evaluated. The evaluation index vector V = (v1, v2, v3) is set to correspond to accuracy, completeness and consistency respectively, and the scoring mechanism based on entropy weight is adopted: for index j, its entropy value e is calculated j =-k×∑ i (p ij ×ln(p ij )),in n is the number of data sources, p ij is the normalized score of the ith data source on indicator j. Calculate indicator weight Calculate a comprehensive quality score for each data source: S i =∑ j w j v ij When dynamically selecting data sources, an improved stratified sampling strategy is adopted: first, data sources are divided into three levels according to their comprehensive quality scores. The quality scores greater than 0.83 are the high-quality layer, between 0.67 and 0.83 are the medium-quality layer, and below 0.67 are the low-quality layer. In each layer, the Neyman optimal allocation method is used to determine the number of samples: Where nh is the number of data sources at the hth layer, S his the standard deviation of the quality score of this layer. For each selected data source, calculate its dynamic weight in the next evaluation cycle: i =S i ×(1+μ×ΔS i ), where ΔS i ) is the rate of change of the quality score, μ is the adjustment coefficient, and its value is 0.43. When switching data sources, a transition strategy based on exponential smoothing is adopted: within the overlapping monitoring window, the weights of the new and old data sources are calculated according to Dynamic adjustment is performed, where τ is the time constant and takes a value of 1.26 seconds. Based on the selected high-quality data source, the network security situation value at the current moment is calculated. First, the feature vector F = (f1, f2, f3, f4, f5) is constructed, where f1 represents the proportion of abnormal traffic, f2 represents the number of alarm events, f3 represents the proportion of attacked assets, f4 represents the vulnerability index, and f5 represents the compliance with the security policy. The improved TOPSIS method is used to calculate the situation value: the feature vector is fuzzy standardized to obtain the standardized matrix R = [r ij ],in Considering the importance of different features, a dynamic weight vector W = (w1, w2, w3, w4, w5) is introduced, where the feature weights are obtained through the hierarchical analysis method: construct a judgment matrix A = [a ij ], aij represents the importance of feature i relative to feature j, and its value range is [1 / 9,9]. The weight vector is obtained by calculating the maximum eigenvalue λmax and the corresponding eigenvector of A and performing normalization. Determine the positive ideal solution S+ and the negative ideal solution S-, where Calculate the distance from the current state to the positive and negative ideal solutions: Final state value The trend of the situation value is smoothed by exponential sliding average: T smooth (t) = α × T (t) + (1-α) × T smooth (t-1), where the smoothing coefficient α = 0.37. When the rate of change of the situation value in three consecutive time windows exceeds the set threshold of 0.28, the non-parametric regression method is used to predict the future situation evolution trend: the improved kernel density estimation method is used, the kernel function selects the Epanechnikov kernel, and the bandwidth is determined to be 0.86 through cross-validation. Finally, based on the time series characteristics of the situation value, the wavelet decomposition method is used for multi-scale analysis, the db4 wavelet base is selected, the decomposition layer number is 3, and the situation characteristics at different scales are extracted to identify potential situation mutation points and evolution patterns.

[0041] In summary, the embodiments of the present application disclose a technical solution that can more accurately process multi-source heterogeneous data with volatility and trend characteristics by introducing an adaptive weight adjustment mechanism, a probability distribution function based on nonlinear mapping, an evidence conflict detection mechanism based on Mahalanobis distance, and a stratified sampling and exponential smoothing transition strategy. This solution takes into account the dynamic characteristics of the data sequence in the calculation of the correlation degree, and improves the accuracy of the association analysis by differentially weighting the positions of the feature points; in the process of evidence fusion, nonlinear mapping and adaptive weight adjustment are used to more reasonably handle evidence conflicts; when selecting data sources, a stratified sampling strategy based on entropy weight is adopted to improve the pertinence of data source selection, and smooth switching of data sources is achieved through exponential smoothing. This technical solution avoids the limitations of traditional methods in processing complex data and discloses more accurate and stable data processing capabilities.

[0042] Figure 2 This is a scatter plot of the fusion effect of the DS evidence theory disclosed in the embodiment of this application. Based on the analysis of the fusion effect of the DS evidence theory, this figure shows the performance comparison of the DS evidence theory and other fusion methods under different conflict conditions. The horizontal axis in the figure represents the conflict degree k between the evidence sources. 12 , the value range is 0 to 1; the vertical axis represents the accuracy of the fusion result; the color depth of the scattered points represents the quality assessment value γ of the data source i , darker colors indicate higher quality.

[0043] In principle, DS evidence theory uses a nonlinear mapping function based on quality assessment to distribute basic probabilities. When there is a conflict between two sources of evidence, an adaptive weight adjustment factor based on quality assessment is introduced. Modify the combination rules. Specifically, for the conflict degree k 12 When the value exceeds 0.83, the modified combination formula is: Where B∩C=A. This processing method enables the system to adaptively adjust the weight of evidence according to the quality of the data source, so as to maintain a good fusion effect in high-conflict scenarios.

[0044] From the data distribution in the figure, it can be observed that in the area with low conflict degree (k 12 <0.83), the performance of DS evidence theory is similar to that of Bayesian fusion, fuzzy set method, and evidence distance method, and the fusion accuracy rate gradually decreases with the increase of conflict degree. However, in the high conflict area (k 12 >0.83), DS evidence theory shows obvious advantages. By introducing the evidence conflict detection mechanism of Mahalanobis distance, when the distance exceeds the set threshold of 1.73, the weight of the corresponding evidence source is attenuated and adjusted: Where λ is the attenuation coefficient, which takes a value of 0.62. This mechanism enables the system to identify and process highly conflicting evidence in a timely manner.

[0045] The distribution characteristics of the scattered points in the figure show that the quality of the data source γ i It has a significant impact on the fusion effect. i When the value is high (the darker points in the figure), the decrease in fusion accuracy is relatively small even in high-conflict scenarios. This verifies the effectiveness of the weight allocation mechanism based on quality assessment. By setting the trust threshold to 0.83, the system can reasonably handle evidence conflicts while ensuring fusion reliability. For conflicting evidence, a conflict redistribution strategy based on data quality is adopted to redistribute the conflicting information according to the quality weight ratio of each data source, further improving the stability of fusion.

[0046] Experimental data show that in the conflict degree k 12 >0.83, the fusion accuracy of DS evidence theory is 8.2 percentage points higher than Bayesian fusion, 7.5 percentage points higher than fuzzy set method, and 9.1 percentage points higher than evidence distance method. i When it is greater than 0.85, even if the conflict degree is close to 1, the DS evidence theory can still maintain a fusion accuracy of no less than 0.78. This shows that this method can effectively handle high-conflict evidence through quality assessment and adaptive weight adjustment.

[0047] Combining the evaluation indicators of data quality in terms of accuracy, completeness and consistency, the system dynamically evaluates the comprehensive quality of each data source. The evaluation cycle is set to 86.4 seconds. At the end of each evaluation cycle, the optimal data source selection scheme is determined by the Bernoulli uniform sampling algorithm based on the accuracy requirement ε (0.15) and confidence requirement δ (0.92) set by the user. This dynamic selection mechanism ensures that the system can continuously obtain high-quality data sources and disclose reliable input for evidence fusion.

[0048] Figure 3 This is a box plot of the situation prediction error disclosed in the embodiment of this application. This figure shows the prediction error analysis of the network security situation awareness method based on DS evidence theory under different prediction time windows. Based on the evaluation cycle, three typical prediction windows of 30 seconds, 60 seconds and 90 seconds were selected for performance comparison. The analysis objects include the method proposed in this article and the traditional TOPSIS method, kernel estimation method and wavelet analysis method.

[0049] At the data foundation level, the system uses a data sampling interval of 0.7 seconds and sets a monitoring time window of 32.6 seconds. The original data is screened by setting a gray correlation threshold of 0.5 as the demarcation coefficient to ensure data quality. The dynamic selection of data sources is based on the Bernoulli uniform sampling algorithm, in which the data quality assessment adopts a hierarchical structure: the quality score greater than 0.83 is the high-quality layer, between 0.67 and 0.83 is the medium-quality layer, and below 0.67 is the low-quality layer.

[0050] In terms of prediction mechanism, the system constructs a five-dimensional feature vector (abnormal traffic proportion, number of alarm events, proportion of attacked assets, vulnerability index, and security policy compliance), and determines the feature weights through the analytic hierarchy process. The situation value calculation adopts the improved TOPSIS method, and uses a smoothing coefficient of 0.37 for exponential sliding average processing. When the rate of change of the situation value in three consecutive time windows exceeds 0.28, the system starts the prediction mechanism. The kernel density estimation uses the Epanechnikov kernel function, and the bandwidth is determined to be 0.86 through cross-validation.

[0051] From the data distribution in the figure, we can observe that: under the 30-second prediction window, the median prediction error of this method is 2.8%, which is significantly lower than the 4.3% of other methods; under the 60-second prediction window, the error of this method is 4.3%, while the other methods reach 5.8%; under the 90-second prediction window, the error of this method is maintained at 6.2%, which is significantly better than the 8.7% of other methods. This performance difference is statistically significant, and the p value is less than 0.05.

[0052] The interquartile range of the box plot reflects the stability of the prediction. The prediction results of this method in each time window show a small fluctuation range. This stability comes from the multi-layer prediction strategy adopted by the system: first, wavelet decomposition (selecting db4 wavelet basis, 3-layer decomposition) is performed to extract multi-scale features; then, non-parametric regression methods are used to predict trends; finally, the prediction results are adjusted through dynamic weights.

[0053] In terms of data source switching, the system adopts a transition strategy based on exponential smoothing, with a time constant of 1.26 seconds, to achieve a smooth transition within a 3.8-second overlapping monitoring window. This mechanism ensures the continuity and stability of data source switching during the prediction process. The number of samples for each level of data source is determined by the Neyman optimal allocation method, and a dynamic weight adjustment factor of μ = 0.43 is introduced to achieve dynamic evaluation and optimal selection of data source quality.

[0054] The improvement of the system's situation prediction effect is mainly based on the following technical elements: first, a data quality evaluation system with three dimensions of accuracy (0.31-0.42), completeness (0.28-0.37) and consistency (0.25-0.35) is adopted; second, a confidence threshold of 0.83 is set for evidence fusion; third, a decay coefficient of 0.62 is introduced in the prediction model to deal with highly conflicting evidence; fourth, a distance threshold of 1.73 is used for evidence conflict detection. The combination of these parameters ensures that the system can maintain high prediction accuracy and stability at different prediction time scales.

[0055] Furthermore, an embodiment of the present application also discloses a network security situation awareness device based on multi-source heterogeneous data, including: a processor, a memory, and a system bus; the processor and the memory are connected via the system bus; the memory is used to store one or more programs, and the one or more programs include instructions, which, when executed by the processor, enable the processor to execute any of the above methods.

[0056] Furthermore, an embodiment of the present application also discloses a computer program product, which, when executed on a terminal device, enables the terminal device to execute any of the above methods.

[0057] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0058] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0059] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0060] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A network security situation awareness method based on multi-source heterogeneous data, characterized in that: include: Configure multiple sets of independent data sources to collect multi-source heterogeneous monitoring data; The collected data is preprocessed by range transformation and dimensionless preprocessing based on the reference sequence, and the grey relational degree is calculated using an adaptive weight mechanism to obtain a quality assessment benchmark value; Establish an evidence weight assignment mechanism based on quality assessment benchmark values, and construct an identification framework to calculate basic probability assignments to fuse and / or redistribute data; Data sources are stratified and sampled by quality score to obtain the situation evolution trend by constructing regression prediction of feature vectors.

2. The network security situation awareness method according to claim 1 is characterized in that: in, Configure multiple independent data sources to collect multi-source heterogeneous monitoring data, including: Set up multiple groups of independent data sources, and configure the accuracy index weight, completeness index weight, and consistency index weight respectively; Hierarchical numbering of data sources based on monitoring areas, and establishment of a time synchronization mechanism to ensure synchronization between data sources; Set a monitoring time window and collect monitoring data including object identification, timestamp and device status according to the sampling interval within the window.

3. The network security situation awareness method according to claim 1 is characterized in that: in, The collected data is preprocessed by range transformation and dimensionless preprocessing based on the reference sequence, and the grey relational degree is calculated using an adaptive weight mechanism to obtain the quality assessment benchmark value, including: Compare the data sequence with the reference sequence and calculate the absolute value of the corresponding element difference; The range transformation method is used to preprocess the sequence data into dimensionless form; Calculate the grey relational degree according to the absolute difference sequence to obtain the quality assessment benchmark value; Weights are assigned to the correlation between the peak and trough positions and the intervals with larger change rates through an adaptive weight adjustment mechanism.

4. The network security situation awareness method according to claim 1 is characterized in that: in, Establish an evidence weight allocation mechanism based on the quality assessment benchmark value, and build an identification framework to calculate the basic probability assignment to fuse and / or redistribute the data, including: Establish an evidence weight allocation mechanism based on data quality, and use the quality assessment value as the evidence weight; Construct an identification framework to calculate the basic probability of different results assigned by evidence; Probability assignment is performed using nonlinear mapping functions, and evidence synthesis is performed through adaptive weight adjustment factors based on quality assessment; The weights of conflicting evidence sources are attenuated and adjusted through a distance-based evidence conflict detection mechanism.

5. The network security situation awareness method according to claim 1 is characterized in that: in, Stratified sampling of data sources by quality score is performed to obtain the situation evolution trend by constructing regression prediction of feature vectors, including: Calculate comprehensive quality indicators of data sources and set quality assessment cycles; Divide data sources into different levels according to their comprehensive quality scores; A stratified sampling strategy was used to determine the number of samples in each stratum; Calculate the dynamic weight of the selected data source and use an exponential smoothing transition strategy to switch the data source; Construct feature vectors and calculate situation values ​​using fuzzy normalization processing; The situation value is smoothed by exponential moving average, and the non-parametric regression method is used to predict the trend of situation evolution.

6. The network security situation awareness method according to claim 3 is characterized in that: in, The grey relational degree is calculated according to the absolute difference sequence to obtain the quality assessment benchmark value, including: The reference sequence and comparison sequence were obtained based on dimensionless preprocessing; Calculating the absolute difference between the reference sequence and the comparison sequence at each sampling point to obtain an absolute difference sequence; Determine the minimum difference between two levels and the maximum difference between two levels in the absolute difference sequence, and divide the sum of the maximum difference between the minimum difference between two levels and the product of the resolution coefficient by the sum of the maximum difference between the absolute difference and the product of the resolution coefficient; In view of the periodic fluctuation of the data sequence, a first weighting coefficient is set for the correlation between the peak and trough positions; In view of the trend change of the data sequence, a second weighting coefficient is set for the correlation degree of the interval with a larger change rate; The average correlation between the data source and the reference sequence is calculated within the entire time window to obtain a quality assessment benchmark value.

7. The network security situation awareness method according to claim 4 is characterized in that: in, Probability assignment is performed using a nonlinear mapping function, and evidence synthesis is performed through an adaptive weight adjustment factor based on quality assessment, including: Using the quality assessment value and characteristic coefficient, the support of the data source for each subset is calculated through an exponential function; Form a basic probability distribution matrix and calculate the degree of conflict between two sources of evidence; When the conflict degree exceeds a preset threshold, the cube root of the quality evaluation values ​​of the two evidence sources is calculated as an adaptive weight adjustment factor; The ratio of the adaptive weight adjustment factor to the evidence combination result is multiplied by the intersection combination of the basic probabilities; The distance matrix between evidence sources is calculated, and the weights of evidence sources exceeding the distance threshold are adjusted exponentially.

8. The network security situation awareness method according to claim 5 is characterized in that: in, Construct feature vectors and calculate situation values ​​using fuzzy normalization, including: Construct a feature vector including abnormal traffic proportion, number of alarm events, proportion of attacked assets, vulnerability index and security policy compliance; Performing fuzzy normalization processing on the eigenvector to obtain a normalized matrix; Construct a feature judgment matrix and obtain a dynamic weight vector by calculating the maximum eigenvalue and its eigenvector; Determine the positive ideal solution and the negative ideal solution, and calculate the Euclidean distance from the current state to the positive and negative ideal solutions; Calculate the ratio of the negative ideal solution distance to the sum of the positive and negative ideal solution distances to obtain the situation value; The situation value is subjected to exponential sliding average processing, and the rate of change within the continuous time window is calculated.

Images