Data security situation awareness system based on multilevel data intelligent fusion

Abstract

The invention relates to the technical field of data security situation analysis, in particular to a data security situation awareness system based on multi-level data intelligent fusion, which comprises a multi-source data center, a data preprocessing module, a processing quality unit, a demand analysis unit, a cross-domain feature fusion module, a dynamic risk awareness unit and a processing response unit. According to the method, the deep learning model is used for automatically extracting and fusing different levels of data features, the complex incidence relation between the data is effectively captured, the discrimination ability of feature representation is improved, and meanwhile, the code semantic feature, the flow space feature, the flow time sequence feature, the log feature and the user operation feature vector are integrated, so that the user experience is improved. According to the method, the limitation of a single data dimension is overcome, the security risk level is divided according to the Hamming distance D, and through visual display and an automatic response mechanism, the workload of security operation and maintenance personnel is reduced, and the accuracy and the real-time performance of security threat identification are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data security situation analysis, and particularly to a data security situation awareness system based on multi-level data intelligent fusion. Background Art

[0002] With the rapid development of Internet technology, network security threats have become increasingly complex and diverse, and data security is facing unprecedented challenges. Most traditional data security situation awareness methods are based on single-dimensional data feature analysis, making it difficult to comprehensively and accurately capture potential risks in the network environment; However, in the existing technology, the scheme that simply relies on the semantic clustering analysis of malware codes has problems such as incomplete feature representation and large cross-domain mapping errors when facing the need for multi-source heterogeneous data fusion, resulting in insufficient recognition accuracy for complex attack patterns. At the same time, it is difficult to conduct regulatory analysis on the data processing process and data processing quality, resulting in the processed data being difficult to meet the subsequent analysis requirements, leading to too large deviations in the analysis results and affecting the accuracy of security situation awareness; In view of the above technical defects, a solution is now proposed. Summary of the Invention

[0003] The purpose of the present invention is to provide a data security situation awareness system based on multi-level data intelligent fusion to solve the above-mentioned technical defects. The present invention initially collects multi-source security data from each data source and stores it, and uses a deep learning model to automatically extract and fuse data features at different levels, effectively capturing the complex correlation relationships between data, improving the robustness and discriminability of feature representation. At the same time, further analysis is carried out from two aspects of data processing and processing quality to ensure that the multi-level data after data processing meets the subsequent fusion and situation awareness requirements. At the same time, by integrating code semantic features, traffic space features, traffic time series features, log features, and user operation feature vectors, a more comprehensive security situation feature representation is constructed, overcoming the limitations of a single data dimension, and dividing the security risk level according to the Hamming distance D. And through intuitive visual display and automated response mechanisms, the workload of security operation and maintenance personnel is reduced, the disposal efficiency of network security incidents is improved, and at the same time, efficient matching of multi-source security data and known malicious features is achieved, improving the accuracy and real-time performance of security threat recognition.

[0004] The purpose of the present invention can be achieved by the following technical solutions: A data security situation awareness system based on multi-level data intelligent fusion, including a multi-source data center, a data preprocessing module, a processing quality unit, a requirement analysis unit, a cross-domain feature fusion module, a dynamic perception risk unit, and a processing response unit; The multi-source data center is used to collect multi-source security data from network devices, security devices, and host devices and store it; The data preprocessing module retrieves multi-source security data from a multi-source data center, and performs data processing and analysis on the multi-source security data to obtain code semantic features, traffic space features, traffic time-series features, log features, and user operation feature vectors; The processing quality unit is used to perform data processing tracking and processing evaluation analysis on the multi-source security data to obtain a processing stability signal or a quality risk signal. When a processing stability signal is generated, the requirement analysis unit is used to perform requirement analysis after data cleaning on the multi-source security data to obtain a qualified signal or an unqualified signal; The cross-domain feature fusion module is used to perform feature fusion processing and analysis on the code semantic features, traffic space features, traffic time-series features, log features, and user operation feature vectors to generate a fused feature vector F. The dynamic risk perception unit is used to perform risk perception analysis on the fused feature vector F to obtain a low-risk or medium-risk or high-risk or normal state.

[0005] Preferably, the data processing and analysis process is as follows: Retrieve the software code in the multi-source security data, perform word segmentation processing on the software code to generate a pure word sequence, train the code vocabulary based on the word sequence to generate word embedding vectors, and then obtain a word embedding vector sequence. Input the word embedding vector sequence to generate a code semantic feature vector sequence; Parse the network traffic data packets in the multi-source security data, extract statistical features and time-series features, and further use a convolutional neural network (CNN) and a recurrent neural network (RNN) to extract 64-dimensional space features and 64-dimensional time-series features from the statistical features and time-series features of the network traffic respectively; Extract key information from the system logs in the multi-source security data, construct a log event sequence, and process the log event sequence to output a 64-dimensional log feature vector; Preprocess the user operation data in the multi-source security data, extract operation features, and perform standardization processing on the operation features to generate user operation feature vectors.

[0006] Preferably, the data processing tracking and processing evaluation analysis process is as follows: Collect the processing period of the multi-source security data, obtain the single data processing time of the data processing device within the processing period and the amount of data that can be processed per unit time, and set the ratio between the amount of data that can be processed per unit time and the value after data normalization processing of the single data processing time as the performance efficiency index; Obtain the maximum occupancy of the memory usage and the average CPU utilization rate of the data processing device during the processing period. Multiply the values obtained after normalizing the maximum occupancy of the memory usage and the average CPU utilization rate to set the product value as the resource occupancy. Compare and analyze the performance efficiency index and the resource occupancy with the preset performance efficiency index threshold and the preset resource occupancy threshold stored internally to obtain a processing stability signal or a quality risk signal.

[0007] Preferably, the process of requirement analysis after data cleaning is as follows: Obtain the ratio of the missing value of each data source after multi-source security data cleaning, and perform discrimination processing on the ratio of the missing value. Set the number of data sources corresponding to the ratio of the missing value greater than or equal to the preset ratio threshold of the missing value as the abnormal data source value; Obtain the ratio corresponding to the total amount of repaired or filtered data after multi-source security data cleaning, and set it as the abnormal processing probability. Perform discrimination processing on the abnormal data source value and the abnormal processing probability to obtain a qualified signal or an unqualified signal.

[0008] Preferably, the process of feature fusion processing and analysis is as follows: Design a gated fusion network to achieve the fusion of code semantic features, traffic space features, traffic time series features, log features, and user operation feature vectors, and generate a fused feature vector F.

[0009] Preferably, the process of risk perception analysis is as follows: Map the fused feature vector F and the feature vectors in the pre-constructed malware feature library to a unified hash space through a hash function to generate hash feature vectors HF and HM; Calculate the Hamming distance D(HF, HM) between the hash feature vectors HF and HM.

[0010] Preferably, compare and analyze the Hamming distance D(HF, HM) with the preset risk assessment threshold T: If the Hamming distance D(HF, HM) ≤ the preset risk assessment threshold T, it is determined that there is a security risk and a risk signal is generated. When a risk signal is generated, obtain the risk level division thresholds T1 and T2, T > T1 > T2. If T1 < D(HF, HM) ≤ the preset risk assessment threshold T, it is determined as a low risk. If T2 < D(HF, HM) < T1, it is determined as a medium risk. If D(HF, HM) < T2, it is determined as a high risk; If the Hamming distance D(HF, HM) > T, it is determined as a normal state.

[0011] The beneficial effects of the present invention are as follows: The present invention initially collects multi-source security data from various data sources and stores it. By using a deep learning model, it automatically extracts and fuses data features at different levels, effectively capturing the complex correlation relationships between data, enhancing the robustness and discriminative power of feature representation. At the same time, it further analyzes from two aspects of data processing and processing quality to ensure that the multi-level data after processing meets the subsequent fusion and situation awareness requirements; The present invention constructs a more comprehensive security situation feature representation by integrating code semantic features, traffic space features, traffic time series features, log features, and user operation feature vectors, overcomes the limitations of a single data dimension, divides the security risk level according to the Hamming distance D, and through intuitive visual display and an automated response mechanism, reduces the workload of security operation and maintenance personnel, improves the handling efficiency of network security incidents, and at the same time realizes the efficient matching of multi-source security data and known malicious features, improving the accuracy and real-time performance of security threat identification. Brief Description of the Drawings

[0012] The present invention will be further described below with reference to the accompanying drawings; Figure 1 is a system flow block diagram of the present invention; Figure 2 is a local analysis reference diagram of the present invention. Detailed Embodiments

[0013] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0014] Referring to "embodiments" herein means that the specific features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of the present invention. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments; Embodiment 1: Please refer to Figures 1 to 2As shown in the figure, the present invention is a data security situation awareness system based on multi-level data intelligent fusion, including a multi-source data center, a data preprocessing module, a processing quality unit, a requirements analysis unit, a cross-domain feature fusion module, a dynamic perception risk unit, and a processing response unit. The multi-source data center is unidirectionally communicatively connected to both the processing quality unit and the requirements analysis unit. The processing quality unit and the requirements analysis unit are both unidirectionally communicatively connected to the processing response unit. The multi-source data center is bidirectionally communicatively connected to the data preprocessing module. The data preprocessing module is unidirectionally communicatively connected to the cross-domain feature fusion module. The cross-domain feature fusion module is unidirectionally communicatively connected to the dynamic perception risk unit. The dynamic perception risk unit is unidirectionally communicatively connected to the processing response unit; The multi-source data center is used to collect and store multi-source security data from multiple data sources such as network devices, security devices, and host devices. The multi-source security data includes software code, network traffic data, system log data, and user operation data; The network traffic data includes source IP, destination IP, etc.; The system log data includes event type, occurrence time, etc.; The user operation data includes access frequency, operation duration, interface click path, etc.; The data preprocessing module retrieves the multi-source security data from the multi-source data center and conducts data processing and analysis on the multi-source security data. The specific data processing and analysis process is as follows: Retrieve the software code in the multi-source security data, perform word segmentation on the software code, remove comments and symbols, that is, match and remove irrelevant characters through regular expressions or lexical analyzers, generate a pure word sequence, train the code vocabulary based on the word sequence to generate word embedding vectors, and then obtain a word embedding vector sequence. Use an encoder (such as a Transformer encoder) to construct a global context semantic encoding model, input the word embedding vector sequence, and generate a code semantic feature vector sequence; Parse the network traffic data packets in the multi-source security data, extract statistical features such as source IP, destination IP, port number, protocol type, traffic rate, etc., and time series features such as session connection duration, packet size, etc. Standardize the statistical features and time series features so that their mean is 0 and variance is 1. Further, use a convolutional neural network (CNN) and a recurrent neural network (RNN) to extract 64-dimensional spatial features and 64-dimensional time series features from the statistical features and time series features of the network traffic respectively; Structurally parse the system logs in the multi-source security data, extract key information such as event type, occurrence time, operation subject, operation object, etc., construct a log event sequence, process the log event sequence (such as a long short-term memory network (LSTM)), and output a 64-dimensional log feature vector; Preprocess the user operation data in the multi-source security data. The preprocessing includes cleaning, desensitization, etc. Extract operation features such as operation frequency and time sequence interval, and perform standardization processing on the operation features to generate user operation feature vectors.

[0015] Embodiment 2: The processing quality unit is used to perform data processing tracking and processing evaluation analysis on multi-source security data, and perform discriminant processing on the obtained performance efficiency index and resource occupancy to obtain a processing stability signal or a quality risk signal. The specific data processing tracking and processing evaluation analysis process is as follows: Collect the processing period of the multi-source security data, obtain the single data processing time of the data processing device within the processing period and the amount of data that can be processed per unit time, and set the ratio between the amount of data that can be processed per unit time and the value after data normalization processing of the single data processing time as the performance efficiency index; Obtain the maximum occupancy of the memory used by the data processing device and the average CPU utilization rate within the processing period, and set the product value obtained by multiplying the maximum occupancy of the memory used and the average CPU utilization rate after data normalization processing as the resource occupancy. It should be noted that the performance efficiency index and the resource occupancy are two influence parameters reflecting the quality risk of data processing; Compare and analyze the performance efficiency index and the resource occupancy with the preset performance efficiency index threshold and the preset resource occupancy threshold stored internally: If the performance efficiency index is greater than or equal to the preset performance efficiency index threshold and the resource occupancy is less than the preset resource occupancy threshold, a processing stability signal is generated. If the performance efficiency index is less than the preset performance efficiency index threshold or the resource occupancy is greater than or equal to the preset resource occupancy threshold, a quality risk signal is generated. The processing response unit is used to respond to the processing stability signal or the quality risk signal and immediately display the preset warning text corresponding to the processing stability signal or the quality risk signal, so as to timely understand whether the quality risk of multi-source security data processing is too high, and then perform reasonable processing on the multi-source security data to reduce the impact of the multi-source security data processing quality on subsequent analysis; When a processing stability signal is generated, the requirement analysis unit is used to perform requirement analysis after data cleaning on the multi-source security data, and perform discriminant processing on the obtained abnormal data source value and abnormal processing probability to obtain a qualified signal or an unqualified signal. The specific requirement analysis process after data cleaning is as follows: Obtain the ratio of the missing value of each data source after cleaning the multi-source security data, and perform discriminant processing on the ratio of the missing value. Set the number of data sources corresponding to the ratio of the missing value greater than or equal to the preset ratio of the missing value threshold as the abnormal data source value; Obtain the ratio corresponding to the total amount of data repaired or filtered after cleaning the multi-source security data, and set it as the abnormal processing probability; Discriminate and process the abnormal data source value and the abnormal processing probability. If the abnormal data source value is equal to zero and the abnormal processing probability is less than the preset abnormal processing probability threshold, a qualified signal is generated. If the abnormal data source value is not equal to zero, or the abnormal processing probability is greater than or equal to the preset abnormal processing probability threshold, an unqualified signal is generated. The processing response unit is used to respond to the qualified signal or the unqualified signal and immediately display the preset warning text corresponding to the qualified signal or the unqualified signal, so as to further conduct risk supervision on the processing of multi-source security data to ensure that the multi-level data after data processing meets the subsequent fusion and situation awareness requirements.

[0016] Embodiment 3: The cross-domain feature fusion module is used to perform feature fusion processing and analysis on the code semantic feature, traffic space feature, traffic time series feature, log feature, and user operation feature vector to generate a fusion feature vector F. The specific feature fusion processing and analysis process is as follows: Design a gated fusion network to automatically learn the importance weights of different-level features through a gating mechanism, realize the adaptive fusion of the code semantic feature, traffic space feature, traffic time series feature, log feature, and user operation feature vector, and generate a fusion feature vector F; Among them, by integrating the code semantic feature, traffic space feature, traffic time series feature, log feature, and user operation feature vector, a more comprehensive security situation feature representation is constructed, overcoming the limitations of a single data dimension; The gated fusion network is a deep learning architecture. The core idea is to dynamically screen and fuse multi-source input information through a gating mechanism, thereby improving the representation ability of the model in complex tasks; it is widely used in fields such as computer vision, natural language processing, and multi-modal learning, especially suitable for scenarios that require integrating different modalities or different-level information (such as image and text fusion, high-level and low-level feature fusion); The dynamic risk perception unit is used to perform risk perception analysis on the fusion feature vector F to obtain a low-risk or medium-risk or high-risk or normal state. The specific risk perception analysis process is as follows: Map the fusion feature vector F and the feature vector in the pre-constructed malware feature library to a unified hash space through a hash function to generate hash feature vectors HF and HM; Calculate the Hamming distance D(HF, HM) between the hash feature vectors HF and HM to measure the similarity between the data to be detected and the known malicious features; Compare and analyze the Hamming distance D(HF, HM) with the preset risk assessment threshold T: If the Hamming distance D(HF, HM) ≤ the preset risk assessment threshold T, it is determined that there is a security risk and a risk signal is generated. When the risk signal is generated, the risk level division thresholds T1 and T2 are obtained, where T > T1 > T2. If T1 < D(HF, HM) ≤ the preset risk assessment threshold T, it is determined as a low risk. If T2 < D(HF, HM) < T1, it is determined as a medium risk. If D(HF, HM) < T2, it is determined as a high risk; If the Hamming distance D(HF, HM) > T, it is determined as a normal state. The processing and response unit is used for low risk or medium risk or high risk or normal state, and immediately displays the preset warning text corresponding to the low risk or medium risk or high risk or normal state. That is, the security risk level is divided according to the Hamming distance D, and through intuitive visual display and automated response mechanism, the workload of security operation and maintenance personnel is reduced, the disposal efficiency of network security incidents is improved, and at the same time, the efficient matching of multi-source security data and known malicious features is realized, improving the accuracy and real-time of security threat identification; In summary, the present invention initially collects multi-source security data from various data sources and stores them. The deep learning model is used to automatically extract and fuse data features at different levels, effectively capturing the complex correlation relationships between data, improving the robustness and discriminability of feature representation. At the same time, further analysis is carried out from two points of data processing and processing quality to ensure that the multi-level data after data processing meets the subsequent fusion and situation awareness requirements. At the same time, by integrating code semantic features, traffic space features, traffic time series features, log features, and user operation feature vectors, a more comprehensive security situation feature representation is constructed, overcoming the limitations of a single data dimension. The security risk level is divided according to the Hamming distance D, and through intuitive visual display and automated response mechanism, the workload of security operation and maintenance personnel is reduced, the disposal efficiency of network security incidents is improved, and at the same time, the efficient matching of multi-source security data and known malicious features is realized, improving the accuracy and real-time of security threat identification.

[0017] The setting of the threshold is for result comparison and analysis to determine whether it is good or bad. Regarding the value of the threshold, it is set and stored according to the combined analysis of the large model of sample data and manual experience, and can also be appropriately adjusted according to seasonal or regular influencing conditions.

[0018] The size of the coefficient is to obtain a specific value by quantifying each parameter for subsequent comparison. Regarding the size of the coefficient, it depends on the amount of sample data and the corresponding operation coefficients initially set by those skilled in the art for each group of sample data; as long as the proportional relationship between the parameter and the quantified value is not affected.

[0019] The above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes should be covered within the protection scope of the present invention.

Claims

1. A data security situation awareness system based on multi-level data intelligent fusion, characterized in that, It includes a multi-source data center, a data preprocessing module, a processing quality unit, a requirements analysis unit, a cross-domain feature fusion module, a dynamic perception risk unit, and a processing response unit; The multi-source data center is used to collect and store multi-source security data from network devices, security devices, and host devices; The data preprocessing module retrieves the multi-source security data from the multi-source data center, and performs data processing and analysis on the multi-source security data to obtain code semantic features, traffic space features, traffic time series features, log features, and user operation feature vectors; The processing quality unit is used to perform data processing tracking and processing evaluation analysis on the multi-source security data to obtain a processing stability signal or a quality risk signal. When a processing stability signal is generated, the requirements analysis unit is used to perform requirements analysis after data cleaning on the multi-source security data to obtain a qualified signal or an unqualified signal; The cross-domain feature fusion module is used to perform feature fusion processing and analysis on the code semantic features, traffic space features, traffic time series features, log features, and user operation feature vectors to generate a fusion feature vector F. The dynamic perception risk unit is used to perform risk perception analysis on the fusion feature vector F to obtain a low risk or medium risk or high risk or normal state.

2. The data security situation awareness system based on multi-level data intelligent fusion according to claim 1, wherein The data processing and analysis process is as follows: Retrieve the software code in the multi-source security data, perform word segmentation processing on the software code to generate a pure word sequence, train the code vocabulary based on the word sequence to generate word embedding vectors, and then obtain a word embedding vector sequence. Input the word embedding vector sequence to generate a code semantic feature vector sequence; Parse the network traffic data packets in the multi-source security data, extract statistical features and time series features, and further use a convolutional neural network (CNN) and a recurrent neural network (RNN) to extract 64-dimensional space features and 64-dimensional time series features from the statistical features and time series features of the network traffic respectively; Extract key information from the system logs in the multi-source security data, construct a log event sequence, and process the log event sequence to output a 64-dimensional log feature vector; Preprocess the user operation data in the multi-source security data, extract operation features, and perform standardization processing on the operation features to generate a user operation feature vector.

3. The data security situation awareness system based on multi-level data intelligent fusion according to claim 1, characterized in that, The data processing tracking and processing evaluation analysis process is as follows: Collect the processing period of the multi-source security data, obtain the single data processing time of the data processing device within the processing period and the amount of data that can be processed per unit time, and set the ratio between the amount of data that can be processed per unit time and the value after data normalization processing of the single data processing time as the performance efficiency index; Obtain the maximum occupancy of the memory used by the data processing device and the average CPU utilization rate within the processing period, set the product value obtained by multiplying the maximum occupancy of the memory used and the average CPU utilization rate after data normalization processing as the resource occupancy. Compare and analyze the performance efficiency index and the resource occupancy with the preset performance efficiency index threshold and the preset resource occupancy threshold stored internally to obtain a processing stability signal or a quality risk signal.

4. The data security situation awareness system based on multi-level data intelligent fusion according to claim 1, characterized in that, The requirements analysis process after data cleaning is as follows: Obtain the ratio of the missing value of each data source after cleaning the multi-source security data, and perform discriminant processing on the ratio of the missing value. Set the number of data sources corresponding to the ratio of the missing value greater than or equal to the preset ratio threshold of the missing value as the abnormal data source value; Obtain the ratio corresponding to the total amount of repaired or filtered data after cleaning the multi-source security data, and set it as the abnormal processing probability. Perform discriminant processing on the abnormal data source value and the abnormal processing probability to obtain a qualified signal or an unqualified signal.

5. The data security situation awareness system based on multi-level data intelligent fusion according to claim 1, characterized in that, The feature fusion processing and analysis process is as follows: Design a gated fusion network to achieve the fusion of code semantic features, traffic space features, traffic time series features, log features, and user operation feature vectors, and generate a fused feature vector F.

6. The data security situation awareness system based on multi-level data intelligent fusion according to claim 1, characterized in that, The risk perception analysis process is as follows: Map the fused feature vector F and the feature vectors in the pre-constructed malware feature library to a unified hash space through a hash function to generate hash feature vectors HF and HM; Calculate the Hamming distance D(HF, HM) between the hash feature vectors HF and HM.

7. The data security situation awareness system based on multi-level data intelligent fusion according to claim 6, characterized in that, Compare and analyze the Hamming distance D(HF, HM) with the preset risk assessment threshold T: If the Hamming distance D(HF, HM) ≤ the preset risk assessment threshold T, it is determined that there is a security risk and a risk signal is generated. When a risk signal is generated, obtain the risk level division thresholds T1 and T2, T > T1 > T2. If T1 < D(HF, HM) ≤ the preset risk assessment threshold T, it is determined to be a low risk. If T2 < D(HF, HM) < T1, it is determined to be a medium risk. If D(HF, HM) < T2, it is determined to be a high risk; If the Hamming distance D(HF, HM) > T, it is determined to be in a normal state.

Images