Data security situation awareness system based on multi-level data intelligent fusion

Abstract

The present invention relates to the technical field of data security situation analysis, and in particular to a data security situation awareness system based on multi-level data intelligent fusion, comprising a multi-source data center, a data preprocessing module, a processing quality unit, a demand analysis unit, a cross-domain feature fusion module, a dynamic risk perception unit, and a processing response unit. The present invention utilizes a deep learning model to automatically extract and fuse data features at different levels, effectively capturing the complex correlations between data and improving the discriminative power of feature representation. At the same time, by integrating code semantic features, traffic spatial features, traffic temporal features, log features, and user operation feature vectors, the limitations of a single data dimension are overcome, and security risk levels are divided according to the Hamming distance D. Furthermore, through intuitive visual display and an automated response mechanism, the workload of security operation and maintenance personnel is reduced, and the accuracy and real-time performance of security threat identification are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data security situation analysis, and in particular to a data security situation awareness system based on multi-level data intelligent fusion. Background Art

[0002] With the rapid development of Internet technology, network security threats are becoming increasingly complex and diverse, and data security faces unprecedented challenges. Traditional data security situation awareness methods are mostly based on single-dimensional data feature analysis, which makes it difficult to comprehensively and accurately capture potential risks in the network environment.

[0003] However, existing solutions that rely solely on semantic clustering analysis of malware code suffer from incomplete feature representation and large cross-domain mapping errors when faced with the need to integrate multi-source heterogeneous data. This leads to insufficient recognition accuracy for complex attack patterns and makes it difficult to monitor and analyze the data processing process and quality. This makes it difficult for the processed data to meet the needs of subsequent analysis, resulting in large deviations in analysis results and affecting the accuracy of security situation awareness.

[0004] In view of the above technical defects, a solution is now proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide a data security situation awareness system based on multi-level data intelligent fusion to solve the technical defects mentioned above. The present invention initially collects and stores multi-source security data from various data sources, and uses a deep learning model to automatically extract and fuse data features at different levels, effectively capturing the complex correlation between data, and improving the robustness and discriminability of feature representation. At the same time, it further analyzes from the two points of data processing and processing quality to ensure that the multi-level data after data processing meets the subsequent fusion and situation awareness requirements. At the same time, by integrating code semantic features, traffic spatial features, traffic temporal features, log features and user operation feature vectors, a more comprehensive security situation feature representation is constructed, overcoming the limitations of a single data dimension, and dividing security risk levels according to the Hamming distance D. Through intuitive visual display and automated response mechanism, the workload of security operation and maintenance personnel is reduced, the efficiency of handling network security incidents is improved, and efficient matching of multi-source security data with known malicious features is achieved, thereby improving the accuracy and real-time performance of security threat identification.

[0006] The object of the present invention can be achieved by the following technical solutions: a data security situation awareness system based on multi-level data intelligent fusion, comprising a multi-source data center, a data preprocessing module, a processing quality unit, a demand analysis unit, a cross-domain feature fusion module, a dynamic risk perception unit, and a processing response unit;

[0007] The multi-source data center is used to collect and store multi-source security data from network devices, security devices, and host devices;

[0008] The data preprocessing module retrieves multi-source security data from multi-source data centers and performs data processing and analysis on the multi-source security data to obtain code semantic features, traffic spatial features, traffic temporal features, log features, and user operation feature vectors;

[0009] The processing quality unit is used to track and evaluate the processing of multi-source safety data to obtain a processing stability signal or a quality risk signal. When a processing stability signal is generated, the demand analysis unit is used to perform a post-data cleaning demand analysis on the multi-source safety data to obtain a qualified signal or a failed signal.

[0010] The cross-domain feature fusion module is used to perform feature fusion processing and analysis on code semantic features, traffic spatial features, traffic temporal features, log features and user operation feature vectors to generate a fused feature vector F. The dynamic risk perception unit is used to perform risk perception analysis on the fused feature vector F to obtain low risk, medium risk, high risk or normal status.

[0011] Preferably, the data processing and analysis process is as follows:

[0012] Retrieve software code from multi-source security data, perform word segmentation on the software code, generate a pure word sequence, train the code vocabulary based on the word sequence, generate word embedding vectors, and then obtain the word embedding vector sequence. Input the word embedding vector sequence to generate the code semantic feature vector sequence;

[0013] Parse network traffic packets from multi-source security data, extract statistical and temporal features, and further use convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to extract 64-dimensional spatial features and 64-dimensional temporal features from the statistical and temporal features of network traffic, respectively.

[0014] Extract key information from system logs in multi-source security data, construct log event sequences, process the log event sequences, and output a 64-dimensional log feature vector;

[0015] Preprocessing is performed on user operation data in multi-source security data, operation features are extracted, and the operation features are standardized to generate user operation feature vectors.

[0016] Preferably, the data processing tracking and processing evaluation analysis process is as follows:

[0017] Collect the processing period of multi-source security data, obtain the single data processing time and the amount of data that can be processed per unit time of the data processing equipment during the processing period, and set the ratio between the amount of data that can be processed per unit time and the single data processing time after data normalization as the performance efficiency index;

[0018] The maximum memory usage and average CPU utilization of the data processing equipment during the processing period are obtained, and the product value obtained by multiplying the maximum memory usage and the average CPU utilization after data normalization is set as the resource occupancy. The performance efficiency index and resource occupancy are compared and analyzed with the preset performance efficiency index threshold and preset resource occupancy threshold stored internally to obtain a processing stability signal or a quality risk signal.

[0019] Preferably, the demand analysis process after data cleaning is as follows:

[0020] Obtain the missing value ratio of each data source after multi-source security data cleaning, perform discrimination processing on the missing value ratio, and set the number of data sources corresponding to the missing value ratio greater than or equal to the preset missing value ratio threshold as abnormal data source values;

[0021] The corresponding proportion of the total amount of data repaired or filtered after multi-source security data cleaning is obtained and set as the exception processing probability. The abnormal data source value and the exception processing probability are discriminated and processed to obtain a qualified signal or an unqualified signal.

[0022] Preferably, the feature fusion processing and analysis process is as follows:

[0023] A gated fusion network is designed to integrate code semantic features, traffic spatial features, traffic temporal features, log features, and user operation feature vectors to generate a fused feature vector F.

[0024] Preferably, the risk perception analysis process is as follows:

[0025] The fusion feature vector F and the feature vectors in the pre-built malware feature library are mapped to a unified hash space through a hash function to generate hash feature vectors HF and HM;

[0026] Calculate the Hamming distance D(HF, HM) between the hash feature vectors HF and HM.

[0027] Preferably, the Hamming distance D(HF, HM) is compared and analyzed with a preset risk assessment threshold T:

[0028] If the Hamming distance D(HF, HM) ≤ the preset risk assessment threshold T, it is determined that there is a security risk and a risk signal is generated. When the risk signal is generated, the risk level classification thresholds T1 and T2 are obtained. T>T1>T2. If T1<D(HF, HM) ≤ the preset risk assessment threshold T, it is determined to be low risk. If T2<D(HF, HM)<T1, it is determined to be medium risk. If D(HF, HM)<T2, it is determined to be high risk.

[0029] If the Hamming distance D(HF, HM)>T, it is determined to be a normal state.

[0030] The beneficial effects of the present invention are as follows:

[0031] This invention initially collects and stores multi-source security data from various data sources, and uses deep learning models to automatically extract and fuse data features at different levels, effectively capturing the complex relationships between data and improving the robustness and discriminative power of feature representation. Furthermore, it further analyzes data processing and processing quality to ensure that the processed multi-level data meets the needs of subsequent fusion and situational awareness.

[0032] The present invention integrates code semantic features, traffic spatial features, traffic temporal features, log features and user operation feature vectors to construct a more comprehensive security situation feature representation, overcomes the limitations of a single data dimension, and divides security risk levels according to the Hamming distance D. Through intuitive visual display and automated response mechanism, the workload of security operation and maintenance personnel is reduced, the efficiency of handling network security incidents is improved, and at the same time, efficient matching of multi-source security data with known malicious features is achieved, thereby improving the accuracy and real-time performance of security threat identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below with reference to the accompanying drawings;

[0034] Figure 1 It is a flow chart of the system of the present invention;

[0035] Figure 2 It is a reference diagram for local analysis of the present invention. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0037] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments;

[0038] Example 1: Please refer to Figures 1 to 2 As shown, the present invention is a data security situation awareness system based on multi-level data intelligent fusion, including a multi-source data center, a data preprocessing module, a processing quality unit, a demand analysis unit, a cross-domain feature fusion module, a dynamic risk perception unit and a processing response unit. The multi-source data center is connected to the processing quality unit and the demand analysis unit in a one-way communication manner, the processing quality unit and the demand analysis unit are connected to the processing response unit in a one-way communication manner, the multi-source data center is connected to the data preprocessing module in a two-way communication manner, the data preprocessing module is connected to the cross-domain feature fusion module in a one-way communication manner, the cross-domain feature fusion module is connected to the dynamic risk perception unit in a one-way communication manner, and the dynamic risk perception unit is connected to the processing response unit in a one-way communication manner;

[0039] The multi-source data center is used to collect and store multi-source security data from multiple data sources such as network devices, security devices, and host devices. The multi-source security data includes software code, network traffic data, system log data, and user operation data.

[0040] Network traffic data includes source IP, destination IP, etc.

[0041] System log data includes event type, occurrence time, etc.

[0042] User operation data includes access frequency, operation duration, interface click path, etc.

[0043] The data preprocessing module retrieves multi-source security data from the multi-source data center and performs data processing and analysis on the multi-source security data. The specific data processing and analysis process is as follows:

[0044] Retrieve software code from multi-source security data, perform word segmentation on the software code, and remove comments and symbols. This involves matching and removing irrelevant characters using regular expressions or lexical analyzers to generate a clean word sequence. The code vocabulary is trained based on the word sequence to generate word embedding vectors, which are then used to obtain a sequence of word embedding vectors. An encoder (such as a Transformer encoder) is used to build a global contextual semantic encoding model. This input takes the word embedding vector sequence and generates a sequence of code semantic feature vectors.

[0045] Parse network traffic packets from multi-source security data, extract statistical features such as source IP, destination IP, port number, protocol type, and traffic rate, as well as temporal features such as session connection duration and packet size. These statistical and temporal features are then normalized to a mean of 0 and a variance of 1. Furthermore, a convolutional neural network (CNN) and a recurrent neural network (RNN) are used to extract 64-dimensional spatial features and 64-dimensional temporal features from the statistical and temporal features of network traffic, respectively.

[0046] Performs structured parsing of system logs from multi-source security data, extracting key information such as event type, occurrence time, operator, and object, constructing and processing log event sequences (e.g., using a long short-term memory (LSTM) network), and outputting a 64-dimensional log feature vector.

[0047] Preprocess the user operation data in the multi-source security data. The preprocessing includes cleaning and desensitization, extracting operation features such as operation frequency and time interval, and standardizing the operation features to generate user operation feature vectors.

[0048] Example 2: The processing quality unit is used to perform data processing tracking and processing evaluation analysis on multi-source security data, and to perform discrimination processing on the obtained performance efficiency index and resource utilization to obtain a processing stability signal or a quality risk signal. The specific data processing tracking and processing evaluation analysis process is as follows:

[0049] Collect the processing period of multi-source security data, obtain the single data processing time and the amount of data that can be processed per unit time of the data processing equipment during the processing period, and set the ratio between the amount of data that can be processed per unit time and the single data processing time after data normalization as the performance efficiency index;

[0050] Obtain the maximum memory usage and average CPU utilization of the data processing device during the processing period. The product of the normalized maximum memory usage and average CPU utilization is set as the resource utilization. It should be noted that the performance efficiency index and resource utilization are two parameters that reflect the risk of data processing quality.

[0051] The performance efficiency index and resource occupancy are compared and analyzed with the preset performance efficiency index threshold and the preset resource occupancy threshold stored internally: if the performance efficiency index is greater than or equal to the preset performance efficiency index threshold, and the resource occupancy is less than the preset resource occupancy threshold, a processing stability signal is generated; if the performance efficiency index is less than the preset performance efficiency index threshold, or the resource occupancy is greater than or equal to the preset resource occupancy threshold, a quality risk signal is generated, and the processing response unit is used to respond to the processing stability signal or the quality risk signal, and immediately display the preset warning text corresponding to the processing stability signal or the quality risk signal, so as to timely understand whether the quality risk of multi-source security data processing is too high, and then rationally process the multi-source security data to reduce the impact of the multi-source security data processing quality on subsequent analysis;

[0052] When a stable processing signal is generated, the demand analysis unit is used to perform post-data cleaning demand analysis on the multi-source security data, discriminate the obtained abnormal data source values ​​and abnormal processing probabilities, and obtain qualified or unqualified signals. The specific post-data cleaning demand analysis process is as follows:

[0053] Obtain the missing value ratio of each data source after multi-source security data cleaning, perform discrimination processing on the missing value ratio, and set the number of data sources corresponding to the missing value ratio greater than or equal to the preset missing value ratio threshold as abnormal data source values;

[0054] Obtain the percentage of the total amount of data repaired or filtered after multi-source security data cleaning, and set it as the exception processing probability;

[0055] The abnormal data source value and the abnormal processing probability are discriminated and processed. If the abnormal data source value is equal to zero and the abnormal processing probability is less than the preset abnormal processing probability threshold, a qualified signal is generated. If the abnormal data source value is not equal to zero, or the abnormal processing probability is greater than or equal to the preset abnormal processing probability threshold, an unqualified signal is generated. The processing response unit is used to respond to the qualified signal or the unqualified signal, and immediately display the preset warning text corresponding to the qualified signal or the unqualified signal, so as to further conduct risk supervision on the processing of multi-source security data to ensure that the multi-level data after data processing meets the subsequent fusion and situational awareness requirements.

[0056] Example 3: The cross-domain feature fusion module is used to perform feature fusion processing and analysis on code semantic features, traffic spatial features, traffic temporal features, log features, and user operation feature vectors to generate a fused feature vector F. The specific feature fusion processing and analysis process is as follows:

[0057] Design a gated fusion network that automatically learns the importance weights of features at different levels through a gating mechanism, achieving adaptive fusion of code semantic features, traffic spatial features, traffic temporal features, log features, and user operation feature vectors to generate a fused feature vector F.

[0058] Among them, by integrating code semantic features, traffic spatial features, traffic temporal features, log features, and user operation feature vectors, a more comprehensive security situation feature representation is constructed, overcoming the limitations of a single data dimension;

[0059] The gated fusion network is a deep learning architecture whose core concept is to dynamically filter and fuse multi-source input information through a gating mechanism, thereby improving the model's representational capabilities for complex tasks. It is widely used in fields such as computer vision, natural language processing, and multimodal learning, and is particularly suitable for scenarios that require integrating information from different modalities or levels (such as image and text fusion, or high- and low-level feature fusion).

[0060] The dynamic risk perception unit is used to perform risk perception analysis on the fused feature vector F to obtain low risk, medium risk, high risk, or normal status. The specific risk perception analysis process is as follows:

[0061] The fusion feature vector F and the feature vectors in the pre-built malware feature library are mapped to a unified hash space through a hash function to generate hash feature vectors HF and HM;

[0062] Calculate the Hamming distance D(HF, HM) between the hash feature vectors HF and HM to measure the similarity between the data to be detected and the known malicious features;

[0063] Compare and analyze the Hamming distance D (HF, HM) with the preset risk assessment threshold T:

[0064] If the Hamming distance D(HF, HM) ≤ the preset risk assessment threshold T, it is determined that there is a security risk and a risk signal is generated. When the risk signal is generated, the risk level classification thresholds T1 and T2 are obtained. T>T1>T2. If T1<D(HF, HM) ≤ the preset risk assessment threshold T, it is determined to be low risk. If T2<D(HF, HM)<T1, it is determined to be medium risk. If D(HF, HM)<T2, it is determined to be high risk.

[0065] If the Hamming distance D(HF, HM)>T, it is determined to be in a normal state. The processing response unit is used for low risk, medium risk, high risk, or normal state, and the preset warning text corresponding to the low risk, medium risk, high risk, or normal state is immediately displayed. That is, the security risk level is divided according to the Hamming distance D. Through intuitive visual display and automated response mechanism, the workload of security operation and maintenance personnel is reduced, the efficiency of handling network security incidents is improved, and at the same time, efficient matching of multi-source security data with known malicious features is achieved, improving the accuracy and real-time performance of security threat identification;

[0066] In summary, the present invention initially collects and stores multi-source security data from various data sources, and uses a deep learning model to automatically extract and fuse data features at different levels, effectively capturing the complex correlation between data, and improving the robustness and discriminability of feature representation. At the same time, it further analyzes from the two points of data processing and processing quality to ensure that the multi-level data after data processing meets the subsequent fusion and situational awareness requirements. At the same time, by integrating code semantic features, traffic space features, traffic time series features, log features and user operation feature vectors, a more comprehensive security situation feature representation is constructed, overcoming the limitations of a single data dimension, and dividing security risk levels according to the Hamming distance D. Through intuitive visual display and automated response mechanism, the workload of security operation and maintenance personnel is reduced, and the efficiency of handling network security incidents is improved. At the same time, efficient matching of multi-source security data with known malicious features is achieved, and the accuracy and real-time performance of security threat identification are improved.

[0067] The threshold is set for result comparison and analysis in order to determine whether it is good or bad. The value of the threshold is set based on a combination of large-scale model analysis of sample data and manual experience to enter and store it. It can also be appropriately adjusted based on seasonal or common sense influencing conditions.

[0068] The size of the coefficient is to quantify each parameter to obtain a specific numerical value, which is convenient for subsequent comparison. The size of the coefficient depends on the amount of sample data and the preliminary setting of the corresponding operating coefficient for each set of sample data by technical personnel in this field; as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0069] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. Data security situation awareness system based on multi-level data intelligent fusion, characterized by: It includes a multi-source data center, a data pre-processing module, a processing quality unit, a demand analysis unit, a cross-domain feature fusion module, a dynamic risk perception unit, and a processing response unit; The multi-source data center is used to collect and store multi-source security data from network devices, security devices, and host devices; The data preprocessing module retrieves multi-source security data from multi-source data centers and performs data processing and analysis on the multi-source security data to obtain code semantic features, traffic spatial features, traffic temporal features, log features, and user operation feature vectors; The processing quality unit is used to track and evaluate the processing of multi-source safety data to obtain a processing stability signal or a quality risk signal. When a processing stability signal is generated, the demand analysis unit is used to perform a post-data cleaning demand analysis on the multi-source safety data to obtain a qualified signal or a failed signal. The cross-domain feature fusion module is used to perform feature fusion processing and analysis on code semantic features, traffic spatial features, traffic temporal features, log features, and user operation feature vectors to generate a fused feature vector F. The dynamic risk perception unit is used to perform risk perception analysis on the fused feature vector F to determine low risk, medium risk, high risk, or normal status. The data processing tracking and processing evaluation analysis process is as follows: Collect the processing period of multi-source security data, obtain the single data processing time and the amount of data that can be processed per unit time of the data processing equipment during the processing period, and set the ratio between the amount of data that can be processed per unit time and the single data processing time after data normalization as the performance efficiency index; The maximum memory usage and average CPU utilization of the data processing equipment during the processing period are obtained, and the product value obtained by multiplying the maximum memory usage and the average CPU utilization after data normalization is set as the resource occupancy. The performance efficiency index and resource occupancy are compared and analyzed with the preset performance efficiency index threshold and preset resource occupancy threshold stored internally to obtain a processing stability signal or a quality risk signal.

2. The data security situation awareness system based on multi-level data intelligent fusion according to claim 1 is characterized in that: The data processing and analysis process is as follows: Retrieve software code from multi-source security data, perform word segmentation on the software code, generate a pure word sequence, train the code vocabulary based on the word sequence, generate word embedding vectors, and then obtain the word embedding vector sequence. Input the word embedding vector sequence to generate the code semantic feature vector sequence; Parse network traffic packets from multi-source security data, extract statistical and temporal features, and further use convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to extract 64-dimensional spatial features and 64-dimensional temporal features from the statistical and temporal features of network traffic, respectively. Extract key information from system logs in multi-source security data, construct log event sequences, process the log event sequences, and output a 64-dimensional log feature vector; Preprocessing is performed on user operation data in multi-source security data, operation features are extracted, and the operation features are standardized to generate user operation feature vectors.

3. The data security situation awareness system based on multi-level data intelligent fusion according to claim 1 is characterized in that: The process of demand analysis after data cleaning is as follows: Obtain the missing value ratio of each data source after multi-source security data cleaning, perform discrimination processing on the missing value ratio, and set the number of data sources corresponding to the missing value ratio greater than or equal to the preset missing value ratio threshold as abnormal data source values; The corresponding proportion of the total amount of data repaired or filtered after multi-source security data cleaning is obtained and set as the exception processing probability. The abnormal data source value and the exception processing probability are discriminated and processed to obtain a qualified signal or an unqualified signal.

4. The data security situation awareness system based on multi-level data intelligent fusion according to claim 1 is characterized in that: The feature fusion processing and analysis process is as follows: A gated fusion network is designed to integrate code semantic features, traffic spatial features, traffic temporal features, log features, and user operation feature vectors to generate a fused feature vector F.

5. The data security situation awareness system based on multi-level data intelligent fusion according to claim 1 is characterized in that: The risk perception analysis process is as follows: The fusion feature vector F and the feature vectors in the pre-built malware feature library are mapped to a unified hash space through a hash function to generate hash feature vectors HF and HM; Calculate the Hamming distance D(HF, HM) between the hash feature vectors HF and HM.

6. The data security situation awareness system based on multi-level data intelligent fusion according to claim 5 is characterized in that: Compare and analyze the Hamming distance D (HF, HM) with the preset risk assessment threshold T: If the Hamming distance D(HF, HM) ≤ the preset risk assessment threshold T, it is determined that there is a security risk and a risk signal is generated. When the risk signal is generated, the risk level classification thresholds T1 and T2 are obtained. T>T1>T2. If T1<D(HF, HM) ≤ the preset risk assessment threshold T, it is determined to be low risk. If T2<D(HF, HM)<T1, it is determined to be medium risk. If D(HF, HM)<T2, it is determined to be high risk. If the Hamming distance D(HF, HM)>T, it is determined to be a normal state.

Images