Data security monitoring method and system

Abstract

The invention relates to the field of network anomaly detection, in particular to a data security monitoring method and system, and the method comprises the steps: carrying out the extraction and grouping of features of a network flow data set, and calculating feature factors, and obtaining a first feature group, a second feature group, a third feature group, and a corresponding first feature factor, second feature factor and third feature factor; training a frequency domain rule of the feature group through an Inception unit, and identifying a time domain rule of the feature group through an LSTM unit; and using the first feature factor, the second feature factor and the third feature factor to respectively carry out weighted fusion on the training results of the feature group to obtain fusion training results, and carrying out classified output on the fusion training results. The method comprises the following steps: extracting and grouping network flow data features, and calculating feature factors; importing the feature group into a network flow detection model for training, and performing judgment in combination with a feature factor; and accurate detection of network traffic abnormity is realized through multiple judgment results.

Description

Technical Field

[0001] This invention relates to the field of network traffic anomaly detection technology, specifically a data security monitoring method and system. Background Technology

[0002] With continuous development and improvement, network technology has greatly promoted social, economic, and technological progress, enriched people's lives, and brought about tremendous changes. However, the development of network technology has also brought security risks. Because traditional laws and regulations are ineffective in the virtual cyberspace, various illegal and criminal activities are rampant; cyberattacks are among the most common. In recent years, cyberattacks have become increasingly intelligent, frequent, and covert, posing a severe challenge to cyberspace. Without proper prevention, this can lead to the theft and modification of important data, the leakage of personal privacy, and economic losses.

[0003] Network traffic serves as the connecting link in cyberspace and is a primary indicator of network status. When a network is attacked or intruded upon, network traffic typically changes, thus necessitating effective management and monitoring methods. To ensure the security and stability of cyberspace and to promptly detect potential intrusion activities, it is essential to seek effective network data monitoring methods. Currently, network traffic anomaly monitoring based on machine learning technology is a major focus, which can be further divided into traditional machine learning algorithms and deep learning algorithms.

[0004] Traditional machine learning algorithms primarily classify and judge network traffic data through clustering, decision trees, Bayesian classification, and support vector machines. For example, patent CN202110912180.0 proposes an intelligent traffic monitoring method, system, and storage medium for the Industrial Internet, which uses an improved hierarchical clustering algorithm to monitor abnormal network traffic and applies it to the intelligentization of the Industrial Internet. Deep learning algorithms, on the other hand, increase the depth of learning and training to discover and detect features in network traffic data. For example, patent CN202110605897.0 describes a deep learning-based method and device for monitoring abnormal traffic in industrial control networks. This method uses historical network traffic data and dynamic thresholds to continuously update and iterate the LSTM prediction model, thereby detecting abnormal traffic in industrial control networks.

[0005] Existing methods have thoroughly considered the detection methods and model parameters when detecting network traffic data; however, they do not conduct in-depth feature selection and segmentation of network data, and the results of anomaly judgment are analyzed from a static perspective. The singularity of data features and the static nature of result analysis lead to low accuracy in network traffic anomaly monitoring and are prone to misjudgment.

[0006] Based on the above factors, the present invention provides a data security monitoring method and system. Summary of the Invention

[0007] The purpose of this invention is to provide a data security monitoring method and system; to collect and extract features from the network traffic data of the monitored object, and to calculate feature factors from the features; to input the data feature group and feature factors into the network traffic detection model constructed by this invention for anomaly judgment, and to calculate the traffic anomaly index based on the anomaly judgment result, thereby obtaining the network status of the monitored object.

[0008] To achieve the above objectives, the present invention provides the following technical solution, including:

[0009] Identify the monitoring target and collect network traffic data of the monitoring target; perform data preprocessing on the network traffic data to obtain the dataset to be detected;

[0010] The dataset to be detected is input into the network traffic detection model for training, and the classification result is output; the classification result is used to determine whether the network traffic data is abnormal; the dataset to be detected passes through the data input layer, feature extraction layer, recognition training layer and classification output layer of the network traffic detection model in sequence;

[0011] The feature extraction layer is used to extract and group features from the dataset to be detected, and to calculate feature factors; to obtain a first feature group, a second feature group, and a third feature group, as well as the corresponding first feature factor, second feature factor, and third feature factor;

[0012] The recognition training layer is used to train the three feature groups and output the training results; the recognition training layer includes a first training structure, a second training structure and a third training structure; the three training structures are respectively used to train the three feature groups; the training structure includes an Inception unit for recognizing the frequency domain pattern of the feature groups and an LSTM unit for recognizing the time domain pattern of the feature groups;

[0013] The classification output layer is used to perform fusion classification on the training results; the classification output layer includes a first feature factor, a second feature factor, and a third feature factor; the three feature factors are respectively weighted on the training results to obtain fusion training results; the fusion training results are classified to obtain classification results, and the classification results are output.

[0014] Data preprocessing includes attribute mapping, standardization, and normalization.

[0015] The attribute mapping process is used to process the non-numerical data in the network traffic data and output numerical network traffic data.

[0016] The standardization process is used to unify the dimensions of the numerical network traffic data and output standardized network traffic data.

[0017]

[0018]

[0019]

[0020] Where, x i The input is numerical network traffic data; x i · This represents the standardized network traffic data output; A is the average value; S is the absolute deviation.

[0021] The normalization process is used to normalize the standardized network traffic data and output the dataset to be detected.

[0022]

[0023] Where, x i · To standardize network traffic data; max(x i · ) represents the maximum value within the same data dimension; min(x) i · ) represents the minimum value within the same data dimension; x i ·· This is the output dataset to be detected after normalization.

[0024] The first feature group includes the outgoing connection speed of the source IP address and the incoming connection speed of the source IP address;

[0025] The outgoing connection rate of the source IP address is defined as the total number of flows generated by the target source IP address within a set time window.

[0026] The inbound connection rate of the source IP address is defined as the total number of flows generated by the target source IP address as the destination IP address within a set time window.

[0027] The second feature group includes the number of streams with different destination IP addresses and the number of streams with different destination ports;

[0028] The number of streams with different destination IP addresses is defined as the number of streams generated by the target source IP address with different destination IP addresses within a set time window;

[0029] The number of different destination port flows is defined as the number of flows from different destination ports generated by the target source IP address within a fixed time window;

[0030] The third feature group includes the number of data packets in the stream, the amount of data in the stream, and the time to life of the stream;

[0031] The number of packets in the stream is defined as the value of packets in the stream generated by the target source IP address within a set time window;

[0032] The data volume of the stream is defined as the value of the data volume in the stream generated by the target source IP address within a set time window;

[0033] The lifetime of a flow is defined as the lifetime of a flow generated by a target source IP address within a set time window.

[0034] The feature factor is used to reflect the relative relationship of features within the corresponding feature group;

[0035] The formula for calculating the feature factor is as follows:

[0036]

[0037]

[0038]

[0039] Where α1 is the first characteristic factor; α2 is the second characteristic factor; α3 is the third characteristic factor; β1 is the first adjustment parameter; β2 is the second adjustment parameter; β 31 The third adjustment parameter; β 32 The fourth adjustment parameter is df1, which is the outgoing speed of the source IP address; df2 is the incoming speed of the source IP address; df3 is the number of streams to different destination IP addresses; df4 is the number of streams to different destination ports; df5 is the number of packets in the stream; df6 is the data volume of the stream; and df7 is the time to live of the stream.

[0040] The data security monitoring method includes a traffic anomaly index for measuring the overall situation of network traffic anomalies; the traffic anomaly index can be updated based on the detection results of the network traffic anomaly detection model.

[0041]

[0042] Where θ represents the traffic anomaly index; a0 represents the number of network traffic anomalies within a specified time window; a is the set threshold for the number of traffic anomalies; t0 represents the interval between the current traffic anomaly and the previous traffic anomaly; t i denoted as , where represents the interval between the i-th and (i+1)-th traffic anomalies; c represents the number of consecutive network traffic anomalies; and e represents a natural constant.

[0043] The network traffic detection model includes a residual learning rate:

[0044]

[0045] Where, γ n γ is the learning rate for the nth model training iteration; γ1 is the initial value of the learning rate, which can be preset; E n denoted as the model loss value for the nth iteration; z represents the fitting parameters; g is the adjustment factor; and n represents the number of training iterations of the model.

[0046] The training process of the network traffic detection model is as follows:

[0047] Obtain a labeled training dataset; perform data preprocessing on the training dataset to obtain a preprocessed training dataset; the data preprocessing process includes attribute mapping, standardization, and normalization; divide the preprocessed training dataset to obtain a training dataset and a test dataset; train the network traffic detection model using the training dataset; test the network traffic detection model using the test dataset; and obtain the network traffic detection model.

[0048] A data security monitoring system includes a data collection module, a data processing module, a feature extraction module, a training and classification module, and a comprehensive judgment module;

[0049] The data collection module is used to collect network traffic data generated by the monitored objects;

[0050] The data processing module is used to preprocess the network traffic data. The preprocessing process includes attribute mapping processing and labeling.

[0051] The standardization and normalization processes are used; the attribute mapping process is used to convert non-numerical data into numerical data; the standardization process is used to convert the dataset into a data distribution with a mean of 0 and a standard deviation of 1; the normalization process is used to map the data range to a specified interval range.

[0052] The feature extraction module is used to extract features from network traffic data, group the data features, and calculate feature factors; the features can be divided into a first feature group, a second feature group, and a third feature group.

[0053] The training classification module trains the features using three training structures, obtains the training results, and outputs the classification results based on the training results.

[0054] The comprehensive judgment module is used to calculate the network traffic anomaly index based on the classification results.

[0055] The feature extraction module includes the three feature groups;

[0056] The first feature group includes the outgoing connection speed of the source IP address and the incoming connection speed of the source IP address;

[0057] The outgoing connection rate of the source IP address is defined as the total number of flows generated by the target source IP address within a set time window.

[0058] The inbound connection rate of the source IP address is defined as the total number of flows generated by the target source IP address as the destination IP address within a set time window.

[0059] The second feature group includes the number of streams with different destination IP addresses and the number of streams with different destination ports;

[0060] The number of streams with different destination IP addresses is defined as the number of streams generated by the target source IP address with different destination IP addresses within a set time window;

[0061] The number of different destination port flows is defined as the number of flows from different destination ports generated by the target source IP address within a fixed time window;

[0062] The third feature group includes the number of data packets in the stream, the amount of data in the stream, and the time to life of the stream;

[0063] The number of packets in the stream is defined as the value of packets in the stream generated by the target source IP address within a set time window;

[0064] The data volume of the stream is defined as the value of the data volume in the stream generated by the target source IP address within a set time window; the lifespan of the stream is defined as the value of the lifespan of the stream generated by the target source IP address within a set time window.

[0065] The feature extraction module includes calculating the corresponding feature factors of the three feature groups to obtain a first feature factor, a second feature factor, and a third feature factor;

[0066]

[0067]

[0068]

[0069] Where α1 is the first characteristic factor; α2 is the second characteristic factor; α3 is the third characteristic factor; β1 is the first adjustment parameter; β2 is the second adjustment parameter; β 31 The third adjustment parameter; β 32The fourth adjustment parameter is df1, which is the outgoing speed of the source IP address; df2 is the incoming speed of the source IP address; df3 is the number of streams to different destination IP addresses; df4 is the number of streams to different destination ports; df5 is the number of packets in the stream; df6 is the data volume of the stream; and df7 is the time to live of the stream.

[0070] The training classification module is used to train the three feature groups and output the training results;

[0071] The training classification module includes a first training structure, a second training structure, and a third training structure; the training structure includes an Inception unit and an LSTM unit; the Inception unit is used to train the frequency domain patterns of the three feature groups; the LSTM unit is used to identify the time domain patterns of the three feature groups; the training results of the feature groups are weighted and fused using the first feature factor, the second feature factor, and the third feature factor respectively; a fused training result is obtained, and the fused training result is classified to obtain a classification result and output.

[0072] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0073] 1. This invention proposes a network traffic detection model to detect abnormal network traffic conditions. The network traffic detection model includes a first training structure, a second training structure, and a third training structure. Each training structure includes an Inception unit and an LSTM unit. The Inception unit is trained to analyze the frequency domain patterns of data features, while the LSTM unit identifies the time domain patterns of data features. This invention combines the Inception unit and the LSTM unit to construct multiple training structures, each trained on different features. Furthermore, by identifying network traffic data features from both the time and frequency domains, better training results can be obtained, ensuring the accuracy of traffic data classification.

[0074] 2. This invention filters network traffic data features to obtain seven features that are significantly correlated with network traffic anomalies. These seven features are then grouped into a first feature group, a second feature group, and a third feature group. Corresponding feature factors are calculated based on the interrelationships within each feature group, resulting in a first feature factor, a second feature factor, and a third feature factor. These three feature factors can reflect network traffic anomalies to a certain extent, and can be adjusted according to the characteristics of different monitoring objects. The three feature factors are then applied to feature classification, weighting the training results of the three feature groups. By grouping and calculating network traffic data features using the characteristics of network traffic, feature groups and feature factors are obtained. The weighting and classification of the trained feature group data using these feature factors improves the accuracy of network traffic data anomaly detection.

[0075] 3. This invention constructs a traffic anomaly index. The traffic anomaly index can be used to measure the anomaly status of network traffic over a period of time. The influencing factors of the traffic anomaly index include three aspects: the number of traffic anomalies, the interval between traffic anomalies, and the number of consecutive traffic anomalies. The number of traffic anomalies is measured by the ratio of the number of network traffic anomalies within a specified time window to a preset threshold. The interval between traffic anomalies is measured by the ratio of the average time interval of several previous traffic anomalies to the time interval of the current traffic anomaly. The number of consecutive traffic anomalies is measured by introducing an exponential function; as the number of consecutive traffic anomalies increases, the severity of network traffic increases exponentially. This invention constructs a traffic anomaly index to comprehensively evaluate the results of anomaly detection within a time window from multiple aspects, and by adjusting the thresholds, it adapts to different scenario environments to obtain a more accurate comprehensive picture of network traffic anomalies. Attached Figure Description

[0076] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0077] Figure 2 This is a schematic diagram of the network traffic detection model structure of the present invention;

[0078] Figure 3 This is a schematic diagram of the Inception unit structure of the present invention;

[0079] Figure 4 This is a schematic diagram of the LSTM cell structure of the present invention;

[0080] Figure 5 This is a schematic diagram illustrating the learning rate comparison experiment results of the present invention;

[0081] Figure 6 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0082] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] In recent years, cyberattacks have become increasingly intelligent, frequent, and covert, posing new challenges to cyberspace. Without proper precautions, this can lead to the theft and modification of important data, leakage of personal privacy, and economic losses. Network traffic is the connecting link in cyberspace and a primary indicator of network status. When a network is attacked or intruded upon, network traffic typically changes; therefore, monitoring anomalies in network traffic has always been a crucial safeguard for network security. Effective management and monitoring methods can be implemented for network traffic. Existing methods, when detecting network traffic data, do not perform extensive feature selection and segmentation of the data, simply inputting the collected network traffic data into a model for training and anomaly detection. The limited range of data features results in low accuracy in detecting network traffic anomalies. Based on these factors, this invention provides a data security monitoring method and system.

[0084] Example 1: The present invention performs feature filtering and grouping on network traffic data, and calculates feature factors.

[0085] Collect network traffic data of the monitored objects; preprocess the network traffic data to obtain the dataset to be detected; extract and classify the features of the dataset to be detected to obtain the first feature group, the second feature group and the third feature group, and calculate the corresponding feature factors; the obtained feature groups and feature factors have a high correlation with network traffic anomalies, which can more accurately judge network data anomalies.

[0086] The data preprocessing process includes attribute mapping, standardization, and normalization. Network traffic data contains various categories of data, including non-numerical data such as protocol type and service type. When using network traffic detection models to detect network traffic anomalies, non-numerical data cannot be processed. To ensure the applicability and accuracy of the model, it is necessary to process the non-numerical data and convert it into numerical data. This invention processes the non-numerical data in the dataset through attribute mapping to output numerical network traffic data.

[0087] Because different types of network traffic data vary greatly in magnitude, it is difficult for different data types to communicate with each other, which can easily lead to inconvenience in operation and low accuracy. Therefore, this invention performs dimensionless processing on numerical network traffic data.

[0088]

[0089]

[0090]

[0091] Where, x i The input is numerical network traffic data; x i · This represents the standardized network traffic data output; A is the average value; S is the absolute deviation.

[0092] The normalization process is used to normalize the standardized network traffic data and output the dataset to be detected.

[0093]

[0094] Where, x i · To standardize network traffic data; max(x i · ) represents the maximum value within the same data dimension; min(x) i · ) represents the minimum value within the same data dimension; x i ·· This is the output dataset to be detected after normalization.

[0095] Feature extraction and processing is one of the important steps in this invention. By extracting features from network traffic data, the process status of network traffic data can be measured and calculated; the data dimensionality of network traffic data can be reduced; and existing network traffic data features can be integrated.

[0096] Network traffic data features include many characteristics, some of which are related to traffic anomalies caused by attacks, while others are not. If the model is trained directly on the overall network traffic data, problems such as excessively long training time and inaccurate training results may occur. By extracting features from network traffic data, data features related to traffic anomalies caused by attacks can be selected, and these selected data features can be used as the focus of subsequent model training.

[0097] In this invention, seven highly relevant data features were selected: outgoing speed of source IP address, incoming speed of source IP address, number of streams with different destination IP addresses, number of streams with different destination ports, number of data packets in a stream, data volume of a stream, and time to life of a stream. In the field of network traffic research, the term "stream" is usually used to describe a group of packets sent from a source to a destination. An IP stream is a group of packets or frames in a network that can be intercepted at a certain point in the network within a time interval. Data packets belonging to the same stream share several common attributes.

[0098] The outgoing connection rate of the source IP address is defined as the total number of flows generated by the target source IP address within a set time window; the incoming connection rate of the source IP address is defined as the total number of flows generated by the target source IP address as the destination IP address within a set time window; the number of flows with different destination IP addresses is defined as the number of flows with different destination IP addresses among the flows generated by the target source IP address within a set time window; the number of flows with different destination ports is defined as the number of flows with different destination ports among the flows generated by the target source IP address within a fixed time window; the number of packets in the flow is defined as the value of the number of packets in the flow generated by the target source IP address within a set time window; the data volume of the flow is defined as the value of the data volume in the flow generated by the target source IP address within a set time window; and the lifespan of the flow is defined as the value of the lifespan of the flow generated by the target source IP address within a set time window.

[0099] Furthermore, in order to better utilize the data features of network traffic data and achieve breakthroughs in training speed and recognition accuracy, this invention groups the above features according to their characteristics to obtain a first feature group, a second feature group, and a third feature group.

[0100] The first feature group includes the outgoing connection speed of the source IP address and the incoming connection speed of the source IP address;

[0101] The second feature group includes the number of streams with different destination IP addresses and the number of streams with different destination ports;

[0102] The third feature group includes the number of data packets in the stream, the amount of data in the stream, and the time to live of the stream.

[0103] The three feature groups are then calculated to obtain corresponding feature factors, namely, the first feature factor, the second feature factor, and the third feature factor. The calculation formulas for the feature factors are as follows:

[0104]

[0105]

[0106]

[0107] Where α1 is the first characteristic factor; α2 is the second characteristic factor; α3 is the third characteristic factor; β1 is the first adjustment parameter; β2 is the second adjustment parameter; β 31 The third adjustment parameter; β 32 The fourth adjustment parameter is df1, which is the outgoing speed of the source IP address; df2 is the incoming speed of the source IP address; df3 is the number of streams to different destination IP addresses; df4 is the number of streams to different destination ports; df5 is the number of packets in the stream; df6 is the data volume of the stream; and df7 is the time to live of the stream.

[0108] The first characteristic factor reflects the relationship between the outbound and inbound connection speeds of source IP addresses in network data. Under normal circumstances, the outbound and inbound connection speeds will show a relatively stable relationship; when the first characteristic factor changes significantly, it indicates that the stable relationship between the outbound and inbound connection speeds has been disrupted. The second characteristic factor reflects the relative relationship between the number of flows to different destination IP addresses and the number of flows to different destination ports. The third characteristic factor reflects the relationship between the number of data packets in a flow, the data volume of a flow, and the lifespan of the flow.

[0109] This invention filters network traffic data features to obtain seven features that are significantly correlated with network traffic anomalies. These seven features are then grouped into a first feature group, a second feature group, and a third feature group. Corresponding feature factors are calculated based on the interrelationships within each feature group. The training results of the three feature groups are then weighted using these three feature factors. By extracting and filtering features, highly correlated features with traffic data anomalies can be selected, while irrelevant influences are eliminated. The feature factors reflect the internal structure of each feature group, adjusting the weight of each feature group on the final classification result and improving the accuracy of network traffic data anomaly detection.

[0110] Example 2: The present invention constructs a network traffic detection model to determine anomalies in network traffic data.

[0111] The network traffic detection model comprises a data input layer, a feature extraction layer, a recognition training layer, and a classification output layer. The specific structure of the network traffic detection model is as follows: Figure 2 As shown.

[0112] The feature extraction layer is used to extract and group features from the dataset to be detected, and to calculate feature factors; thus, the first feature group, the second feature group, and the third feature group, as well as the corresponding first feature factor, second feature factor, and third feature factor are obtained.

[0113] The recognition training layer is used to train the three feature groups and output the training results; the recognition training layer includes a first training structure, a second training structure and a third training structure; the three training structures are respectively used to train the three feature groups; the training structure includes an Inception unit for recognizing the frequency domain pattern of the feature groups and an LSTM unit for recognizing the time domain pattern of the feature groups;

[0114] The Inception unit assembles multiple convolutional or pooling operations into a single network module, comprising four 1x1 convolutional layers, one 3x3 convolutional layer, one 5x5 convolutional layer, and one max-pooling layer. Its specific structure is as follows: Figure 3 As shown, when the Inception unit is applied to network traffic detection, it can be trained on the frequency domain patterns of network traffic data, which can both improve the recognition performance of network traffic data and ensure the efficiency of computing resource utilization.

[0115] The LSTM unit is derived from an improved RNN model and is used for training and recognition of time series data; the structure of the LSTM unit is as follows. Figure 4 As shown; the LSTM unit contains a forget gate, an input gate, and an output gate; these three gate structures control the retention and forgetting of information, extracting useful data information. Wherein, X t Enter information for the current moment; h t-1 Indicates the hidden state at the previous time step; h t The hidden state to be passed to the next time step; σ is the sigmoid function; tanh is the tanh function, which can transform the data into values ​​in the range [-1, 1]; C t It is in the cellular state.

[0116] The classification output layer is used to perform fusion classification on the training results; the classification output layer includes a first feature factor, a second feature factor, and a third feature factor; the three feature factors are respectively weighted on the training results to obtain fusion training results; the fusion training results are classified to obtain classification results, and the classification results are output.

[0117] The learning rate is one of the most important hyperparameters in neural networks. During neural network model training, the learning rate significantly affects the speed at which information accumulates in the model, thus impacting model performance. A high learning rate leads to excessively fast convergence, making it difficult to identify local optima; a low learning rate results in slower convergence, requiring more training time. Therefore, setting an appropriate learning rate is crucial for ensuring the performance of a backpropagation (BP) neural network model. Commonly used learning rates include exponential decay, fractional decay, cosine decay, and adaptive learning rates. Network traffic data exhibits strong temporal correlations, showing certain patterns depending on the activity habits of the monitored objects. For example, campus network traffic is much more active and congested at night than in the morning, and more active on weekends than on weekdays. For office buildings, network traffic activity is higher on weekdays and relatively lower on weekends and holidays. Therefore, this invention proposes a residual learning rate during model training, constructed using residual structures.

[0118] Based on the temporal and continuous characteristics of network traffic data, this invention proposes a residual learning rate:

[0119]

[0120] Where, γ n γ is the learning rate for the nth model training iteration; γ1 is the initial value of the learning rate, which can be preset; E n Let z be the model loss value at the nth iteration; z be the fitting parameter; g be the adjustment factor; and n represent the number of training iterations. To verify the actual effect of the improved fluctuating learning rate, this invention uses the controlled variable method for research, including a first learning rate model using a residual learning rate, a second learning rate model using a fixed learning rate, a third learning rate model using an exponentially decaying learning rate, and a fourth learning rate model using a cosine decaying learning rate; all four models use a network traffic detection model, and only the learning rate in the models differs.

[0121] Using the initial dataset collected and established, after preprocessing, 80% of the data in the dataset is selected by random sampling for each experiment, and it is divided into an experimental training set and an experimental test set.

[0122] The first, second, third, and fourth learning rate models were trained using the experimental training set, and then tested using the experimental test set to obtain the accuracy rate of the network traffic detection model. The accuracy rate is the ratio of the number of times the network traffic detection model correctly identified the network traffic to the total number of identifications in the experimental test set data. This experimental process was repeated 10 times, and the accuracy rate of each model was recorded.

[0123] The final experimental results are as follows Figure 5 As shown in the figure, the horizontal axis represents the number of experiments conducted, and the vertical axis represents the model's recognition accuracy.

[0124] Overall, the experimental results show that the first learning rate model using residual learning rate has the highest average accuracy of 94.09%, followed by the second learning rate model using fixed learning rate with an average accuracy of 90.65%. Next is the fourth learning rate model using cosine decay learning rate with an average accuracy of 86.81%, and finally the third learning rate model using exponential decay learning rate with an average accuracy of 86.55%.

[0125] The classification output layer is used to perform fusion classification on the training results; the classification output layer includes a first feature factor, a second feature factor, and a third feature factor; the three feature factors are respectively weighted on the training results to obtain fusion training results; the fusion training results are classified to obtain classification results, and the classification results are output.

[0126] The training process of the network traffic detection model is as follows:

[0127] Obtain a labeled training dataset; perform data preprocessing on the training dataset to obtain a preprocessed training dataset; the data preprocessing process includes attribute mapping, standardization, and normalization; divide the preprocessed training dataset to obtain a training dataset and a test dataset; train the network traffic detection model using the training dataset; test the network traffic detection model using the test dataset; and obtain the network traffic detection model.

[0128] Example 3: This invention proposes a traffic anomaly index to measure the severity of network traffic anomalies.

[0129] Since network traffic is constantly in flux, and network attacks and defenses are unpredictable and ever-present, it is difficult to accurately and effectively assess network security based solely on a single anomaly detection of network data. Therefore, this invention proposes a traffic anomaly index to measure the severity of network traffic anomalies over a period of time.

[0130] The data security monitoring method includes a traffic anomaly index for measuring the overall situation of network traffic anomalies; the traffic anomaly index can be updated based on the detection results of the network traffic anomaly detection model.

[0131]

[0132] Where θ represents the traffic anomaly index; a0 represents the number of network traffic anomalies within a specified time window; a is the set threshold for the number of traffic anomalies; t0 represents the interval between the current traffic anomaly and the previous traffic anomaly; ti Let represent the interval between the i-th and (i+1)-th traffic anomalies; c is the number of consecutive network traffic anomalies; and e is a natural constant. This invention constructs a traffic anomaly index from three aspects: the number of traffic anomalies, the traffic anomaly interval, and the number of consecutive traffic anomalies.

[0133] The number of traffic anomalies is measured by the ratio of the number of network traffic anomalies within a specified time window to a preset threshold for the number of traffic anomalies. When the number of network traffic anomalies within the specified time window is greater than the preset threshold, the ratio is greater than 1, thus increasing the traffic anomaly index. When the number of network traffic anomalies within the specified time window is less than the preset threshold, the ratio is less than 1, thus decreasing the traffic anomaly index.

[0134] The traffic anomaly interval is measured by the ratio of the average time interval of several previous traffic anomalies to the current traffic anomaly interval. If the time interval between the current traffic anomaly and the previous traffic anomaly is less than the average of the previous several anomaly intervals, it indicates that the time interval between anomalies is decreasing and the attacks are increasing, thus generating an impact weight greater than 1 and increasing the traffic anomaly index. If the time interval between the current traffic anomaly and the previous traffic anomaly is greater than the average of the previous several anomaly intervals, then generating an impact weight less than 1 and decreasing the traffic anomaly index.

[0135] Regarding the number of consecutive traffic anomalies, an exponential function is introduced to measure the severity. As the number of consecutive traffic anomalies increases, the severity of the network traffic situation grows exponentially. The more consecutive attacks a network receives, the more severe the anomaly becomes.

[0136] Example 4: The present invention also proposes a data security monitoring system.

[0137] The data security monitoring system includes a data collection module, a data processing module, a feature extraction module, a training and classification module, and a comprehensive judgment module; the structure of the data security monitoring system is as follows: Figure 6 As shown.

[0138] The data collection module is used to collect network traffic data generated by the monitored objects;

[0139] The data processing module is used to preprocess the network traffic data. The preprocessing process includes attribute mapping, standardization, and normalization. The attribute mapping is used to convert non-numerical data into numerical data. The standardization is used to convert the dataset into a data distribution with a mean of 0 and a standard deviation of 1. The normalization is used to map the data range to a specified interval.

[0140] The feature extraction module is used to extract features from network traffic data, group the data features, and calculate feature factors;

[0141] The data characteristics include the outgoing connection speed of the source IP address, the incoming connection speed of the source IP address, the number of streams with different destination IP addresses, the number of streams with different destination ports, the number of data packets in the stream, the data volume of the stream, and the time to life of the stream.

[0142] The outgoing connection rate of the source IP address is defined as the total number of flows generated by the target source IP address within a set time window; the incoming connection rate of the source IP address is defined as the total number of flows generated by the target source IP address as the destination IP address within a set time window; the number of flows with different destination IP addresses is defined as the number of flows with different destination IP addresses among the flows generated by the target source IP address within a set time window; the number of flows with different destination ports is defined as the number of flows with different destination ports among the flows generated by the target source IP address within a fixed time window; the number of packets in the flow is defined as the value of the number of packets in the flow generated by the target source IP address within a set time window; the data volume of the flow is defined as the value of the data volume in the flow generated by the target source IP address within a set time window; and the lifespan of the flow is defined as the value of the lifespan of the flow generated by the target source IP address within a set time window.

[0143] The features can be divided into a first feature group, a second feature group, and a third feature group.

[0144] Furthermore, in order to better utilize the data features of network traffic data and achieve breakthroughs in training speed and recognition accuracy, this invention groups the above features according to their characteristics to obtain a first feature group, a second feature group, and a third feature group, and calculates the corresponding feature factors for the three feature groups, namely the first feature factor, the second feature factor, and the third feature factor.

[0145]

[0146]

[0147]

[0148] Where α1 is the first characteristic factor; α2 is the second characteristic factor; α3 is the third characteristic factor; β1 is the first adjustment parameter; β2 is the second adjustment parameter; β 31 The third adjustment parameter; β 32 The fourth adjustment parameter is df1, which is the outgoing speed of the source IP address; df2 is the incoming speed of the source IP address; df3 is the number of streams to different destination IP addresses; df4 is the number of streams to different destination ports; df5 is the number of packets in the stream; df6 is the data volume of the stream; and df7 is the time to live of the stream.

[0149] The training classification module is used to identify the features, obtain training results, and classify them based on the training results;

[0150] The training classification module is used to train the three feature groups and output the training results;

[0151] The training classification module includes a first training structure, a second training structure, and a third training structure; the training structure includes an Inception unit and an LSTM unit; the Inception unit is used to train the frequency domain patterns of the three feature groups; the LSTM unit is used to identify the time domain patterns of the three feature groups.

[0152] The training results of the feature group are weighted and fused using the first feature factor, the second feature factor, and the third feature factor, respectively; the fused training result is obtained; the fused training result is then classified to obtain a classification result and output.

[0153] The comprehensive judgment module is used to calculate the network traffic anomaly index based on the classification results.

[0154] In summary, this invention improves the accuracy of detecting network traffic anomalies by selecting and processing network traffic data features, constructing a network traffic detection model, and calculating a traffic anomaly index. Specifically, the extraction and grouping of data features effectively filters for detecting traffic anomalies. Then, by calculating the feature factors of the feature groups, the importance between feature groups is adjusted. Next, the network traffic detection model verifies the feature groups, identifying their frequency and time domain patterns using Inception and LSTM units. Finally, the network traffic anomaly index is updated based on the obtained judgment results to accurately determine the severity of network traffic anomalies.

[0155] The above are specific embodiments of the present invention. For those skilled in the art, various changes, modifications, substitutions and variations made to these embodiments without departing from the principles and spirit of the present invention are all protected within the scope defined by the appended claims and their equivalents.

Claims

1. A data security monitoring method, characterized in that: Identify the monitoring targets and collect network traffic data for the monitored targets; The network traffic data is preprocessed to obtain the dataset to be detected; The dataset to be detected is input into the network traffic detection model for training, and the classification result is output. The classification result is used to determine whether the network traffic data is abnormal; the dataset to be detected passes sequentially through the data input layer, feature extraction layer, recognition training layer and classification output layer of the network traffic detection model; The feature extraction layer is used to extract and group features from the dataset to be detected, and to calculate feature factors; to obtain a first feature group, a second feature group, and a third feature group, as well as the corresponding first feature factor, second feature factor, and third feature factor; The recognition training layer is used to train the three feature groups and output the training results; the recognition training layer includes a first training structure, a second training structure and a third training structure; The three training structures are used to train the three feature groups respectively; each training structure includes an Inception unit for identifying frequency domain patterns of the feature groups and an LSTM unit for identifying time domain patterns of the feature groups. The classification output layer is used to perform fusion classification on the training results; the classification output layer includes a first feature factor, a second feature factor, and a third feature factor; the three feature factors are respectively weighted on the training results to obtain fusion training results; the fusion training results are classified to obtain classification results, and the classification results are output.

2. The data security monitoring method according to claim 1, characterized in that: Data preprocessing includes attribute mapping, standardization, and normalization. The attribute mapping process is used to process the non-numerical data in the network traffic data and output numerical network traffic data. The standardization process is used to unify the dimensions of the numerical network traffic data and output standardized network traffic data. Where, x i The input is numerical network traffic data; x i · This represents the standardized network traffic data output; A is the average value; S is the absolute deviation. The normalization process is used to normalize the standardized network traffic data and output the dataset to be detected. Where, x i · To standardize network traffic data; max(x i · ) represents the maximum value in the data; min(x) i · ) represents the minimum value in the data; x i ·· This is the output dataset to be detected after normalization.

3. The data security monitoring method according to claim 1, characterized in that: The first feature group includes the outgoing connection speed of the source IP address and the incoming connection speed of the source IP address; The outgoing connection rate of the source IP address is defined as the total number of flows generated by the target source IP address within a set time window. The inbound connection rate of the source IP address is defined as the total number of flows generated by the target source IP address as the destination IP address within a set time window. The second feature group includes the number of streams with different destination IP addresses and the number of streams with different destination ports; The number of streams with different destination IP addresses is defined as the number of streams generated by the target source IP address with different destination IP addresses within a set time window; The number of different destination port flows is defined as the number of flows from different destination ports generated by the target source IP address within a fixed time window; The third feature group includes the number of data packets in the stream, the amount of data in the stream, and the time to life of the stream; The number of packets in the stream is defined as the value of packets in the stream generated by the target source IP address within a set time window; The data volume of the stream is defined as the value of the data volume in the stream generated by the target source IP address within a set time window; The lifetime of a flow is defined as the lifetime of a flow generated by a target source IP address within a set time window.

4. The data security monitoring method according to claim 1, characterized in that: The feature factor is used to reflect the relationship between features within the feature group; The formula for calculating the feature factor is as follows: Where α1 is the first characteristic factor; α2 is the second characteristic factor; α3 is the third characteristic factor; β1 is the first adjustment parameter; β2 is the second adjustment parameter; β 31 The third adjustment parameter; β 32 The fourth adjustment parameter is df1, which is the outgoing speed of the source IP address; df2 is the incoming speed of the source IP address; df3 is the number of streams to different destination IP addresses; df4 is the number of streams to different destination ports; df5 is the number of packets in the stream; df6 is the data volume of the stream; and df7 is the time to live of the stream.

5. The data security monitoring method according to claim 1, characterized in that: The data security monitoring method includes a traffic anomaly index for measuring the overall situation of network traffic anomalies; the traffic anomaly index can be updated based on the detection results of the network traffic anomaly detection model. Where θ represents the traffic anomaly index; a0 represents the number of network traffic anomalies within a specified time window; a is the set threshold for the number of traffic anomalies; t0 represents the interval between the current traffic anomaly and the previous traffic anomaly; t i denoted as , where represents the interval between the i-th and (i+1)-th traffic anomalies; c represents the number of consecutive network traffic anomalies; and e represents a natural constant.

6. The data security monitoring method according to claim 1, characterized in that: The training process of the network traffic detection model is as follows: Obtain the labeled training dataset; The training dataset is preprocessed to obtain a preprocessed training dataset; the data preprocessing process includes attribute mapping, standardization and normalization. The preprocessed training dataset is divided into a training dataset and a test dataset; The network traffic detection model is trained using the training dataset. The network traffic monitoring model was tested using the test dataset.

7. A data security monitoring system, characterized in that: The data security monitoring system includes a data collection module, a data processing module, a feature extraction module, a training and classification module, and a comprehensive judgment module. The data collection module is used to collect network traffic data generated by the monitored objects; The data processing module is used to preprocess the network traffic data. The preprocessing process includes attribute mapping, standardization, and normalization. The attribute mapping process is used to convert non-numerical data into numerical data; the standardization process is used to convert the dataset into a data distribution with a mean of 0 and a standard deviation of 1; the normalization process is used to map the data range to a specified interval range. The feature extraction module is used to extract features from network traffic data, group the data features, and calculate feature factors; The features can be divided into a first feature group, a second feature group, and a third feature group; The training classification module trains the features using three training structures, obtains the training results, and outputs the classification results based on the training results. The comprehensive judgment module is used to calculate the network traffic anomaly index based on the classification results.

8. The data security monitoring system according to claim 7, characterized in that: The feature extraction module includes the three feature groups; The first feature group includes the outgoing connection speed of the source IP address and the incoming connection speed of the source IP address; The outgoing connection rate of the source IP address is defined as the total number of flows generated by the target source IP address within a set time window. The inbound connection rate of the source IP address is defined as the total number of flows generated by the target source IP address as the destination IP address within a set time window. The second feature group includes the number of streams with different destination IP addresses and the number of streams with different destination ports; The number of streams with different destination IP addresses is defined as the number of streams generated by the target source IP address with different destination IP addresses within a set time window; The number of different destination port flows is defined as the number of flows from different destination ports generated by the target source IP address within a fixed time window; The third feature group includes the number of data packets in the stream, the amount of data in the stream, and the time to life of the stream; The number of packets in the stream is defined as the value of packets in the stream generated by the target source IP address within a set time window; The data volume of the stream is defined as the value of the data volume in the stream generated by the target source IP address within a set time window; the lifespan of the stream is defined as the value of the lifespan of the stream generated by the target source IP address within a set time window.

9. A data security monitoring system according to claim 7, characterized in that: The feature extraction module includes calculating the corresponding feature factors of the three feature groups to obtain a first feature factor, a second feature factor, and a third feature factor; Where α1 is the first characteristic factor; α2 is the second characteristic factor; α3 is the third characteristic factor; β1 is the first adjustment parameter; β2 is the second adjustment parameter; β 31 The third adjustment parameter; β 32 The fourth adjustment parameter is df1, which is the outgoing speed of the source IP address; df2 is the incoming speed of the source IP address; df3 is the number of streams to different destination IP addresses; df4 is the number of streams to different destination ports; df5 is the number of packets in the stream; df6 is the data volume of the stream; and df7 is the time to live of the stream.

10. A data security monitoring system according to claim 7, characterized in that: The training classification module is used to train the three feature groups and output the training results; The training classification module includes a first training structure, a second training structure, and a third training structure; the training structure includes an Inception unit and an LSTM unit; the Inception unit is used to train the frequency domain patterns of the three feature groups; the LSTM unit is used to identify the time domain patterns of the three feature groups. The training results of the feature group are weighted and fused using the first feature factor, the second feature factor, and the third feature factor, respectively. The fusion training results are obtained, and the fusion training results are classified to obtain the classification results and output.

Images