A real-time traffic security detection method and system based on network intrusion

Abstract

The present invention discloses a real-time traffic security detection method and system based on network intrusion, the method comprising: capturing traffic data at each moment in real time; detecting the traffic data at the current moment, if it exceeds a preset traffic threshold range, marking it as first traffic data to be detected, inputting it into a first abnormal traffic detection model, so that the first abnormal traffic detection model extracts the features to be detected of the first traffic data to be detected, and outputs the data type and impact duration of the first traffic data to be detected in combination with an abnormal feature database; if it does not exceed the preset traffic threshold range, marking it as second traffic data to be detected; when data anomaly is detected, training the second abnormal traffic detection model with all second traffic data to be detected and unlearned abnormal features within the impact range when the data anomaly occurs, so that the second abnormal traffic detection model outputs unlearned abnormal features and impact duration, and updates the abnormal feature database.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a real-time traffic security detection method and system based on network intrusion. Background Art

[0002] With the further development of network technology, some incidental network security issues have also emerged in people's sight. One of the main aspects of network security maintenance is to perform security detection on network traffic, and even if abnormal data or intrusion data is found in network traffic, it can be processed to maintain network security.

[0003] The existing technology for detecting abnormal traffic mainly collects network traffic data, log data and network security data, cleans and preprocesses the collected network traffic data, log data and network security data, and extracts feature elements based on the preprocessed data to generate an effective feature data set; constructs a deep learning model as a detection model, and then trains the deep learning model with the effective feature data set and known network attack samples, and applies the trained model as a detection model to the anomaly detection of network traffic. The model trained in this way cannot dynamically learn new features, making it difficult to detect newly generated factors that cause network traffic anomalies. Summary of the invention

[0004] The embodiments of the present invention provide a real-time traffic security detection method and system based on network intrusion, which can quickly and accurately detect abnormal traffic in real-time traffic, dynamically learn new abnormal features, improve the comprehensiveness of abnormal detection, and enhance network security.

[0005] An embodiment of the present invention provides a real-time traffic security detection method based on network intrusion, comprising:

[0006] Capture traffic data at every moment in real time;

[0007] Detect the flow data at the current moment. If the flow data at the current moment exceeds the preset flow threshold range, mark the flow data at the current moment as the first flow data to be detected, isolate the first flow data to be detected, and input it into the first abnormal flow detection model, so that the first abnormal flow detection model extracts the features to be detected of the first flow data to be detected, and outputs the data type and impact duration of the first flow data to be detected in combination with the abnormal feature database; wherein the data types include normal data, abnormal data and suspicious data;

[0008] If the flow data at the current moment does not exceed the preset flow threshold range, the flow data at the current moment is marked as the second flow data to be detected;

[0009] When any second flow data to be detected is monitored to have a data anomaly, all second flow data to be detected and unlearned abnormal features within the influence range of the data anomaly are obtained, and the second abnormal flow detection model is trained with all second flow data to be detected and unlearned abnormal features within the influence range of the data anomaly, so that the second abnormal flow detection model outputs unlearned abnormal features and impact duration, and the abnormal feature database is updated with the unlearned abnormal features and impact duration;

[0010] Execute the corresponding network security strategy according to the data type of the current traffic data;

[0011] Wherein, the construction of the first abnormal traffic detection model includes:

[0012] Obtain a historical traffic data sample set; wherein the historical traffic data sample set includes abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments; the abnormal traffic data sample at each historical moment includes abnormal features;

[0013] An initial first abnormal traffic detection model is constructed, and the initial first abnormal traffic detection model is iteratively trained with the historical traffic data sample set until the initial first abnormal traffic detection model converges to generate the first abnormal traffic detection model; wherein, during the iteration process, the initial first abnormal traffic detection model extracts feature vectors from the historical traffic data sample set, determines the data type and impact duration based on the feature vector, and adds the abnormal features and impact duration to the abnormal feature database.

[0014] Furthermore, the obtaining of a sample set of historical traffic data includes:

[0015] Obtain abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments;

[0016] Randomly decompose the abnormal traffic data samples at several historical moments into abnormal traffic data fragment samples at several historical moments;

[0017] Merge abnormal traffic data fragment samples at several historical moments and normal traffic data samples at several historical moments to obtain suspicious samples containing abnormal traffic data fragment samples at several historical moments and the suspicious probability of each suspicious sample; wherein the suspicious probability is determined according to the abnormal traffic data fragment samples, the suspicious samples and the abnormal traffic data samples at the historical moments;

[0018] A historical traffic data sample set is constructed using abnormal traffic data samples at several historical moments, normal traffic data samples at several historical moments, and suspicious samples at several historical moments.

[0019] Further, the initial first abnormal traffic detection model is iteratively trained with the historical traffic data sample set until the initial first abnormal traffic detection model converges to generate the first abnormal traffic detection model, including:

[0020] Taking abnormal traffic data samples at several historical moments as input and data types corresponding to the abnormal traffic data samples at several historical moments as output, iteratively training the first abnormal traffic detection model so that the first abnormal traffic detection model extracts abnormal features corresponding to the abnormal traffic data samples at each historical moment;

[0021] Taking abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments as input and taking the data type corresponding to each sample as output, iteratively training the first abnormal traffic detection model, and determining the first sample weight when the first abnormal traffic detection model converges;

[0022] Taking suspicious samples at several historical moments as input and the data type corresponding to each suspicious sample as output, the first abnormal traffic detection model is iteratively trained, and when the first abnormal traffic detection model converges, the second sample weight is determined; wherein, when the suspicious probability is greater than a preset probability threshold, the data type corresponding to the suspicious sample is abnormal data;

[0023] Taking abnormal traffic data samples at several historical moments, normal traffic data samples at several historical moments, and suspicious samples at several historical moments as input, and taking the data type corresponding to each sample as output, iteratively training the first abnormal traffic detection model, and determining a third sample weight when the first abnormal traffic detection model converges;

[0024] The fusion weight of the first abnormal traffic detection model is determined according to the first sample weight, the second sample weight and the third sample weight.

[0025] Further, the second abnormal flow detection model is trained with all the second flow data to be detected within the impact range when the data is abnormal and the unlearned abnormal features, so that the second abnormal flow detection model outputs the unlearned abnormal features and the impact duration, and the abnormal feature database is updated with the unlearned abnormal features and the impact duration, including:

[0026] Constructing a training sample set according to all second flow data to be detected and unlearned abnormal features within the impact range when the data is abnormal; wherein the training sample set includes a number of flow data samples containing unlearned abnormal features and a number of flow data samples not containing unlearned abnormal features;

[0027] The second abnormal traffic detection model is trained with the training sample set until the second abnormal traffic detection model converges, and the weights to be fused are determined;

[0028] Perform simulation operation with all second flow data to be detected and unlearned abnormal features within the impact range when the data is abnormal, and determine the impact duration of the unlearned abnormal features;

[0029] The unlearned abnormal features, the duration of influence of the unlearned abnormal features and the weights to be fused are transmitted to the first abnormal traffic detection model, so that the first abnormal traffic detection model updates the fusion weights with the weights to be fused to obtain updated fusion weights, and the unlearned abnormal features and the duration of influence of the unlearned abnormal features are stored in the abnormal feature database;

[0030] Initialize the second abnormal traffic detection model.

[0031] Furthermore, the step of randomly decomposing the abnormal traffic data samples at a number of historical moments into abnormal traffic data fragment samples at a number of historical moments includes:

[0032] Performing stratified sampling on abnormal traffic data samples at several historical moments to obtain selected abnormal traffic data samples of several groups; wherein the abnormal characteristics of the selected abnormal traffic data samples included in each group are different;

[0033] Fusing several selected abnormal traffic data samples in each group to obtain selected fusion feature samples of several groups;

[0034] Randomly intercept the selected fusion feature samples of each group to obtain abnormal traffic data fragment samples at several historical moments.

[0035] Furthermore, the abnormal traffic data fragment samples at several historical moments are merged with the normal traffic data samples at several historical moments to obtain suspicious samples containing abnormal traffic data fragment samples at several historical moments and the suspicious probability of each suspicious sample, including:

[0036] Merge abnormal traffic data fragment samples at several historical moments with normal traffic data samples at several historical moments to obtain suspicious samples containing abnormal traffic data fragment samples at several historical moments;

[0037] Determine the first segmented suspicious probability according to the proportion of abnormal traffic data fragment samples in the suspicious samples at several historical moments to the abnormal traffic data samples at the corresponding historical moments;

[0038] Determine the second segmented suspicious probability according to the proportion of abnormal traffic data fragment samples in the suspicious samples at several historical moments;

[0039] The suspicious probabilities of suspicious samples at a number of historical moments are determined according to each of the first subdivided suspicious probabilities and each of the second subdivided suspicious probabilities.

[0040] Further, executing a corresponding network security policy according to the data type of the flow data at the current moment includes:

[0041] When the data type is normal data, the isolation of the current traffic data is lifted;

[0042] When the data type is suspicious data, the current flow data is marked and monitored within a preset period. If it does not turn into abnormal data within the preset period, the isolation of the current flow data is lifted;

[0043] When the data type is abnormal data, the abnormal data is processed according to the abnormal characteristics of the abnormal data. If the abnormal data can be converted into normal data after processing, the isolation of the traffic data at the current moment is released. If the abnormal data cannot be converted into normal data after processing, the traffic data at the current moment is eliminated.

[0044] Based on the above method embodiment, the present invention provides a corresponding system embodiment;

[0045] An embodiment of the present invention provides a real-time traffic security detection system based on network intrusion, including: a traffic data capture module, a detection module, a feature update module, a security management module and a first abnormal traffic detection model construction module;

[0046] The flow data capture module is used to capture the flow data at each moment in real time;

[0047] The detection module is used to detect the flow data at the current moment. If the flow data at the current moment exceeds the preset flow threshold range, the flow data at the current moment is marked as the first flow data to be detected, the first flow data to be detected is isolated, and input into the first abnormal flow detection model, so that the first abnormal flow detection model extracts the to-be-detected features of the first flow data to be detected, and outputs the data type and impact duration of the first flow data to be detected in combination with the abnormal feature database; wherein the data type includes; normal data, abnormal data and suspicious data; if the flow data at the current moment does not exceed the preset flow threshold range, the flow data at the current moment is marked as the second flow data to be detected;

[0048] The feature updating module is used to obtain all the second flow data to be detected and the unlearned abnormal features within the influence range of the data abnormality when monitoring any second flow data to be detected has a data abnormality, train the second abnormal flow detection model with all the second flow data to be detected and the unlearned abnormal features within the influence range of the data abnormality, so that the second abnormal flow detection model outputs the unlearned abnormal features and the impact duration, and update the abnormal feature database with the unlearned abnormal features and the impact duration;

[0049] The security control module is used to execute the corresponding network security strategy according to the data type of the flow data at the current moment;

[0050] The first abnormal traffic detection model construction module is used to obtain a historical traffic data sample set; wherein the historical traffic data sample set includes abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments; the abnormal traffic data sample at each historical moment includes abnormal features; an initial first abnormal traffic detection model is constructed, and the initial first abnormal traffic detection model is iteratively trained with the historical traffic data sample set until the initial first abnormal traffic detection model converges to generate the first abnormal traffic detection model; wherein, during the iteration process, the initial first abnormal traffic detection model extracts feature vectors from the historical traffic data sample set, determines the data type and impact duration based on the feature vectors, and adds the abnormal features and impact duration to the abnormal feature database.

[0051] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the real-time traffic security detection method based on network intrusion described in the above-mentioned embodiment of the invention.

[0052] Another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a real-time traffic security detection method based on network intrusion as described in the above-mentioned embodiment of the invention.

[0053] The following beneficial effects are achieved by implementing the present invention:

[0054] The present invention provides a real-time traffic security detection method and system based on network intrusion. The detection method performs security detection on the traffic data captured at each moment in real time by capturing the traffic data at each moment in real time; based on the characteristic that the traffic data fluctuates greatly during network intrusion, a quick preliminary judgment is made on the traffic data at the current moment within a preset traffic threshold range; when the traffic data at the current moment exceeds the preset traffic threshold range, that is, when the network traffic fluctuation is abnormal, the traffic data at the current moment is used as the first traffic data to be detected, and the data is isolated to avoid the first traffic data to be detected from spreading in the further refined detection process; after isolation, the first traffic data to be detected is detected in combination with a first abnormal traffic detection model to determine the data type and impact duration of the first traffic data to be detected;

[0055] Furthermore, if the flow data at the current moment does not exceed the preset flow threshold range, the flow data at the current moment is marked as the second flow data to be detected, and the marked second flow data to be detected is monitored. When data abnormality occurs in any second flow to be detected, all second flow data to be detected and unlearned abnormal features within the impact range of the data abnormality are obtained, and the second abnormal flow detection model is trained with the above as training samples, so that the second abnormal flow detection model learns the unlearned abnormal features, determines the impact duration of the unlearned abnormal features, and updates the abnormal feature database of the first abnormal flow detection model with the learning results. The traffic data in which no abnormality is found is monitored, and when an abnormality occurs, the traffic data within the affected range is used as samples to train the second abnormal traffic detection model, and then the training results are shared with the first abnormal traffic detection model. This makes it unnecessary for the first abnormal traffic detection model to be retrained when new abnormal features need to be learned, and the amount of training data for the second abnormal traffic detection model used to learn new abnormal features is small, and the computing power and complexity during training are lower, which is convenient for dynamically learning new abnormal features and dynamically updating the abnormal feature database, which can improve the generalization recognition ability and detection accuracy of the first abnormal traffic detection model, thereby further improving network security. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The present invention is a flowchart of a method for real-time traffic security detection based on network intrusion provided by an embodiment of the present invention.

[0057] Figure 2 It is a structural diagram of a real-time traffic security detection system based on network intrusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] like Figure 1 As shown, a real-time traffic security detection method based on network intrusion is provided in one embodiment of the present invention, comprising:

[0060] Step S1: Capture traffic data at each moment in real time;

[0061] Step S2: Detect the flow data at the current moment. If the flow data at the current moment exceeds the preset flow threshold range, mark the flow data at the current moment as the first flow data to be detected, isolate the first flow data to be detected, and input it into the first abnormal flow detection model, so that the first abnormal flow detection model extracts the features to be detected of the first flow data to be detected, and outputs the data type and impact duration of the first flow data to be detected in combination with the abnormal feature database; wherein the data types include normal data, abnormal data and suspicious data;

[0062] Step S3: if the flow data at the current moment does not exceed the preset flow threshold range, the flow data at the current moment is marked as the second flow data to be detected;

[0063] Step S4: when any second flow data to be detected is monitored to have data anomaly, all second flow data to be detected and unlearned abnormal features within the influence range of the data anomaly are obtained, and the second abnormal flow detection model is trained with all second flow data to be detected and unlearned abnormal features within the influence range of the data anomaly, so that the second abnormal flow detection model outputs unlearned abnormal features and impact duration, and the abnormal feature database is updated with the unlearned abnormal features and impact duration;

[0064] Step S5: Execute the corresponding network security policy according to the data type of the flow data at the current moment;

[0065] For step S1, real-time capture of traffic data at each moment in the network can be performed using the Wireshark network protocol analyzer. Wireshark can capture data packets in the network and display detailed data packet information in real time.

[0066] Regarding step S2, it should be explained in advance that, when the network is in a normal state, the traffic data required to transmit data packets in the network is in a steady state, that is, the fluctuation of the network traffic data during the transmission process is within a preset normal range; when the network is invaded, such as by virus attacks and human malicious operations, the manifestation on the data packets is usually that the data packets are too long and the number of requests is frequent, which in turn causes network fluctuations. Therefore, abnormal fluctuations in network traffic data are the most intuitive way to determine whether the network has been invaded.

[0067] In the present invention, in order to quickly determine whether the network is abnormal and ensure network security, when detecting the captured traffic data at the current moment, first determine whether there is a possibility of abnormality through the fluctuation of the network traffic data. If the traffic data at the current moment exceeds the fluctuation range (that is, the above-mentioned preset traffic threshold range), it is considered that it is very likely to be abnormal data; in order to prevent the abnormal data from further spreading to the network over time, before performing detailed abnormality detection on the traffic data at the current moment, the traffic data at the current moment is calibrated as the first traffic data to be detected, and at the same time, the first traffic data to be detected is isolated so that it will not have further impact on the network during the detection process, thereby ensuring the normal operation of the network.

[0068] The detection of the first flow data to be detected is mainly carried out through the first abnormal flow detection model. The first flow data to be detected is input into the first abnormal flow detection model so that the first abnormal flow detection model extracts the feature vector (i.e., the above-mentioned feature to be detected) in the first flow data to be detected, and matches the feature vector with the abnormal feature stored in the abnormal feature database of the first abnormal flow detection model. If the matching probability is not higher than the first preset threshold, it is considered to be normal data; if the matching probability is higher than the first preset threshold and lower than the second preset threshold, it is considered to be suspicious data; if the matching probability is higher than the second preset threshold, it is considered to be abnormal data; the first preset threshold is usually set to 50%, and the second preset threshold is usually set to 100%. The first abnormal flow detection model outputs the data type of the first flow data to be detected according to the matching result. Determine the duration of its impact on the network based on the data type. If it is normal data or suspicious data, it has no impact on the network at the current moment. If it is abnormal data, the matching impact duration can be retrieved in the abnormal feature database based on the corresponding abnormal feature. The impact duration is obtained by clustering the impact duration of similar abnormal features in the network in the past. By determining the impact duration, the time required to isolate the data can be determined, thereby optimizing the execution time of the security policy.

[0069] The construction of the first abnormal traffic detection model includes the following steps:

[0070] Step S101: Obtain a historical traffic data sample set; wherein the historical traffic data sample set includes abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments; the abnormal traffic data sample at each historical moment includes abnormal features;

[0071] Step S201: construct an initial first abnormal traffic detection model, and iteratively train the initial first abnormal traffic detection model with the historical traffic data sample set until the initial first abnormal traffic detection model converges to generate the first abnormal traffic detection model; wherein, during the iteration process, the initial first abnormal traffic detection model extracts feature vectors from the historical traffic data sample set, determines the data type and impact duration based on the feature vector, and adds the abnormal features and impact duration to the abnormal feature database.

[0072] For step S101, the acquired historical traffic data sample set needs to include abnormal traffic data samples at several historical moments. The samples are samples with known abnormal features and can be considered as calibration samples. In order to enable the first abnormal traffic detection model to learn as many abnormal features as possible during training, the selected abnormal traffic data samples should cover the known abnormal features before the current training moment, for example, known virus data before the current training moment, abnormal behaviors known to cause network congestion, etc.; in addition, the historical traffic data sample set also needs to include normal traffic data at several historical moments.

[0073] In a preferred embodiment, the acquisition of the historical traffic data sample set includes: acquiring abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments; randomly decomposing the abnormal traffic data samples at several historical moments into abnormal traffic data fragment samples at several historical moments; fusing the abnormal traffic data fragment samples at several historical moments with the normal traffic data samples at several historical moments to obtain suspicious samples containing abnormal traffic data fragment samples at several historical moments and the suspicious probability of each suspicious sample; wherein the suspicious probability is determined based on the abnormal traffic data fragment samples, suspicious samples and abnormal traffic data samples at historical moments; and constructing the historical traffic data sample set with the abnormal traffic data samples at several historical moments, the normal traffic data samples at several historical moments and the suspicious samples at several historical moments.

[0074] Specifically, for abnormal traffic data samples at several historical moments, they are randomly decomposed into abnormal traffic data fragment samples at several historical moments, so that they form multiple fragment samples containing some abnormal features.

[0075] In a preferred embodiment, the method of randomly decomposing abnormal traffic data samples at several historical moments into abnormal traffic data fragment samples at several historical moments includes: performing stratified sampling on the abnormal traffic data samples at several historical moments to obtain selected abnormal traffic data samples of several groups; wherein the selected abnormal traffic data samples contained in each group have different abnormal features; fusing several selected abnormal traffic data samples in each group to obtain selected fused feature samples of several groups; and randomly intercepting the selected fused feature samples of each group to obtain abnormal traffic data fragment samples at several historical moments.

[0076] Specifically, firstly, stratified sampling is performed on the abnormal traffic data samples at historical moments. During stratified sampling, the abnormal features corresponding to the abnormal traffic data samples at historical moments contained in each group are different, that is, the abnormal traffic data samples at historical moments with several different abnormal features are contained in the same group, so that each group sample can contain more abnormal features. According to the determined grouping, the abnormal traffic data samples at historical moments in each group are fused to obtain selected fusion feature samples of several groups. For the selected fusion feature samples of each group, the selected fusion feature samples are randomly intercepted to obtain abnormal traffic data fragment samples at several historical moments. The fragment samples obtained in this way can carry one or more local abnormal features.

[0077] In a preferred embodiment, the method of fusing abnormal traffic data fragment samples at several historical moments with normal traffic data samples at several historical moments to obtain suspicious samples containing abnormal traffic data fragment samples at several historical moments and the suspicious probability of each suspicious sample includes: fusing abnormal traffic data fragment samples at several historical moments with normal traffic data samples at several historical moments to obtain suspicious samples containing abnormal traffic data fragment samples at several historical moments; determining a first subdivided suspicious probability according to the proportion of abnormal traffic data fragment samples in the suspicious samples at several historical moments to the abnormal traffic data samples at the corresponding historical moments; determining a second subdivided suspicious probability according to the proportion of abnormal traffic data fragment samples in the suspicious samples at several historical moments in the suspicious samples; and determining the suspicious probability of the suspicious samples at several historical moments according to each of the first subdivided suspicious probabilities and each of the second subdivided suspicious probabilities.

[0078] Specifically, the obtained fragment samples are fused with normal traffic data samples at historical moments to obtain several suspicious samples with local and diverse abnormal features; for the suspicious probability corresponding to each suspicious sample, the proportion of the abnormal traffic data fragment samples in the corresponding original abnormal traffic data samples is determined according to the abnormal traffic data fragment samples contained in the suspicious sample, so as to determine the first subdivided suspicious probability; if the abnormal traffic data fragment samples contain multiple abnormal features, the proportion of each abnormal feature in the corresponding abnormal traffic data sample is determined respectively, and the first subdivided suspicious probability is determined comprehensively with multiple proportion ratios; preferably, the determination method can be weighted fusion, median value, etc. Further, the second subdivided suspicious probability is determined according to the proportion of the abnormal traffic data fragment samples in the suspicious sample, and the first subdivided suspicious probability and the second subdivided suspicious probability are processed by weighted fusion to obtain the suspicious probability of the suspicious sample.

[0079] A historical wandering data sample set is constructed based on the obtained suspicious samples, abnormal traffic data samples and normal traffic data samples at several historical moments as training data for the first abnormal traffic detection model.

[0080] For step S201, an initial first abnormal traffic detection model is constructed, and the initial first abnormal traffic detection model is trained with the historical traffic data sample set constructed in step S101; in each iteration process, a sample in the historical traffic data sample set is used as input, and the data type corresponding to the sample is used as output, the initial first abnormal traffic detection model extracts the feature vector of the input sample, predicts the data type according to the feature vector, compares the predicted result with the actual result of the sample label, and adjusts the model parameters according to the prediction error; the error is gradually reduced through iterative training, and when the error is minimum or the maximum number of iterations is reached, the initial first abnormal traffic detection model converges to generate the first abnormal traffic detection model.

[0081] In a preferred embodiment, the initial first abnormal traffic detection model is iteratively trained with the historical traffic data sample set until the initial first abnormal traffic detection model converges to generate the first abnormal traffic detection model, including: taking abnormal traffic data samples at several historical moments as input, taking the data types corresponding to the abnormal traffic data samples at several historical moments as output, and iteratively training the first abnormal traffic detection model so that the first abnormal traffic detection model extracts the abnormal features corresponding to the abnormal traffic data samples at each historical moment; taking abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments as input, taking the data types corresponding to each sample as output, and iteratively training the first abnormal traffic detection model, and when the first abnormal traffic detection model converges, determining The first sample weight; taking suspicious samples at several historical moments as input and the data type corresponding to each suspicious sample as output, iteratively training the first abnormal traffic detection model, and determining the second sample weight when the first abnormal traffic detection model converges; wherein, when the suspicious probability is greater than a preset probability threshold, the data type corresponding to the suspicious sample is abnormal data; taking abnormal traffic data samples at several historical moments, normal traffic data samples at several historical moments and suspicious samples at several historical moments as input and the data type corresponding to each sample as output, iteratively training the first abnormal traffic detection model, and determining the third sample weight when the first abnormal traffic detection model converges; determining the fusion weight of the first abnormal traffic detection model according to the first sample weight, the second sample weight and the third sample weight.

[0082] Specifically, the training of the initial first abnormal traffic detection model mainly includes four rounds of training. First, the first abnormal traffic detection model is trained with abnormal traffic data samples at several historical moments as input and corresponding data types as output. In this round of training, the first abnormal traffic detection model extracts abnormal features of abnormal traffic data samples at each historical moment and learns each abnormal feature, so that the initial first abnormal traffic detection model memorizes the abnormal features.

[0083] After completing the abnormal feature memory learning, the abnormal traffic data samples and normal traffic data samples at historical moments are used as input, and the corresponding data types are used as output. At this time, the output data types can only be abnormal data and normal data. The initial first abnormal traffic detection model is trained so that it can have the ability to identify normal data and abnormal data through training, and at the same time, the first sample weight of the model determined during this training is determined. Further, the abnormal traffic detection capability of the initial first abnormal traffic detection model is trained. The suspicious samples at historical moments are used as input, and the data types corresponding to each suspicious sample are used as output. The first abnormal traffic detection model is trained so that the initial first abnormal traffic detection model can extract abnormal feature fragments from suspicious samples, determine their suspicious probability based on the abnormal feature fragments, and then judge the data type based on the suspicious probability, so as to further optimize the accuracy of the model for feature extraction. After the training is completed, the second sample weight is determined.

[0084] Finally, taking abnormal traffic data samples at several historical moments, normal traffic data samples at several historical moments and suspicious samples at several historical moments as input and the data type corresponding to each sample as output, the first abnormal traffic detection model is trained as a whole, and the third sample weight when the model converges is determined, and the fusion weight of the first abnormal traffic detection model is determined according to the first sample weights, the second sample weights and the third sample weights of multiple rounds of training. At this time, the training of the first abnormal traffic detection model is completed, and the abnormal feature database corresponding to the first abnormal traffic detection model stores all abnormal features involved in the training process and the impact duration corresponding to each abnormal feature. After the training is completed, the first abnormal traffic detection model will no longer be retrained during the application process.

[0085] For steps S3 and S4, when the flow data at the current moment does not exceed the preset flow threshold range, that is, when there is no obvious network intrusion feature, the flow data at the current moment is calibrated, calibrated as the second flow data to be detected, and monitored. During the monitoring process, when any second flow data to be detected has a data anomaly, all second flow data to be detected involved in the impact range of the data anomaly are obtained; it should be noted that as data that has an abnormality during the monitoring process, the abnormal feature is usually an unlearned feature, therefore, after monitoring the data anomaly, the network operation and maintenance personnel will usually investigate it to determine the abnormal feature, therefore, after knowing that the data anomaly has occurred, the corresponding abnormal feature (that is, the above-mentioned unlearned abnormal feature) can be obtained. The main purpose of the second abnormal flow detection model is to learn the unlearned abnormal feature and assist in updating the abnormal feature database of the first abnormal flow detection model.

[0086] In a preferred embodiment, the second abnormal traffic detection model is trained with all the second traffic data to be detected and the unlearned abnormal features within the influence range when the data is abnormal, so that the second abnormal traffic detection model outputs the unlearned abnormal features and the impact duration, and the abnormal feature database is updated with the unlearned abnormal features and the impact duration, including: constructing a training sample set according to all the second traffic data to be detected and the unlearned abnormal features within the influence range when the data is abnormal; wherein the training sample set includes a number of traffic data samples containing the unlearned abnormal features and a number of traffic data samples not containing the unlearned abnormal features; training the second abnormal traffic detection model with the training sample set until the second abnormal traffic detection model converges, and determining the weights to be fused; performing simulation operation with all the second traffic data to be detected and the unlearned abnormal features within the influence range when the data is abnormal, and determining the impact duration of the unlearned abnormal features; transmitting the unlearned abnormal features, the impact duration of the unlearned abnormal features, and the weights to be fused to the first abnormal traffic detection model, so that the first abnormal traffic detection model updates the fusion weights with the weights to be fused to obtain the updated fusion weights, and storing the unlearned abnormal features and the impact duration of the unlearned abnormal features in the abnormal feature database; initializing the second abnormal traffic detection model.

[0087] Specifically, when a new unlearned abnormal feature appears, a training sample set is constructed based on all the second flow data to be detected and the unlearned abnormal feature within the influence range when the data is abnormal; the training sample set includes the second flow data to be detected that does not contain the unlearned abnormal feature and the second flow data to be detected that contains the unlearned abnormal feature. The second abnormal flow detection model is trained with the training sample set, with any second flow data to be detected in the training sample set as input, and the corresponding data type as output, and the unlearned abnormal feature is learned during the training process, and when the second abnormal flow detection model converges, its weight to be fused is determined. Simulation operation is performed with all the second flow data to be detected and the unlearned abnormal feature within the influence range when the data is abnormal, and the impact duration of the unlearned abnormal feature is determined. The unlearned abnormal feature and the impact duration of the unlearned abnormal feature are transmitted to the abnormal feature database, and the weight to be fused is transmitted to the first abnormal flow detection model, so that the first abnormal flow detection model updates the fusion weight with the weight to be fused, and obtains the updated fusion weight, so as to realize the dynamic update of the abnormal feature of the first abnormal flow detection model. Preferably, the second flow data to be detected containing unlearned abnormal features is obtained and input into the first abnormal flow detection model to verify the accuracy of the first abnormal flow detection model in detecting the unlearned abnormal features. If the error is large, the network parameters of the first abnormal flow detection model are locally adjusted to reduce the error while ensuring that the accuracy of the detection of other abnormal features is not affected.

[0088] Initialize the network parameters of the second abnormal traffic detection model so that it can be used in the next learning of unlearned abnormal features.

[0089] The second abnormal traffic detection model can dynamically update the abnormal features of the first abnormal traffic detection model, making the first abnormal traffic detection model more generalizable. When there are unlearned abnormal features, the second abnormal traffic detection model learns the features, avoiding the need to retrain the first abnormal traffic detection model when data is updated, and reducing the loss of computing power.

[0090] For step S5, execute the corresponding network security policy according to the data type of the flow data at the current moment. In a preferred embodiment, the execution of the corresponding network security policy according to the data type of the flow data at the current moment includes: when the data type is normal data, release the isolation of the flow data at the current moment; when the data type is suspicious data, mark the flow data at the current moment, and monitor the flow data at the current moment within a preset period of time, if it is not converted into abnormal data within the preset period of time, release the isolation of the flow data at the current moment; when the data type is abnormal data, process the abnormal data according to the abnormal characteristics of the abnormal data, if the abnormal data can be converted into normal data after processing, release the isolation of the flow data at the current moment, if the abnormal data cannot be converted into normal data after processing, disinfect the flow data at the current moment.

[0091] By determining the data type and combining it with processing strategies to de-isolate data, abnormal data can be controlled in advance before it spreads, thereby improving network security.

[0092] Based on the above method embodiments, the present invention provides corresponding system embodiments.

[0093] like Figure 2 As shown, an embodiment of the present invention provides a real-time traffic security detection system based on network intrusion, including: a traffic data capture module, a detection module, a feature update module, a security management module and a first abnormal traffic detection model construction module;

[0094] The flow data capture module is used to capture the flow data at each moment in real time;

[0095] The detection module is used to detect the flow data at the current moment. If the flow data at the current moment exceeds the preset flow threshold range, the flow data at the current moment is marked as the first flow data to be detected, the first flow data to be detected is isolated, and input into the first abnormal flow detection model, so that the first abnormal flow detection model extracts the to-be-detected features of the first flow data to be detected, and outputs the data type and impact duration of the first flow data to be detected in combination with the abnormal feature database; wherein the data type includes; normal data, abnormal data and suspicious data; if the flow data at the current moment does not exceed the preset flow threshold range, the flow data at the current moment is marked as the second flow data to be detected;

[0096] The feature updating module is used to obtain all the second flow data to be detected and the unlearned abnormal features within the influence range of the data abnormality when monitoring any second flow data to be detected has a data abnormality, train the second abnormal flow detection model with all the second flow data to be detected and the unlearned abnormal features within the influence range of the data abnormality, so that the second abnormal flow detection model outputs the unlearned abnormal features and the impact duration, and update the abnormal feature database with the unlearned abnormal features and the impact duration;

[0097] The security control module is used to execute the corresponding network security strategy according to the data type of the flow data at the current moment;

[0098] The first abnormal traffic detection model construction module is used to obtain a historical traffic data sample set; wherein the historical traffic data sample set includes abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments; the abnormal traffic data sample at each historical moment includes abnormal features; an initial first abnormal traffic detection model is constructed, and the initial first abnormal traffic detection model is iteratively trained with the historical traffic data sample set until the initial first abnormal traffic detection model converges to generate the first abnormal traffic detection model; wherein, during the iteration process, the initial first abnormal traffic detection model extracts feature vectors from the historical traffic data sample set, determines the data type and impact duration based on the feature vectors, and adds the abnormal features and impact duration to the abnormal feature database.

[0099] It should be noted that the system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the system embodiment provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0100] Those skilled in the art can clearly understand that for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0101] Based on the above method item embodiments, the present invention provides corresponding terminal device item embodiments.

[0102] An embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a real-time traffic security detection method based on network intrusion as described in any one of the present invention is implemented.

[0103] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0104] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0105] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0106] Based on the above method item embodiments, the present invention provides a corresponding storage medium item embodiment.

[0107] An embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a real-time traffic security detection method based on network intrusion as described in any one of the present inventions.

[0108] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0109] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A real-time traffic security detection method based on network intrusion, characterized in that: include: Capture traffic data at every moment in real time; Detect the flow data at the current moment. If the flow data at the current moment exceeds the preset flow threshold range, mark the flow data at the current moment as the first flow data to be detected, isolate the first flow data to be detected, and input it into the first abnormal flow detection model, so that the first abnormal flow detection model extracts the features to be detected of the first flow data to be detected, and outputs the data type and impact duration of the first flow data to be detected in combination with the abnormal feature database; wherein the data types include normal data, abnormal data and suspicious data; If the flow data at the current moment does not exceed the preset flow threshold range, the flow data at the current moment is marked as the second flow data to be detected; When any second flow data to be detected is monitored to have a data anomaly, all second flow data to be detected and unlearned abnormal features within the influence range of the data anomaly are obtained, and the second abnormal flow detection model is trained with all second flow data to be detected and unlearned abnormal features within the influence range of the data anomaly, so that the second abnormal flow detection model outputs unlearned abnormal features and impact duration, and the abnormal feature database is updated with the unlearned abnormal features and impact duration; Execute the corresponding network security strategy according to the data type of the current traffic data; Wherein, the construction of the first abnormal traffic detection model includes: Obtain a historical traffic data sample set; wherein the historical traffic data sample set includes abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments; the abnormal traffic data sample at each historical moment includes abnormal features; Constructing an initial first abnormal traffic detection model, iteratively training the initial first abnormal traffic detection model with the historical traffic data sample set until the initial first abnormal traffic detection model converges, and generating the first abnormal traffic detection model; wherein, during the iteration process, the initial first abnormal traffic detection model extracts feature vectors from the historical traffic data sample set, determines the data type and impact duration according to the feature vector, and adds the abnormal feature and impact duration to the abnormal feature database; The method of training the second abnormal traffic detection model with all the second traffic data to be detected and the unlearned abnormal features within the influence range when the data is abnormal, so that the second abnormal traffic detection model outputs the unlearned abnormal features and the influence duration, and updating the abnormal feature database with the unlearned abnormal features and the influence duration, includes: constructing a training sample set according to all the second traffic data to be detected and the unlearned abnormal features within the influence range when the data is abnormal; wherein the training sample set includes a number of traffic data samples containing the unlearned abnormal features and a number of traffic data samples not containing the unlearned abnormal features; training the second abnormal traffic detection model with the training sample set until the second abnormal traffic detection model converges, and determining the weights to be fused; performing simulation operation with all the second traffic data to be detected and the unlearned abnormal features within the influence range when the data is abnormal, and determining the influence duration of the unlearned abnormal features; transmitting the unlearned abnormal features, the influence duration of the unlearned abnormal features and the weights to be fused to the first abnormal traffic detection model, so that the first abnormal traffic detection model updates the fusion weights with the weights to be fused to obtain the updated fusion weights, and storing the unlearned abnormal features and the influence duration of the unlearned abnormal features in the abnormal feature database; and initializing the second abnormal traffic detection model.

2. A real-time traffic security detection method based on network intrusion as claimed in claim 1, characterized in that: The step of obtaining a sample set of historical traffic data includes: Obtain abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments; Randomly decompose the abnormal traffic data samples at several historical moments into abnormal traffic data fragment samples at several historical moments; Merge abnormal traffic data fragment samples at several historical moments and normal traffic data samples at several historical moments to obtain suspicious samples containing abnormal traffic data fragment samples at several historical moments and the suspicious probability of each suspicious sample; wherein the suspicious probability is determined according to the abnormal traffic data fragment samples, the suspicious samples and the abnormal traffic data samples at the historical moments; A historical traffic data sample set is constructed using abnormal traffic data samples at several historical moments, normal traffic data samples at several historical moments, and suspicious samples at several historical moments.

3. A real-time traffic security detection method based on network intrusion as claimed in claim 2, characterized in that: Iteratively training the initial first abnormal traffic detection model with the historical traffic data sample set until the initial first abnormal traffic detection model converges to generate the first abnormal traffic detection model, including: Taking abnormal traffic data samples at several historical moments as input and data types corresponding to the abnormal traffic data samples at several historical moments as output, iteratively training the first abnormal traffic detection model so that the first abnormal traffic detection model extracts abnormal features corresponding to the abnormal traffic data samples at each historical moment; Taking abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments as input and taking the data type corresponding to each sample as output, iteratively training the first abnormal traffic detection model, and determining the first sample weight when the first abnormal traffic detection model converges; Taking suspicious samples at several historical moments as input and the data type corresponding to each suspicious sample as output, the first abnormal traffic detection model is iteratively trained, and when the first abnormal traffic detection model converges, the second sample weight is determined; wherein, when the suspicious probability is greater than a preset probability threshold, the data type corresponding to the suspicious sample is abnormal data; Taking abnormal traffic data samples at several historical moments, normal traffic data samples at several historical moments, and suspicious samples at several historical moments as input, and taking the data type corresponding to each sample as output, iteratively training the first abnormal traffic detection model, and determining a third sample weight when the first abnormal traffic detection model converges; The fusion weight of the first abnormal traffic detection model is determined according to the first sample weight, the second sample weight and the third sample weight.

4. A real-time traffic security detection method based on network intrusion as claimed in claim 3, characterized in that: The method of randomly decomposing abnormal traffic data samples at a number of historical moments into abnormal traffic data fragment samples at a number of historical moments includes: Performing stratified sampling on abnormal traffic data samples at several historical moments to obtain selected abnormal traffic data samples of several groups; wherein the abnormal characteristics of the selected abnormal traffic data samples included in each group are different; Fusing several selected abnormal traffic data samples in each group to obtain selected fusion feature samples of several groups; Randomly intercept the selected fusion feature samples of each group to obtain abnormal traffic data fragment samples at several historical moments.

5. A real-time traffic security detection method based on network intrusion as claimed in claim 4, characterized in that: The method of fusing abnormal traffic data fragment samples at several historical moments with normal traffic data samples at several historical moments to obtain suspicious samples containing abnormal traffic data fragment samples at several historical moments and the suspicious probability of each suspicious sample includes: Merge abnormal traffic data fragment samples at several historical moments with normal traffic data samples at several historical moments to obtain suspicious samples containing abnormal traffic data fragment samples at several historical moments; Determine the first segmented suspicious probability according to the proportion of abnormal traffic data fragment samples in the suspicious samples at several historical moments to the abnormal traffic data samples at the corresponding historical moments; Determine the second segmented suspicious probability according to the proportion of abnormal traffic data fragment samples in the suspicious samples at several historical moments; The suspicious probabilities of suspicious samples at a number of historical moments are determined according to each of the first subdivided suspicious probabilities and each of the second subdivided suspicious probabilities.

6. A real-time traffic security detection method based on network intrusion as claimed in claim 1, characterized in that: The executing of the corresponding network security strategy according to the data type of the flow data at the current moment includes: When the data type is normal data, the isolation of the current traffic data is released; When the data type is suspicious data, the current flow data is marked and monitored within a preset period. If it does not turn into abnormal data within the preset period, the isolation of the current flow data is lifted; When the data type is abnormal data, the abnormal data is processed according to the abnormal characteristics of the abnormal data. If the abnormal data can be converted into normal data after processing, the isolation of the traffic data at the current moment is released. If the abnormal data cannot be converted into normal data after processing, the traffic data at the current moment is eliminated.

7. A real-time traffic security detection system based on network intrusion, characterized in that: include: Traffic data capture module, detection module, feature update module, security management module and first abnormal traffic detection model construction module; The flow data capture module is used to capture the flow data at each moment in real time; The detection module is used to detect the flow data at the current moment. If the flow data at the current moment exceeds the preset flow threshold range, the flow data at the current moment is marked as the first flow data to be detected, the first flow data to be detected is isolated, and input into the first abnormal flow detection model, so that the first abnormal flow detection model extracts the to-be-detected features of the first flow data to be detected, and outputs the data type and impact duration of the first flow data to be detected in combination with the abnormal feature database; wherein the data type includes; normal data, abnormal data and suspicious data; if the flow data at the current moment does not exceed the preset flow threshold range, the flow data at the current moment is marked as the second flow data to be detected; The feature updating module is used to obtain all the second flow data to be detected and the unlearned abnormal features within the influence range of the data abnormality when monitoring any second flow data to be detected has a data abnormality, train the second abnormal flow detection model with all the second flow data to be detected and the unlearned abnormal features within the influence range of the data abnormality, so that the second abnormal flow detection model outputs the unlearned abnormal features and the impact duration, and update the abnormal feature database with the unlearned abnormal features and the impact duration; The security control module is used to execute the corresponding network security strategy according to the data type of the flow data at the current moment; The first abnormal traffic detection model construction module is used to obtain a historical traffic data sample set; wherein the historical traffic data sample set includes abnormal traffic data samples at several historical moments and normal traffic data samples at several historical moments; the abnormal traffic data sample at each historical moment includes abnormal features; construct an initial first abnormal traffic detection model, iteratively train the initial first abnormal traffic detection model with the historical traffic data sample set until the initial first abnormal traffic detection model converges, and generate the first abnormal traffic detection model; wherein, during the iteration process, the initial first abnormal traffic detection model extracts feature vectors from the historical traffic data sample set, determines the data type and impact duration according to the feature vectors, and adds the abnormal features and impact duration to the abnormal feature database; The method of training the second abnormal traffic detection model with all the second traffic data to be detected and the unlearned abnormal features within the influence range when the data is abnormal, so that the second abnormal traffic detection model outputs the unlearned abnormal features and the influence duration, and updating the abnormal feature database with the unlearned abnormal features and the influence duration, includes: constructing a training sample set according to all the second traffic data to be detected and the unlearned abnormal features within the influence range when the data is abnormal; wherein the training sample set includes a number of traffic data samples containing the unlearned abnormal features and a number of traffic data samples not containing the unlearned abnormal features; training the second abnormal traffic detection model with the training sample set until the second abnormal traffic detection model converges, and determining the weights to be fused; performing simulation operation with all the second traffic data to be detected and the unlearned abnormal features within the influence range when the data is abnormal, and determining the influence duration of the unlearned abnormal features; transmitting the unlearned abnormal features, the influence duration of the unlearned abnormal features and the weights to be fused to the first abnormal traffic detection model, so that the first abnormal traffic detection model updates the fusion weights with the weights to be fused to obtain the updated fusion weights, and storing the unlearned abnormal features and the influence duration of the unlearned abnormal features in the abnormal feature database; and initializing the second abnormal traffic detection model.

8. A terminal device, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a real-time traffic security detection method based on network intrusion is implemented as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a real-time traffic security detection method based on network intrusion as described in any one of claims 1 to 6.

Images