Encrypted traffic threat detection method and system based on deep learning

Abstract

The invention relates to the technical field of threat detection, in particular to an encrypted traffic threat detection method and system based on deep learning, and the method comprises the following steps: based on encrypted traffic data, carrying out the time sequence analysis of a data stream, extracting traffic behavior features, training an LSTM network through the features, matching the behavior pattern change, and carrying out the prediction analysis of future data. And obtaining a behavior pattern data set. According to the invention, through the long-short-term memory network, the capability of encrypted traffic threat detection is improved, traffic behavior characteristics can be extracted through time sequence analysis on the premise of not decrypting data, and a system is allowed to capture tiny changes of behavior patterns, so that abnormal behaviors hidden in encrypted traffic are effectively identified, and the security of the system is improved. The abnormal threshold is automatically adjusted through the abnormal score algorithm, the false alarm rate is reduced, the automation degree of data processing is improved, the LSTM network structure can automatically adapt to the change of the network environment through the genetic algorithm, the dynamic monitoring capability is enhanced, and the threat detection efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of threat detection technology, and in particular to an encrypted traffic threat detection method and system based on deep learning. Background Art

[0002] Threat detection is a core area of ​​network security technology, which aims to identify, evaluate and respond to various forms of network threats, such as malware, viruses, phishing attacks, and zero-day vulnerabilities. This field integrates a variety of technologies and methods, including signature-based detection, behavioral analysis, anomaly detection, and automated detection technology. Threat detection usually requires the ability to analyze large amounts of data in real time and quickly and accurately identify potential security threats. With the complexity of the network environment and the increase in encrypted traffic, it is also necessary to adapt to the parsing of encrypted data in order to effectively identify malicious activities hidden in encrypted communications.

[0003] Among them, the encrypted traffic threat detection method is a technology that identifies and responds to security threats in encrypted network traffic. In the modern network environment, most data transmissions use encryption technology to protect data privacy, but this also brings difficulties to traditional threat detection methods, because encrypted communications can easily hide malicious content. The encrypted traffic threat detection method analyzes the metadata, traffic patterns, time series and other information of encrypted data to detect potential malicious activities without decrypting the content. Its main purpose is to improve the level of network security protection, reduce the exploitation of security vulnerabilities, and protect personal and corporate data from network attackers.

[0004] Existing technologies mainly rely on signature-based detection and behavioral analysis, which have deficiencies when processing encrypted traffic, especially the limited ability to identify unknown threats and zero-day attacks. Existing methods are difficult to accurately evaluate behavioral patterns in encrypted data without decrypting the communication content, resulting in limited real-time and accuracy when processing large amounts of encrypted data. As a result, the level of network security protection is difficult to meet the needs of the modern network environment, and is prone to data leakage or system intrusion due to untimely response or inaccurate identification, posing serious risks to security protection. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings existing in the prior art and propose an encrypted traffic threat detection method and system based on deep learning.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: an encrypted traffic threat detection method based on deep learning, comprising the following steps:

[0007] S1: Based on encrypted traffic data, perform time series analysis on the data stream, extract traffic behavior features, use the features to train the LSTM network, match behavior pattern changes, and perform predictive analysis on future data to obtain a behavior pattern data set;

[0008] S2: Based on the behavior pattern data set, an anomaly score algorithm is used to calculate an anomaly score for each data packet, an anomaly threshold is automatically set and adjusted, and anomaly data is screened and marked to obtain an anomaly marked data packet;

[0009] S3: Based on the abnormal marked data packets, perform deep feature analysis on the data packets, extract key features, optimize the LSTM network structure using genetic algorithms, refine the abnormal detection capabilities, and obtain optimized detection configurations;

[0010] S4: Based on the optimized detection configuration, the threat detection capability of the LSTM network is tested using encrypted traffic data, the performance data is compared with the performance standard, and the LSTM network parameters are adjusted according to the test results to obtain an adjusted detection model.

[0011] The present invention is improved in that the step of extracting the traffic behavior characteristics is specifically as follows:

[0012] S111: Based on the encrypted traffic data, the autocorrelation function is used to perform time series analysis on the encrypted traffic data and perform window processing, using the formula:

[0013]

[0014] Get the window feature vector, where μ is the weighted average autocorrelation coefficient in the window, r j is the autocorrelation coefficient of the jth data point in the window, w is the window size, β j is the weight of the jth data point in the window;

[0015] S112: Based on the window feature vector, the skewness and kurtosis of each window are calculated using the formula:

[0016]

[0017] and

[0018]

[0019] The key statistical features are obtained, where S represents the skewness calculated from the window, K represents the kurtosis calculated from the window, and σ is the standard deviation of the data in the window.

[0020] The present invention is improved in that the steps of acquiring the behavior pattern data set are specifically as follows:

[0021] S121: Train the LSTM network using the features, which include mean, standard deviation, skewness and kurtosis, using the formula:

[0022]

[0023] Get the trained LSTM network model, where y i is the target output, V i is the i-th input feature vector, N is the total number of samples, and E is the average error during training;

[0024] S122: Using the trained LSTM network model, predict the feature vector of the new data set using the formula:

[0025]

[0026] Get predicted behavior patterns Among them, V new is the feature vector in the new data set, and M is the trained LSTM network model.

[0027] The present invention is improved in that the step of calculating the anomaly score is specifically as follows:

[0028] S211: Based on the behavior pattern data set, extract key features of the data packet, including flow size and duration, and calculate anomaly scores for the data packet using the formula:

[0029]

[0030] Get the abnormal score SE of the i-th data packet i , where βQ k is the weight of feature k, is the average value of feature k in the training data, is the kth eigenvalue in data packet i, K E is the total number of features;

[0031] S212: Statistically analyze the abnormal scores to determine the distribution characteristics of the abnormal scores, using the formula:

[0032]

[0033] and

[0034]

[0035] Get the average of the anomaly scores and standard deviation Among them, N E is the total number of packets, SE iis the anomaly score of the ith packet.

[0036] The present invention is improved in that the step of acquiring the abnormal marked data packet is specifically as follows:

[0037] S221: Automatically set an abnormality threshold using the calculated abnormality score, using the formula:

[0038]

[0039] Get the abnormal threshold T a ,in, is the average of the anomaly scores, is the standard deviation of the anomaly score, k a is the standard deviation multiplier;

[0040] S222: Based on the abnormal threshold, each data packet is screened and marked, and the abnormal score of the data packet is compared with the abnormal threshold, using the formula:

[0041]

[0042] Get the abnormal marked data packet, where is the abnormal flag of packet i, SE i is the anomaly score of packet i, T a is the abnormal threshold.

[0043] The present invention is improved in that the steps of performing the deep feature analysis are specifically as follows:

[0044] S311: Based on the abnormal marked data packet, extract preliminary features, including communication size, duration and protocol type, and calculate the feature index of each data packet using the formula:

[0045]

[0046] The characteristic index FD is obtained, where and is the feature weight, which is assigned according to the characteristics of the data packet. and are the communication size, duration and protocol type of the ith data packet, n D is the total number of packets;

[0047] S312: Based on the characteristic index, perform deep characteristic analysis to identify abnormal patterns using the formula:

[0048]

[0049] The deep analysis result FD′ is obtained to identify packets that deviate from the normal state, where FD i is the characteristic index of packet i, is the average value of the characteristic index, n D is the total number of packets;

[0050] S313: Based on the in-depth analysis results, select features that are higher or lower than the average level to determine key features.

[0051] The present invention is improved in that the step of obtaining the optimized detection configuration is specifically as follows:

[0052] S321: Using the genetic algorithm, adjust the LSTM network structure using the formula:

[0053] LH new =K H ·θ H

[0054] Get the new network parameters LH new , where K H Represents the value of the key feature set, θ H is the parameter vector;

[0055] S322: Based on the new network parameters, the performance of the optimized model is verified using the formula:

[0056]

[0057] The performance index PH of the model verification phase is obtained, where yh i is the current label value of the ith data point, is the label value predicted by the model for the i-th data point, n H is the total number of samples in the dataset;

[0058] S323: Based on the performance indicators of the model verification phase, the anomaly detection capability of the LSTM network model is refined to obtain an optimized detection configuration.

[0059] The present invention is improved in that the steps of obtaining the adjusted detection model are specifically as follows:

[0060] S411: Based on the optimized detection configuration, the threat detection capability of the LSTM network is tested using encrypted traffic data to determine whether the model before adjustment meets the performance standard, using the judgment formula:

[0061] CB=(SB t ≤∈ b )

[0062] The performance comparison result CB is obtained, where SB t is the average error in the test phase, ∈ b is the performance threshold;

[0063] S412: Based on the performance comparison result, the model parameters are adjusted using the gradient descent method, using the formula:

[0064]

[0065] Get the adjusted model parameters in, is the model parameter before adjustment, η B is the learning rate, is the gradient of the loss function J with respect to the parameter θ.

[0066] An encrypted traffic threat detection system based on deep learning, the system comprising:

[0067] The feature extraction module performs time series analysis on the data stream based on encrypted traffic data, extracts traffic behavior features, trains long-term and short-term memory networks, matches behavior pattern changes, and generates behavior pattern data sets;

[0068] The anomaly detection module calculates the anomaly score of each data packet based on the behavior pattern data set through an anomaly score algorithm, automatically sets and adjusts the anomaly threshold, and performs anomaly screening and marking of the data packet to generate an anomaly marked data packet;

[0069] The network optimization module performs in-depth feature analysis on the data packets based on the abnormally marked data packets, optimizes the LSTM network structure using a genetic algorithm, refines the abnormality detection performance, and generates an optimized detection configuration;

[0070] The performance evaluation module tests the threat detection performance of the long short-term memory network using encrypted traffic data based on the optimized detection configuration, compares the results with the performance standard, matches the current threat environment, and generates an adjusted detection model;

[0071] The configuration and deployment module evaluates the model effectiveness based on the adjusted detection model and key performance indicators, including network traffic and error rate, and determines the parameter settings before the detection model is deployed to generate the detection model deployment results.

[0072] Compared with the prior art, the advantages and positive effects of the present invention are:

[0073] In the present invention, the ability of encrypted traffic threat detection is improved through the long short-term memory network. The traffic behavior characteristics can be extracted through time series analysis without decrypting the data, allowing the system to capture slight changes in behavior patterns, thereby effectively identifying abnormal behaviors hidden in encrypted traffic. The anomaly threshold is automatically adjusted through the anomaly score algorithm to reduce the false alarm rate and improve the degree of automation of data processing. The genetic algorithm enables the LSTM network structure to automatically adapt to changes in the network environment, enhance the dynamic monitoring capability, and improve the efficiency and accuracy of threat detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A flowchart of a method for detecting encrypted traffic threats based on deep learning is proposed for the present invention;

[0075] Figure 2 It is a flow chart for extracting traffic behavior characteristics in the present invention;

[0076] Figure 3 A flowchart for obtaining a behavior pattern data set in the present invention;

[0077] Figure 4 A flowchart for calculating anomaly scores in the present invention;

[0078] Figure 5 This is a flow chart for obtaining abnormally marked data packets in the present invention;

[0079] Figure 6 A flowchart for performing deep feature analysis in the present invention;

[0080] Figure 7 A flowchart for obtaining the optimized detection configuration in the present invention;

[0081] Figure 8 This is a flow chart for obtaining the adjusted detection model in the present invention. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0083] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating positions or positional relationships, are based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0084] Example

[0085] See also Figure 1 The present invention provides a technical solution: an encrypted traffic threat detection method based on deep learning, comprising the following steps:

[0086] S1: Based on encrypted traffic data, perform time series analysis on the data stream, extract traffic behavior features, use the features to train the LSTM network, match behavior pattern changes, and perform predictive analysis on future data to obtain a behavior pattern data set;

[0087] S2: Based on the behavior pattern data set, the anomaly score algorithm is used to calculate the anomaly score for each data packet, automatically set and adjust the anomaly threshold, and perform anomaly data screening and marking to obtain anomaly marked data packets;

[0088] S3: Based on the abnormally marked data packets, perform in-depth feature analysis on the data packets, extract key features, use genetic algorithms to optimize the LSTM network structure, refine the anomaly detection capabilities, and obtain optimized detection configurations;

[0089] S4: Based on the optimized detection configuration, the threat detection capability of the LSTM network is tested using encrypted traffic data, the performance data is compared with the performance standard, and the LSTM network parameters are adjusted according to the test results to obtain the adjusted detection model.

[0090] The behavior pattern data set includes pattern change trends, abnormal activity time, and behavior consistency measurement. The abnormal marking data packet includes the marked timestamp, marked priority, and correlation scoring results. The optimized detection configuration includes hierarchical adjustment results, parameter adjustment records, and performance improvement points.

[0091] See also Figure 2 ,The specific steps for extracting traffic behavior features are as follows:

[0092] S111: Based on the encrypted traffic data, the autocorrelation function is used to perform time series analysis on the encrypted traffic data and perform window processing, using the formula:

[0093]

[0094] Get the window feature vector, where μ is the weighted average autocorrelation coefficient in the window, r j is the autocorrelation coefficient of the jth data point in the window, w is the window size, i.e. the number of consecutive data points, and β j is the weight of the jth data point in the window;

[0095] S112: Based on the window feature vector, the skewness and kurtosis of each window are calculated using the formula:

[0096]

[0097] and

[0098]

[0099] The key statistical features are obtained, where S represents the skewness calculated from the window, K represents the kurtosis calculated from the window, and σ is the standard deviation of the data in the window.

[0100] β j Defined as

[0101] The resulting autocorrelation coefficient array is:

[0102] {-0.2485,0.1902,-0.0534,0.3001,-0.1163}

[0103] Select window size w=3 and set γ=0.5;

[0104] The process of calculating the weighted average μ with the window center at r2 is as follows:

[0105] Calculating Beta j :

[0106]

[0107] β2=1(center point)

[0108] β3=0.8889(symmetric weight)

[0109] Calculate μ:

[0110] μ=β1×r1+β2×r2+β3×r3

[0111] μ=0.8×(-0.2485)+1×0.1902+0.8×(-0.0534)

[0112] μ=-0.1988+0.1902-0.04272≈-0.05132

[0113] The average autocorrelation coefficient μ in the window is -0.0781, indicating that within this window, the correlation of the time series tends to be negatively correlated, but the effect is relatively mild. This indicator can help identify local characteristics of data behavior.

[0114] The calculation method of σ is

[0115] Using the above data and, the process of calculating skewness S and kurtosis K is as follows:

[0116] Calculate σ:

[0117] σ 2 =β1×(r1-μ) 2 +β2×(r2-μ) 2 +β3×(r3-μ) 2

[0118] σ 2 =0.8×(-0.2485+0.0781) 2 +1×(0.1902+0.0781) 2

[0119] +0.8×(-0.0534+0.0781) 2

[0120] σ 2 =0.8×0.0290 2 +1×0.2683 2 +0.8×0.0247 2

[0121] σ 2 =0.8×0.000841+0.072019+0.8×0.000610≈0.073

[0122]

[0123] Calculate S and K:

[0124]

[0125] The results show a skewness of S ≈ 0.7803 and a kurtosis of K ≈ -2.306, indicating that within the selected window, the distribution of the autocorrelation coefficient is slightly biased towards negative values, the distribution is relatively flat and has a slight peak, which helps to identify abnormal behaviors or prominent features in the data.

[0126] See also Figure 3 ,The specific steps for obtaining the behavior pattern dataset are:

[0127] S121: Train the LSTM network using features including mean, standard deviation, skewness and kurtosis using the formula:

[0128]

[0129] Get the trained LSTM network model, where y i is the target output, V i is the i-th input feature vector, N is the total number of samples, and E is the average error during training;

[0130] S122: Use the trained LSTM network model to predict the feature vector of the new data set using the formula:

[0131]

[0132] Get predicted behavior patterns Among them, V new is the feature vector in the new data set, and M is the trained LSTM network model.

[0133] There are 5 observations with data points:

[0134] y={2,4,5,6,7}

[0135] And the corresponding predicted value:

[0136]

[0137] The process of calculating the average error is as follows:

[0138] Calculate the squared error for each data point:

[0139] (2-2.1) 2 =0.01

[0140] (4-3.9) 2 =0.01

[0141] (5-4.8) 2 =0.04

[0142] (6-6.1) 2 =0.01

[0143] (7-7.2) 2 =0.04

[0144] Sum the squared errors:

[0145] 0.01+0.01+0.04+0.01+0.04=0.11

[0146] Calculate the average error E:

[0147]

[0148] The result shows that the average error of the LSTM model on the data set is 0.022, which means that the model can fit the actual data well with a small error.

[0149] The feature vector V of the new data collected new ={3.5,2.5,0.5,1.0};

[0150] Use this vector to make predictions in the trained LSTM model:

[0151]

[0152] Based on the current data characteristics, the model predicts future behavior with an output of 5.5, representing a quantitative assessment of a specific behavioral pattern or trend.

[0153] See also Figure 4 , the specific steps for calculating the anomaly score are:

[0154] S211: Based on the behavior pattern data set, extract the key features of the data packet, including the traffic size and duration, and calculate the anomaly score of the data packet using the formula:

[0155]

[0156] Get the abnormal score SE of the i-th data packet i , where βQ k is the weight of feature k, is the average value of feature k in the training data, is the kth eigenvalue in data packet i, K E is the total number of features;

[0157] S212: Statistically analyze the abnormal scores to determine the distribution characteristics of the abnormal scores using the formula:

[0158]

[0159] and

[0160]

[0161] Get the average of the anomaly scores and standard deviation Among them, N E is the total number of packets, SE i is the anomaly score of the ith packet.

[0162] Three features are collected: packet size, duration, and signal strength, and the corresponding weights βQ k They are 0.5, 0.3, and 0.2 respectively;

[0163] The weights are based on data analysis, where packet size has the greatest variability and therefore the highest weight, and the historical average for each feature They are 100 bytes, 50 ms, -70 dBm respectively;

[0164] There is a data packet i, whose characteristic value They are 110 bytes, 45ms, and -75dBm respectively.

[0165] Fractional calculation of packet size:

[0166]

[0167] Fractional calculation of duration:

[0168]

[0169] Score calculation for signal strength:

[0170]

[0171] Total score SE i for:

[0172] SE i =50+7.5+5=62.5

[0173] Calculation result SE i = 62.5 indicates the degree of deviation of packet i from normal behavior. A high anomaly score indicates that the packet may be abnormal and requires further review or action.

[0174] βQ3 is the weight of the signal strength feature, which is 0.2;

[0175] is the signal strength characteristic value of the current data packet, which is -75dBm;

[0176] It is the average value of signal strength in historical data, which is -70dBm.

[0177] Calculate the deviation between the signal strength of the current packet and the historical average:

[0178]

[0179] Indicates that the signal strength of the current data packet is 5dBm lower than the average value.

[0180] Square the deviation values ​​to remove the negative sign and amplify the difference:

[0181] (-5) 2 =25

[0182] The squared value is 25, which indicates the quantization of the deviation, regardless of the direction of the deviation (positive or negative).

[0183] Multiply the calculated squared deviation by the weight of the signal strength to get the weighted anomaly score for the feature:

[0184] βQ3·25=0.2·25=5

[0185] The result of multiplying the weight by the square of the deviation is 5, which is the contribution to the overall anomaly score;

[0186] The resulting anomaly score value of 5 indicates that the packet has a significant deviation in signal strength from the historical average.

[0187] See also Figure 5 ,The specific steps for obtaining abnormal marked data packets are:

[0188] S221: Use the calculated anomaly score to automatically set the anomaly threshold using the formula:

[0189]

[0190] Get the abnormal threshold T a ,in, is the average of the anomaly scores, is the standard deviation of the anomaly score, k a is the standard deviation multiplier;

[0191] S222: Based on the abnormal threshold, each data packet is screened and marked, and the abnormal score of the data packet is compared with the abnormal threshold, using the formula:

[0192]

[0193] Get the abnormal marked data packet, where is the abnormal flag of packet i, SE i is the anomaly score of packet i, T a is the abnormal threshold.

[0194] The anomaly scores obtained from 100 packets are as follows (100 values ​​in total):

[0195] SE i ={20,22,19,18,30,...}

[0196] Calculate the average

[0197]

[0198] Assume that the calculation result is

[0199] Calculating the standard deviation

[0200]

[0201] Assume that the calculation result is

[0202] Set the threshold T a :

[0203] T a =25+3·5=40

[0204] Select k a = 3 to ensure that the threshold is high enough to reduce false positives.

[0205] T a =40 means that any packet with anomaly score higher than 40 will be considered abnormal;

[0206] Since the threshold is 40, the collected anomaly scores are (42, 35, 45, 25, 50):

[0207] Packet score 42: 42 ≥ 40 = 1, abnormal;

[0208] Packet score 35: 35 ≥ 40 = 0, normal;

[0209] Packet score 45: 45 ≥ 40 = 1, abnormal;

[0210] Packet score 25: 25 ≥ 40 = 0, normal;

[0211] Packet score 50: 50 ≥ 40 = 1, abnormal;

[0212] This process marks which packets are abnormal, providing a basis for further analysis.

[0213] See also Figure 6 , the specific steps for deep feature analysis are:

[0214] S311: Based on the abnormally marked data packets, preliminary features are extracted, including communication size, duration, and protocol type, and the feature index of each data packet is calculated using the formula:

[0215]

[0216] The characteristic index FD is obtained, where and is the feature weight, which is assigned according to the characteristics of the data packet. and are the communication size, duration and protocol type of the ith data packet, n D is the total number of packets;

[0217] S312: Based on the feature index, perform deep feature analysis to identify abnormal patterns using the formula:

[0218]

[0219] The deep analysis result FD′ is obtained to identify packets that deviate from the normal state, where FD i is the characteristic index of packet i, is the average value of the characteristic index, n D is the total number of packets;

[0220] S313: Based on the in-depth analysis results, select features that are above or below average and determine key features.

[0221] The monitoring information of three data packets is as follows:

[0222] Packet 1:

[0223]

[0224] Packet 2:

[0225]

[0226] Packet 3:

[0227]

[0228] The weight is set to

[0229] Calculate FD:

[0230] FD=(0.5×1500+1.5×0.2+1×1)+(0.5×1000+1.5×0.1+1×2)

[0231] +(0.5×1200+1.5×0.3+1×1)

[0232] FD=(750+0.3+1)+(500+0.15+2)+(600+0.45+1)

[0233] FD=751.3+502.15+601.45

[0234] FD=1854.9

[0235] FD=1854.9 indicates the weighted sum of the features of the three data packets, representing the feature strength of the overall data packet.

[0236] Using the aforementioned combination and 3 packets of data;

[0237] Calculate the average

[0238]

[0239] Calculate the variance:

[0240]

[0241] The results show that the dispersion of feature values ​​is quite high, pointing to some abnormal activities or behaviors in the data packets. In network security and monitoring systems, feature data with high variance may indicate network attacks, data leaks or other security threats.

[0242] See also Figure 7 , the steps to obtain the optimized detection configuration are as follows:

[0243] S321: Use genetic algorithm to adjust the LSTM network structure using the formula:

[0244] LH new =K H ·θ H

[0245] Get the new network parameters LH new , where K H Represents the value of the key feature set, θ H is the parameter vector;

[0246] S322: Based on the new network parameters, the performance of the optimized model is verified using the formula:

[0247]

[0248] The performance index PH of the model verification phase is obtained, where yh i is the current label value of the ith data point, is the label value predicted by the model for the i-th data point, n H is the total number of samples in the dataset;

[0249] S323: Based on the performance indicators of the model verification phase, the anomaly detection capability of the LSTM network model is refined to obtain an optimized detection configuration.

[0250] Let K Hand θ H The dimensions of can be matched, and the dot multiplication operation can be performed directly. The specific values ​​are as follows:

[0251] K H =[1.5,2.0,0.5]

[0252] Represents the importance score of key features extracted from abnormal data packets;

[0253] θ H =[0.5,1.5,0.75]

[0254] Represents the optimization parameters obtained by the genetic algorithm, and the parameters correspond to the adjustment weights of different features

[0255] Performing a dot product operation, the adjusted weight of each feature is multiplied by its importance score:

[0256] LH new =[1.5×0.5,2.0×1.5,0.5×0.75]=[0.75,3.0,0.375]

[0257] The calculation results [0.75, 3.0, 0.375] show the weight of each feature in the optimized model, which will directly affect the model's analysis and prediction of the data.

[0258] The data set for model validation contains 100 data points;

[0259] The actual and predicted labels for the data points are shown below:

[0260] The value of the current label yh is [1, 0, 1, 1, 0, 1, 1, 0, …, 1];

[0261] Model predictions The value of is [1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, …, 1];

[0262] Suppose the differences between the model predictions and the current values ​​are as follows (first 10):

[0263] difference Values:

[0264] [0, -1, 0, 1, 0, 0, 0, 1, -1, …, 0]

[0265] Calculate the sum of squared errors:

[0266]

[0267] Substitute this value into the formula to calculate the performance index PH:

[0268]

[0269] The lower pH value indicates that the prediction error of the model is relatively small, indicating that the model has good prediction ability.

[0270] See also Figure 8 , the specific steps for obtaining the adjusted detection model are:

[0271] S411: Based on the optimized detection configuration, the threat detection capability of the LSTM network is tested using encrypted traffic data to determine whether the model before adjustment meets the performance standards. The judgment formula is:

[0272] CB=(SB t ≤∈ b )

[0273] The performance comparison result CB is obtained, where SB t is the average error in the test phase, ∈ b is the performance threshold;

[0274] S412: Based on the performance comparison results, the model parameters are adjusted using the gradient descent method using the formula:

[0275]

[0276] Get the adjusted model parameters in, is the model parameter before adjustment, η B is the learning rate, is the gradient of the loss function J with respect to the parameter θ.

[0277] In the actual test, the prediction results of the model on 100 samples were compared with the actual data. The specific values ​​are as follows:

[0278] SB t =0.15

[0279] ∈ b =0.1

[0280] Judgment process:

[0281] CB=(0.15≤0.1)

[0282] CB=False

[0283] A false comparison result means that the performance of the model does not meet the predetermined standard, so further parameter adjustments are required to optimize the model.

[0284] Obtained from a model configuration or a previous optimization iteration;

[0285] ηB Determined through the debugging phase of the model to achieve fast and stable convergence;

[0286] Calculated during model training using the back-propagation algorithm;

[0287] The following data was collected:

[0288]

[0289] η B =0.01;

[0290]

[0291] Calculation process:

[0292]

[0293] The new parameter values ​​represent the fine-tuned model parameters, which help improve the prediction performance of the model and ensure the effectiveness and reliability of the detection model in practical applications.

[0294] The encrypted traffic threat detection system based on deep learning includes:

[0295] The feature extraction module performs time series analysis on the data stream based on encrypted traffic data, extracts traffic behavior features, trains long-term and short-term memory networks, matches behavior pattern changes, and generates behavior pattern data sets;

[0296] The anomaly detection module calculates the anomaly score of each data packet based on the behavior pattern data set through the anomaly score algorithm, automatically sets and adjusts the anomaly threshold, and performs anomaly screening and marking of data packets to generate anomaly marked data packets;

[0297] The network optimization module performs in-depth feature analysis on data packets based on anomaly-marked data packets, optimizes the LSTM network structure using genetic algorithms, refines anomaly detection performance, and generates optimized detection configurations;

[0298] The performance evaluation module tests the threat detection performance of the LSTM network based on the optimized detection configuration and using encrypted traffic data, compares the results with the performance criteria, matches the current threat environment, and generates an adjusted detection model;

[0299] The configuration and deployment module evaluates the model effectiveness based on the adjusted detection model and key performance indicators, including network traffic and error rate, and determines the parameter settings before the detection model is deployed to generate the detection model deployment results.

[0300] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. The encrypted traffic threat detection method based on deep learning is characterized by: The following steps are involved: Based on encrypted traffic data, we conduct time series analysis on the data stream, extract traffic behavior features, use the features to train the LSTM network, match the behavior pattern changes, and perform predictive analysis on future data to obtain a behavior pattern data set. Based on the behavior pattern data set, an anomaly score algorithm is used to calculate an anomaly score for each data packet, an anomaly threshold is automatically set and adjusted, and anomaly data is screened and marked to obtain an anomaly marked data packet; Based on the abnormal marked data packets, deep feature analysis is performed on the data packets to extract key features, and the LSTM network structure is optimized using a genetic algorithm to refine the abnormal detection capability and obtain an optimized detection configuration; Based on the optimized detection configuration, the threat detection capability of the LSTM network is tested using encrypted traffic data, the performance data is compared with the performance standard, and the LSTM network parameters are adjusted according to the test results to obtain an adjusted detection model.

2. The encrypted traffic threat detection method based on deep learning according to claim 1 is characterized in that: The steps of extracting the traffic behavior features are specifically as follows: Based on the encrypted traffic data, the autocorrelation function is used to perform time series analysis on the encrypted traffic data and perform window processing using the formula: Get the window feature vector, where μ is the weighted average autocorrelation coefficient in the window, r j is the autocorrelation coefficient of the jth data point in the window, w is the window size, β j is the weight of the jth data point in the window; Based on the window feature vector, the skewness and kurtosis of each window are calculated using the formula: and The key statistical features are obtained, where S represents the skewness calculated from the window, K represents the kurtosis calculated from the window, and σ is the standard deviation of the data in the window.

3. The encrypted traffic threat detection method based on deep learning according to claim 1 is characterized in that: The steps for obtaining the behavior pattern data set are specifically as follows: The LSTM network is trained using the features, including mean, standard deviation, skewness and kurtosis, using the formula: Get the trained LSTM network model, where y i is the target output, V i is the i-th input feature vector, N is the total number of samples, and E is the average error during training; Using the trained LSTM network model, the feature vector of the new data set is predicted using the formula: Get predicted behavior patterns Among them, V new is the feature vector in the new data set, and M is the trained LSTM network model.

4. The encrypted traffic threat detection method based on deep learning according to claim 1 is characterized in that: The steps of calculating the anomaly score are specifically as follows: Based on the behavior pattern dataset, the key features of the data packet, including the traffic size and duration, are extracted, and the anomaly score of the data packet is calculated using the formula: Get the abnormal score SE of the i-th data packet i , where βQ k is the weight of feature k, is the average value of feature k in the training data, is the kth eigenvalue in data packet i, K E is the total number of features; The anomaly scores are statistically analyzed to determine the distribution characteristics of the anomaly scores, using the formula: and Get the average of the anomaly scores and standard deviation Among them, N E is the total number of packets, SE i is the anomaly score of the ith packet.

5. The encrypted traffic threat detection method based on deep learning according to claim 1 is characterized in that: The steps of obtaining the abnormal marking data packet are specifically as follows: Using the calculated anomaly score, the anomaly threshold is automatically set using the formula: Get the abnormal threshold T a ,in, is the average of the anomaly scores, is the standard deviation of the anomaly score, k a is the standard deviation multiplier; Based on the anomaly threshold, each data packet is screened and marked, and the anomaly score of the data packet is compared with the anomaly threshold, using the formula: Get the abnormal marked data packet, where is the abnormal flag of packet i, SE i is the anomaly score of packet i, T a is the abnormal threshold.

6. The encrypted traffic threat detection method based on deep learning according to claim 1 is characterized in that: The steps of performing deep feature analysis are specifically as follows: Based on the abnormally marked data packets, preliminary features are extracted, including communication size, duration and protocol type, and the feature index of each data packet is calculated using the formula: The characteristic index FD is obtained, where and is the feature weight, which is assigned according to the characteristics of the data packet. and are the communication size, duration and protocol type of the ith data packet, n D is the total number of packets; Based on the characteristic index, deep feature analysis is performed to identify abnormal patterns using the formula: The deep analysis result FD′ is obtained to identify packets that deviate from the normal state, where FD i is the characteristic index of packet i, is the average value of the characteristic index, n D is the total number of packets; Based on the in-depth analysis results, features that are above or below average are selected to determine key features.

7. The encrypted traffic threat detection method based on deep learning according to claim 1 is characterized in that: The steps for obtaining the optimized detection configuration are specifically as follows: Using the genetic algorithm, the LSTM network structure is adjusted using the formula: LH new =K H ·i H Get the new network parameters LH new , where K H Represents the value of the key feature set, θ H is the parameter vector; Based on the new network parameters, the performance of the optimized model is verified using the formula: The performance index PH of the model verification phase is obtained, where yh i is the current label value of the ith data point, is the label value predicted by the model for the i-th data point, n H is the total number of samples in the dataset; Based on the performance indicators of the model verification phase, the anomaly detection capability of the LSTM network model is refined to obtain an optimized detection configuration.

8. The encrypted traffic threat detection method based on deep learning according to claim 1 is characterized in that: The steps for obtaining the adjusted detection model are specifically as follows: Based on the optimized detection configuration, the threat detection capability of the LSTM network is tested using encrypted traffic data to determine whether the model before adjustment meets the performance standards. The judgment formula is: CB=(SB t ≤∈ b ) The performance comparison result CB is obtained, where SB t is the average error in the test phase, ∈ b is the performance threshold; Based on the performance comparison results, the model parameters are adjusted using the gradient descent method using the formula: Get the adjusted model parameters in, is the model parameter before adjustment, η B is the learning rate, is the gradient of the loss function J with respect to the parameter θ.

9. The encrypted traffic threat detection system based on deep learning is characterized by: According to the method for detecting encrypted traffic threats based on deep learning according to any one of claims 1 to 8, the system comprises: The feature extraction module performs time series analysis on the data stream based on encrypted traffic data, extracts traffic behavior features, trains long-term and short-term memory networks, matches behavior pattern changes, and generates behavior pattern data sets; The anomaly detection module calculates the anomaly score of each data packet based on the behavior pattern data set through an anomaly score algorithm, automatically sets and adjusts the anomaly threshold, and performs anomaly screening and marking of the data packet to generate an anomaly marked data packet; The network optimization module performs in-depth feature analysis on the data packets based on the abnormally marked data packets, optimizes the LSTM network structure using a genetic algorithm, refines the abnormality detection performance, and generates an optimized detection configuration; The performance evaluation module tests the threat detection performance of the long short-term memory network using encrypted traffic data based on the optimized detection configuration, compares the results with the performance standard, matches the current threat environment, and generates an adjusted detection model; The configuration and deployment module evaluates the model effectiveness based on the adjusted detection model and key performance indicators, including network traffic and error rate, and determines the parameter settings before the detection model is deployed to generate the detection model deployment results.

Images