A method, device, equipment and medium for detecting abnormal network traffic

Abstract

The present application discloses a method, apparatus, device and medium for detecting abnormal network traffic, which relates to the field of computer technology, and includes: obtaining network traffic data to be detected, and using the exponential smoothing method to calculate the weighted average of the network traffic data to obtain the predicted value of the network traffic at the next time point; determining the current service cycle value based on the preset sliding time window size, and using the exponential smoothing method to predict the service cycle factor corresponding to the current service cycle value; predicting the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size and the service cycle factor, and evaluating the network traffic predicted value and the corresponding network traffic observed value to obtain a scoring value; determining whether the scoring value exceeds a preset scoring threshold, and if so, determining that there is abnormal network traffic data in the network traffic data, and outputting the abnormal network traffic data. It can detect abnormal network traffic for networks with periodic changes.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and particularly to a method, apparatus, device and medium for detecting abnormal network traffic. Background Art

[0002] The exponential smoothing method is a commonly used method in production forecasting. It is also used for forecasting medium- and short-term economic development trends. Among all forecasting methods, exponential smoothing is the most widely used. The simple full-period average method equally utilizes all past data in a time series without omission; the moving average method does not consider data in the more distant past and gives greater weight to recent data in the weighted moving average method; while the exponential smoothing method combines the advantages of the full-period average and the moving average, does not discard past data, but only gives a gradually weakening degree of influence, that is, as the data moves away, weights that gradually converge to zero are assigned. That is to say, the exponential smoothing method is a time series analysis and forecasting method developed on the basis of the moving average method. It forecasts the future of a phenomenon by calculating the exponential smoothing value and matching a certain time series forecasting model. Its principle is that the exponential smoothing value of any period is the weighted average of the actual observed value of this period and the exponential smoothing value of the previous period.

[0003] According to the number of smoothing times, the exponential smoothing method is divided into: the first-order exponential smoothing method, the second-order exponential smoothing method, the third-order exponential smoothing method, etc. The basic idea is that the predicted value is the weighted sum of previous observed values, and different weights are given to different data, with larger weights for new data and smaller weights for old data. The selection of the exponential smoothing method can generally be determined according to the trend presented by the scatter plot of the original sequence. For example, if a linear trend is presented, the second-order exponential smoothing method is selected; if a parabolic trend is presented, the third-order exponential smoothing method is selected; or, when there is still curvature after the data of the time series is processed by the second-order exponential smoothing, the third-order exponential smoothing method is applied. In the calculation of the exponential smoothing method, the key is the value of the exponential coefficient, but the value of the exponential coefficient is easily affected subjectively. Therefore, it is very important to reasonably determine the value of the exponential coefficient. Generally, if the data fluctuates greatly, a larger value should be taken for the exponential coefficient to increase the influence of recent data on the prediction result; if the data fluctuates smoothly, a smaller value should be taken for the value of the exponential coefficient.

[0004] However, traditional single-layer or shallow statistical function models, such as single exponential smoothing, moving average model, etc., are difficult to be applied to business scenarios with periodic variation rules, such as network attack changes occurring during day-night cycles or special business hours, due to their low statistical level and small number of learnable parameters. For more advanced artificial intelligence or machine learning methods, for continuous signals such as network traffic, in addition to the large number of parameters to be solved, high algorithm complexity, and large computational resource overhead, a certain number of continuous signal sampling samples are usually required to accumulate the dataset to achieve model generalization, and a large amount of manpower is needed to continuously adjust some hyperparameters in the proprietary business scenario for the detection effect. This is not practical for a situation awareness system with high real-time requirements and other concurrent businesses to deploy large AI models.

[0005] In summary, how to perform anomaly detection on network traffic with business scenario periodicity and improve the robustness of the detection is an issue to be solved currently. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a network traffic anomaly detection method, device, equipment and medium, which can perform anomaly detection on network traffic with business scenario periodicity and improve the robustness of the detection. The specific solutions are as follows:

[0007] In a first aspect, the present application discloses a network traffic anomaly detection method, including:

[0008] Obtain the network traffic data to be detected, and use the exponential smoothing method to perform weighted average calculation on the network traffic data to obtain the predicted value of the network traffic at the next time point;

[0009] Determine the current business cycle value based on the preset sliding time window size, and use the exponential smoothing method to predict the business cycle factor corresponding to the current business cycle value;

[0010] Predict the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size and the business cycle factor to obtain a network traffic prediction value, and evaluate the network traffic prediction value and the corresponding network traffic observation value to obtain a score value;

[0011] Judge whether the score value exceeds a preset score threshold. If it exceeds, determine that there is abnormal network traffic data in the network traffic data, and output the abnormal network traffic data.

[0012] Optionally, the obtaining the network traffic data to be detected includes:

[0013] Obtain the network traffic data to be detected under different scenario types; wherein, the scenario types include any one or several of network session fluctuation scenario, network failure scenario, server communication scenario, server downtime scenario, and DDOS attack scenario.

[0014] Optionally, the determining the current service cycle value based on the preset sliding time window size includes:

[0015] Determine the service environment parameters under the current scenario type;

[0016] Based on the preset sliding time window size and the service environment parameters, determine the current service cycle value.

[0017] Optionally, the using the exponential smoothing method to perform weighted average calculation on the network traffic data includes:

[0018] Use the exponential smoothing method and perform weighted average calculation on the network traffic data based on the first exponential coefficient;

[0019] Correspondingly, the using the exponential smoothing method to predict the service cycle factor corresponding to the current service cycle value includes:

[0020] Use the exponential smoothing method and predict the service cycle factor corresponding to the current service cycle value based on the second exponential coefficient.

[0021] Optionally, the predicting the network traffic after a preset number of future cycles to obtain the network traffic prediction value based on the predicted value at the next time point, the sliding time window size, and the service cycle factor includes:

[0022] Determine whether to introduce a trend factor according to the current service environment; the trend factor includes a third exponential coefficient;

[0023] If introduced, based on the trend factor, the predicted value at the next time point, the sliding time window size, and the service cycle factor, predict the network traffic after a preset number of future cycles to obtain the network traffic prediction value.

[0024] Optionally, the network traffic anomaly detection method further includes:

[0025] Pre-obtain the network traffic training data under different scenarios, and use the exponential smoothing method to predict the network traffic training data to determine the training data prediction values including the first exponential coefficient, the second exponential coefficient, and the third exponential coefficient;

[0026] Determine the prediction error between the predicted value of the training data and the observed value of the corresponding training data, and solve it using a grid search method based on the stochastic gradient descent method based on the prediction error to determine the values of the first exponential coefficient, the second exponential coefficient, and the third exponential coefficient corresponding to the minimum prediction error.

[0027] Optionally, the output of the abnormal network traffic data includes:

[0028] Determine the target time point corresponding to the abnormal network traffic data, and determine the abnormal level corresponding to the score value according to the pre-set abnormal level division rule;

[0029] Output data information including the abnormal traffic data, the target time point, and the abnormal level, and visually display the data information according to the preset display rule.

[0030] In a second aspect, the present application discloses a network traffic anomaly detection device, including:

[0031] A first exponential smoothing processing module, configured to obtain network traffic data to be detected, and perform weighted average calculation on the network traffic data using the exponential smoothing method to obtain a predicted value of the network traffic at the next time point;

[0032] A second exponential smoothing processing module, configured to determine the current service cycle value based on a pre-set sliding time window size, and predict the service cycle factor corresponding to the current service cycle value using the exponential smoothing method;

[0033] A network traffic prediction module, configured to predict the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size, and the service cycle factor to obtain a network traffic prediction value, and evaluate the network traffic prediction value and the corresponding network traffic observed value to obtain a score value;

[0034] An anomaly detection module, configured to determine whether the score value exceeds a preset score threshold. If it exceeds, it is determined that there is abnormal network traffic data in the network traffic data, and the abnormal network traffic data is output.

[0035] In a third aspect, the present application discloses an electronic device, including:

[0036] A memory, configured to store a computer program;

[0037] A processor, configured to execute the computer program to implement the steps of the network traffic anomaly detection method disclosed above.

[0038] Fourthly, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the network traffic anomaly detection method disclosed above are implemented.

[0039] It can be seen that the present application obtains the network traffic data to be detected, and uses the exponential smoothing method to perform weighted average calculation on the network traffic data to obtain the predicted value of the network traffic at the next time point; determines the current service cycle value based on the preset sliding time window size, and uses the exponential smoothing method to predict the service cycle factor corresponding to the current service cycle value; predicts the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size, and the service cycle factor to obtain the network traffic predicted value, and evaluates the network traffic predicted value and the corresponding network traffic observed value to obtain a score value; determines whether the score value exceeds a preset score threshold, and if it exceeds, determines that there is abnormal network traffic data in the network traffic data, and outputs the abnormal network traffic data. Thus, it can be seen that the present application first obtains the predicted value of the network traffic at the next time point through the exponential smoothing method, and predicts the service cycle factor based on the sliding time window size using the exponential smoothing method, then further predicts the network traffic after a preset number of future cycles to obtain the network traffic predicted value, and then evaluates the network traffic predicted value and the actual network traffic observed value to obtain a score value, and determines whether the score value exceeds the preset score threshold, and if it exceeds, determines that there is abnormal network traffic data and outputs it. In this way, the exponential smoothing method can be used to detect anomalies in network traffic with business scenario periodicity and improve the robustness of detection. Description of the Drawings

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0041] Figure 1 It is a flowchart of a network traffic anomaly detection method disclosed in the present application;

[0042] Figure 2 It is a flowchart of a specific network traffic anomaly detection method disclosed in the present application;

[0043] Figure 3 It is a schematic diagram of a specific exponential smoothing model structure disclosed in the present application;

[0044] Figure 4Structural schematic diagram of a network traffic anomaly detection device disclosed in this application;

[0045] Figure 5 Structural diagram of an electronic device disclosed in this application. Detailed implementation manners

[0046] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] Currently, traditional single-layer or shallow statistical function models, such as single exponential smoothing, moving average model, etc., are difficult to be applied to business scenarios with periodic change rules, such as network attacks occurring during day and night or special business periods, due to low statistical levels and few learnable parameters. For more advanced artificial intelligence or machine learning methods, for continuous signals such as network traffic, in addition to the large number of parameters to be solved, high algorithm complexity, and large computational resource overhead, a certain number of continuous signal sampling samples are usually required to accumulate a data set to achieve model generalization, and a large amount of manpower is required to continuously adjust some hyperparameters in a proprietary business scenario for the detection effect. This is not practical for a situation awareness system with high real-time requirements and other concurrent services when deploying a large AI model. Therefore, the embodiments of this application disclose a network traffic anomaly detection method, device, equipment, and medium, which can perform anomaly detection on network traffic with business scenario periodicity and improve the robustness of detection.

[0048] See Figure 1 As shown, the embodiments of this application disclose a network traffic anomaly detection method, which includes:

[0049] Step S11: Obtain the network traffic data to be detected, and use the exponential smoothing method to perform weighted average calculation on the network traffic data to obtain the predicted value of the network traffic at the next time point.

[0050] In this embodiment, first, the network traffic data to be detected is obtained. The optional network traffic data set may include anomaly detection of the number of domain name requests, anomaly detection of failed website access, anomaly detection of the number of network sessions, and anomaly detection of inBound traffic. In addition, it is necessary to vectorize the network traffic data. Denote the network traffic at time point t as S t , and then use the exponential smoothing method to perform weighted average calculation on the network traffic data to obtain the predicted value of the network traffic at the next time point, that is, take the weighted average of all past network traffic observation values and use it as the prediction of the current distribution average value, and denote it as

[0051] Step S12: Determine the current business cycle value based on the preset sliding time window size, and use the exponential smoothing method to predict the business cycle factor corresponding to the current business cycle value.

[0052] In this embodiment, the current business cycle value is determined based on the preset sliding time window size. Herein, the sliding time window (i.e., WindowSize) is set by the user, and can be set to 30 days, 7 days, 1 day, etc. Then, the exponential smoothing method is used to predict the business cycle factor corresponding to the current business cycle factor. It should be noted that, according to the actual business environment, the trend and business cycle factor influence factors are introduced, and a multiplicative / additive model is established. If the periodic pattern has nothing to do with the network traffic level of the business environment, the additive model is selected. In most cases, the periodic pattern and the network traffic level of the business environment are more suitable for using the multiplicative model as the default algorithm of the situation awareness system.

[0053] Step S13: Predict the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size, and the business cycle factor to obtain a network traffic prediction value, and evaluate the network traffic prediction value and the corresponding network traffic observation value to obtain a score value.

[0054] In this embodiment, based on the predicted value at the next time point, the sliding time window size, and the business cycle factor, the network traffic after a preset number of future cycles can be predicted to obtain a network traffic prediction value, and then the network traffic observation value corresponding to the network traffic prediction value, that is, the actual network traffic, is determined. Then, the network traffic prediction value and the network traffic observation value are evaluated to obtain an evaluation value.

[0055] Step S14: Determine whether the score value exceeds a preset score threshold. If it exceeds, it is determined that there is abnormal network traffic data in the network traffic data, and the abnormal network traffic data is output.

[0056] In this embodiment, it is determined whether the evaluation value exceeds a preset score threshold. Specifically, the preset score threshold can be set to 0.5. If it exceeds this threshold, it can be determined that there is abnormal network traffic data in the network traffic data and output.

[0057] Further, the output of the abnormal network traffic data includes: determining a target time point corresponding to the abnormal network traffic data, and determining an abnormal level corresponding to the score value according to a pre-set abnormal level division rule; outputting data information including the abnormal traffic data, the target time point, and the abnormal level, and visually displaying the data information according to a pre-set display rule. That is, after determining that there is abnormal network traffic data, the target time point at which the abnormality occurs can be further determined, and the abnormal level corresponding to the score value can be determined according to the pre-set abnormal level division rule. It should be noted that the range between the maximum score and the pre-set score threshold can be divided into three equal parts, namely 0: no abnormality; 1: general abnormality, and a yellow warning is displayed; 2: poisoning abnormality, and an orange warning is displayed; 3: serious abnormality, and a red warning is displayed. Then, data information including abnormal traffic data, target time point, and abnormal level is output. In addition, time series of network traffic data, reference intermediate indicators, abnormal point feature vector matrices, etc. can also be output. Among them, the feature vector matrix can also include data such as the number of principal components, the contribution ratio of principal components, and the list of principal component eigenvalues. Finally, the above data information is visually displayed according to the pre-set display rule. Specifically, the score value of real data anomaly detection is visually displayed through line charts, bar charts, and swimlane charts.

[0058] It can be seen that this application obtains the network traffic data to be detected, and uses the exponential smoothing method to perform weighted average calculation on the network traffic data to obtain the predicted value of the network traffic at the next time point; determines the current service cycle value based on the pre-set sliding time window size, and uses the exponential smoothing method to predict the service cycle factor corresponding to the current service cycle value; predicts the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size, and the service cycle factor to obtain the network traffic prediction value, and evaluates the network traffic prediction value and the corresponding network traffic observation value to obtain a score value; determines whether the score value exceeds the pre-set score threshold. If it exceeds, it is determined that there is abnormal network traffic data in the network traffic data, and the abnormal network traffic data is output. Thus, this application first obtains the predicted value of the network traffic at the next time point through the exponential smoothing method, and predicts the service cycle factor based on the sliding time window size using the exponential smoothing method, then further predicts the network traffic after a preset number of future cycles to obtain the network traffic prediction value, and then evaluates the network traffic prediction value and the actual network traffic observation value to obtain a score value, and determines whether the score value exceeds the pre-set score threshold. If it exceeds, it is determined that there is abnormal network traffic data and output. In this way, the exponential smoothing method can be used to detect anomalies in network traffic with business scenario periodicity and improve the robustness of the detection.

[0059] SeeFigure 2 As shown in the figure, an embodiment of the present application discloses a specific network traffic anomaly detection method. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, it includes:

[0060] Step S21: Obtain the network traffic data to be detected under different scenario types, and use the exponential smoothing method to perform weighted average calculation on the network traffic data to obtain the predicted value of the network traffic at the next time point.

[0061] In this embodiment, the network traffic data to be detected under different scenario types can be obtained, where the scenario types include any one or several of the network session fluctuation scenario, network failure scenario, server communication scenario, server downtime scenario, and DDOS attack scenario. For example, for the network session fluctuation scenario and network failure scenario, time series analysis and prediction can be performed on the mirror traffic network flow session statistics to detect network session fluctuations and even failures, and focus on discovering potential intrusion attack behaviors that may cause the above problems; for the server communication scenario, the number of domain names requested by the original log host can be counted to detect abnormal request behaviors such as a large number of domain name resolutions hidden in normal requests by malware, such as through DGA technology; for the server downtime scenario, component failure detection scenario, or malicious scan attack detection scenario, the original logs with unsuccessful website return response codes can be counted and analyzed and predicted based on the existing time series data to promptly detect server self-abnormalities or malicious scan attacks; for the DDOS (i.e., distributed denial of service attack) attack scenario, time series analysis and prediction can be performed on the inbound traffic volume to promptly detect DDOS attacks in view of the extremely high traffic peak characteristics generated by DDOS attacks.

[0062] Furthermore, it should be noted that the above-mentioned weighted average calculation of the network traffic data using the exponential smoothing method includes: using the exponential smoothing method and based on the first exponential coefficient to perform weighted average calculation on the network traffic data to obtain the predicted value of the network traffic at the next time point, where the exponential smoothing method can specifically be triple exponential smoothing (i.e., Triple Exponential Smoothing). Specifically, reference can be made to Figure 3 shown in Figure 3 which is a schematic diagram of a specific exponential smoothing model structure disclosed in the present application. In this embodiment, the weighted average of the actual observed value in the current period and the exponential smoothing value in the previous period can be taken, and its expression is as follows:

[0063]

[0064] Where is the predicted value, S t is the actual observed value in the current period, is the exponential smoothing value of the previous period, A is the first exponential coefficient, and A ∈ [0, 1];

[0065] In addition, let Thus, it can be obtained that:

[0066]

[0067] At this time, is an unbiased estimator of E(S), and the prediction for any period is:

[0068]

[0069] When M → ∞, Then there is Therefore, when the stochastic process is non-stationary, it is not strictly established that E(S) is an unbiased estimator of .

[0070] It should be noted that the value of the first exponential coefficient is very important. In some specific embodiments, it can be judged by the empirical judgment method. This method mainly depends on the development trend of the time series and the experience of the predictor to make a judgment. For example, when the time series shows a relatively stable horizontal trend, a smaller A value should be selected, generally in the range of 0.05 - 0.20; when the time series has fluctuations but the long-term trend changes little, a slightly larger A value can be selected, usually in the range of 0.1 - 0.4; when the time series fluctuates greatly, the long-term trend changes significantly, showing an obvious and rapid upward or downward trend, a larger A value should be selected, such as in the range of 0.6 - 0.8, so that the prediction model has higher sensitivity and can quickly keep up with the changes of the data; when the time series data is of an upward (or downward) development trend type, A should take a larger value, in the range of 0.6 - 1. In some other specific embodiments, the trial method can also be used, that is, according to the specific time series situation, referring to the empirical judgment method, to roughly determine the rated value range, and then take several A values for trial calculation, compare the prediction standard errors under different A values, and select the A with the smallest prediction standard error. In practical applications, the predictor should make a qualitative judgment in combination with the change law of the prediction object and calculate the prediction error, and it must be considered that the prediction sensitivity and prediction accuracy are contradictory to each other, and a compromise A value must be given to both.

[0071] Step S22: Determine the business environment parameters under the current scenario type, and determine the current business cycle value based on the preset sliding time window size and the business environment parameters, and use the exponential smoothing method to predict the business cycle factor corresponding to the current business cycle value.

[0072] In this embodiment, the business environment parameters under the current scenario type are denoted as p, and the current business cycle value L is determined based on the sliding time window size and the business environment parameters. The business cycle can be quarterly, annual, non-natural calendar production cycle law, etc. The specific expression is:

[0073] L = p * WindowSize;

[0074] Further, predicting the business cycle factor corresponding to the current business cycle value by using the exponential smoothing method includes: predicting the business cycle factor F corresponding to the current business cycle value by using the exponential smoothing method and based on the second exponential coefficient t , and the specific expression is:

[0075]

[0076] where the second exponential coefficient B ∈ [0, 1], and F t-L is the previous window estimated value.

[0077] Further, the predicted value with the current business cycle value is:

[0078]

[0079] Then the prediction S for the next cycle t,1 is:

[0080]

[0081] The prediction S for the next T cycles t,T is:

[0082] where T ≤ L 3 ;

[0083] where, let J be the largest integer less than or equal to M / L, then according to the function F of all past observations with variables, exponential coefficients A and B, and prediction initial conditions t and is:

[0084]

[0085]

[0086] It can be understood that the influence of usually decays faster than the influence of the initial F because t is corrected every WindowSize cycle, but F

[0087] Step S23: Determine whether to introduce a trend factor according to the current business environment; the trend factor includes a third exponential coefficient.

[0088] In this embodiment, it is determined whether to introduce a trend factor according to the current business environment. If this prediction model with periodicity but no trend influence is applied to the network traffic time series whose average value is affected by long-term and short-term system changes or trends, the periodic factor F will soon no longer be a simple periodic factor, and it will contain some noise incorporated with trend influence. Therefore, if there is a linear trend influence in the network traffic of the business environment, such as the company expands over time and the network traffic increases as the business grows, etc., a specific trend factor R must be introduced:

[0089]

[0090] Among them, the third exponential coefficient C ∈ [0, 1].

[0091] In addition, the above method further includes: pre-obtaining network traffic training data under different scenarios, and using the exponential smoothing method to predict the network traffic training data to determine the predicted values of the training data including the first exponential coefficient, the second exponential coefficient, and the third exponential coefficient; determining the prediction error between the predicted value of the training data and the corresponding observed value of the training data, and using the grid search method based on the stochastic gradient descent method to solve based on the prediction error to determine the values of the first exponential coefficient, the second exponential coefficient, and the third exponential coefficient corresponding to the minimum prediction error. That is, this application determines the values of the first exponential coefficient, the second exponential coefficient, and the third exponential coefficient through the pre-training data. First, determine the prediction error e t,T :

[0092] e t,T = S t+T - S t,T ; where S t,T is the predicted value of the training data, and S t+T is the observed value of the training data.

[0093] Then, within a time window, the prediction standard error is:

[0094]

[0095] Among them, N is the number of observed values. And in the prediction of the network traffic in the near future, the accuracy of the predicted value is the most important, while the importance in the long-term future decreases continuously. Then, according to this strategy, a function U is used to weight the prediction value error:

[0096] where a 1 > a2 >a 3 .

[0097] In this embodiment, since there are more than one parameters to be determined, it is necessary to use a grid search method based on the stochastic gradient descent method to solve. The stochastic gradient descent method is first used to solve the range of multiple local optimal solutions under a certain minimum step size. If the total network traffic can be divided into multiple different subtypes, the grid exhaustive parameter adjustment loop is used to traverse each possibility of each free hyperparameter in the "flat" area (i.e., the learning step size is small). Each set of parameter values ​​will get a corresponding model. According to the objective function, the solution with the smallest error between the observed value and the predicted value is selected as the optimal parameter.

[0098] Step S24: If introduced, the network traffic after a preset number of cycles in the future is predicted based on the trend factor, the predicted value at the next time point, the sliding time window size and the business cycle factor to obtain a network traffic prediction value, and the network traffic prediction value and the corresponding network traffic observation value are evaluated to obtain a scoring value.

[0099] In this embodiment, after the trend factor is introduced, the network traffic after T cycles in the future is predicted based on the trend factor, the predicted value at the next time point, the sliding time window size and the service cycle factor to obtain the predicted network traffic value, which is expressed as follows:

[0100] Among them, T = 1, 2,…, L.

[0101] It should be pointed out that after the trend factor is introduced, the initial values ​​are determined as follows:

[0102] Let the special statistical period of a certain indicator D = WindowSize, let have to The exponential model is then used on the first part of the network traffic time series (i.e., t = 1, 2, ... H) in the same way as in the second part, where no predictions are made and therefore no prediction error is measured. The values ​​of F and R are considered as the initial values ​​of the second part of the sequence (i.e. t = H + 1, H + 2, ..., L), and the prediction is performed and the error is calculated. In order to eliminate the influence of the arbitrarily selected initial values ​​as much as possible, the complexity of using the first part of the sequence twice is increased. In the case of the business cycle factor F, because each F is only recalculated once at each L; as the weight becomes smaller, it becomes more and more important for the exponential smoothing, periodicity, and trend characteristics of each factor. The initialization formula is as follows:

[0103] Calculate the average network traffic V of each event window in each business cycle i , i identifies each large cycle;

[0104] R last For the average trends of the first major cycle and the current major cycle at the start of the statistics, its forward estimation:

[0105]

[0106] Calculate the periodicity factor in each time window, which is the ratio of the actual network traffic in the first part of the network traffic time series (i.e., t = 1, 2,..., H) to the periodically adjusted average network traffic of this period, and further use the trend factor R last After adjustment, we have:

[0107] j = 1, 2,..., L;

[0108] Average the periodicity factors of the corresponding time windows in each initial cycle to obtain the periodicity factors of each time window in a business cycle;

[0109] Finally, to ensure that within a cycle, the periodicity factor only undergoes periodic adjustment and does not increase or decrease the average level of network traffic, normalize the periodic data:

[0110]

[0111] Step S25: Determine whether the scoring value exceeds a preset scoring threshold. If it does, determine that there is abnormal network traffic data in the network traffic data, and output the abnormal network traffic data.

[0112] Among them, for the more specific processing process of the above step S25, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated here.

[0113] It can be seen that in the embodiments of the present application, through the exponential smoothing method of medium- and short-term trend prediction, after filtering and screening the received network traffic and its packet content for analysis, a statistical model is applied to the field of network security, thereby realizing the prediction analysis of traffic data to detect abnormal alarms. Triple exponential smoothing can well solve the problems of periodic variation factors and trend components, which is usually used to remove high-frequency signals from the studied time series. This method can strengthen the role of recent observed values in the observation period on the predicted value by increasing the weights of recent observed values, and at the same time can control the change rate of the weights, and has better detection accuracy for the medium- and short-term of business scenarios than existing methods. And in the solution of multiple parameters, using the stochastic gradient descent grid search algorithm can improve the calculation accuracy, and can also effectively solve the global optimal solution of multiple parameters according to the network traffic of special services and different types of network traffic with less computational resource overhead. In addition, different real-time network data sets can be flexibly configured for model training and anomaly detection. In this way, the present solution improves the robustness of detecting abnormal data in the time series of network traffic with business scenario periodicity.

[0114] See Figure 4 As shown, the embodiments of the present application disclose a network traffic anomaly detection device, which includes:

[0115] The first exponential smoothing processing module 11 is used to obtain the network traffic data to be detected, and perform weighted average calculation on the network traffic data by using the exponential smoothing method to obtain the predicted value of the network traffic at the next time point;

[0116] The second exponential smoothing processing module 12 is used to determine the current business cycle value based on the preset sliding time window size, and predict the business cycle factor corresponding to the current business cycle value by using the exponential smoothing method;

[0117] The network traffic prediction module 13 is used to predict the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size, and the business cycle factor to obtain a network traffic prediction value, and evaluate the network traffic prediction value and the corresponding network traffic observation value to obtain a score value;

[0118] The anomaly detection module 14 is used to determine whether the score value exceeds a preset score threshold. If it exceeds, it is determined that there is abnormal network traffic data in the network traffic data, and the abnormal network traffic data is output.

[0119] It can be seen that this application obtains the network traffic data to be detected, and uses the exponential smoothing method to perform weighted average calculation on the network traffic data to obtain the predicted value of the network traffic at the next time point; determines the current business cycle value based on the preset sliding time window size, and uses the exponential smoothing method to predict the business cycle factor corresponding to the current business cycle value; predicts the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size, and the business cycle factor to obtain the network traffic prediction value, and evaluates the network traffic prediction value and the corresponding network traffic observation value to obtain a score value; determines whether the score value exceeds the preset score threshold, and if it exceeds, determines that there is abnormal network traffic data in the network traffic data and outputs the abnormal network traffic data. Thus, this application first obtains the predicted value of the network traffic at the next time point through the exponential smoothing method, and predicts the business cycle factor based on the sliding time window size using the exponential smoothing method, then further predicts the network traffic after a preset number of future cycles to obtain the network traffic prediction value, and then evaluates the network traffic prediction value and the actual network traffic observation value to obtain a score value, and determines whether the score value exceeds the preset score threshold, and if it exceeds, determines that there is abnormal network traffic data and outputs it. In this way, the exponential smoothing method can be used to detect anomalies in network traffic with business scenario periodicity and improve the robustness of the detection.

[0120] Figure 5 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the network traffic anomaly detection method executed by the electronic device disclosed in any of the foregoing embodiments.

[0121] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of this application, and no specific limitation is imposed on it here; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitation is imposed here.

[0122] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for the rendering and drawing of the content to be displayed on the display screen. In some embodiments, the processor 21 may further include an AI (Artificial Intelligence) processor, which is used to process computational operations related to machine learning.

[0123] In addition, as a carrier for resource storage, the memory 22 may be a read-only memory, a random access memory, a magnetic disk, or an optical disc, etc. The resources stored thereon include an operating system 221, a computer program 222, data 223, etc., and the storage method may be temporary storage or permanent storage.

[0124] Among them, the operating system 221 is used to manage and control each hardware device and the computer program 222 on the electronic device 20, so as to implement the operation and processing of the massive data 223 in the memory 22 by the processor 21. It may be Windows, Unix, Linux, etc. In addition to the computer program that can be used to complete the network traffic anomaly detection method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program that can be used to complete other specific tasks. In addition to the data transmitted by external devices received by the electronic device, the data 223 may also include data collected by its own input / output interface 25, etc.

[0125] Furthermore, the embodiments of the present application also disclose a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the method steps executed in the network traffic anomaly detection process disclosed in any of the foregoing embodiments are implemented.

[0126] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0127] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0128] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0129] Finally, it should also be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0130] The above has introduced in detail a network traffic anomaly detection method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for detecting abnormal network traffic, characterized in that, it includes: Obtain the network traffic data to be detected, and use the exponential smoothing method to perform weighted average calculation on the network traffic data to obtain the predicted value of the network traffic at the next time point; Determine the current business cycle value based on the preset sliding time window size, and use the exponential smoothing method to predict the business cycle factor corresponding to the current business cycle value; Predict the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size, and the business cycle factor to obtain the network traffic prediction value, and evaluate the network traffic prediction value and the corresponding network traffic observation value to obtain a score value; Judge whether the score value exceeds the preset score threshold. If it exceeds, determine that there is abnormal network traffic data in the network traffic data, and output the abnormal network traffic data.

2. The method for detecting abnormal network traffic according to claim 1, characterized in that, the obtaining of the network traffic data to be detected includes: Obtain the network traffic data to be detected under different scenario types; wherein, the scenario types include any one or several of the network session fluctuation scenario, the network failure scenario, the server communication scenario, the server downtime scenario, and the DDOS attack scenario.

3. The method for detecting abnormal network traffic according to claim 2, characterized in that, the determining of the current business cycle value based on the preset sliding time window size includes: Determine the business environment parameters under the current scenario type; Determine the current business cycle value based on the preset sliding time window size and the business environment parameters.

4. The method for detecting abnormal network traffic according to claim 1, characterized in that, the using of the exponential smoothing method to perform weighted average calculation on the network traffic data includes: Use the exponential smoothing method and perform weighted average calculation on the network traffic data based on the first exponential coefficient; Correspondingly, the using of the exponential smoothing method to predict the business cycle factor corresponding to the current business cycle value includes: Use the exponential smoothing method and predict the business cycle factor corresponding to the current business cycle value based on the second exponential coefficient.

5. The method for detecting abnormal network traffic according to claim 4, characterized in that, the predicting of the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size, and the business cycle factor to obtain the network traffic prediction value includes: Determine whether to introduce a trend factor according to the current business environment; the trend factor includes a third exponential coefficient; If introduced, predict the network traffic after a preset number of future cycles based on the trend factor, the predicted value at the next time point, the sliding time window size, and the business cycle factor to obtain the network traffic prediction value.

6. The method for detecting abnormal network traffic according to claim 5, characterized in that, it further includes: Pre-acquire network traffic training data under different scenarios, and use the exponential smoothing method to predict the network traffic training data to determine the training data prediction value including the first exponential coefficient, the second exponential coefficient, and the third exponential coefficient; Determine the prediction error between the training data prediction value and the corresponding training data observation value, and use the grid search method based on the stochastic gradient descent method to solve based on the prediction error to determine the values of the first exponential coefficient, the second exponential coefficient, and the third exponential coefficient corresponding to the minimum prediction error.

7. The network traffic anomaly detection method according to any one of claims 1 to 6, characterized in that, The output of the abnormal network traffic data includes: Determine the target time point corresponding to the abnormal network traffic data, and determine the abnormal level corresponding to the score value according to the pre-set abnormal level division rule; Output data information including the abnormal traffic data, the target time point, and the abnormal level, and visually display the data information according to the preset display rule.

8. A network traffic anomaly detection device, characterized in that, comprising: A first exponential smoothing processing module, configured to acquire network traffic data to be detected, and use the exponential smoothing method to perform weighted average calculation on the network traffic data to obtain a predicted value of the network traffic at the next time point; A second exponential smoothing processing module, configured to determine the current service cycle value based on the pre-set sliding time window size, and use the exponential smoothing method to predict the service cycle factor corresponding to the current service cycle value; A network traffic prediction module, configured to predict the network traffic after a preset number of future cycles based on the predicted value at the next time point, the sliding time window size, and the service cycle factor to obtain a network traffic prediction value, and evaluate the network traffic prediction value and the corresponding network traffic observation value to obtain a score value; An anomaly detection module, configured to determine whether the score value exceeds a preset score threshold. If it exceeds, it is determined that there is abnormal network traffic data in the network traffic data, and the abnormal network traffic data is output.

9. An electronic device, characterized in that, comprising: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the steps of the network traffic anomaly detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, used to store a computer program; wherein, when the computer program is executed by a processor, the steps of the network traffic anomaly detection method according to any one of claims 1 to 7 are implemented.

Images