Model input adjustment method, anomaly traffic detection method, apparatus, device and medium

Abstract

Disclosed in the embodiments of the present invention are a model input adjustment method, an anomaly traffic detection method, an apparatus, a device and a medium. The model input adjustment method comprises: acquiring a traffic sample containing traffic, the traffic sample comprising a sign of anomaly traffic among the traffic; inputting the traffic sample within a time window into a trained anomaly traffic detection model; on the basis of the sign and a detection result of the anomaly traffic output by the trained anomaly traffic detection model, determining the detection accuracy of the anomaly traffic detection model in respect of the traffic sample within the time window; and, on the basis of the detection accuracy, adjusting the time window. The present invention optimizes input data volumes of models by means of an adjustable time window, so as to achieve a good balance between detection accuracy and real-time performance, thus helping to detect online traffic in real time. On the basis of models with optimized and adjusted time windows, online traffic can be detected in real time with high accuracy.

Description

Model input adjustment and abnormal flow detection method, device, equipment and medium Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a method, device, equipment and medium for adjusting model input and detecting abnormal traffic. Background Art

[0002] In the context of industrial internet, digitalization, and cloud platforms, network security is becoming increasingly important. The attacks and threats faced by various assets, including factories, are becoming more severe, creating an urgent need for detection mechanisms that can promptly detect abnormal traffic (e.g., malicious traffic). Traffic anomaly detection involves analyzing network traffic data to detect and identify traffic behavior that is inconsistent with normal traffic patterns. Artificial intelligence (AI) algorithms can be used to quickly detect abnormal traffic by learning and optimizing data models.

[0003] Most existing abnormal traffic detection methods are offline. These methods require downloading traffic data and then detecting abnormal traffic offline based on AI models, which has the disadvantage of low timeliness.

[0004] Summary of the Invention

[0005] The embodiments of the present invention provide a method, device, equipment and medium for adjusting model input and detecting abnormal flow.

[0006] A method for adjusting model input, comprising:

[0007] Acquire a traffic sample containing traffic, wherein the traffic sample contains a marker of abnormal traffic in the traffic;

[0008] Inputting the traffic samples within the time window into the trained abnormal traffic detection model;

[0009] Determining the detection accuracy of the abnormal traffic detection model for the traffic sample within the time window based on the abnormal traffic detection result output by the trained abnormal traffic detection model and the label;

[0010] Based on the detection accuracy, the time window is adjusted.

[0011] Therefore, optimizing the input data volume of the model with a dynamically adjustable time window can achieve a good compromise between detection accuracy and real-time performance, laying a good foundation for real-time detection of abnormal traffic based on the model.

[0012] In one embodiment, adjusting the time window based on the detection accuracy includes at least one of the following:

[0013] When the difference between the detection accuracy and the first threshold is greater than a second threshold, reducing the length of the time window;

[0014] When the difference between the first threshold and the detection accuracy is greater than the second threshold, increasing the length of the time window;

[0015] When the difference between the detection accuracy and the first threshold is not greater than the second threshold and the difference between the first threshold and the detection accuracy is not greater than the second threshold, the length of the time window is maintained.

[0016] Therefore, by simply comparing the detection accuracy with the first threshold, the length of the time window can be adjusted quickly and accurately, thereby improving the adjustment efficiency.

[0017] In one embodiment, a training process of the abnormal traffic detection model is included, and the training process includes:

[0018] Determining a training sample containing training traffic, wherein the training sample contains a marker of abnormal traffic in the training traffic;

[0019] Inputting the training samples into a neural network model;

[0020] Obtaining a detection result of abnormal traffic output by the neural network model;

[0021] Determining a loss function value of the neural network model based on a difference between the detection result and the label;

[0022] Configuring model parameters of the neural network model so that the loss function value is lower than a preset threshold;

[0023] The neural network model is determined as a trained abnormal traffic detection model.

[0024] Therefore, the embodiment of the present invention also implements the training process of the abnormal traffic detection model.

[0025] A method for detecting abnormal traffic, comprising:

[0026] Get real-time traffic;

[0027] Inputting the real-time traffic into a trained abnormal traffic detection model according to the time window adjusted by any of the above methods;

[0028] Obtaining the abnormal traffic detection result output by the abnormal traffic detection model.

[0029] It can be seen that the model based on the optimized adjustment of the time window can achieve high-accuracy real-time detection for online traffic.

[0030] In one embodiment, it includes:

[0031] Determining the detection accuracy of the abnormal traffic detection model for the real-time traffic within the time window;

[0032] When the detection accuracy is lower than a preset third threshold, the time window is adjusted according to any of the above methods.

[0033] Therefore, based on the detection accuracy during the abnormal traffic detection process, the adjustment process of the time window can be triggered.

[0034] In one embodiment, it includes:

[0035] Counting the number of times the time window is adjusted;

[0036] When the number of adjustments is greater than or equal to a preset third threshold, the abnormal traffic detection model is retrained.

[0037] It can be seen that when the time window is adjusted multiple times, the detection accuracy can be improved by retraining the abnormal traffic detection model, which is especially helpful for discovering abnormal traffic with unknown patterns.

[0038] A device for adjusting model input, comprising:

[0039] An acquisition module, configured to acquire a flow sample containing flow, wherein the flow sample includes a mark of abnormal flow in the flow;

[0040] An input module, configured to input the traffic samples within the time window into a trained abnormal traffic detection model;

[0041] a determination module, configured to determine the detection accuracy of the abnormal traffic detection model for the traffic sample within the time window based on the abnormal traffic detection result output by the trained abnormal traffic detection model and the mark;

[0042] An adjustment module is configured to adjust the time window based on the detection accuracy.

[0043] Therefore, optimizing the input data volume of the model with a dynamically adjustable time window can achieve a good compromise between detection accuracy and real-time performance, laying a good foundation for real-time detection of abnormal traffic based on the model.

[0044] In one embodiment, the adjustment module is configured to perform at least one of the following:

[0045] When the difference between the detection accuracy and the first threshold is greater than a second threshold, reducing the length of the time window;

[0046] When the difference between the first threshold and the detection accuracy is greater than the second threshold, increasing the length of the time window;

[0047] When the difference between the detection accuracy and the first threshold is not greater than the second threshold and the difference between the first threshold and the detection accuracy is not greater than the second threshold, the length of the time window is maintained.

[0048] Therefore, by simply comparing the detection accuracy with the first threshold, the length of the time window can be adjusted quickly and accurately, thereby improving the adjustment efficiency.

[0049] In one embodiment, it includes:

[0050] A training module is used to determine a training sample containing training traffic, wherein the training sample contains a label of abnormal traffic in the training traffic; input the training sample into a neural network model; obtain a detection result of abnormal traffic output by the neural network model; determine a loss function value of the neural network model based on the difference between the detection result and the label; configure model parameters of the neural network model so that the loss function value is lower than a preset threshold; and determine the neural network model as a trained abnormal traffic detection model.

[0051] Therefore, the embodiment of the present invention also implements the training process of the abnormal traffic detection model.

[0052] An abnormal flow detection device, comprising:

[0053] The first acquisition module is used to obtain real-time traffic;

[0054] An input module, configured to input the real-time traffic into a trained abnormal traffic detection model according to a time window adjusted according to any of the above methods;

[0055] The second acquisition module is used to obtain the abnormal traffic detection result output by the abnormal traffic detection model.

[0056] It can be seen that the model based on the optimized adjustment of the time window can achieve high-accuracy real-time detection for online traffic.

[0057] In one embodiment, it includes:

[0058] An adjustment module is used to determine the detection accuracy of the abnormal traffic detection model for the real-time traffic within the time window; when the detection accuracy is lower than a preset third threshold, adjust the time window according to any of the methods described above.

[0059] Therefore, based on the detection accuracy during the abnormal traffic detection process, the adjustment process of the time window can be triggered.

[0060] In one embodiment, it includes:

[0061] A retraining module is used to count the number of adjustments of the time window; when the number of adjustments is greater than or equal to a preset third threshold, retrain the abnormal traffic detection model.

[0062] It can be seen that when the time window is adjusted multiple times, the detection accuracy can be improved by retraining the abnormal traffic detection model, which is especially helpful for discovering abnormal traffic with unknown patterns.

[0063] An electronic device, comprising:

[0064] processor;

[0065] a memory for storing executable instructions of the processor;

[0066] The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement any of the above-described model input adjustment methods or any of the above-described abnormal traffic flow detection methods.

[0067] A computer-readable storage medium stores computer instructions thereon, wherein when the computer instructions are executed by a processor, the method for adjusting the model input described above or the abnormal flow detection method described above are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, so that those skilled in the art will understand the above and other features and advantages of the present invention more clearly. In the accompanying drawings:

[0069] FIG1 is an exemplary flow chart of a method for adjusting a model input according to an embodiment of the present invention.

[0070] FIG2 is a schematic diagram of a training process of an abnormal traffic detection model according to an embodiment of the present invention.

[0071] FIG3 is a schematic diagram of an exemplary process of adjusting a time window according to an embodiment of the present invention.

[0072] FIG4 is an exemplary flow chart of an abnormal traffic flow detection method according to an embodiment of the present invention.

[0073] FIG5 is a flowchart of an exemplary process of an abnormal traffic detection method according to an embodiment of the present invention.

[0074] FIG6 is an exemplary structural diagram of a device for adjusting model input according to an embodiment of the present invention.

[0075] FIG7 is an exemplary structural diagram of an abnormal flow detection device according to an embodiment of the present invention.

[0076] FIG8 is an exemplary structural diagram of an electronic device according to an embodiment of the present invention.

[0077] The accompanying drawings are numerals as follows: DETAILED DESCRIPTION

[0078] To make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail with reference to the following embodiments. Nouns and pronouns related to people in this patent application are not limited to specific genders.

[0079] For the sake of brevity and intuitiveness in description, the solution of the present invention is explained below by describing several representative implementations. A large number of details in the implementations are only used to help understand the solution of the present invention. However, it is obvious that the technical solution of the present invention may not be limited to these details when implemented. In order to avoid unnecessarily obscuring the solution of the present invention, some implementations are not described in detail, but only a framework is given. Hereinafter, "including" means "including but not limited to", and "according to..." means "at least according to..., but not limited to only according to...". Due to the language habits of Chinese, when the number of a component is not specifically specified below, it means that the component can be one or more, or can be understood as at least one.

[0080] In existing technologies, offline detection is commonly used to detect abnormal traffic. In this method, network traffic is first downloaded locally, and then an AI model with abnormal traffic detection capabilities (usually a trained abnormal traffic detection model) is used to detect abnormal traffic. This has the disadvantage of delay, making it difficult to detect abnormal traffic in a timely manner.

[0081] The applicant attempted to use an AI model with abnormal traffic detection capabilities to detect abnormal traffic in online traffic in real time in an online detection manner. However, the applicant found in the process of trying that detection accuracy and real-time performance are usually a pair of parameters that restrict and influence each other. For example, in order to ensure the real-time performance of detection, the amount of traffic to be inspected input into the AI ​​model during each detection process should be relatively small (because: when the amount of traffic to be inspected is large, it means that more online traffic needs to be cached, thereby affecting the real-time performance of detection). However, less traffic to be inspected may lead to reduced detection accuracy (because: with less traffic to be inspected as the analysis object, it is difficult for the AI ​​model to accurately detect the abnormal traffic therein). Similarly, in order to ensure high detection accuracy, the amount of traffic to be inspected input into the AI ​​model should be relatively large (because: when the amount of traffic to be inspected is small, it is difficult to accurately detect the abnormal traffic therein). However, more traffic to be inspected may lead to reduced real-time performance (because: when the amount of traffic to be inspected is large, it means that more online traffic needs to be cached, thereby affecting the real-time performance of detection).

[0082] The applicants also discovered that optimizing the AI ​​model's input data volume (i.e., the size of the traffic flow to be inspected) with an adjustable dynamic time window can achieve a good compromise between detection accuracy and real-time performance, balancing the seemingly unadjustable contradictions mentioned above. This lays a good foundation for real-time detection of online traffic based on AI models. Furthermore, AI models with optimized and adjusted time windows can achieve highly accurate real-time detection of online traffic.

[0083] The above disclosure details the technical defects in the prior art, the causes of these defects, and the analytical process for overcoming them. In reality, the understanding of these technical defects is not common knowledge in the field, but rather a novel discovery made by the applicant during their research. Furthermore, the tracing of the causes of these defects and the analytical process for overcoming them are the result of gradual analysis conducted by the applicant during their actual research and are not common knowledge in the field.

[0084] FIG1 is an exemplary flow chart of a method for adjusting a model input according to an embodiment of the present invention. As shown in FIG1 , the method includes:

[0085] Step 101: Obtain a traffic sample containing traffic, where the traffic sample contains a mark of abnormal traffic in the traffic.

[0086] Here, the traffic sample is the traffic that needs to be input into the trained abnormal traffic detection model to adjust the time window (the time window determines the size of the traffic input into the abnormal traffic detection model during each abnormal traffic detection process).

[0087] For example, traffic can be collected from as many scenarios as possible (including normal traffic and abnormal traffic caused by various attacks), and then abnormal traffic in the traffic can be marked (for example, manually or automatically based on machine learning) to form traffic samples.

[0088] Specifically, the traffic sample may include: (1) normal traffic; (2) abnormal traffic; (3) a mark of abnormal traffic. Normal traffic and abnormal traffic together constitute the traffic to be input into the abnormal traffic detection model. In one embodiment, the mark of abnormal traffic is used to characterize that the corresponding traffic belongs to abnormal traffic, that is, the marked traffic is abnormal traffic. Preferably, the mark of abnormal traffic is used to characterize the type of abnormal traffic to which the corresponding traffic belongs, such as the marked traffic is Domain Name System (DNS) attack traffic or Server Message Block (SMB) attack traffic, etc.

[0089] Step 102: Input the traffic samples within the time window into the trained abnormal traffic detection model.

[0090] When the process shown in Figure 1 is first executed, the length of the time window is set to the initial value. Traffic samples within the time window equal to this initial value are fed into the trained abnormal traffic detection model. For example, if the initial value is 3 seconds, traffic samples for 3 seconds are fed into the trained abnormal traffic detection model.

[0091] The trained abnormal traffic detection model has the ability to detect abnormal traffic. Based on the model training method in the AI ​​field, the neural network model can be trained to obtain the trained abnormal traffic detection model.

[0092] In one embodiment, the method shown in FIG1 includes a training process of an abnormal traffic detection model. FIG2 is a schematic diagram of the training process of an abnormal traffic detection model according to an embodiment of the present invention. As shown in FIG2, the training process includes:

[0093] Sub-step (1): Determine a training sample 11 containing training traffic. The training sample 11 contains a label of abnormal traffic in the training traffic. Here, the training sample 11 is training data that needs to be input into an untrained abnormal traffic detection model 12 (usually a neural network model) so that the neural network model has the ability to detect abnormal traffic. First, collect training traffic from as many scenarios as possible (including normal traffic and abnormal traffic caused by various attacks), and then add labels to the abnormal traffic in the training traffic (for example, manually add labels or automatically add labels based on machine learning) to form training samples. For example, the training sample may include: (1) normal traffic; (2) abnormal traffic; (3) labels of abnormal traffic. In one embodiment, the label of abnormal traffic is used to characterize that the corresponding traffic belongs to abnormal traffic, that is, to determine that the labeled traffic is abnormal traffic. At this time, the trained abnormal traffic detection model can detect abnormal traffic (but usually cannot identify the type of abnormal traffic). Preferably, the labeling of abnormal traffic is used to characterize the type of abnormal traffic to which the corresponding traffic belongs (for example, determining that the labeled traffic is DNS attack traffic or SMB attack traffic, etc.). At this time, the trained abnormal traffic detection model can not only detect abnormal traffic, but also identify the specific type of abnormal traffic.

[0094] Sub-step (2): Input the training sample 11 into the neural network model 12.

[0095] Sub-step (3): Obtain the abnormal traffic detection result 13 output by the neural network model 12. Here, when the abnormal traffic mark in the training sample 11 is used to indicate that the corresponding traffic belongs to abnormal traffic, the detection result 13 includes the detected abnormal traffic; when the abnormal traffic mark in the training sample 11 is used to indicate the abnormal traffic type to which the corresponding traffic belongs, the detection result 13 can not only detect the abnormal traffic but also identify the type of abnormal traffic.

[0096] Sub-step (4): Based on the difference between the detection result 13 and the label, determine the loss function value of the neural network model 12. For example, the loss function may include: Cross Entropy Loss, Hinge Loss, 0 / 1 loss function, exponential loss, logarithmic loss / log-likelihood loss, etc.

[0097] Sub-step (5): configuring the model parameters of the neural network model 12 so that the loss function value is lower than a preset threshold.

[0098] For example, the model parameters of the neural network model 12 are updated through algorithms such as back propagation and gradient descent so that the value of the loss function continues to decrease. Back propagation calculates the gradient of the loss function for each model parameter through the chain rule of derivatives, and updates the model parameters according to the gradient.

[0099] Sub-step (6): The neural network model 12 after the model parameters are configured is determined as a trained abnormal traffic detection model.

[0100] For example, the neural network model can be a time series-based network model, such as the Long Short-Term Memory (LSTM) network. LSTM is a time-recurrent neural network that can solve the long-term dependency problem of general recurrent neural networks (RNNs).

[0101] After further research, the applicant also found that: time series-based network models often have the disadvantage of time lag and are generally not suitable for online detection of abnormal traffic. In the online detection scenario of abnormal traffic, the Bidirectional Encoder Representations from Transformers (BERT) model or various variants of the BERT model (such as ET-BERT) can be preferably used to improve the detection accuracy of small samples of malicious traffic in multiple scenarios, and can detect abnormal traffic in real time based on a dynamic time window mechanism.

[0102] The following describes the training details of the abnormal traffic detection model according to an embodiment of the present invention, using the ET-BERT model as an example. First, traffic is collected from as many scenarios as possible, including normal traffic and abnormal traffic caused by various attacks. After cleaning and processing the traffic data, it is used to train the abnormal traffic detection model. The downstream classification task of the abnormal traffic detection model can be configured as multiple classification tasks. In this way, the type of traffic and attack can be identified. Finally, fine-tuning is performed using real-world scenarios to obtain the desired trained ET-BERT model.

[0103] Table 1 is a typical diagram of the training set during the training process and the test set after the training is completed.

[0104] Table 1

[0105] The above describes in detail a typical example of a neural network model and its training process. Those skilled in the art will appreciate that this description is merely exemplary and is not intended to limit the scope of protection of the embodiments of the present invention.

[0106] Step 103: Based on the abnormal traffic detection results and labels output by the trained abnormal traffic detection model, determine the detection accuracy of the abnormal traffic detection model for the traffic samples within the time window.

[0107] Here, the meaning of detection accuracy is: the ratio between the number of abnormal flows accurately detected by the abnormal flow detection model and the total abnormal flows in the flow samples within the time window.

[0108] For example: Assume that the traffic samples within the time window (for example, the initial value is 3 seconds) include 13,334 abnormal traffic data, and the detection results of the abnormal traffic detection model accurately detect these 13,334 abnormal traffic, then the detection accuracy is: (13,334 / 13,334)*100%=100%.

[0109] Another example: Assume that the traffic samples within the time window (for example, the initial value is 3 seconds) include 15,374 abnormal traffic data, of which 13,361 abnormal traffic are accurately detected in the detection results output by the abnormal traffic detection model. The detection accuracy is: (13,361 / 15,374)*100%=86.91%.

[0110] Step 104: Adjust the time window based on the detection accuracy.

[0111] In one embodiment, adjusting the time window based on the detection accuracy includes:

[0112] (1): When the difference between the detection accuracy and the first threshold is greater than the second threshold, the length of the time window is reduced.

[0113] For example, the first threshold can be a predetermined expected detection accuracy. When the difference between the detection accuracy and the first threshold is greater than a predetermined second threshold (e.g., the second threshold is 0.05), the detection accuracy is deemed sufficient, and the length of the time window can be reduced. Then, during subsequent abnormal flow detection, the flow to be detected within the reduced time window can be input into the abnormal flow detection model, thereby improving real-time performance.

[0114] For example, assume the initial time window length is 3 seconds, the detection accuracy is A, the first threshold is B, and the second threshold is C. When (A-B) is greater than C, the time window length is reduced by a fixed step size (e.g., 0.5 seconds), so that the time window length is adjusted to 3-0.5=2.5 seconds. Then, in the subsequent abnormal traffic detection process, 2.5 seconds of the traffic to be detected are input into the abnormal traffic detection model each time, thereby improving real-time performance.

[0115] (2): When the difference between the first threshold and the detection accuracy is greater than the second threshold, the length of the time window is increased.

[0116] For example, the first threshold can be a predetermined desired detection accuracy. When the difference between the first threshold and the detection accuracy is greater than a second threshold (e.g., the second threshold is 0.05), the detection process is deemed sufficiently real-time, and the length of the time window can be increased to improve detection accuracy. Subsequently, during subsequent abnormal traffic flow detection, the flow to be detected within the increased time window can be input into the abnormal traffic flow detection model, thereby improving detection accuracy.

[0117] For example, assume the initial time window length is 3 seconds, the detection accuracy is A, the first threshold is B, and the second threshold is C. When (B-A) is greater than C, the time window length is increased by a fixed step size (e.g., 0.5 seconds), so the time window length is adjusted to 3 + 0.5 = 3.5 seconds. Then, in the subsequent abnormal traffic detection process, 3.5 seconds of the traffic to be detected are input into the abnormal traffic detection model each time, thereby improving detection accuracy.

[0118] (3): When the difference between the detection accuracy and the first threshold is not greater than the second threshold and the difference between the first threshold and the detection accuracy is not greater than the second threshold, the length of the time window is maintained.

[0119] Here, when the difference between the detection accuracy and the first threshold is not greater than the second threshold and the difference between the first threshold and the detection accuracy is not greater than the second threshold, the length of the time window is deemed reasonable and the length of the time window is maintained (i.e., kept unchanged at the initial value).

[0120] The method shown in FIG1 may be executed repeatedly to adjust the time window multiple times. The method shown in FIG1 may be terminated based on a predetermined loop termination condition (e.g., the number of adjustments reaches a predetermined number or the time window length no longer changes).

[0121] FIG3 is a schematic diagram of an exemplary process of adjusting a time window according to an embodiment of the present invention. The process shown in FIG3 includes:

[0122] Step 21: Set the initial value t0 of the time window length and the expected detection accuracy α.

[0123] Step 22: Set the length ti of the current time window (when setting for the first time: ti is equal to t0).

[0124] Step 23: Input the traffic samples within the length ti of the current time window into the trained abnormal traffic detection model to determine the detection accuracy P of the abnormal traffic detection model for the traffic samples within the length ti of the current time window.

[0125] Step 24: When the difference between the detection accuracy P and α (i.e., P-α) is greater than a predetermined threshold (for example, 0.05) (corresponding to the "Y branch"), execute step 27 and its subsequent steps; otherwise (corresponding to the "N branch"), execute step 25 and its subsequent steps.

[0126] Step 25: When the difference between α and the detection accuracy P (i.e., α-P) is greater than a predetermined threshold (e.g., 0.05) (corresponding to the "Y branch"), execute step 28 and its subsequent steps; otherwise (corresponding to the "N branch"), execute step 26 and its subsequent steps.

[0127] Step 26: Set the current time window length ti as the final time window length and exit this process.

[0128] Step 27: Based on the predetermined step length (n), reduce the length ti of the current time window, for example, ti=ti-n, and return to step 22.

[0129] Step 28: Based on the predetermined step length (n), increase the length ti of the current time window, for example, ti=ti+n, and return to step 22.

[0130] Based on the process shown in Figure 1, the amount of data input into the AI ​​model can be dynamically optimized and adjusted, enabling high-accuracy real-time detection of online traffic.

[0131] FIG4 is an exemplary flow chart of an abnormal traffic detection method according to an embodiment of the present invention. As shown in FIG4 , the method includes:

[0132] Step 401: Obtain real-time traffic.

[0133] For example, real-time traffic in application environments such as the Industrial Internet, cloud platforms, and factories can be obtained at switches, routers, or firewalls.

[0134] Step 402: According to the time window adjusted according to the method shown in FIG1 , the real-time traffic is input into the trained abnormal traffic detection model.

[0135] For example, assuming that the length of the time window adjusted according to the method shown in FIG1 is T, the real-time traffic with the time window length of T is input into the trained abnormal traffic detection model.

[0136] Step 403: Acquire the abnormal traffic detection result output by the abnormal traffic detection model.

[0137] Here, the trained abnormal traffic detection model performs abnormal traffic detection on the real-time traffic with an input time window length of T based on its own abnormal traffic detection capabilities, and outputs the detection results.

[0138] Therefore, by optimizing the model's input data volume with a dynamically adjustable time window, a good compromise is achieved between detection accuracy and real-time performance. Based on a model with an optimized time window, high-accuracy real-time detection of online traffic can be achieved.

[0139] In one embodiment, the process includes: determining the detection accuracy of the abnormal traffic detection model for real-time traffic within a time window; and when the detection accuracy falls below a predetermined third threshold, adjusting the time window according to the method illustrated in FIG1 . For example, the detection accuracy of the abnormal traffic detection model for real-time traffic within each time window is manually calculated, and when the detection accuracy falls below the predetermined third threshold, the time window is readjusted according to the method illustrated in FIG1 . Therefore, the time window adjustment process can be triggered based on the detection accuracy during the abnormal traffic detection process.

[0140] In one embodiment, the method includes: counting the number of time window adjustments; and retraining the abnormal traffic detection model when the number of adjustments is greater than or equal to a pre-set third threshold. This indicates that retraining the abnormal traffic detection model when the time window is adjusted multiple times can improve detection accuracy, particularly facilitating the detection of abnormal traffic patterns with unknown patterns.

[0141] FIG5 is a flow chart of an exemplary process of an abnormal traffic detection method according to an embodiment of the present invention. As shown in FIG5 , the method includes:

[0142] Step 301: Train an abnormal traffic detection model with the ET-BERT model architecture.

[0143] Step 302: Based on the method shown in FIG1 , an optimization adjustment is performed on the time window of the traffic to be input into the abnormal traffic detection model.

[0144] Step 303: Perform anomaly detection on online traffic based on the trained anomaly traffic detection model and the optimized time window. Here, each instance of online traffic within the optimized time window is input into the trained anomaly traffic detection model, and the trained anomaly traffic detection model performs anomaly detection on each instance of input online traffic.

[0145] Step 304: Manually count the detection accuracy of each detection result output by the abnormal traffic detection model. The meaning of detection accuracy is: the ratio between the number of abnormal traffic accurately detected by the abnormal traffic detection model and the total abnormal traffic in the optimized and adjusted time window.

[0146] Step 305: Determine whether the detection accuracy calculated in step 304 meets the preset conditions (for example, whether it is greater than or equal to a preset threshold); if so (corresponding to the "Y" branch), determine that the detection accuracy of the abnormal traffic detection model meets the requirements, and return to execute step 303 and its subsequent steps; otherwise (corresponding to the "N" branch), adjust the time window based on the method shown in Figure 1.

[0147] Step 306: Determine whether the number of time window adjustments is greater than a smaller preset value (e.g., 2). If so (corresponding to the "Y" branch), execute step 307 and subsequent steps; otherwise (corresponding to the "N" branch), return to step 302. Here, if it is found that the number of time window adjustments is greater than a smaller preset value, the abnormal traffic detection model is retrained to improve detection accuracy.

[0148] Step 307: Determine whether the number of time window adjustments is greater than a larger preset value (e.g., 4). If so (corresponding to the "Y" branch), proceed to step 308; otherwise (corresponding to the "N" branch), return to step 301 (i.e., retrain the abnormal traffic detection model).

[0149] Step 308: Manually detect abnormal traffic. Here, when it is found that the number of times the time window is adjusted is greater than a larger preset value, it is determined that the abnormal traffic detection model is no longer suitable for detecting abnormal traffic. At this time, manual detection of abnormal traffic is performed to ensure detection accuracy.

[0150] FIG6 is an exemplary structural diagram of a device for adjusting model input according to an embodiment of the present invention. As shown in FIG6 , the device for adjusting model input 600 includes: an acquisition module 601 for acquiring a flow sample containing flow, wherein the flow sample contains a mark of abnormal flow in the flow; an input module 602 for inputting the flow sample within the time window into a trained abnormal flow detection model 606; a determination module 603 for determining the detection accuracy of the abnormal flow detection model 606 for the flow sample within the time window based on the detection results and marks of abnormal flow output by the trained abnormal flow detection model 606; and an adjustment module 604 for adjusting the time window based on the detection accuracy. The abnormal flow detection model 606 can be deployed inside the device 600 or can be an external model that has a communication connection with the device 600.

[0151] In one embodiment, the adjustment module 604 is used to perform at least one of the following: when the difference between the detection accuracy and the first threshold is greater than the second threshold, reducing the length of the time window; when the difference between the first threshold and the detection accuracy is greater than the second threshold, increasing the length of the time window; when the difference between the detection accuracy and the first threshold is not greater than the second threshold and the difference between the first threshold and the detection accuracy is not greater than the second threshold, maintaining the length of the time window.

[0152] In one embodiment, the device 600 includes: a training module 605, which is used to determine a training sample containing training traffic, and the training sample contains a label of abnormal traffic in the training traffic; input the training sample into a neural network model; obtain the detection result of the abnormal traffic output by the neural network model; based on the difference between the detection result and the label, determine the loss function value of the neural network model; configure the model parameters of the neural network model so that the loss function value is lower than a preset threshold; and determine the neural network model as a trained abnormal traffic detection model 606.

[0153] FIG7 is an exemplary structural diagram of an abnormal flow detection device according to an embodiment of the present invention. As shown in FIG7 , the abnormal flow detection device 700 includes:

[0154] The first acquisition module 701 is used to obtain real-time traffic; the input module 702 is used to input the real-time traffic into the trained abnormal traffic detection model according to the time window adjusted by any of the above methods; the second acquisition module 703 is used to obtain the abnormal traffic detection result output by the abnormal traffic detection model.

[0155] In one embodiment, the apparatus 700 includes:

[0156] The adjustment module 704 is configured to determine the detection accuracy of the abnormal traffic detection model for real-time traffic within the time window; when the detection accuracy is lower than a preset third threshold, adjust the time window according to any of the above methods.

[0157] In one embodiment, the apparatus 700 includes: a retraining module 705 for counting the number of adjustments of the time window; and retraining the abnormal traffic detection model when the number of adjustments is greater than or equal to a preset third threshold.

[0158] The embodiment of the present invention also proposes an electronic device with a processor-memory architecture. Figure 8 is a structural diagram of an electronic device according to an embodiment of the present invention. As shown in Figure 8, the electronic device 800 includes a processor 801, a memory 802, and a computer program stored on the memory 802 and executable on the processor 801. When the computer program is executed by the processor 801, it implements any of the above-mentioned model input adjustment methods or abnormal flow detection methods. Among them, the memory 802 can be specifically implemented as a variety of storage media such as an electrically erasable programmable read-only memory (EEPROM), a flash memory (Flash memory), and a programmable read-only memory (PROM). The processor 801 can be implemented as including one or more central processing units or one or more field programmable gate arrays, wherein the field programmable gate array integrates one or more central processing unit cores. Specifically, the central processing unit or the central processing unit core can be implemented as a CPU, an MCU, or a DSP, and the like.

[0159] It should be noted that not all steps and modules in the above processes and structure diagrams are required, and certain steps or modules can be omitted based on actual needs. The execution order of the steps is not fixed and can be adjusted as needed. The division of the modules is merely for the convenience of describing the functional division adopted. In actual implementation, a module can be implemented by multiple modules, and the functions of multiple modules can be implemented by the same module. These modules can be located in the same device or in different devices.

[0160] The hardware modules in each embodiment can be implemented mechanically or electronically. For example, a hardware module may include a specially designed permanent circuit or logic device (such as a dedicated processor, such as an FPGA or ASIC) for performing a specific operation. The hardware module may also include a programmable logic device or circuit (such as a general-purpose processor or other programmable processor) temporarily configured by software to perform a specific operation. As for whether to implement the hardware module mechanically, or using a dedicated permanent circuit, or using a temporarily configured circuit (such as configured by software), it can be decided based on cost and time considerations.

[0161] The above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for adjusting model input, characterized in that: include; Obtaining a flow sample containing flow, wherein the flow sample contains a mark of abnormal flow in the flow (101); Inputting the traffic samples within the time window into a trained abnormal traffic detection model (102); Determining the detection accuracy of the abnormal traffic detection model for the traffic sample within the time window based on the abnormal traffic detection result output by the trained abnormal traffic detection model and the label (103); Based on the detection accuracy, the time window (104) is adjusted.

2. The method according to claim 1, characterized in that The adjusting the time window (104) based on the detection accuracy includes at least one of the following: When the difference between the detection accuracy and the first threshold is greater than a second threshold, reducing the length of the time window; When the difference between the first threshold and the detection accuracy is greater than the second threshold, increasing the length of the time window; When the difference between the detection accuracy and the first threshold is not greater than the second threshold and the difference between the first threshold and the detection accuracy is not greater than the second threshold, the length of the time window is maintained.

3. The method according to any one of claims 1 to 2, characterized in that The training process of the abnormal traffic detection model includes: Determining a training sample containing training traffic, wherein the training sample contains a marker of abnormal traffic in the training traffic; Inputting the training samples into a neural network model; Obtaining a detection result of abnormal traffic output by the neural network model; Determining a loss function value of the neural network model based on a difference between the detection result and the label; Configuring model parameters of the neural network model so that the loss function value is lower than a preset threshold; The neural network model is determined as a trained abnormal traffic detection model.

4. A method for detecting abnormal traffic, characterized in that: include: Get real-time traffic (201); Inputting the real-time traffic into a trained abnormal traffic detection model (202) according to the time window adjusted according to the method according to any one of claims 1 to 3; The abnormal traffic detection result output by the abnormal traffic detection model is obtained (203).

5. The method according to claim 4, characterized in that include: Determining the detection accuracy of the abnormal traffic detection model for the real-time traffic within the time window; When the detection accuracy is lower than a preset third threshold, the time window is adjusted according to the method according to any one of claims 1-3.

6. The method according to claim 5, characterized in that include: Counting the number of times the time window is adjusted; When the number of adjustments is greater than or equal to a preset third threshold, the abnormal traffic detection model is retrained.

7. A model input adjustment device, characterized in that: include; An acquisition module (601) is used to acquire a flow sample containing flow, wherein the flow sample contains a mark of abnormal flow in the flow; An input module (602) is used to input the traffic samples within the time window into a trained abnormal traffic detection model (606); a determination module (603) for determining the detection accuracy of the abnormal traffic detection model (606) for the traffic sample within the time window based on the abnormal traffic detection result output by the trained abnormal traffic detection model (606) and the label; An adjustment module (604) is configured to adjust the time window based on the detection accuracy.

8. The device according to claim 7, characterized in that The adjustment module (604) is configured to perform at least one of the following: When the difference between the detection accuracy and the first threshold is greater than a second threshold, reducing the length of the time window; When the difference between the first threshold and the detection accuracy is greater than the second threshold, increasing the length of the time window; When the difference between the detection accuracy and the first threshold is not greater than the second threshold and the difference between the first threshold and the detection accuracy is not greater than the second threshold, the length of the time window is maintained.

9. The device according to claim 7 or 8, characterized in that include: A training module (605) is configured to determine a training sample comprising training traffic, wherein the training sample comprises a marker of abnormal traffic in the training traffic; input the training sample into a neural network model; obtain a detection result of abnormal traffic output by the neural network model; determine a loss function value of the neural network model based on a difference between the detection result and the marker; configure model parameters of the neural network model so that the loss function value is lower than a preset threshold; and determine the neural network model as a trained abnormal traffic detection model (606).

10. An abnormal flow detection device, characterized in that: include: A first acquisition module (701) is used to acquire real-time traffic; An input module (702) for inputting the real-time traffic into a trained abnormal traffic detection model according to a time window adjusted according to the method according to any one of claims 1 to 3; The second acquisition module (703) is used to obtain the abnormal traffic detection result output by the abnormal traffic detection model.

11. The device according to claim 10, characterized in that include: An adjustment module (704) is used to determine the detection accuracy of the abnormal traffic detection model for the real-time traffic within the time window; when the detection accuracy is lower than a pre-set third threshold, adjust the time window according to the method according to any one of claims 1-3.

12. The device according to claim 10 or 11, characterized in that include: A retraining module (705), configured to count the number of times the time window is adjusted; When the number of adjustments is greater than or equal to a preset third threshold, the abnormal traffic detection model is retrained.

13. An electronic device, characterized in that: include: Processor (801); a memory (802) for storing executable instructions of the processor (801); The processor (801) is configured to read the executable instructions from the memory (802) and execute the executable instructions to implement the model input adjustment method according to any one of claims 1 to 3 or the abnormal flow detection method according to any one of claims 4 to 6.

14. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the method for adjusting the model input according to any one of claims 1 to 3 or the method for detecting abnormal flow according to any one of claims 4 to 6 is implemented.

Images