Abnormal flow detection method, device, electronic equipment and storage medium
By combining the abnormal flow detection method of binary classification model and multi-classification model, the problem of excessive abnormal flow detection operation time in the prior art is solved, and fast and accurate detection is achieved, which is suitable for scenarios with high security and real-time requirements.
Patent Information
- Application Number
- CN202211305165.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-24
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-10-24
AI Technical Summary
The existing outlier traffic detection method based on deep learning has too long operation time and cannot meet the security and real-time requirements.
An abnormal flow detection method combining binary classification model and multi-classification model is used to determine whether the traffic to be detected meets specific conditions through preset judgment conditions. If it is satisfied, a binary classification model will be used for preliminary identification. If it is not satisfied, a multi-classification model will be used for Detailed identification to achieve fast and accurate abnormal traffic detection.
While ensuring the accuracy of detection, the speed of abnormal flow detection is significantly improved, and is suitable for scenarios with high security and real-time requirements.
Smart Images

Figure CN115622787B_ABST
Abstract
Description
Background Art
[0002] Traffic detection is an important way to identify network attacks. The traditional traffic detection method is to match the feature library, which has the advantage of fast speed. With the development of the Internet, network attack methods are emerging in an endless stream, and network traffic has become more complex and changeable. An effective method is needed to detect abnormal traffic.
[0003] In the existing technology, abnormal traffic detection based on deep learning can accurately detect abnormal traffic and can deal with unknown abnormal traffic. However, abnormal traffic detection based on deep learning requires more parameters, resulting in too long calculation time and more delays, which cannot meet the scenarios with high requirements for security and real-time performance. Therefore, how to quickly and accurately identify abnormal network traffic based on deep learning technology is a problem that needs to be solved urgently.
[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0005] The present disclosure provides an abnormal traffic detection method, device, electronic device and storage medium, which at least to a certain extent overcome the problem that abnormal traffic cannot be detected quickly in the related art.
[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.
[0007] According to one aspect of the present disclosure, there is provided an abnormal traffic detection method, including: obtaining data information of the traffic to be detected; judging whether the data information of the traffic to be detected meets a preset judgment condition, wherein the preset judgment condition is determined according to abnormal traffic proportion information; if the data information of the traffic to be detected meets the preset judgment condition, sending the data information of the traffic to be detected to a binary classification model for a first identification, and if the first identification is normal traffic, forwarding it normally; if the first identification is abnormal traffic, sending the abnormal traffic and feature information corresponding to the abnormal traffic to a multi-classification model for a second identification, and performing different processing according to the second identification result.
[0008] In one embodiment of the present disclosure, the method further includes: if the data information of the traffic to be detected does not meet the preset judgment conditions, sending the feature information of the traffic to be detected to a multi-classification model for a second identification, and performing different processing according to the second identification result.
[0009] In one embodiment of the present disclosure, the first identification includes: a binary classification model performs model calculation on the characteristic information of the traffic to be detected and outputs a confidence level; when the confidence level is greater than a preset confidence threshold, it is identified as normal traffic; when the confidence level is less than a preset confidence threshold, it is identified as abnormal traffic.
[0010] In one embodiment of the present disclosure, the second identification includes: a multi-classification model performs model calculation on the characteristic information of the flow to be detected, and outputs a multi-dimensional confidence vector, wherein each element in the multi-dimensional confidence vector is a value between 0 and 1; and a second identification result is determined according to the arrangement sequence number of the elements, wherein the arrangement sequence number of the elements is the category corresponding to the flow, and the category corresponding to the largest sequence number of the elements is the second identification result.
[0011] In one embodiment of the present disclosure, the method further includes: determining normal traffic quantity information and abnormal traffic quantity information within a preset time period according to a preset time period; determining abnormal traffic proportion information according to normal traffic quantity information and abnormal traffic quantity information within the preset time period.
[0012] In one embodiment of the present disclosure, the method also includes: obtaining the traffic to be detected; determining characteristic information of the traffic to be detected based on the traffic to be detected; determining data information of the traffic to be detected based on the characteristic information of the traffic to be detected, wherein the data information is used to characterize the traffic to be detected and the characteristic information of the traffic to be detected, and the data information is in a data form recognizable by a machine learning model.
[0013] In one embodiment of the present disclosure, performing different processing according to the second identification result includes: if the second identification result is abnormal traffic, intercepting and processing the abnormal traffic; if the second identification result is normal traffic, forwarding the normal traffic normally.
[0014] According to another aspect of the present disclosure, an abnormal traffic detection device is provided, including: a data information acquisition module, which acquires data information of the traffic to be detected; a data information judgment module, which is used to judge whether the data information of the traffic to be detected meets a preset judgment condition, wherein the preset judgment condition is determined according to abnormal traffic proportion information; a dual-model recognition module, which is used to send the data information of the traffic to be detected to a binary classification model for a first recognition, and if the first recognition is normal traffic, it is forwarded normally; if the first recognition is abnormal traffic, the abnormal traffic and feature information corresponding to the abnormal traffic are sent to a multi-classification model for a second recognition, and different processing is performed according to the second recognition result.
[0015] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the above-mentioned abnormal traffic detection method by executing the executable instructions.
[0016] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned abnormal traffic detection method is implemented.
[0017] According to another aspect of the present disclosure, a computer program product is provided, including computer instructions, wherein the computer instructions are stored in a computer-readable storage medium, and when the computer instructions are executed by a processor, the operating instructions of any of the above-mentioned abnormal traffic detection methods are implemented.
[0018] The embodiments of the present disclosure provide an abnormal traffic detection method, device, electronic device and storage medium, which obtain data information of the traffic to be detected; determine whether the data information of the traffic to be detected meets the preset judgment conditions, wherein the preset judgment conditions are determined according to the abnormal traffic proportion information; if the data information of the traffic to be detected meets the preset judgment conditions, the data information of the traffic to be detected is sent to a binary classification model for a first identification, and if the first identification is normal traffic, it is forwarded normally; if the first identification is abnormal traffic, the abnormal traffic and the feature information corresponding to the abnormal traffic are sent to a multi-classification model for a second identification, and different processing is performed according to the second identification result. In the embodiments of the present disclosure, since the binary classification model and the multi-classification model are combined and the advantages of different models are integrated, the speed is improved while the accuracy of traffic detection is guaranteed, and rapid detection of abnormal traffic is achieved.
[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0021] Figure 1 A schematic diagram showing an exemplary system architecture of an abnormal traffic detection method in an embodiment of the present disclosure;
[0022] Figure 2 An exemplary system working principle diagram of an abnormal traffic detection method in an embodiment of the present disclosure is shown;
[0023] Figure 3 A flow chart of an abnormal traffic detection method according to an embodiment of the present disclosure is shown;
[0024] Figure 4 A flow chart of an abnormal traffic detection method according to an embodiment of the present disclosure is shown;
[0025] Figure 5 A flow chart of an abnormal traffic detection method according to an embodiment of the present disclosure is shown;
[0026] Figure 6 A flow chart of an abnormal traffic detection method according to an embodiment of the present disclosure is shown;
[0027] Figure 7 A flow chart of an abnormal traffic detection method according to an embodiment of the present disclosure is shown;
[0028] Figure 8 A flowchart showing a specific example of an abnormal traffic detection method in an embodiment of the present disclosure;
[0029] Fig. 9 A flow diagram showing an abnormal flow detection method in an embodiment of the present disclosure;
[0030] Fig.10 A schematic diagram of an abnormal flow detection device in an embodiment of the present disclosure is shown;
[0031] Fig.11 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown;
[0032] Fig.12 A schematic diagram showing a computer-readable storage medium in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0033] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0034] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0035] Figure 1 A schematic diagram of an exemplary system architecture that can be applied to an abnormal traffic flow detection method or an abnormal traffic flow detection device according to an embodiment of the present disclosure is shown.
[0036] like Figure 1 As shown, the system architecture 100 may include terminal devices 101 , 102 , 103 , 104 , a secondary relay device 105 , a primary relay device 106 , a gateway server 107 , a local area management server 108 , a network 109 and the Internet 110 .
[0037] Among them, the network 109 is used to provide a medium for communication links between the terminal devices 101, 102, 103, 104, the secondary relay device 105, the primary relay device 106, the gateway server 107, and the local area management server 108, and can be a wired network or a wireless network.
[0038] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.
[0039] The terminal devices 101, 102, 103, 104 may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, wearable devices, augmented reality devices, virtual reality devices, etc.
[0040] Optionally, the client of the application installed in different terminal devices 101, 102, 103, 104 is the same, or is a client of the same type of application based on different operating systems. Based on different terminal platforms, the specific form of the client of the application can also be different, for example, the application client can be a mobile client, a PC client, etc.
[0041] In the near-field area network (P-RAN), relay devices provide Internet access to other devices. In this scenario, the first-level relay device can deploy a traffic feature processing module and a small model (binary classification model), the gateway server can deploy a large model (multi-class model), and the near-field management server can deploy a detection and control module.
[0042] In the multi-model identification mode, the first-level relay device identifies and marks the traffic as normal and abnormal, and sends them to the gateway server. The gateway server uses a large model to perform detailed identification of only the traffic marked as abnormal, forwards the traffic identified as normal and marked as normal to the Internet, and performs subsequent security processing on the traffic identified as abnormal.
[0043] In the single model recognition mode, only the traffic feature processing module in the first-level relay device is used. After processing, all traffic and features are sent to the gateway server. The gateway server uses a large model to perform detailed recognition of all traffic.
[0044] The near-domain management server will receive the recognition results regularly, and the detection control module therein will switch the recognition mode according to the current abnormal traffic proportion.
[0045] Those skilled in the art will know that Figure 1 The number of terminal devices, secondary relay devices, primary relay devices, gateway servers, near-domain management servers, and networks in the example are merely illustrative, and any number of terminal devices, secondary relay devices, primary relay devices, gateway servers, near-domain management servers, and networks may be provided according to actual needs. The embodiments of the present disclosure are not limited thereto.
[0046] Figure 2 An exemplary system operating principle diagram that can be applied to an abnormal flow detection method or an abnormal flow detection device in an embodiment of the present disclosure is shown.
[0047] like Figure 2 As shown, the system working principle diagram 200 may include a feature extraction processing module 201, a data preprocessing module 202, a detection control module network 203, a small model 204 and a large model 205.
[0048] The data feature extraction and processing module is used to extract and process traffic features. After the traffic enters the device, the model cannot directly identify the traffic. The traffic features must be extracted first and converted into a form that the model can recognize. The data feature extraction and processing module extracts the traffic model, port number, protocol number and other content and sends it to the data preprocessing module.
[0049] The data preprocessing module is used to convert the features into a form recognizable by the model, which is a value between 0 and 1, and then send the flow characteristics (equivalent to the above-mentioned value between 0 and 1) to the control module.
[0050] There are two traffic identification modes: multi-model identification and single-model identification. The small model and the large model send the number of normal and abnormal traffic identified within the preset time period to the detection control module to determine whether the set conditions in the detection control module are met. If the set conditions are met, multi-model identification is adopted. The small model is used to identify the normal traffic first, and the normal traffic is forwarded normally. The remaining traffic is then identified by the large model in multiple categories. If the set conditions are not met, single model identification is used, and all traffic is identified and classified by the large model. The set conditions can be derived based on parameters such as the normal traffic recall rate, abnormal traffic recall rate, and traffic identification time of the large model and the small model.
[0051] The small model is only used to distinguish between normal traffic and abnormal traffic. It is a binary classification model that is responsible for forwarding most normal traffic normally and can reduce the amount of traffic identified by the large model. After receiving the traffic feature data, the small model performs model calculation on the feature and outputs the confidence, which is a value between 0 and 1. If the confidence is greater than the set confidence threshold, it is judged as normal traffic. If the confidence is less than the confidence threshold, it is judged as abnormal traffic. The small model forwards normal traffic normally and sends the remaining traffic and features to the large model. The small model has the characteristics of fast speed and high binary classification accuracy.
[0052] The large model is responsible for detailed identification and classification of traffic. It needs to identify the types of normal traffic and various abnormal traffic. It is a multi-classification model. The input of the large model is the pre-processed traffic characteristics, and the output is a multi-dimensional confidence vector. Each element in the confidence vector is a value between 0 and 1. The sequence number of the element arrangement is the category corresponding to the traffic. The category corresponding to the largest element sequence number in the vector is the identification result. The large model will identify normal traffic and forward it normally, and intercept abnormal traffic for subsequent processing. The large model has the characteristics of high multi-classification accuracy.
[0053] An abnormal traffic detection method is provided in an embodiment of the present disclosure. The method can be executed by any electronic device with computing and processing capabilities.
[0054] Figure 3 A flow chart of an abnormal flow detection method in an embodiment of the present disclosure is shown as follows: Figure 3 As shown, the abnormal traffic detection method provided in the embodiment of the present disclosure includes the following steps:
[0055] S302, obtaining data information of the traffic to be detected.
[0056] It should be noted that the above-mentioned data information can be characteristic information used to characterize the traffic to be detected and the traffic to be detected, and the storage format of the above-mentioned data information is a data form that can be recognized by the machine learning model.
[0057] For example, characteristic information such as the traffic model, port number, and protocol number is extracted, and the characteristic information is converted into a form recognizable by the model, for example, the characteristic information is converted into a numerical value between 0 and 1.
[0058] S304, determining whether the data information of the traffic to be detected meets a preset determination condition, wherein the preset determination condition is determined based on the abnormal traffic proportion information.
[0059] It should be noted that the above abnormal traffic ratio information can be the ratio of abnormal traffic to total traffic. For example, the preset judgment condition can be derived based on the normal traffic recall rate, abnormal traffic recall rate, traffic identification time and other parameters of the binary classification model (equivalent to the above small model) and the multi-classification model (equivalent to the above large model).
[0060] S306: If the data information of the traffic to be detected meets the preset judgment conditions, the data information of the traffic to be detected is sent to the binary classification model for the first identification. If the first identification is normal traffic, it is forwarded normally; if the first identification is abnormal traffic, the abnormal traffic and the feature information corresponding to the abnormal traffic are sent to the multi-classification model for the second identification, and different processing is performed according to the second identification result.
[0061] It should be noted that the above-mentioned normal forwarding can be to send the traffic to the corresponding port according to the forwarding rules of the traffic itself. The above-mentioned binary classification model can be a pre-trained machine learning model that can identify the input traffic as normal traffic or abnormal traffic. The above-mentioned multi-classification model can be a pre-trained machine learning model that can identify the input traffic as normal traffic or abnormal traffic. Responsible for detailed identification and classification of traffic, it is necessary to identify the types of normal traffic and various abnormal traffic.
[0062] In the specific implementation, the binary classification model and the multi-classification model are combined to integrate the advantages of different models. While ensuring the accuracy of traffic detection, the speed is improved and the rapid detection of abnormal traffic is achieved.
[0063] In one embodiment of the present disclosure, Figure 4 As shown, the abnormal traffic detection method provided in the embodiment of the present disclosure can determine the abnormal traffic through the following steps, without the need for a binary classification model to detect the traffic, which can improve the detection speed:
[0064] S402: If the data information of the traffic to be detected does not meet the preset judgment conditions, the feature information of the traffic to be detected is sent to the multi-classification model for a second identification, and different processing is performed according to the second identification result.
[0065] In one embodiment of the present disclosure, the first identification includes: a binary classification model performs model calculation on the characteristic information of the traffic to be detected and outputs a confidence level; when the confidence level is greater than a preset confidence threshold, it is identified as normal traffic; when the confidence level is less than a preset confidence threshold, it is identified as abnormal traffic.
[0066] In one embodiment of the present disclosure, the second identification includes: a multi-classification model performs model calculation on the characteristic information of the detected traffic, and outputs a multi-dimensional confidence vector, wherein each element in the multi-dimensional confidence vector is a value between 0 and 1; according to the arrangement sequence number of the elements, the second identification result is determined, wherein the arrangement sequence number of the elements is the category corresponding to the traffic, and the category corresponding to the largest sequence number of the elements is the second identification result.
[0067] In one embodiment of the present disclosure, Figure 5 As shown, the abnormal traffic detection method provided in the embodiment of the present disclosure can determine the abnormal traffic proportion information through the following steps, and can dynamically update the preset judgment conditions:
[0068] S502, according to a preset time period, determining normal flow quantity information and abnormal flow quantity information within the preset time period;
[0069] S504: Determine abnormal traffic proportion information according to normal traffic quantity information and abnormal traffic quantity information within a preset time period.
[0070] In one embodiment of the present disclosure, Figure 6 As shown, the abnormal traffic detection method provided in the embodiment of the present disclosure can determine the data information of the traffic to be detected through the following steps, and can convert the traffic into a data format that can be recognized by the model:
[0071] S602, obtaining the traffic to be detected;
[0072] S604, determining characteristic information of the flow to be detected according to the flow to be detected;
[0073] S606, determining data information of the traffic to be detected based on the characteristic information of the traffic to be detected, wherein the data information is used to characterize the traffic to be detected and the characteristic information of the traffic to be detected, and the data information is in a data form recognizable by a machine learning model.
[0074] In one embodiment of the present disclosure, Figure 7 As shown, the abnormal traffic detection method provided in the embodiment of the present disclosure can perform different processing on the second recognition result through the following steps, so as to quickly identify and process the abnormal traffic:
[0075] S702, the second identification result is abnormal traffic, and the abnormal traffic is intercepted and processed;
[0076] S704: The second identification result is normal traffic, and the normal traffic is forwarded normally.
[0077] Figure 8 A flowchart of a specific example of an abnormal flow detection method according to an embodiment of the present disclosure is shown as follows: Figure 8 As shown, the abnormal traffic detection method provided in the embodiment of the present disclosure includes the following steps:
[0078] S801, the traffic is subjected to feature extraction and preprocessing by a feature extraction and preprocessing module, and the traffic features are converted into a form recognizable by the model.
[0079] S802, initializing abnormal traffic ratio, k=0.
[0080] S803, the time difference Δt is the total time of single model recognition minus the total time of multi-model recognition. When Δt>0, the multi-model recognition mode is used (enter S804), otherwise the single model recognition mode is used (enter S805). Therefore, based on Δt>0, the judgment condition with the abnormal traffic proportion k as the variable can be derived.
[0081] S804, the small model forwards the traffic identified as normal (confidence c ≥ threshold) normally, and sends the remaining traffic to the large model.
[0082] S805, the large model forwards the traffic identified as normal (Max{C} corresponding sequence number = normal traffic sequence number) normally, and performs subsequent processing on the traffic identified as abnormal.
[0083] S806, preset a time period, count the number of normal traffic N1 and the number of abnormal traffic N2 in the time period at regular intervals, and calculate the current abnormal traffic proportion k based on N1 and N2.
[0084] S807: Use the new k value as a basis for determining the subsequent recognition mode to determine whether the recognition mode needs to be switched.
[0085] In the above step S803, based on Δt>0, the judgment condition with abnormal flow ratio k as a variable can be derived, and the flow diagram is as follows: Fig. 9 As shown (where the recall rate r of the small model for normal traffic is known a , the recall rate of abnormal traffic r b, the time t for identifying a single flow; the time T for the large model to identify a single flow), deriving the judgment condition with the abnormal flow ratio k as a variable includes the following steps: obtaining N flows (equivalent to the data information of the above-mentioned flow to be detected), for N flows, the proportion of abnormal flows is k, k∈[0,1], so the real flow number is N(1-k), and the real abnormal flow number is Nk. In the single model recognition mode, only the large model is used for recognition, and the time required is NT. In the multi-model recognition mode, all flows will be recognized by the small model, and the time is Nt. The number of real normal flows is N(1-k), of which the number identified as abnormal flows is N(1-k)(1-r a ), the number of abnormal traffic identified in the real abnormal traffic is Nkr b , so finally a large model is needed to identify the total remaining flow as Nkr b +N(1-k)(1-r a )], the time required is [kr b +(1-k)(1-r a )]NT, so the total time of multi-model recognition is: Nt+[kr b +(1-k)(1-r a )]NT. Subtract the multi-model recognition time from the total single-model recognition time to get the time difference Δt. After simplifying Δt, we get the following formula. If Δt>0, the multi-model recognition mode is faster. If Δt<0, the single-model recognition mode is faster. Δt=N[k(1-r a -r b )+r a ]T-Nt. Based on Δt>0, the judgment condition (equivalent to the above-mentioned preset judgment condition) is obtained by taking the abnormal flow ratio k as the variable:
[0086] When a +r b <1,
[0087] When a +r b >1 hour,
[0088] When a +r b =1,
[0089] In response to the problem that current traffic detection based on deep learning technology is time-consuming, the present invention proposes an abnormal traffic detection method based on multiple models, which mainly includes a detection control module, a small binary classification model and a large multi-classification model. The detection control module selects different identification methods in real time based on the abnormal traffic proportion. After feature extraction and preprocessing, the traffic is jointly detected using a small binary classification model and a large multi-classification model under specific conditions, and only the large multi-classification model is used for detection under other conditions. This method combines the advantages of different models, selects the fastest traffic detection method in real time according to the abnormal traffic proportion in the traffic, and improves the speed while ensuring the accuracy of traffic detection.
[0090] Based on the same inventive concept, the present disclosure also provides an abnormal flow detection device, as described in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.
[0091] Fig.10 A schematic diagram of an abnormal flow detection device in an embodiment of the present disclosure is shown. Fig.10 As shown, the device includes: a data information acquisition module 1001, a data information judgment module 1002, a dual-model recognition module 1003, a single-model recognition module 1004, an abnormal traffic proportion information determination module 1005 and a traffic preprocessing module 1006 to be detected.
[0092] The data information acquisition module 1001 acquires data information of the traffic to be detected;
[0093] The data information judgment module 1002 is used to judge whether the data information of the flow to be detected meets the preset judgment condition, wherein the preset judgment condition is determined according to the abnormal flow ratio information;
[0094] The dual-model identification module 1003 is used to send the data information of the traffic to be detected to the binary classification model for a first identification when the data information of the traffic to be detected meets the preset judgment conditions. If the first identification is normal traffic, it is forwarded normally; if the first identification is abnormal traffic, the abnormal traffic and the feature information corresponding to the abnormal traffic are sent to the multi-classification model for a second identification, and different processing is performed according to the second identification result.
[0095] In one embodiment of the present disclosure, the above-mentioned abnormal traffic detection device also includes a single model identification module 1004: when the data information of the traffic to be detected does not meet the preset judgment conditions, the characteristic information of the traffic to be detected is sent to the multi-classification model for a second identification, and different processing is performed according to the second identification result.
[0096] In one embodiment of the present disclosure, the first identification in the above-mentioned dual-model identification module 1003 includes: the binary classification model performs model calculation on the characteristic information of the traffic to be detected and outputs the confidence; when the confidence is greater than the preset confidence threshold, it is identified as normal traffic; when the confidence is less than the preset confidence threshold, it is identified as abnormal traffic.
[0097] In one embodiment of the present disclosure, the second identification in the above-mentioned dual-model identification module 1003 includes: a multi-classification model performs model calculation on the characteristic information of the detected traffic, and outputs a multi-dimensional confidence vector, wherein each element in the multi-dimensional confidence vector is a value between 0 and 1; according to the arrangement sequence number of the elements, a second identification result is determined, wherein the arrangement sequence number of the elements is the category corresponding to the traffic, and the category corresponding to the largest sequence number of the elements is the second identification result.
[0098] In one embodiment of the present disclosure, the above-mentioned abnormal traffic detection device also includes an abnormal traffic proportion information determination module 1005: used to determine the normal traffic quantity information and the abnormal traffic quantity information within a preset time period according to a preset time period; determine the abnormal traffic proportion information according to the normal traffic quantity information and the abnormal traffic quantity information within the preset time period.
[0099] In one embodiment of the present disclosure, the above-mentioned abnormal flow detection device also includes a flow preprocessing module 1006 for detection: obtaining the flow to be detected; determining characteristic information of the flow to be detected based on the flow to be detected; determining data information of the flow to be detected based on the characteristic information of the flow to be detected, wherein the data information is used to characterize the flow to be detected and the characteristic information of the flow to be detected, and the data information is in a data form recognizable by a machine learning model.
[0100] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".
[0101] Refer to the following Fig.11 1100 according to this embodiment of the present disclosure is described. Fig.11 The electronic device 1100 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0102] like Fig.11As shown, the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 may include but are not limited to: at least one processing unit 1110, at least one storage unit 1120, and a bus 1130 connecting different system components (including the storage unit 1120 and the processing unit 1110).
[0103] The storage unit stores program codes, which can be executed by the processing unit 1110, so that the processing unit 1110 executes the steps described in the above “exemplary method” section of this specification according to various exemplary embodiments of the present disclosure.
[0104] For example, the processing unit 1110 can execute the following steps of the above method embodiment: obtaining data information of the traffic to be detected; judging whether the data information of the traffic to be detected meets the preset judgment conditions, wherein the preset judgment conditions are determined according to the abnormal traffic proportion information; if the data information of the traffic to be detected meets the preset judgment conditions, the data information of the traffic to be detected is sent to the binary classification model for a first identification, and if the first identification is normal traffic, it is forwarded normally; if the first identification is abnormal traffic, the abnormal traffic and the feature information corresponding to the abnormal traffic are sent to the multi-classification model for a second identification, and different processing is performed according to the second identification result.
[0105] For example, the processing unit 1110 can also execute the following steps of the above method embodiment: if the data information of the traffic to be detected does not meet the preset judgment conditions, the feature information of the traffic to be detected is sent to the multi-classification model for a second identification, and different processing is performed according to the second identification result.
[0106] In one example, the processing unit 1110 executes the first identification in the steps of the above method embodiment, including: a binary classification model performs model calculation on the characteristic information of the traffic to be detected and outputs a confidence level; when the confidence level is greater than a preset confidence threshold, it is identified as normal traffic; when the confidence level is less than a preset confidence threshold, it is identified as abnormal traffic.
[0107] In one instance, the processing unit 1110 executes the steps of the above method embodiment in the second identification, including: a multi-classification model performs model calculation on the characteristic information of the flow to be detected, and outputs a multi-dimensional confidence vector, wherein each element in the multi-dimensional confidence vector is a value between 0 and 1; according to the arrangement sequence number of the elements, a second identification result is determined, wherein the arrangement sequence number of the elements is the category corresponding to the flow, and the category corresponding to the largest sequence number of the elements is the second identification result.
[0108] For example, the processing unit 1110 can also execute the following steps of the above method embodiment: determine the normal traffic quantity information and abnormal traffic quantity information within the preset time period according to the preset time period; determine the abnormal traffic proportion information according to the normal traffic quantity information and abnormal traffic quantity information within the preset time period.
[0109] For example, the processing unit 1110 can also execute the following steps of the above method embodiment: obtaining the traffic to be detected; determining the characteristic information of the traffic to be detected based on the traffic to be detected; determining the data information of the traffic to be detected based on the characteristic information of the traffic to be detected, wherein the data information is used to characterize the traffic to be detected and the characteristic information of the traffic to be detected, and the data information is in a data form recognizable by a machine learning model.
[0110] For example, the processing unit 1110 may also perform the following steps of the above method embodiment: if the second identification result is abnormal traffic, the abnormal traffic is intercepted and processed; if the second identification result is normal traffic, the normal traffic is forwarded normally.
[0111] The storage unit 1120 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 11201 and / or a cache storage unit 11202 , and may further include a read-only storage unit (ROM) 11203 .
[0112] The storage unit 1120 may also include a program / utility 11204 having a set (at least one) of program modules 11205, such program modules 11205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0113] Bus 1130 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0114] The electronic device 1100 may also communicate with one or more external devices 1140 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1100, and / or communicate with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1150. Furthermore, the electronic device 1100 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1160. As shown, the network adapter 1160 communicates with other modules of the electronic device 1100 via a bus 1130. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0115] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.
[0116] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is further provided, which may be a readable signal medium or a readable storage medium. Fig.12 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is shown. Fig.12 As shown, a program product capable of implementing the above method of the present disclosure is stored on the computer-readable storage medium 1200. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.
[0117] For example, when the program product in the embodiment of the present disclosure is executed by a processor, the following steps are implemented: obtaining data information of the traffic to be detected; judging whether the data information of the traffic to be detected meets a preset judgment condition, wherein the preset judgment condition is determined based on the abnormal traffic proportion information; if the data information of the traffic to be detected meets the preset judgment condition, sending the data information of the traffic to be detected to a binary classification model for a first identification, and if the first identification is normal traffic, forwarding it normally; if the first identification is abnormal traffic, sending the abnormal traffic and the feature information corresponding to the abnormal traffic to a multi-classification model for a second identification, and performing different processing according to the second identification result.
[0118] In some embodiments, the program product in the embodiments of the present disclosure implements the following steps when executed by a processor: if the data information of the traffic to be detected does not meet the preset judgment conditions, the feature information of the traffic to be detected is sent to a multi-classification model for a second identification, and different processing is performed based on the second identification result.
[0119] In one example, when the program product in the embodiment of the present disclosure is executed by a processor, the first identification includes: a binary classification model performs model calculation on the characteristic information of the traffic to be detected and outputs a confidence level; when the confidence level is greater than a preset confidence threshold, it is identified as normal traffic; when the confidence level is less than a preset confidence threshold, it is identified as abnormal traffic.
[0120] In one instance, when the program product in the embodiment of the present disclosure is executed by the processor, the second identification includes: a multi-classification model performs model calculation on the characteristic information of the flow to be detected, and outputs a multi-dimensional confidence vector, wherein each element in the multi-dimensional confidence vector is a value between 0 and 1; according to the arrangement sequence number of the elements, the second identification result is determined, wherein the arrangement sequence number of the elements is the category corresponding to the flow, and the category corresponding to the largest sequence number of the elements is the second identification result.
[0121] In some embodiments, the program product in the embodiments of the present disclosure implements the following steps when executed by a processor: determining normal traffic quantity information and abnormal traffic quantity information within a preset time period based on a preset time period; determining abnormal traffic proportion information based on normal traffic quantity information and abnormal traffic quantity information within the preset time period.
[0122] In some embodiments, the program product in the embodiments of the present disclosure implements the following steps when executed by a processor: obtaining the traffic to be detected; determining characteristic information of the traffic to be detected based on the traffic to be detected; determining data information of the traffic to be detected based on the characteristic information of the traffic to be detected, wherein the data information is used to characterize the traffic to be detected and the characteristic information of the traffic to be detected, and the data information is in a data form recognizable by a machine learning model.
[0123] In some embodiments, the program product in the embodiments of the present disclosure implements the following method when executed by a processor: if the second identification result is abnormal traffic, the abnormal traffic is intercepted and processed; if the second identification result is normal traffic, the normal traffic is forwarded normally.
[0124] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0125] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0126] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0127] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).
[0128] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0129] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.
[0130] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.
[0131] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.
Claims
1. A method for detecting abnormal traffic, characterized in that: include: Obtain data information of the traffic to be detected; Determine whether the data information of the flow to be detected meets a preset judgment condition, wherein the preset judgment condition is determined according to the abnormal flow ratio information; If the data information of the traffic to be detected meets the preset judgment condition, the data information of the traffic to be detected is sent to the two-classification model for first identification. If the first identification is normal traffic, it is forwarded normally. If the first identification is abnormal traffic, the data information of the abnormal traffic is sent to the multi-classification model for second identification, and different processing is performed according to the second identification result; If the data information of the flow to be detected does not meet the preset judgment condition, the feature information of the flow to be detected is sent to the multi-classification model for second identification, and different processing is performed according to the second identification result; The preset judgment condition is: When a +r b <1 hour, When a +r b >1 hour, When a +r b =1, Among them, r a is the recall rate of the binary classification model for normal traffic, r b is the recall rate of the binary classification model for abnormal traffic, t is the time it takes for the binary classification model to identify a single traffic, T is the time it takes for the multi-classification model to identify a single traffic, and k is the proportion of abnormal traffic.
2. The abnormal flow detection method according to claim 1, characterized in that: The first identification includes: The binary classification model performs model calculation on the characteristic information of the flow to be detected and outputs a confidence level; When the confidence level is greater than the preset confidence threshold, it is identified as normal traffic; When the confidence level is less than the preset confidence threshold, it is identified as abnormal traffic.
3. The abnormal flow detection method according to claim 1, characterized in that: The second identification includes: The multi-classification model performs model calculation on the characteristic information of the traffic to be detected, and outputs a multi-dimensional confidence vector, wherein each element in the multi-dimensional confidence vector is a value between 0 and 1; A second identification result is determined according to the arrangement sequence number of the element, wherein the arrangement sequence number of the element is the category corresponding to the flow, and the category corresponding to the largest sequence number of the element is the second identification result.
4. The abnormal flow detection method according to claim 1, characterized in that: The method further comprises: According to a preset time period, determine normal flow quantity information and abnormal flow quantity information within the preset time period; The abnormal traffic ratio is determined based on the normal traffic volume information and the abnormal traffic volume information within a preset time period.
5. The abnormal flow detection method according to claim 1, characterized in that: The method further comprises: Get the traffic to be tested; Determining characteristic information of the flow to be detected according to the flow to be detected; According to the characteristic information of the flow to be detected, the data information of the flow to be detected is determined, wherein the data information is used to characterize the flow to be detected and the characteristic information of the flow to be detected, and the data information is in a data form recognizable by a machine learning model.
6. The abnormal flow detection method according to claim 1, characterized in that: Different processing based on the second recognition result includes: The second identification result is abnormal traffic, and the abnormal traffic is intercepted and processed; The second identification result is normal traffic, and the normal traffic is forwarded normally.
7. An abnormal flow detection device, characterized in that: include: A data information acquisition module is used to acquire data information of the traffic to be detected; A data information judgment module, used to judge whether the data information of the flow to be detected meets a preset judgment condition, wherein the preset judgment condition is determined according to the abnormal flow ratio information; The dual-model identification module is used to send the data information of the traffic to be detected to the binary classification model for first identification when the data information of the traffic to be detected meets the preset judgment condition. If the first identification is normal traffic, it is forwarded normally. If the first identification is abnormal traffic, the abnormal traffic and the feature information corresponding to the abnormal traffic are sent to the multi-classification model for second identification, and different processing is performed according to the second identification result; Single model identification module: when the data information of the traffic to be detected does not meet the preset judgment conditions, the feature information of the traffic to be detected is sent to the multi-classification model for a second identification, and different processing is performed according to the second identification result; The preset judgment condition is: When a +r b <1 hour, When a +r b >1 hour, When a +r b =1, Among them, r a is the recall rate of the binary classification model for normal traffic, r b is the recall rate of the binary classification model for abnormal traffic, t is the time it takes for the binary classification model to identify a single traffic, T is the time it takes for the multi-classification model to identify a single traffic, and k is the proportion of abnormal traffic.
8. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the abnormal traffic detection method according to any one of claims 1 to 6 by executing the executable instructions.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the abnormal traffic flow detection method according to any one of claims 1 to 6 is implemented.
