Abnormal traffic identification method, storage medium and electronic equipment

Abstract

The invention provides an abnormal traffic identification method, a storage medium and electronic equipment, and belongs to the technical field of computers. The method comprises the following steps: acquiring current traffic information of a current network request to be identified; detecting whether the current traffic information and a preset static abnormal traffic dictionary accord with a first matching property or not; when the current traffic information does not accord with the first matching property, detecting whether the current traffic information and the latest dynamic abnormal traffic dictionary accord with a second matching property or not; and when the current network request accords with the first matching performance or the second matching performance, judging that the current network request is abnormal traffic, and updating a preset dynamic abnormal traffic dictionary based on the current traffic information which is judged to be abnormal traffic. According to the invention, efficiency and comprehensiveness of abnormal traffic identification can be considered.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to an abnormal traffic recognition method, a storage medium, and an electronic device. Background Art

[0002] With the rapid development of Internet technology, as the core channel for data interaction, the security of APIs (Application Programming Interfaces) has become a key area of network security. Abnormal traffic (such as malicious access behaviors like DDoS attacks, SQL injections, and fraudulent crawlers) initiates realistic access through automated tools, and the attack methods are becoming increasingly concealed and diversified, posing a serious threat to the security and stability of API assets. How to efficiently identify and block abnormal traffic while reducing the impact on normal services has become an important challenge in the current field of network security.

[0003] Currently, the mainstream abnormal traffic detection technologies are mainly divided into the following two categories: One is the method based on feature rule matching. This method constructs a static rule library (such as blacklists and whitelists) by predefined traffic parameters of abnormal traffic (such as specific URL paths, abnormal parameter combinations, etc.), and filters the traffic through string matching or regular expressions. The other is the method based on machine learning. This method learns abnormal behavior patterns from historical traffic data by training classification models (such as supervised learning) or clustering models (such as unsupervised learning).

[0004] For the first type of method, although it can quickly identify abnormal traffic, the establishment of its rule library highly depends on manual experience, and the coverage rate of abnormal traffic recognition is not high; for the second type of method, the accuracy of machine learning highly depends on the quality and representativeness of training data, has strict requirements for data annotation, and at the same time, since machine learning consumes a large amount of computing resources, it will lead to untimely determination of whether an access request is abnormal, and it is difficult to meet the real-time abnormal traffic detection requirements of high throughput and low latency. Summary of the Invention

[0005] The purpose of the present invention is to provide a new abnormal traffic recognition method, a storage medium, and an electronic device to solve at least one technical problem in the prior art and balance the accuracy and real-time performance of abnormal traffic recognition.

[0006] In the first aspect of this application, an abnormal traffic recognition method is provided. The method includes: Obtain the current traffic information of the current network request to be recognized; Detect whether the current traffic information conforms to a first matching property with a preset static abnormal traffic dictionary; When it does not conform to the first matching property, detect whether the current traffic information conforms to a second matching property with the latest dynamic abnormal traffic dictionary; When the first matching condition or the second matching condition is met, it is determined that the current network request is abnormal traffic, and the preset dynamic abnormal traffic dictionary is updated based on the current traffic information determined to be abnormal traffic.

[0007] Optionally, detecting whether the second matching condition is met between the current traffic information and the latest dynamic abnormal traffic dictionary includes: Identifying whether one or more traffic parameters identical to the current traffic information are included in the latest dynamic abnormal traffic dictionary, where the traffic parameters include a combination of one or more of source IP, destination IP, source port, destination port, request header, return result, and return status code; Comparing whether the frequency or number of occurrences of the same one or more traffic parameters in the dynamic abnormal traffic dictionary exceeds the corresponding frequency threshold or number threshold, and when it exceeds the corresponding frequency threshold or number threshold, it is determined that the current traffic information meets the second matching condition.

[0008] Optionally, the traffic parameters include main traffic parameters and auxiliary traffic parameters, the frequency thresholds include a first frequency threshold and a second frequency threshold, and the number thresholds include a first number threshold and a second number threshold; The comparing whether the frequency or number of occurrences of the same one or more traffic parameters in the dynamic abnormal traffic dictionary exceeds the corresponding frequency threshold or number threshold, and when it exceeds the corresponding frequency threshold or number threshold, it is determined that the current traffic information meets the second matching condition, includes: When the same traffic parameters include at least one main traffic parameter, if the frequency or number of occurrences of any one main traffic parameter in the dynamic abnormal traffic dictionary exceeds the corresponding first frequency threshold or first number threshold, it is determined that the current traffic information meets the second matching condition; When the same traffic parameters include multiple auxiliary traffic parameters, if more than a preset number of auxiliary traffic parameters have a frequency or number of occurrences in the dynamic abnormal traffic dictionary that exceeds the corresponding first frequency threshold or first number threshold, it is determined that the current traffic information meets the second matching condition.

[0009] Optionally, the current traffic information includes a request time, and the validity period of the latest dynamic abnormal traffic dictionary is within a preset time sliding window corresponding to the request time. The updating of the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic includes: Adding the current traffic information to the preset dynamic abnormal traffic dictionary, where the dynamic abnormal traffic dictionary contains traffic information determined to be abnormal traffic within the time sliding window.

[0010] Optionally, after updating the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic, the method further includes: Re-extracting the historical traffic information of historical network requests within the time sliding window that have not been determined to be abnormal traffic; Detecting whether there is a second matching between the historical traffic information and the latest dynamic abnormal traffic dictionary; Determining the historical network requests corresponding to the historical traffic information that meets the second matching as abnormal traffic, and updating the preset dynamic abnormal traffic dictionary based on the historical traffic information determined to be abnormal traffic.

[0011] Optionally, the static abnormal traffic dictionary includes a first static abnormal traffic dictionary. Detecting whether there is a first matching between the current traffic information and the preset static abnormal traffic dictionary includes: detecting whether any abnormal traffic feature existing in the first static abnormal traffic dictionary is included in the current traffic information. If it is included, it is determined that the first matching is met.

[0012] Optionally, the static abnormal traffic dictionary includes a second static abnormal traffic dictionary. Detecting whether there is a first matching between the current traffic information and the preset static abnormal traffic dictionary includes: performing fuzzy matching between the current traffic information and the sensitive words in the second static abnormal traffic dictionary. When the fuzzy matching is successful, it is determined that the first matching is met.

[0013] Optionally, the method further includes: obtaining a traffic information sample set including abnormal traffic labels and normal traffic labels; extracting abnormal traffic features from the traffic information sample set; and constructing a static abnormal traffic dictionary based on the extracted abnormal traffic features.

[0014] In a second aspect of the present application, there is provided a computer-readable storage medium, on which executable instructions are stored. When the executable instructions are executed by a processor, the processor is caused to execute the abnormal traffic recognition method as described in any embodiment of the present application.

[0015] In a third aspect of the present application, there is provided an electronic device, including: one or more processors; a memory for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the abnormal traffic recognition method as described in any one of the embodiments of the present application.

[0016] The abnormal traffic identification method storage medium and electronic device in the present application set up two modes, static identification and dynamic identification. First, the network requests that obviously belong to abnormal traffic are quickly identified through the static identification method, and the dynamic abnormal traffic dictionary is updated for the identified abnormal traffic. The network requests that cannot be quickly identified as abnormal traffic are detected through the dynamic abnormal traffic dictionary to avoid omissions in abnormal traffic identification, thereby taking into account both the efficiency and comprehensiveness of abnormal traffic identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope of the present application.

[0018] Figure 1 A schematic diagram of a flow chart of an abnormal traffic identification method in one embodiment; Figure 2 A schematic diagram of a process of backtracking anomaly detection for historical traffic requests in one embodiment; Figure 3 A schematic diagram of a process for constructing a static abnormal traffic dictionary in one embodiment; Figure 4 FIG. 1 is a schematic diagram of the structure of an electronic device in an embodiment. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] All terms (including technical and scientific terms) used in this application have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used here should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0021] For example, the terms "first", "second", etc. used in this application are only used to distinguish similar objects and to differentiate the first object from another object, rather than to describe a specific order or sequence, and cannot be understood as indicating or implying relative importance.

[0022] This application proposes a method for identifying abnormal traffic. Figure 1 As shown, the method includes: Step 110, obtaining current traffic information of the current network request to be identified.

[0023] In this embodiment, a network request refers to a data request sent by a client (such as a browser, an APP, etc.) to a server, which is used to obtain resources on the server or perform certain operations. A network request usually contains one or more of the following contents: a request method (such as GET, POST, etc.), a request URL, a request header (Header), and a request body (Body).

[0024] Traffic information refers to the packet information generated during network transmission, which records the detailed data of network requests and responses. Traffic information may include one or more traffic parameters such as a source IP address, a destination IP address, a source port, a destination port, a protocol type, a timestamp, a request URL, a request header (Header), and a request body (Body), a return result, and a return status code. By analyzing the traffic information, abnormal behaviors or malicious traffic (i.e., abnormal traffic) in the network can be identified.

[0025] For example, when a user enters a website address (URL) in a browser, the browser sends a DNS query request to a DNS server to request the resolution of the IP address corresponding to the URL. Then this network request is a DNS query request; when the user clicks the "Login" button on a web page, the browser sends a POST request to the server, and the request contains the user's login information (such as a username, a password, etc.). Then this network request is a POST request; when the user clicks the "Refresh Data" button in an APP, the APP sends a GET request to the server to request the latest data. Then this network request is a GET request.

[0026] The traffic information corresponding to the DNS query request may include information such as a source IP address (the user's IP address), a destination IP address (the server's IP address), a source port (usually a randomly assigned temporary port), a destination port (usually the default port 80 of the HTTP protocol or the default port 443 of the HTTPS protocol), a protocol type (TCP), a timestamp (the time when the request is sent), an HTTP request header, and a request body; the traffic information corresponding to the POST request, in addition to including the above-mentioned several kinds of information, may also include the login information submitted by the user, such as a username, a password, etc.; if the server responds to the user's request, the returned traffic information includes one or more of the following traffic parameters: a source IP address (the server's IP address), a destination IP address (the user's IP address), a source port (the server's port, the same as the destination port during the request), a destination port (the user's port, the same as the source port during the request or a randomly assigned port), a protocol type (TCP), a timestamp (the time when the response is sent), an HTTP response header, and a response body.

[0027] An electronic device may receive a large number of network requests at the same time or within a short period of time, and it is necessary to identify whether these network requests belong to malicious / abnormal requests (i.e., abnormal traffic). For example, the abnormal traffic may be fuzz attack traffic, SQL injection traffic, malicious interface traffic, etc.

[0028] The network requests received within the current time period are the current network requests. The network requests to be identified are the network requests for which abnormal traffic analysis needs to be performed, and the traffic information of the current network requests is the current traffic information.

[0029] Among them, the current time period can be any suitable time period set in advance. For example, it can be the time period of the most recent duration, or any suitable time period divided according to a preset duration threshold. For example, the duration of each time period is 1 minute, 2 minutes, 5 minutes, etc., or all the network requests received within the most recent 1 minute are regarded as the current network requests.

[0030] Step 120: Detect whether there is a first match between the current traffic information and a preset static abnormal traffic dictionary. When there is no first match, execute step 130. When there is a first match, execute step 140.

[0031] In this embodiment, the static abnormal traffic dictionary is a pre-established database containing abnormal traffic characteristics. The characteristics in the static abnormal traffic dictionary can be derived from historical attack cases, the experience summary of security experts, or industry-wide malicious traffic identification rules, and are extracted from the traffic parameters in the abnormal traffic information. The characteristics in the static abnormal traffic dictionary are usually described in ways such as regular expressions and string matching to facilitate quick comparison with the traffic information. The static abnormal traffic dictionary is a relatively fixed dictionary, and the characteristics in it remain unchanged or are not updated within a certain period of time.

[0032] When it is detected that the traffic parameters in the current traffic information match the characteristics in the static abnormal traffic dictionary, it is determined that there is a first match. The first match can include one or more of the following ways: complete match, partial match, fuzzy match, or match based on behavior rules. For example, when there is a complete match with a certain characteristic in the static abnormal traffic dictionary, it is determined that there is a first match. Or when there is an incomplete match with another or several characteristics in the static abnormal traffic dictionary, but the fuzzy match is satisfied, it is still determined that there is a first match. The characteristics in the traffic information can be one or more of the above traffic parameters such as source IP address, destination IP address, source port, destination port, protocol type, timestamp, HTTP response header, response body, etc.

[0033] The static abnormal traffic dictionary may specifically include one or more of a URL dictionary, a ResBody dictionary, and a fuzz dictionary. Among them, the URL dictionary includes a URL base path dictionary and a URL query parameter dictionary, and the ResBody dictionary is a sensitive word dictionary for the interface response body.

[0034] For example, the sensitive words (i.e., features) included in the URL dictionary may be npm-group, Centos, CentOS, LinuX, scLo, openEuler-22.03-LTS-SP1, etc.; the sensitive words (i.e., features) included in the ResBody dictionary may be "Turn down", "Disagree", "503 Service Unavailable", "418 I'm a teapot", "400 BadRequest", etc.; the sensitive words (i.e., features) included in the fuzz dictionary may be "~SyS~#", "~tmp ~user@", "~webmaster^", " / \bxss.me%", "_debug_ / render_panel!", etc. When any one or more of these features are included in the traffic parameters of a network request, it can be determined that the first matching is met.

[0035] Step 130, detect whether the current traffic information and the latest dynamic abnormal traffic dictionary meet the second matching. When the second matching is met, execute Step 140, otherwise, execute Step 150.

[0036] In this embodiment, similar to the features in the static abnormal traffic dictionary, the dynamic abnormal traffic dictionary also includes the features of network requests suspected of being abnormal traffic. Among them, the dynamic abnormal traffic dictionary is a traffic dictionary that is set to be continuously updated, and the features in it are in a continuously updated state. For example, the features in the dynamic abnormal traffic dictionary may include the source IP, the destination IP, the source port, the destination port, and the request header.

[0037] In one implementation, in addition to being continuously updated dynamically, the features recorded in the dynamic abnormal traffic dictionary further update the cumulative count of each feature in the dictionary.

[0038] The second matching rule can be the same as or similar to the first matching rule. For example, it can also be one or more of exact matching, partial matching, fuzzy matching, or behavior rule-based matching, etc. It can also include a combination of one or several matching methods such as the number of times and frequency of the matching features appearing in the dynamic abnormal traffic dictionary, and the number of traffic parameters of the matching features. For example, the dynamic abnormal traffic dictionary contains features A, B, C, D, E, etc. By continuously updating the dynamic abnormal traffic dictionary, at the current moment, the cumulative number (or cumulative times) of feature A recorded in it is 5, the cumulative times of feature B is 8, and the cumulative times of feature C is 10. When it is detected that the traffic parameters of the current traffic information also contain feature C that appears 10 times in the dynamic abnormal traffic dictionary, it can be determined that the second matching rule is met. Or when it is detected that the traffic parameters of the current traffic information contain features A, B, D, and E in the dynamic abnormal traffic dictionary, that is, 4 features in the dynamic abnormal traffic dictionary appear, it can be determined that the second matching rule is met.

[0039] Step 140: Determine that the current network request is abnormal traffic, and update the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic.

[0040] In this embodiment, when the network request is determined to be abnormal traffic, the dynamic abnormal traffic dictionary is updated according to the traffic information of the abnormal traffic. Among them, one or more features can be extracted from the traffic parameters of the abnormal traffic and updated to the dynamic abnormal traffic dictionary. At the same time, the number of the same feature appearing in the dynamic abnormal traffic dictionary can also be updated. For example, when a certain network request is identified as abnormal traffic, the traffic parameters such as the source IP, source port, target IP, target port, response success and / or failure corresponding to the network request are updated to the dynamic abnormal traffic dictionary as features in the dynamic abnormal traffic dictionary. Furthermore, the number of times (i.e., the cumulative times) of each feature appearing is recorded. For example, at the current moment, a certain source IP in the dynamic abnormal traffic dictionary has appeared 100 times.

[0041] Step 150: Determine that the current network request is normal traffic.

[0042] In this embodiment, when it is recognized that a certain network request does not conform to the first matching property nor the second matching property, it can be directly determined that it belongs to normal traffic, and a normal response can be made to the network request. Or other methods can also be used to further identify abnormal traffic. When it is recognized that it does not belong to abnormal traffic, it is determined to be normal traffic.

[0043] When it belongs to abnormal traffic, detailed log information is recorded, including request time, source IP address, destination IP address, request type, request content, etc., for subsequent analysis and tracing; according to the recorded log information and context, the cause of the abnormal traffic is deeply analyzed to determine whether there are security vulnerabilities or malicious behaviors. Block the network requests of abnormal traffic to prevent further damage to the system; send an alert to the system administrator or security team to notify the occurrence of abnormal traffic and provide the necessary log information and context for quick response and handling.

[0044] In the abnormal traffic recognition method of this application, by setting two methods of static recognition and dynamic recognition, first, the network requests that are obviously abnormal traffic are quickly recognized through the static recognition method. For the recognized abnormal traffic, the dynamic abnormal traffic dictionary is updated. For the network requests that cannot be quickly recognized as abnormal traffic, they are detected through the dynamic abnormal traffic dictionary, avoiding omission of abnormal traffic recognition, so as to balance the efficiency and comprehensiveness of abnormal traffic recognition.

[0045] In one embodiment, the current traffic information includes the request time, and the validity period of the latest dynamic abnormal traffic dictionary is within a preset time sliding window corresponding to the request time; updating the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic includes: adding the current traffic information to the preset dynamic abnormal traffic dictionary, and the dynamic abnormal traffic dictionary contains the traffic information determined to be abnormal traffic within the time sliding window.

[0046] In this embodiment, the length of the sliding time window can be adjusted and set according to actual needs, such as it can be set to any appropriate duration such as 10 minutes, 5 minutes, 20 minutes, 1 hour, 2 hours, etc.

[0047] In one embodiment, the length of the sliding time window can be determined according to the timestamps of abnormal traffic with an association relationship. Generally speaking, the abnormal network requests initiated by the same source IP have a certain persistence. The electronic device forms a network traffic dataset D by collecting the three-dimensional network features x of the timestamp t, source IP, and destination IP of the abnormal traffic. D = {x1, x2, …, xn}, n≥2, and each network feature x in the traffic dataset D contains information in three dimensions: timestamp t, source IP, and destination IP. By analyzing the duration interval between the timestamps t between the same source IP and destination IP, the length of the corresponding sliding time window is calculated.

[0048] For example, a continuous sequence of the same abnormal traffic can be identified from the dataset, and the time difference between the earliest request time and the latest request time in each continuous sequence of the same abnormal traffic can be calculated. Based on each calculated time difference, the length of the sliding time window is determined. For example, the length can be the weighted average of each time difference. Among them, the abnormal traffic in the continuous sequence of the same abnormal traffic has the same source IP and destination IP, and when the abnormal traffic in the sequence is sorted in ascending or descending order according to the time stamp t, the time stamp difference between adjacent abnormal traffic is within a preset duration threshold. The duration threshold is any appropriately set threshold.

[0049] For the determined length, the time period of each sliding time window or the validity period of the sliding time window can be determined. The validity period of the sliding time window is a dynamic validity period or a static validity period determined according to the above length. The static validity period can be a fixed time period. Taking the determined duration of 5 minutes as an example, the static validity period of the sliding time window is 5 minutes. A sliding time window is divided every 5 minutes, and according to the time period in which the current moment is located, the sliding time window corresponding to the current moment is determined. After entering a new sliding time window, the dynamic abnormal traffic dictionary can be cleared and the dynamic abnormal traffic dictionary can be updated again.

[0050] The dynamic time period can be a time period of the most recent duration (such as 5 minutes) with the current moment as the cut-off moment. The traffic information of the abnormal traffic outside the most recent duration is discarded, and the traffic information of the abnormal traffic within the most recent duration is added to the dynamic abnormal traffic dictionary, and the traffic information of the abnormal traffic outside the most recent duration is cleared from the dynamic abnormal traffic dictionary. For one or more features extracted from the traffic parameters of the abnormal traffic, if they are within the validity period, they are added to the dynamic abnormal traffic dictionary, and if they are outside the validity period, the originally added features are deleted. At the same time, the quantity of the same feature appearing in the dynamic abnormal traffic dictionary can also be updated (increased or decreased).

[0051] Since network malicious behavior is a continuous behavior, when abnormal behavior is found in the flow of a source IP, it is very likely that the network behavior in the adjacent time period (time sliding window) is still abnormal. Therefore, by setting the validity period of the dynamic abnormal traffic dictionary and updating the dynamic abnormal traffic dictionary within this validity period, the recognition accuracy of continuous abnormal traffic can be improved.

[0052] In one embodiment, the static abnormal traffic dictionary includes a first static abnormal traffic dictionary. Detecting whether there is a first match between the current traffic information and the preset static abnormal traffic dictionary includes: detecting whether any abnormal traffic feature existing in the first static abnormal traffic dictionary is included in the current traffic information. If it is included, it is determined that there is a first match.

[0053] In this embodiment, the abnormal traffic features included in the first static abnormal traffic dictionary are features that are clearly within abnormal traffic. The abnormal traffic features may include specific URL paths, request parameters, keywords or phrases in the response body content that are clearly abnormal traffic, such as the above-mentioned CentOS, "400 Bad Request" and other features. For the current traffic information, it can be detected whether any feature in the static abnormal traffic dictionary is included. If it is included, it is determined that there is a first match. This inclusion means that there is a feature in the current traffic information that is exactly the same as any abnormal traffic feature in the static abnormal traffic dictionary. When there is a first match, it can be determined that the current network request is abnormal traffic.

[0054] In one embodiment, the static abnormal traffic dictionary includes a second static abnormal traffic dictionary. Detecting whether there is a first match between the current traffic information and the preset static abnormal traffic dictionary includes: performing fuzzy matching between the current traffic information and the sensitive words in the second static abnormal traffic dictionary. When the fuzzy matching is successful, it is determined that there is a first match.

[0055] In this embodiment, the second static abnormal traffic dictionary is a supplement to the first static abnormal traffic dictionary. The abnormal traffic features in the second static abnormal traffic dictionary mainly target the features that are difficult to match with the abnormal traffic features in the first static abnormal traffic dictionary. This is mainly because of the existence of relevant special characters (such as!@#$ etc.) that make it difficult to match. For example, for a network request that does not meet the first match with the first static abnormal traffic dictionary, its first match with the second static abnormal traffic dictionary can be detected. At this time, it can be detected whether the relevant features in the traffic information are fuzzy matched with any abnormal traffic feature in the second static abnormal traffic dictionary. If a fuzzy match can be made, it is determined that the network request meets the first match. Among them, the matching method can be carried out according to the rules of regular expressions.

[0056] By setting the second static abnormal traffic dictionary to include abnormal traffic features that are difficult to fully match due to special characters, etc., and using the method of fuzzy matching to perform the first match detection, the comprehensiveness of the features included in the static abnormal traffic dictionary is further improved, and the comprehensiveness of static detection is improved.

[0057] In one embodiment, detecting whether there is a second match between the current traffic information and the latest dynamic abnormal traffic dictionary includes: identifying whether one or more traffic parameters identical to the current traffic information are included in the latest dynamic abnormal traffic dictionary, where the traffic parameters include one or more combinations of source IP, destination IP, source port, destination port, request header, return result, and return status code; comparing whether the frequency or number of occurrences of the same one or more traffic parameters in the dynamic abnormal traffic dictionary exceeds the corresponding frequency threshold or number threshold, and when it exceeds the corresponding frequency threshold or number threshold, determining that the current traffic information meets the second match.

[0058] In this embodiment, the traffic parameter can be a single traffic parameter or a combination of multiple traffic parameters. For example, the traffic parameter can be a single parameter such as source IP, destination IP, source port, destination port, etc., or a traffic parameter combination formed by 4 single parameters such as source IP, destination IP, source port, and destination port.

[0059] Since the dynamic abnormal traffic dictionary includes not only the traffic parameters themselves but also the statistical information of the current traffic parameters. For example, the number of times a certain source IP-A is recorded in the dynamic abnormal traffic dictionary is 100 times, which means that in the corresponding time sliding window, 100 network requests with the source IP being source IP-A are determined to be abnormal traffic.

[0060] The frequency thresholds or number thresholds corresponding to each traffic parameter are not necessarily the same. For example, the number threshold corresponding to the source IP is 10,000 times, and the number threshold corresponding to the traffic parameter combination formed by source IP, destination IP, source port, and destination port is 100 times.

[0061] For example, when it is detected that in the current network request, the traffic parameter combination formed by source IP, destination IP, source port, and destination port also exists in the dynamic abnormal traffic dictionary and the number of occurrences exceeds 100 times, it indicates that the current network request meets the second match. Since the traffic parameters appearing in the dynamic abnormal traffic dictionary are all parameters extracted from abnormal traffic, the same traffic parameters also exist in the current network request, and when the corresponding number threshold or frequency threshold is exceeded, it indicates that the current network request is also abnormal traffic. If there are the same traffic parameters, but each same traffic parameter does not reach the corresponding frequency threshold or number threshold, the network request is not determined to be abnormal traffic.

[0062] The introduction of the dynamic abnormal traffic dictionary enables the system to flexibly respond to constantly changing network attack means, improving the accuracy and timeliness of abnormal traffic identification. At the same time, through the methods of combined matching and frequency / number comparison, the detection ability and robustness of the system are further enhanced.

[0063] In one embodiment, the traffic parameters include a primary traffic parameter and a secondary traffic parameter, the frequency thresholds include a first frequency threshold and a second frequency threshold, and the count thresholds include a first count threshold and a second count threshold. Compare whether the frequency or count of the same one or more traffic parameters appearing in the dynamic abnormal traffic dictionary exceeds the corresponding frequency threshold or count threshold. When it exceeds the corresponding frequency threshold or count threshold, it is determined that the current traffic information meets the second matching criterion, including: when the same traffic parameters include at least one primary traffic parameter, if the frequency or count of any primary traffic parameter appearing in the dynamic abnormal traffic dictionary exceeds the corresponding first frequency threshold or first count threshold, it is determined that the current traffic information meets the second matching criterion; when the same traffic parameters include multiple secondary traffic parameters, if more than a preset number of secondary traffic parameters have a frequency or count of appearance in the dynamic abnormal traffic dictionary that exceeds the corresponding first frequency threshold or first count threshold, it is determined that the current traffic information meets the second matching criterion.

[0064] In this embodiment, the traffic parameters in the dynamic abnormal traffic dictionary are further divided into two parameter types, namely the primary traffic parameter and the secondary traffic parameter. Among them, the primary traffic parameter refers to a parameter that has a decisive influence on the identification of abnormal traffic, such as the source IP, destination IP, request header, etc.; while the secondary traffic parameter is a supplement to the primary traffic parameter, such as the source port, destination port, return status code, etc. Further, the primary traffic parameter is a combination of multiple single traffic parameters. For example, a traffic parameter combination formed by the above-mentioned source IP, destination IP, source port, and destination port is a primary traffic parameter. And the secondary traffic parameter is usually a single traffic parameter.

[0065] As described above, the first frequency threshold / first count threshold (for the primary traffic parameter) and the second frequency threshold / second count threshold corresponding to different traffic parameters (including the primary traffic parameter and the secondary traffic parameter) are not necessarily the same.

[0066] If any primary traffic parameter in the current network request appears in the dynamic abnormal traffic dictionary and the count / frequency of its appearance exceeds the corresponding first frequency threshold / first count threshold of the primary traffic parameter, it is directly determined that the current network request is abnormal traffic.

[0067] If there is no primary traffic parameter in the current network request that appears in the dynamic abnormal traffic dictionary, or the existing primary traffic parameters do not exceed the corresponding abnormal traffic, then detect whether the current network request includes secondary traffic parameters in the dynamic abnormal traffic dictionary. If so, count the number of times / frequency of appearance of each same secondary traffic parameter in the dynamic abnormal traffic dictionary, count the same secondary traffic parameters that exceed the corresponding frequency / count of appearance, and determine whether it belongs to abnormal traffic based on the same secondary traffic parameters that exceed the corresponding frequency / count of appearance.

[0068] For example, in a current network request, the same auxiliary traffic parameters that exceed the corresponding occurrence frequency / number of occurrences respectively include the source IP, source port, access result of response failure (return status code), and strings in a certain same URL path. Then, it is determined that the network request belongs to abnormal traffic. Among them, the preset quantity can be a fixed quantity, or determined according to specific same auxiliary traffic parameters and their occurrence times / frequencies. Different auxiliary traffic parameters and occurrence times / frequencies are not necessarily the same. Specifically, the higher the occurrence times / frequency of the auxiliary traffic parameter, the fewer the corresponding preset quantity.

[0069] In this embodiment, by setting the main traffic parameter and the auxiliary traffic parameter, it is possible to not only ensure the comprehensiveness of abnormal traffic identification, but also improve the accuracy of abnormal traffic identification.

[0070] In one embodiment, as Figure 2 shown, after updating the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic, the method further includes retrospective abnormal detection for historical traffic requests. The process of the retrospective abnormal detection includes: Step 210, re-extract the historical traffic information of historical network requests that have not been determined to be abnormal traffic within the time sliding window.

[0071] In this embodiment, abnormal detection retrospective can be performed immediately after each update of the dynamic abnormal traffic dictionary, or according to a preset retrospective frequency. For example, taking the time sliding window as a fixed time period as an example, at a certain moment before the expiration of the current time sliding window (such as 1 minute or 30 seconds or 10 seconds before expiration, etc., any appropriate moment), extract the traffic information of all network requests that have not been determined to be abnormal traffic during this time period. When the time sliding window has a dynamic validity period, historical traffic information can be extracted every 1 / 2 or 1 / 3 of the window duration. Specifically, the network requests extracted are the network requests re-extracted for the first time.

[0072] Step 220, detect whether there is a second matching between the historical traffic information and the latest dynamic abnormal traffic dictionary.

[0073] Step 230, determine the historical network requests corresponding to the historical traffic information that meets the second matching as abnormal traffic, and update the preset dynamic abnormal traffic dictionary based on the historical traffic information determined to be abnormal traffic.

[0074] In this embodiment, the detection method for the second matching of the re-extracted historical network requests is the same as that for the second matching of the current traffic request, except that the dynamic abnormal traffic dictionary at this time is the updated traffic dictionary.

[0075] Specifically, it is identified whether one or more traffic parameters identical to the re-extracted historical traffic information are included in the latest dynamic abnormal traffic dictionary; it is compared whether the frequency or number of occurrences of the identical one or more traffic parameters in the dynamic abnormal traffic dictionary exceeds the corresponding frequency threshold or number threshold. When the corresponding frequency threshold or number threshold is exceeded, it is determined that the current traffic information conforms to the second matching property.

[0076] In one embodiment, when the identical traffic parameters in the re-extracted historical traffic information include at least one main traffic parameter, if the frequency or number of occurrences of any one main traffic parameter in the dynamic abnormal traffic dictionary exceeds the corresponding first frequency threshold or first number threshold, it is determined that the current traffic information conforms to the second matching property; when the identical traffic parameters in the re-extracted historical traffic information include multiple auxiliary traffic parameters, if the number of auxiliary traffic parameters exceeding a preset quantity has a frequency or number of occurrences in the dynamic abnormal traffic dictionary that exceeds the corresponding first frequency threshold or first number threshold, it is determined that the current traffic information conforms to the second matching property.

[0077] By performing retrospective detection on historical traffic requests, since the dynamic abnormal traffic dictionary has been updated, requests that were not previously identified but actually conform to abnormal characteristics can be discovered, thereby improving the coverage rate of abnormal detection, promptly performing abnormal processing on historical traffic re-determined as abnormal traffic, and avoiding greater losses. Updating the dictionary based on the newly identified abnormal traffic information can further enhance the system's ability to identify subsequent abnormal traffic.

[0078] In one embodiment, as Figure 3 shown, the above method further includes a process of constructing a static abnormal traffic dictionary, and this process includes: Step 310, obtaining a traffic information sample set containing abnormal traffic labels and normal traffic labels.

[0079] Step 320, extracting abnormal traffic characteristics from the traffic information sample set.

[0080] Step 330, constructing a static abnormal traffic dictionary based on the extracted abnormal traffic characteristics.

[0081] The sample set can come from the historical data of a network traffic monitoring system, or from a known network attack data set or a publicly available network traffic data set. These sample sets should contain a sufficient number of abnormal traffic and normal traffic to ensure the accuracy and representativeness of feature extraction.

[0082] Each traffic information in the sample set is labeled with one of the labels of abnormal traffic or normal traffic. These labels can be determined through manual annotation, automatic annotation based on known attack patterns, or reports generated by using other network security tools.

[0083] Before extracting features, it is necessary to determine which features are important for distinguishing abnormal traffic from normal traffic. These features can include packet size, connection duration, number of packets, source / destination IP addresses, source / destination port numbers, protocol types, etc.

[0084] Specifically, machine learning algorithms (such as decision trees, random forests, support vector machines, etc.) or statistical methods (such as clustering analysis, principal component analysis, etc.) can be used to extract the features of abnormal traffic from the sample set. Through relevant algorithms, it is possible to identify which features have significant differences between abnormal traffic and normal traffic. Among them, for the features that are clearly identified as only belonging to abnormal traffic, they are used as the abnormal traffic features in the static abnormal traffic dictionary and added to the static abnormal traffic dictionary.

[0085] In one embodiment, after step 310, it further includes: determining the traffic parameters of the dynamic abnormal traffic dictionary from the traffic information sample set.

[0086] For the non-abnormal traffic features among them, further identification can be performed to extract the features that appear frequently in abnormal traffic, and for the frequency or number of occurrences of these features, appropriate number thresholds or frequency thresholds are set, and the extracted features are combined. Appropriate number thresholds or frequency thresholds are also set for the combined features to form matching rules for the dynamic abnormal traffic dictionary.

[0087] Through the above steps, the parameters applicable to the static abnormal traffic dictionary and the parameters of the dynamic abnormal traffic dictionary can be determined from the traffic information sample set. These parameters are not only representative but also capable of capturing the dynamic changes of abnormal traffic, providing strong support for subsequent abnormal traffic identification. At the same time, by continuously optimizing and updating the parameters, the accuracy and adaptability of the static abnormal traffic dictionary and the dynamic abnormal traffic dictionary can be ensured, and the effectiveness of network security protection can be improved.

[0088] The abnormal traffic recognition method in this application adds a dynamic abnormal traffic dictionary on the basis of the static abnormal traffic dictionary. On the premise of ensuring the high throughput computing performance of the system, it increases the detection coverage of abnormal behaviors, increases the dynamic regulation of real-time abnormal traffic, and enhances the adaptability to unknown traffic. This application is applied to the interface aggregation scenario of API asset sorting, which can effectively reduce the workload of API asset sorting and reduce the interference of abnormal traffic; based on historical network traffic behaviors and industry experience, it constructs a behavior rule library for abnormal traffic to effectively identify abnormal traffic; by introducing dynamic rules, it adaptively updates the sensitive word dictionary library and the behavior rule library to identify unknown traffic and solve the problem of poor adaptability to unknown traffic of the sensitive word filtering method. At the same time, it introduces a time sliding window to increase the dynamic regulation of real-time abnormal traffic and reduce the influence range and false alarm rate of dynamic rules.

[0089] In one embodiment, a computer storage medium is provided, on which executable instructions are stored. When the instructions are executed by a processor, the processor executes the steps in the above-mentioned embodiments of the abnormal traffic recognition method.

[0090] In one embodiment, an electronic device is proposed, which includes a memory and a processor. When the computer program stored in the memory is executed by the processor, the processor executes the steps of the abnormal traffic recognition method in any of the above embodiments.

[0091] In one embodiment, an electronic device is provided. The electronic device can specifically be a terminal or a server. As Figure 4 shown, the electronic device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 into the random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other through a bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0092] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as required. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 410 as required so that a computer program read therefrom is installed into the storage section 408 as required.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0094] In addition, those skilled in the art can understand that although some of the embodiments herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments means that it is within the scope of the present application and forms different embodiments. For example, all the above embodiments can be used in any combination. The information disclosed in this background art section is only intended to deepen the understanding of the overall background art of the present application, and should not be regarded as an admission or any form of implication that this information constitutes the prior art already known to those skilled in the art.

Claims

1. An abnormal traffic recognition method, characterized in that, The method includes: Obtaining the current traffic information of a current network request to be recognized; Detecting whether there is a first match between the current traffic information and a preset static abnormal traffic dictionary; When there is no first match, detecting whether there is a second match between the current traffic information and the latest dynamic abnormal traffic dictionary; When there is a first match or a second match, determining that the current network request is abnormal traffic, and updating the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic.

2. The abnormal traffic recognition method according to claim 1, wherein The detecting whether there is a second match between the current traffic information and the latest dynamic abnormal traffic dictionary includes: Identifying whether the latest dynamic abnormal traffic dictionary contains one or more traffic parameters that are the same as the current traffic information, where the traffic parameters include a combination of one or more of source IP, destination IP, source port, destination port, request header, return result, and return status code; Comparing whether the frequency or number of occurrences of the same one or more traffic parameters in the dynamic abnormal traffic dictionary exceeds the corresponding frequency threshold or number threshold, and when it exceeds the corresponding frequency threshold or number threshold, determining that the current traffic information meets the second match.

3. The abnormal traffic recognition method according to claim 2, wherein The traffic parameters include main traffic parameters and auxiliary traffic parameters, the frequency threshold includes a first frequency threshold and a second frequency threshold, and the number threshold includes a first number threshold and a second number threshold; The comparing whether the frequency or number of occurrences of the same one or more traffic parameters in the dynamic abnormal traffic dictionary exceeds the corresponding frequency threshold or number threshold, and when it exceeds the corresponding frequency threshold or number threshold, determining that the current traffic information meets the second match includes: When the same traffic parameters include at least one main traffic parameter, if the frequency or number of occurrences of any one main traffic parameter in the dynamic abnormal traffic dictionary exceeds the corresponding first frequency threshold or first number threshold, determining that the current traffic information meets the second match; When the same traffic parameters include multiple auxiliary traffic parameters, if the number of auxiliary traffic parameters exceeding a preset quantity has a frequency or number of occurrences in the dynamic abnormal traffic dictionary that exceeds the corresponding first frequency threshold or first number threshold, determining that the current traffic information meets the second match.

4. The abnormal traffic recognition method according to claim 1, wherein The current traffic information includes a request time, and the validity period of the latest dynamic abnormal traffic dictionary is within a preset time sliding window corresponding to the request time; The updating the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic includes: Adding the current traffic information to the preset dynamic abnormal traffic dictionary, where the dynamic abnormal traffic dictionary contains the traffic information determined to be abnormal traffic within the time sliding window.

5. The abnormal traffic recognition method according to claim 4, wherein After the updating the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic, the method further includes: Re-extracting the historical traffic information of historical network requests within the time sliding window that have not been determined to be abnormal traffic; Check whether there is a second matching between the historical traffic information and the latest dynamic abnormal traffic dictionary; Determine the historical network requests corresponding to the historical traffic information that meets the second matching as abnormal traffic, and update the preset dynamic abnormal traffic dictionary based on the historical traffic information determined as abnormal traffic.

6. The abnormal traffic recognition method according to claim 1, wherein The static abnormal traffic dictionary includes a first static abnormal traffic dictionary. The step of checking whether there is a first matching between the current traffic information and the preset static abnormal traffic dictionary includes: Check whether any abnormal traffic feature existing in the first static abnormal traffic dictionary is included in the current traffic information. If it is included, it is determined that the first matching is met.

7. The abnormal traffic recognition method according to claim 1, wherein The static abnormal traffic dictionary includes a second static abnormal traffic dictionary. The step of checking whether there is a first matching between the current traffic information and the preset static abnormal traffic dictionary includes: Perform fuzzy matching between the current traffic information and the sensitive words in the second static abnormal traffic dictionary. When the fuzzy matching is successful, it is determined that the first matching is met.

8. The abnormal traffic identification method according to any one of claims 1 to 7, characterized in that The method further includes: Obtain a traffic information sample set containing abnormal traffic labels and normal traffic labels; Extract abnormal traffic features from the traffic information sample set; Construct a static abnormal traffic dictionary based on the extracted abnormal traffic features.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores executable instructions, and when the executable instructions are executed by a processor, the processor executes the abnormal traffic recognition method according to any one of claims 1 to 8.

10. An electronic device, characterized in that, It includes: One or more processors; A memory for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors execute the abnormal traffic recognition method according to any one of claims 1 to 8.

Images