Abnormal traffic identification method, storage medium and electronic device

Abstract

The present application provides an abnormal traffic identification method, storage medium and electronic device, which belongs to the field of computer technology. The method includes: obtaining the current traffic information of the current network request to be identified; detecting whether the current traffic information and the preset static abnormal traffic dictionary meet the first matching; when it does not meet the first matching, detecting whether the current traffic information and the latest dynamic abnormal traffic dictionary meet the second matching; when it meets the first matching or the second matching, determining that the current network request is abnormal traffic, and updating the preset dynamic abnormal traffic dictionary based on the current traffic information that is determined to be abnormal traffic. The present application can take into account both the efficiency and comprehensiveness of abnormal traffic identification.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an abnormal traffic identification method, storage medium, and electronic device. Background Art

[0002] With the rapid development of internet technology, API (Application Programming Interface) security, as the core channel for data exchange, has become a critical area of ​​network security. Abnormal traffic (such as DDoS attacks, SQL injections, fraudulent crawlers, and other malicious access behaviors) is being simulated through automated tools. The attack methods are becoming increasingly covert and diverse, posing a serious threat to the security and stability of API assets. Efficiently identifying and blocking abnormal traffic while minimizing the impact on normal business operations has become a major challenge in the current network security field.

[0003] Currently, mainstream abnormal traffic detection technologies fall into two categories: The first is based on feature-based rule matching. This method predefines abnormal traffic parameters (such as specific URL paths and abnormal parameter combinations) to build a static rule base (such as blacklists and whitelists), and then filters traffic through string matching or regular expressions. The second is based on machine learning. This method learns abnormal behavior patterns from historical traffic data by training classification models (such as supervised learning) or clustering models (such as unsupervised learning).

[0004] For the first type of method, although it can quickly identify abnormal traffic, the establishment of its rule base is highly dependent on manual experience, and the coverage of abnormal traffic identification is not high; for the second type of method, the accuracy of machine learning is highly dependent on the quality and representativeness of the training data, and has strict requirements on data labeling. At the same time, because machine learning requires a lot of computing resources, it will lead to the judgment of whether the access request is abnormal not being timely enough, which makes it difficult to meet the needs of high-throughput, low-latency real-time abnormal traffic detection. Summary of the Invention

[0005] The purpose of the present invention is to provide a new abnormal traffic identification method, storage medium and electronic device to solve at least one technical problem in the prior art, so as to take into account both the accuracy and real-time performance of abnormal traffic identification.

[0006] In a first aspect of the present application, a method for identifying abnormal traffic is provided, the method comprising:

[0007] Get the current traffic information of the current network request to be identified;

[0008] Detecting whether the current traffic information meets a first matching requirement with a preset static abnormal traffic dictionary;

[0009] When the first matching is not met, detecting whether the current traffic information and the latest dynamic abnormal traffic dictionary meet the second matching;

[0010] When the first matching or the second matching is met, the current network request is determined to be abnormal traffic, and the preset dynamic abnormal traffic dictionary is updated based on the current traffic information determined to be abnormal traffic.

[0011] Optionally, detecting whether the current traffic information meets a second matching requirement with the latest dynamic abnormal traffic dictionary includes:

[0012] Identify whether the latest dynamic abnormal traffic dictionary contains one or more traffic parameters that are the same as the current traffic information, where the traffic parameters include one or more combinations of source IP, destination IP, source port, destination port, request header, return result, and return status code;

[0013] Compare whether the frequency or number of occurrences of the same one or more flow parameters in the dynamic abnormal flow dictionary exceeds the corresponding frequency threshold or number threshold; when it exceeds the corresponding frequency threshold or number threshold, determine that the current flow information meets the second matching.

[0014] Optionally, the flow parameter includes a primary flow parameter and an auxiliary flow parameter, the frequency threshold includes a first frequency threshold and a second frequency threshold, and the number threshold includes a first number threshold and a second number threshold;

[0015] The comparing whether the frequency or number of occurrences of the same one or more traffic parameters in the dynamic abnormal traffic dictionary exceeds a corresponding frequency threshold or number threshold, and determining that the current traffic information meets the second matching when the frequency threshold or number threshold is exceeded, includes:

[0016] When the same flow parameters include at least one main flow parameter, if the frequency or number of occurrences of any main flow parameter in the dynamic abnormal flow dictionary exceeds the corresponding first frequency threshold or first number threshold, it is determined that the current flow information meets the second matching condition;

[0017] When the same flow parameter includes multiple auxiliary flow parameters, if the auxiliary flow parameters exceed a preset number, and the frequency or number of occurrences in the dynamic abnormal flow dictionary exceeds the corresponding first frequency threshold or first number threshold, the current flow information is determined to meet the second matching.

[0018] Optionally, the current traffic information includes a request time, and the validity period of the latest dynamic abnormal traffic dictionary is within a preset time sliding window corresponding to the request time. The updating of the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic includes:

[0019] The current traffic information is added to a preset dynamic abnormal traffic dictionary, where the dynamic abnormal traffic dictionary contains traffic information that is determined to be abnormal traffic within the time sliding window.

[0020] Optionally, after updating the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic, the method further includes:

[0021] Re-extracting historical traffic information of historical network requests that are not determined to be abnormal traffic within the time sliding window;

[0022] Detecting whether the historical traffic information and the latest dynamic abnormal traffic dictionary meet the second matching requirement;

[0023] The historical network request corresponding to the historical traffic information meeting the second matching is determined to be abnormal traffic, and the preset dynamic abnormal traffic dictionary is updated based on the historical traffic information determined to be abnormal traffic.

[0024] Optionally, the static abnormal traffic dictionary includes a static abnormal traffic first dictionary, and the detection of whether the current traffic information meets the first matching requirement with the preset static abnormal traffic dictionary includes: detecting whether the current traffic information contains any abnormal traffic feature existing in the static abnormal traffic first dictionary; if so, determining that the first matching requirement is met.

[0025] Optionally, the static abnormal traffic dictionary includes a static abnormal traffic second dictionary, and the detection of whether the current traffic information meets the first matching requirement with the preset static abnormal traffic dictionary includes: fuzzy matching the current traffic information with sensitive words in the static abnormal traffic second dictionary, and when the fuzzy match is successful, determining that the first matching requirement is met.

[0026] Optionally, the method further includes: obtaining a flow information sample set including abnormal flow labels and normal flow labels; extracting abnormal flow features from the flow information sample set; and constructing a static abnormal flow dictionary based on the extracted abnormal flow features.

[0027] In a second aspect of the present application, a computer-readable storage medium is provided, on which executable instructions are stored. When the executable instructions are executed by a processor, the processor executes the abnormal traffic identification method as described in any embodiment of the present application.

[0028] In a third aspect of the present application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the abnormal traffic identification method as described in any one of the embodiments of the present application.

[0029] The abnormal traffic identification method storage medium and electronic device in the present application set up two modes, static identification and dynamic identification. First, the network requests that obviously belong to abnormal traffic are quickly identified through the static identification method, and the dynamic abnormal traffic dictionary is updated for the identified abnormal traffic. The network requests that cannot be quickly identified as abnormal traffic are detected through the dynamic abnormal traffic dictionary to avoid omissions in abnormal traffic identification, thereby taking into account both the efficiency and comprehensiveness of abnormal traffic identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope of the present application.

[0031] Figure 1 1 is a flow chart of a method for identifying abnormal traffic in one embodiment;

[0032] Figure 2 1 is a flow chart of retrospective anomaly detection for historical traffic requests in one embodiment;

[0033] Figure 3 A schematic diagram of a process for constructing a static abnormal traffic dictionary in one embodiment;

[0034] Figure 4 FIG. 1 is a schematic structural diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0036] All terms (including technical and scientific terms) used in this application have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0037] For example, the terms "first" and "second" used in this application are only used to distinguish similar objects and to differentiate the first object from another object, rather than to describe a specific order or sequence, and cannot be understood as indicating or implying relative importance.

[0038] This application proposes a method for identifying abnormal traffic, combining Figure 1 As shown, the method includes:

[0039] Step 110: Obtain current traffic information of the current network request to be identified.

[0040] In this embodiment, a network request refers to a data request sent by a client (such as a browser or app) to a server to obtain resources or perform certain operations on the server. A network request typically includes one or more of the following: a request method (such as GET or POST), a request URL, a request header, and a request body.

[0041] Traffic information refers to the data packet information generated during network transmission, recording detailed data about network requests and responses. Traffic information can include one or more traffic parameters, such as source IP address, destination IP address, source port, destination port, protocol type, timestamp, request URL, request header and body, return result, and return status code. By analyzing traffic information, abnormal network behavior or malicious traffic (i.e., abnormal traffic) can be identified.

[0042] For example, when a user enters a URL in a browser, the browser sends a DNS query request to the DNS server, requesting to resolve the IP address corresponding to the URL. This network request is a DNS query request. When a user clicks the "Login" button on a web page, the browser sends a POST request to the server, which contains the user's login information (such as user name, password, etc.). This network request is a POST request. When a user clicks the "Refresh Data" button in an APP, the APP sends a GET request to the server, requesting to obtain the latest data. This network request is a GET request.

[0043] The traffic information corresponding to the DNS query request may include the source IP address (user's IP address), destination IP address (server's IP address), source port (usually a randomly assigned temporary port), destination port (usually the default port 80 for HTTP or the default port 443 for HTTPS), protocol type (TCP), timestamp (time when the request is sent), HTTP request header, request body, and other information; in addition to the above-mentioned information, the traffic information corresponding to the POST request may also include login information submitted by the user, such as user name, password, and other information; if the server responds to the user's request, the returned traffic information includes: source IP address (server's IP address), destination IP address (user's IP address), source port (server's port, the same as the destination port at the time of the request), destination port (user's port, the same as the source port at the time of the request or a randomly assigned port), protocol type (TCP), timestamp (time when the response is sent), HTTP response header, response body, and other one or more traffic parameters.

[0044] Electronic devices may receive a large number of network requests at the same time or within a short period of time. It is necessary to identify whether these network requests are malicious or abnormal requests (i.e., abnormal traffic). Examples of abnormal traffic include fuzz attack traffic, SQL injection traffic, and interface malicious traffic.

[0045] The network request received within the current time period is the current network request. The network request to be identified is the network request that requires abnormal traffic analysis, and the traffic information of the current network request is the current traffic information.

[0046] Among them, the current time period can be a pre-set time period of any appropriate length, such as the most recent time period, or any appropriate time period divided according to a preset time threshold, such as each time period is 1 minute, 2 minutes, 5 minutes, etc., or all network requests received within the last 1 minute are regarded as current network requests.

[0047] Step 120 , detecting whether the current traffic information meets a first matching requirement with a preset static abnormal traffic dictionary. If the first matching requirement is not met, step 130 is executed; if the first matching requirement is met, step 140 is executed.

[0048] In this embodiment, the static abnormal traffic dictionary is a pre-established database containing abnormal traffic features. The features in this static abnormal traffic dictionary can be derived from historical attack cases, security experts' experience, or industry-standard malicious traffic identification rules, and are extracted from traffic parameters in abnormal traffic information. Features in the static abnormal traffic dictionary are typically described using regular expressions, string matching, or other methods to facilitate rapid comparison with traffic information. The static abnormal traffic dictionary is a relatively fixed dictionary, where the features remain unchanged over a period of time or are not updated.

[0049] When it is detected that the traffic parameters in the current traffic information match the features in the static abnormal traffic dictionary, it is determined to meet the first matching. The first matching may include one or more of the following methods: full matching, partial matching, fuzzy matching, or matching based on behavioral rules. For example, when it fully matches a feature in the static abnormal traffic dictionary, it is determined to meet the first matching, or when it does not fully match another or several features in the static abnormal traffic dictionary, but satisfies the fuzzy matching, it is still determined to meet the first matching. The features in the traffic information may be one or more traffic parameters such as the above-mentioned source IP address, destination IP address, source port, destination port, protocol type, timestamp, HTTP response header, response body, etc.

[0050] The static abnormal traffic dictionary may specifically include one or more of a URL dictionary, a ResBody dictionary, and a fuzz dictionary. The URL dictionary includes a URL base path dictionary and a URL query parameter dictionary, and the ResBody dictionary is a dictionary of sensitive words in the interface response body.

[0051] For example, sensitive words (i.e., features) included in the URL dictionary might include npm-group, Centos, CentOS, LinuX, scLo, and openEuler-22.03-LTS-SP1; sensitive words (i.e., features) included in the ResBody dictionary might include "Turn down," "Disagree," "503 Service Unavailable," "418 I'm a teapot," and "400 BadRequest"; and sensitive words (i.e., features) included in the fuzz dictionary might include "~SyS~#," "~tmp ~user@," "~webmaster^," " / \bxss.me%," and "_debug_ / render_panel!". When the traffic parameters of a network request contain any one or more of these features, the request is considered to meet the first match.

[0052] Step 130 : Check whether the current traffic information and the latest dynamic abnormal traffic dictionary meet the second matching requirement. If the second matching requirement is met, execute step 140 ; ​​otherwise, execute step 150 .

[0053] In this embodiment, similar to the features in the static abnormal traffic dictionary, the dynamic abnormal traffic dictionary also contains features of network requests suspected of abnormal traffic. The dynamic abnormal traffic dictionary is a continuously updated traffic dictionary, and the features within it are constantly updated. For example, the features in the dynamic abnormal traffic dictionary may include source IP, destination IP, source port, destination port, and request header.

[0054] In one implementation, in addition to continuously and dynamically updating the features recorded in the dynamic abnormal traffic dictionary, the cumulative number of times each feature appears in the dictionary is further updated.

[0055] The second matching rule can be the same or similar to the first matching rule, for example, one or more of a full match, partial match, fuzzy match, or behavior-based match. It can also include one or a combination of matching methods, such as the number of occurrences of the matching feature in the dynamic abnormal traffic dictionary, the frequency of occurrences, and the number of traffic parameters of the matching feature. For example, the dynamic abnormal traffic dictionary contains features A, B, C, D, and E. By continuously updating the dynamic abnormal traffic dictionary, at the current moment, the cumulative number (or cumulative occurrences) of feature A is 5, the cumulative occurrences of feature B is 8, and the cumulative occurrences of feature C is 10. If the traffic parameters of the current traffic information also contain feature C, which appears 10 times in the dynamic abnormal traffic dictionary, then the second matching rule is determined to be met. Alternatively, if the traffic parameters of the current traffic information contain features A, B, D, and E, i.e., four of the features in the dynamic abnormal traffic dictionary are present, then the second matching rule is determined to be met.

[0056] Step 140 : Determine whether the current network request is abnormal traffic, and update a preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic.

[0057] In this embodiment, when a network request is determined to be abnormal traffic, the dynamic abnormal traffic dictionary is updated based on the traffic information of the abnormal traffic. Among them, one or more features can be extracted from the traffic parameters of the abnormal traffic and updated to the dynamic abnormal traffic dictionary. At the same time, the number of the same feature appearing in the dynamic abnormal traffic dictionary can also be updated. For example, when a network request is identified as abnormal traffic, the source IP, source port, target IP, target port, response success and / or failure and other traffic parameters corresponding to the network request are updated to the dynamic abnormal traffic dictionary as features in the dynamic abnormal traffic dictionary. Furthermore, the number of times each feature appears (i.e., the cumulative number) is recorded. For example, at the current moment, a source IP in the dynamic abnormal traffic dictionary has appeared 100 times.

[0058] Step 150: Determine whether the current network request is normal traffic.

[0059] In this embodiment, when a network request is identified as not meeting either the first or second matching criteria, it can be directly determined to be normal traffic and the network request can be responded to normally. Alternatively, further abnormal traffic identification can be performed through other methods, and if it is determined not to be abnormal traffic, it can be determined to be normal traffic.

[0060] When abnormal traffic is detected, detailed log information is recorded, including request time, source IP address, destination IP address, request type, and request content, for subsequent analysis and tracing. Based on the recorded log information and context, the cause of the abnormal traffic is analyzed in depth to determine whether there are security vulnerabilities or malicious behavior. Network requests for abnormal traffic are blocked to prevent further damage to the system. Alerts are sent to system administrators or security teams to notify them of the abnormal traffic and provide the necessary log information and context for rapid response and resolution.

[0061] The abnormal traffic identification method in the present application sets up two modes, static identification and dynamic identification. First, the network requests that are obviously abnormal traffic are quickly identified through the static identification method, and the dynamic abnormal traffic dictionary is updated for the identified abnormal traffic. The network requests that cannot be quickly identified as abnormal traffic are detected through the dynamic abnormal traffic dictionary to avoid omissions in abnormal traffic identification, thereby taking into account both the efficiency and comprehensiveness of abnormal traffic identification.

[0062] In one embodiment, the current traffic information includes the request time, and the validity period of the latest dynamic abnormal traffic dictionary is within a preset time sliding window corresponding to the request time; the preset dynamic abnormal traffic dictionary is updated based on the current traffic information that is judged to be abnormal traffic, including: adding the current traffic information to the preset dynamic abnormal traffic dictionary, and the dynamic abnormal traffic dictionary contains the traffic information that is judged to be abnormal traffic within the time sliding window.

[0063] In this embodiment, the length of the sliding time window can be adjusted and set according to actual needs, for example, it can be set to any appropriate length such as 10 minutes, 5 minutes, 20 minutes, 1 hour, 2 hours, etc.

[0064] In one embodiment, the length of the sliding time window can be determined based on the timestamps of associated abnormal traffic. Generally speaking, abnormal network requests initiated by the same source IP address have a certain degree of persistence. The electronic device collects the three-dimensional network features x of the abnormal traffic, including the timestamp t, source IP address, and destination IP address, to form a network traffic dataset D. D = {x1, x2, …, xn}, where n ≥ 2. Each network feature x in the traffic dataset D contains information from three dimensions: timestamp t, source IP address, and destination IP address. By analyzing the time interval between timestamps t between requests with the same source IP address and destination IP address, the length of the corresponding sliding time window is calculated.

[0065] For example, a continuous sequence of identical abnormal traffic can be identified from the data set, and the time difference between the earliest request time and the latest request time in each continuous sequence of identical abnormal traffic can be calculated, and the length of the sliding time window can be determined based on each calculated time difference. For example, the length can be a weighted average of the various time differences. The abnormal traffic in the continuous sequence of identical abnormal traffic has the same source IP and destination IP, and when the abnormal traffic in the sequence is sorted in ascending or descending order according to the timestamp t, the difference in the timestamps between adjacent abnormal traffic flows is within a preset time threshold. The time threshold is an arbitrarily set appropriate threshold.

[0066] For the determined length, the time period of each sliding time window or the validity period of the sliding time window can be determined. The validity period of the sliding time window is a dynamic validity period or a static validity period determined according to the above length. The static validity period can be a fixed time period. Taking the determined time length of 5 minutes as an example, the static validity period of the sliding time window is 5 minutes, and a sliding time window is divided into 5 minutes. According to the time period of the current moment, the sliding time window corresponding to the current moment is determined. After entering a new sliding time window, the dynamic abnormal traffic dictionary can be cleared and the dynamic abnormal traffic dictionary can be updated again.

[0067] The dynamic time period can be a time period of the most recent duration (e.g., 5 minutes) with the current moment as the end time. Data outside of the most recent duration is discarded, and the flow information of abnormal traffic within the most recent duration is added to the dynamic abnormal traffic dictionary, while the flow information of abnormal traffic outside the most recent duration is removed from the dynamic abnormal traffic dictionary. For one or more features extracted from the flow parameters of abnormal traffic, if they are within the validity period, they are added to the dynamic abnormal traffic dictionary. If they are outside the validity period, the previously added features are deleted. At the same time, the number of the same feature appearing in the dynamic abnormal traffic dictionary can also be updated (increased or decreased).

[0068] Since malicious network behavior is continuous, when abnormal behavior is found in the flow of a source IP, it is highly likely that the network behavior in the adjacent time period (time sliding window) will still be abnormal. Therefore, based on the temporal locality characteristics of network request behavior, by setting the validity period of the dynamic abnormal traffic dictionary and updating the dynamic abnormal traffic dictionary within the validity period, the accuracy of identifying continuous abnormal traffic can be improved.

[0069] In one embodiment, the static abnormal traffic dictionary includes a static abnormal traffic first dictionary, and detecting whether the current traffic information meets the first matching between the preset static abnormal traffic dictionary includes: detecting whether the current traffic information contains any abnormal traffic feature existing in the static abnormal traffic first dictionary; if so, determining that the first matching is met.

[0070] In this embodiment, the abnormal traffic features included in the first dictionary of static abnormal traffic are features that clearly belong to abnormal traffic. The abnormal traffic features may include specific URL paths, request parameters, and keywords or phrases in the response body content that are clearly abnormal traffic, such as the above-mentioned CentOS, "400 Bad Request" and other features. For the current traffic information, it can be detected whether it contains any feature in the static abnormal traffic dictionary. If it does, it is determined to meet the first matching. The inclusion indicates that the previous traffic information has a feature that is completely consistent with any abnormal traffic feature in the static abnormal traffic dictionary. When the first matching is met, the current network request can be determined to be abnormal traffic.

[0071] In one embodiment, the static abnormal traffic dictionary includes a static abnormal traffic second dictionary, which detects whether the current traffic information meets the first matching requirement with the preset static abnormal traffic dictionary, including: fuzzy matching the current traffic information with the sensitive words in the static abnormal traffic second dictionary, and when the fuzzy match is successful, it is determined that the first matching requirement is met.

[0072] In this embodiment, the second static abnormal traffic dictionary supplements the first static abnormal traffic dictionary. The abnormal traffic features in the second static abnormal traffic dictionary primarily target features that are difficult to match with the first static abnormal traffic dictionary. This difficulty is primarily due to the presence of special characters (such as !@#$, etc.). For example, a network request that does not meet the first match criteria with the first static abnormal traffic dictionary can be tested for a first match with the second static abnormal traffic dictionary. This test checks whether the relevant features in the traffic information fuzzily match any abnormal traffic feature in the second static abnormal traffic dictionary. If a fuzzy match is found, the network request is determined to meet the first match criteria. The matching method can employ regular expression rules.

[0073] By setting up a second dictionary of static abnormal traffic to include abnormal traffic features that are difficult to fully match due to special characters, etc., and using fuzzy matching to perform the first matching detection, the comprehensiveness of the features included in the static abnormal traffic dictionary is further improved, and the comprehensiveness of static detection is improved.

[0074] In one embodiment, detecting whether the current traffic information and the latest dynamic abnormal traffic dictionary meet the second matching includes: identifying whether the latest dynamic abnormal traffic dictionary contains one or more traffic parameters that are the same as the current traffic information, the traffic parameters including one or more combinations of source IP, target IP, source port, target port, request header, return result, and return status code; comparing whether the frequency or number of occurrences of the same one or more traffic parameters in the dynamic abnormal traffic dictionary exceeds the corresponding frequency threshold or number threshold, and when it exceeds the corresponding frequency threshold or number threshold, determining that the current traffic information meets the second matching.

[0075] In this embodiment, the traffic parameter can be a single traffic parameter or a combination of multiple traffic parameters. For example, the traffic parameter can be a single parameter such as source IP, destination IP, source port, and destination port, or a traffic parameter combination formed by four single parameters such as source IP, destination IP, source port, and destination port.

[0076] Because the dynamic abnormal traffic dictionary includes not only the traffic parameters themselves but also the statistics of the current traffic parameters, for example, if a source IP-A appears 100 times in the dynamic abnormal traffic dictionary, this means that within the corresponding time sliding window, 100 network requests with source IP-A were identified as abnormal traffic.

[0077] The frequency threshold or number threshold corresponding to each traffic parameter is not necessarily the same. For example, the number threshold corresponding to the source IP is 10,000 times, and the number threshold corresponding to the traffic parameter combination formed by the source IP, destination IP, source port, and destination port is 100 times.

[0078] For example, when it is detected that the traffic parameter combination formed by the source IP, target IP, source port, and target port in the current network request also exists in the dynamic abnormal traffic dictionary, and the number of occurrences exceeds 100 times, it means that the current network request meets the second matching. Since the traffic parameters appearing in the dynamic abnormal traffic dictionary are all parameters extracted from abnormal traffic, the same traffic parameters also exist in the current network request, and when the corresponding number threshold or frequency threshold is exceeded, it means that the current network request is also abnormal traffic. If the same traffic parameters exist, but each of the same traffic parameters reaches the corresponding frequency threshold or number threshold, the network request is not determined to be abnormal traffic.

[0079] The introduction of a dynamic abnormal traffic dictionary enables the system to flexibly respond to ever-changing network attack methods, improving the accuracy and timeliness of abnormal traffic identification. At the same time, through combined matching and frequency / count comparison, the system's detection capabilities and robustness are further enhanced.

[0080] In one embodiment, the flow parameters include a main flow parameter and an auxiliary flow parameter, the frequency threshold includes a first frequency threshold and a second frequency threshold, and the number threshold includes a first number threshold and a second number threshold. Comparing whether the frequency or number of occurrences of the same or multiple flow parameters in the dynamic abnormal flow dictionary exceeds the corresponding frequency threshold or number threshold, when the corresponding frequency threshold or number threshold is exceeded, it is determined that the current flow information meets the second matching, including: when the same flow parameters include at least one main flow parameter, if the frequency or number of occurrences of any main flow parameter in the dynamic abnormal flow dictionary exceeds the corresponding first frequency threshold or first number threshold, it is determined that the current flow information meets the second matching; when the same flow parameters include multiple auxiliary flow parameters, if the frequency or number of occurrences of the auxiliary flow parameters exceeding a preset number in the dynamic abnormal flow dictionary exceeds the corresponding first frequency threshold or first number threshold, it is determined that the current flow information meets the second matching.

[0081] In this embodiment, the traffic parameters in the dynamic abnormal traffic dictionary are further divided into two parameter types: primary traffic parameters and secondary traffic parameters. Among them, the primary traffic parameters are parameters that have a decisive influence on the identification of abnormal traffic, such as source IP, target IP, request header, etc.; while the secondary traffic parameters are supplementary to the primary traffic parameters, such as source port, target port, return status code, etc. Furthermore, the primary traffic parameters are a combination of multiple single traffic parameters. For example, a traffic parameter combination formed by the above-mentioned source IP, target IP, source port, and target port is a primary traffic parameter. The secondary traffic parameters are usually single traffic parameters.

[0082] As described above, the first frequency threshold / first number threshold (for the main flow parameter) and the second frequency threshold / second number threshold corresponding to different flow parameters (including the main flow parameter and the auxiliary flow parameter) are not necessarily the same.

[0083] When any main traffic parameter in the current network request appears in the dynamic abnormal traffic dictionary, and the number of occurrences / frequency exceeds the first frequency threshold / first number threshold corresponding to the main traffic parameter, the current network request is directly determined to be abnormal traffic.

[0084] If there is no main flow parameter in the current network request that appears in the dynamic abnormal flow dictionary, or the existing main flow parameters do not exceed the corresponding abnormal flow, then check whether the current network request includes auxiliary flow parameters in the dynamic abnormal flow dictionary. If so, count the number of occurrences / frequency of occurrence of each identical auxiliary flow parameter in the dynamic abnormal flow dictionary, count the identical auxiliary flow parameters that exceed the corresponding frequency / number of occurrences, and determine whether it belongs to abnormal flow based on the identical auxiliary flow parameters that exceed the corresponding frequency / number of occurrences.

[0085] For example, if the current network request contains identical auxiliary traffic parameters exceeding the corresponding frequency / number of occurrences, including the source IP address, source port, failed access response result (return status code), and a string within the same URL path, the network request is considered abnormal traffic. The preset number can be fixed or determined based on the specific identical auxiliary traffic parameters and their frequency / number of occurrences. Different auxiliary traffic parameters and their frequency / number of occurrences may not necessarily be the same. Specifically, the higher the frequency / number of occurrences of the auxiliary traffic parameters, the smaller the corresponding preset number.

[0086] In this embodiment, by setting the primary flow parameters and the secondary flow parameters, the accuracy of abnormal flow identification can be improved in addition to ensuring the comprehensiveness of abnormal flow identification.

[0087] In one embodiment, Figure 2 As shown, after updating the preset dynamic abnormal traffic dictionary based on the current traffic information determined to be abnormal traffic, the method further includes retrospective anomaly detection for historical traffic requests, and the process of retrospective anomaly detection includes:

[0088] Step 210: re-extract historical traffic information of historical network requests that are not determined to be abnormal traffic within the time sliding window.

[0089] In this embodiment, anomaly detection backtracking can be performed each time the dynamic abnormal traffic dictionary is updated, or anomaly detection backtracking can be performed according to a preset backtracking frequency. For example, taking the time sliding window as an example, the traffic information of all network requests that are not determined to be abnormal traffic in the time period can be extracted at a certain moment before the current time sliding window expires (such as 1 minute before expiration, 30 seconds before expiration, or 10 seconds before expiration, or any other appropriate moment). When the time sliding window is a dynamic validity period, historical traffic information can be extracted every 1 / 2 or 1 / 3 of the window duration. Specifically, the extracted network request is the network request that is re-extracted for the first time.

[0090] Step 220 : Check whether the historical traffic information and the latest dynamic abnormal traffic dictionary meet the second matching requirement.

[0091] Step 230 : Determine the historical network request corresponding to the historical traffic information that meets the second matching as abnormal traffic, and update the preset dynamic abnormal traffic dictionary based on the historical traffic information determined to be abnormal traffic.

[0092] In this embodiment, the second matching detection method for the re-extracted historical network request is the same as the second matching detection method for the current traffic request, except that the dynamic abnormal traffic dictionary is an updated traffic dictionary.

[0093] Specifically, identify whether the latest dynamic abnormal traffic dictionary contains one or more traffic parameters that are the same as the re-extracted historical traffic information; compare whether the frequency or number of occurrences of the same one or more traffic parameters in the dynamic abnormal traffic dictionary exceeds the corresponding frequency threshold or number threshold; when it exceeds the corresponding frequency threshold or number threshold, determine that the current traffic information meets the second matching.

[0094] In one embodiment, when the same traffic parameters in the re-extracted historical traffic information include at least one main traffic parameter, if the frequency or number of occurrences of any main traffic parameter in the dynamic abnormal traffic dictionary exceeds the corresponding first frequency threshold or first number threshold, the current traffic information is determined to meet the second matching; when the same traffic parameters in the re-extracted historical traffic information include multiple auxiliary traffic parameters, if more than a preset number of auxiliary traffic parameters appear in the dynamic abnormal traffic dictionary at a frequency or number that exceeds the corresponding first frequency threshold or first number threshold, the current traffic information is determined to meet the second matching.

[0095] By retrospectively detecting historical traffic requests and updating the dynamic abnormal traffic dictionary, we can discover previously unidentified requests that actually meet abnormal characteristics, thereby improving the coverage of anomaly detection and promptly handling historical traffic that has been re-identified as abnormal, thus avoiding further losses. Updating the dictionary based on newly identified abnormal traffic information can further enhance the system's ability to identify subsequent abnormal traffic.

[0096] In one embodiment, Figure 3 As shown, the above method also includes a process of constructing a static abnormal traffic dictionary, which includes:

[0097] Step 310: Acquire a flow information sample set including abnormal flow labels and normal flow labels.

[0098] Step 320: Extract abnormal traffic features from the traffic information sample set.

[0099] Step 330: construct a static abnormal traffic dictionary based on the extracted abnormal traffic features.

[0100] The sample set can come from historical data of the network traffic monitoring system, known network attack datasets, or public network traffic datasets. These sample sets should contain a sufficient amount of abnormal traffic and normal traffic to ensure the accuracy and representativeness of feature extraction.

[0101] Each flow in the sample set is labeled as either abnormal or normal. These labels can be determined by manual annotation, automatic annotation based on known attack patterns, or reports generated by other network security tools.

[0102] Before extracting features, it is necessary to determine which features are important for distinguishing abnormal traffic from normal traffic. These features can include packet size, connection duration, number of packets, source / destination IP addresses, source / destination port numbers, protocol type, etc.

[0103] Specifically, machine learning algorithms (such as decision trees, random forests, and support vector machines) or statistical methods (such as cluster analysis and principal component analysis) can be used to extract abnormal traffic features from the sample set. These algorithms can identify which features significantly differ between abnormal and normal traffic. Features that are clearly identified as belonging solely to abnormal traffic are added to the static abnormal traffic dictionary as abnormal traffic features.

[0104] In one embodiment, after step 310 , the method further includes: determining traffic parameters of a dynamic abnormal traffic dictionary from the traffic information sample set.

[0105] For the non-abnormal traffic features, further identification can be performed to extract the features that appear more frequently in the abnormal traffic, and set corresponding appropriate number thresholds or frequency thresholds according to the frequency or number of times these features appear. These extracted features are combined, and appropriate number thresholds or frequency thresholds are also set for the combined features to form matching rules for the dynamic abnormal traffic dictionary.

[0106] Through the above steps, parameters suitable for the static and dynamic anomaly traffic dictionaries can be determined from the traffic information sample set. These parameters are not only representative but also able to capture the dynamic changes in anomaly traffic, providing strong support for subsequent anomaly traffic identification. Furthermore, through continuous optimization and updating of parameters, the accuracy and adaptability of the static and dynamic anomaly traffic dictionaries can be ensured, improving the effectiveness of network security protection.

[0107] The abnormal traffic identification method in this application adds a dynamic abnormal traffic dictionary on the basis of the static abnormal traffic dictionary. Under the premise of ensuring the high-throughput computing performance of the system, it increases the detection coverage of abnormal behavior, increases the dynamic regulation of real-time abnormal traffic, and enhances the adaptability to unknown traffic. This application is applied to the interface aggregation scenario of API asset combing, which can effectively reduce the workload of API asset combing and reduce the interference of abnormal traffic; based on historical network traffic behavior and industry experience, it builds a behavioral rule library for abnormal traffic to effectively identify abnormal traffic; by introducing dynamic rules, adaptively updating the sensitive word dictionary library and the behavioral rule library, it identifies unknown traffic and solves the problem of poor adaptability to unknown traffic based on the sensitive word filtering method. At the same time, a time sliding window is introduced to increase the dynamic regulation of real-time abnormal traffic, reducing the impact range and false alarm rate of dynamic rules.

[0108] In one embodiment, a computer storage medium is provided, on which executable instructions are stored. When the instructions are executed by a processor, the processor executes the steps in the above-mentioned abnormal traffic identification method embodiments.

[0109] In one embodiment, an electronic device is provided, including a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the abnormal traffic identification method in any of the above embodiments.

[0110] In one embodiment, an electronic device is provided, which may be a terminal or a server. Figure 4As shown, electronic device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 402 or programs loaded from storage unit 408 into random access memory (RAM) 403. Various programs and data required for the operation of electronic device 400 are also stored in RAM 403. CPU 401, ROM 402, and RAM 403 are connected to each other via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0111] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, mouse, and the like; an output section 407 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 408 including devices such as a hard disk; and a communication section 409 including a network interface card such as a LAN card or a modem. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. Removable media 411, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 410 as needed, so that computer programs read from the removable media can be installed in the storage section 408 as needed.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

[0113] Furthermore, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this application and to form different embodiments. For example, all of the above embodiments may be used in any combination. The information disclosed in this background section is intended solely to enhance understanding of the overall background of this application and should not be construed as an admission or any form of implication that such information constitutes prior art known to those skilled in the art.

Claims

1. A method for identifying abnormal traffic, characterized in that: The method comprises: Obtaining current traffic information of a current network request to be identified, the current traffic information including the request time; Detecting whether the current traffic information meets a first matching requirement with a preset static abnormal traffic dictionary, where the first matching requirement includes one or more of a complete match, a partial match, a fuzzy match, or a match based on a behavior rule; When the first matching is not met, detecting whether the current traffic information meets the second matching between the latest dynamic abnormal traffic dictionary, where the validity period of the latest dynamic abnormal traffic dictionary is within a preset time sliding window corresponding to the request time, and the dynamic abnormal traffic dictionary is a traffic dictionary that is set to be continuously updated; When the first matching condition or the second matching condition is met, the current network request is determined to be abnormal traffic, and the current traffic information is added to a preset dynamic abnormal traffic dictionary, where the dynamic abnormal traffic dictionary contains traffic information determined to be abnormal traffic within the time sliding window; Re-extracting historical traffic information of historical network requests that are not determined to be abnormal traffic within the time sliding window; Detecting whether the historical traffic information and the latest dynamic abnormal traffic dictionary meet the second matching requirement; The historical network request corresponding to the historical traffic information meeting the second matching is determined to be abnormal traffic, and the preset dynamic abnormal traffic dictionary is updated based on the historical traffic information determined to be abnormal traffic.

2. The abnormal traffic identification method according to claim 1, characterized in that: The detecting whether the current traffic information meets the second matching requirement with the latest dynamic abnormal traffic dictionary includes: Identify whether the latest dynamic abnormal traffic dictionary contains one or more traffic parameters that are the same as the current traffic information, where the traffic parameters include one or more combinations of source IP, destination IP, source port, destination port, request header, return result, and return status code; Compare whether the frequency or number of occurrences of the same one or more flow parameters in the dynamic abnormal flow dictionary exceeds the corresponding frequency threshold or number threshold; when it exceeds the corresponding frequency threshold or number threshold, determine that the current flow information meets the second matching.

3. The abnormal traffic identification method according to claim 2, characterized in that: The flow parameters include a primary flow parameter and an auxiliary flow parameter, the frequency threshold includes a first frequency threshold and a second frequency threshold, and the number threshold includes a first number threshold and a second number threshold; The comparing whether the frequency or number of occurrences of the same one or more traffic parameters in the dynamic abnormal traffic dictionary exceeds a corresponding frequency threshold or number threshold, and determining that the current traffic information meets the second matching when the frequency threshold or number threshold is exceeded, includes: When the same flow parameters include at least one main flow parameter, if the frequency or number of occurrences of any main flow parameter in the dynamic abnormal flow dictionary exceeds the corresponding first frequency threshold or first number threshold, it is determined that the current flow information meets the second matching condition; When the same flow parameter includes multiple auxiliary flow parameters, if the auxiliary flow parameters exceed a preset number, and the frequency or number of occurrences in the dynamic abnormal flow dictionary exceeds the corresponding first frequency threshold or first number threshold, the current flow information is determined to meet the second matching.

4. The abnormal traffic identification method according to claim 1, characterized in that: The static abnormal traffic dictionary includes a first static abnormal traffic dictionary, and detecting whether the current traffic information meets a first matching requirement with a preset static abnormal traffic dictionary includes: It is detected whether the current traffic information contains any abnormal traffic feature in the static abnormal traffic first dictionary. If it does, it is determined that the first matching is met.

5. The abnormal traffic identification method according to claim 1, characterized in that: The static abnormal traffic dictionary includes a static abnormal traffic second dictionary, and detecting whether the current traffic information meets the first matching requirement with the preset static abnormal traffic dictionary includes: The current traffic information is fuzzy matched with the sensitive words in the second dictionary of the static abnormal traffic. When the fuzzy match is successful, it is determined that the first matching is met.

6. The abnormal traffic identification method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtain a traffic information sample set including abnormal traffic labels and normal traffic labels; Extracting abnormal traffic features from the traffic information sample set; A static abnormal traffic dictionary is constructed based on the extracted abnormal traffic features.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores executable instructions, and when the executable instructions are executed by a processor, the processor executes the abnormal traffic identification method according to any one of claims 1 to 6.

8. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to execute the abnormal traffic identification method according to any one of claims 1 to 6.

Images