An abnormal flow detection method, device, electronic equipment, and storage medium

Abstract

This application discloses an abnormal traffic detection method, apparatus, electronic device, and storage medium, relating to the field of network security technology. The method includes: first, acquiring N user requests within a set time window, and segmenting each of the N user requests into words based on a vocabulary to obtain M segmented user requests; then, calculating the similarity between any two user requests among the M user requests, and classifying the M user requests based on each similarity to obtain P sets of user requests; finally, determining the anomaly detection result for each of the N user requests based on any one user request from each of the P sets of user requests, thereby improving the efficiency of abnormal traffic detection.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to an abnormal traffic detection method, device, electronic device, and storage medium. Background Technology

[0002] With the rapid development and application of the internet, people's lives have become more convenient, but cybersecurity issues are also emerging one after another, such as attacks and vulnerabilities. These issues often cause changes in network traffic, and in order to improve network security, it is usually necessary to analyze the traffic data generated by network users to detect abnormal behavior.

[0003] In abnormal traffic detection scenarios, pre-set rules and trained detection models are typically used to check each user request one by one. However, for the same website, due to the limited scope of business, there are usually a large number of duplicate or highly similar user requests. Checking them one by one would waste resources and reduce overall detection efficiency. Summary of the Invention

[0004] This application provides an abnormal flow detection method, apparatus, electronic device, and storage medium to improve the efficiency of abnormal flow detection.

[0005] Firstly, an abnormal traffic detection method is provided, including:

[0006] N user requests within a set time window are obtained, and the N user requests are segmented into words based on a vocabulary to obtain M segmented user requests; wherein, the vocabulary is used to record multiple characters of independent word segmentation, and N equals M, which are integers greater than 0;

[0007] Calculate the similarity between any two user requests among the M user requests, and classify the M user requests based on each similarity to obtain P user request sets; wherein, a user request set includes user requests of the same class, and P is an integer less than or equal to M;

[0008] Based on any one user request from each of the P user request sets, determine the anomaly detection results for each of the N user requests.

[0009] In this embodiment, firstly, N user requests within a set time window are obtained, and the N user requests are segmented into words based on a vocabulary to obtain M segmented user requests. Then, the similarity between any two user requests among the M user requests is calculated, and the M user requests are classified based on each similarity to obtain P sets of user requests. Finally, based on any one user request in each of the P sets of user requests, the anomaly detection result of each of the N user requests is determined. Compared with the prior art, it is not necessary to detect each user request one by one, which can reduce the number of detections, reduce resource consumption, and thus improve the efficiency of abnormal traffic detection.

[0010] In some embodiments, the step of segmenting the N user requests based on a vocabulary to obtain M segmented user requests includes:

[0011] Based on the set character replacement rules, the interfering characters in each of the N user requests are replaced to obtain N target user requests; wherein, the interfering characters are used to represent characters that are not related to anomaly detection;

[0012] The N target user requests are segmented into words based on the vocabulary to obtain M segmented user requests.

[0013] By using the above method, characters unrelated to risk detection can be replaced, which can improve the deduplication rate and the accuracy of subsequent anomaly detection results.

[0014] In some embodiments, the character replacement rule includes at least one of the following:

[0015] Replace the Chinese characters in the user's request with the first preset character;

[0016] Replace the email address in the user's request with the second preset character;

[0017] Replace the numeric string in the user's request with a third preset character;

[0018] Replace the domain name in the user's request with the fourth preset character;

[0019] Merge multiple placeholders in the user request.

[0020] In some embodiments, calculating the similarity between any two user requests among the M user requests, and classifying the M user requests based on each similarity to obtain a set of P user requests, includes:

[0021] Each character of the first user request is added to the first matrix, and each character of the second user request is added to the second matrix, and the similarity between the first matrix and the second matrix is ​​calculated; wherein, the first user request and the second user request are any two different user requests among the M user requests;

[0022] If the similarity is greater than the similarity threshold, then the first user request and the second user request are classified into the same set of user requests;

[0023] If the similarity is less than or equal to the similarity threshold, then the first user request and the second user request are classified into different user request sets.

[0024] The above method can effectively identify similar user requests, thereby classifying all user requests. This makes it easier to perform traffic anomaly detection on any one of the user requests in the same category, and thus determine the anomaly detection results for all user requests in that category.

[0025] In some embodiments, the vocabulary is constructed using the BPE algorithm.

[0026] In some embodiments, determining the anomaly detection result of each of the N user requests based on any one of the P user request sets includes:

[0027] If the detection engine detects that any user request in the first user request set of the P user request sets has abnormal traffic, then it is determined that the abnormal detection results of all user requests included in the first user request set are abnormal traffic.

[0028] Secondly, an abnormal flow detection device is provided, comprising:

[0029] The word segmentation module is used to acquire N user requests within a set time window, and to segment the N user requests into words based on a word list to obtain M user requests after word segmentation; wherein, the word list is used to record multiple characters of independent word segmentation, and N equals M, which are integers greater than 0;

[0030] The similarity calculation module is used to calculate the similarity between any two user requests among the M user requests, and classify the M user requests based on each similarity to obtain P user request sets; wherein, a user request set includes user requests of the same class, and P is an integer less than or equal to M;

[0031] The anomaly detection module is used to determine the anomaly detection result of each of the N user requests based on any one of the P user request sets.

[0032] In some embodiments, the word segmentation module is specifically used for:

[0033] Based on the set character replacement rules, the interfering characters in each of the N user requests are replaced to obtain N target user requests; wherein, the interfering characters are used to represent characters that are not related to anomaly detection;

[0034] The N target user requests are segmented into words based on the vocabulary to obtain M segmented user requests.

[0035] Thirdly, an electronic device is provided, comprising:

[0036] A memory for storing computer programs; a processor for executing the computer programs stored in the memory to implement the method steps described in any one of the first aspects.

[0037] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the method steps described in any one of the first aspects.

[0038] For the various aspects of the second to fourth aspects mentioned above, and the technical effects that each aspect may achieve, please refer to the above description of the technical effects that can be achieved for the first aspect or the various possible solutions in the first aspect, which will not be repeated here. Attached Figure Description

[0039] Figure 1 This is a schematic diagram illustrating the application scenario to which this application applies;

[0040] Figure 2 A flowchart illustrating an abnormal traffic detection method provided in this application embodiment;

[0041] Figure 3 A logical diagram illustrating an abnormal traffic detection method provided in an embodiment of this application;

[0042] Figure 4 This is a schematic diagram of the structure of an abnormal flow detection device provided in an embodiment of this application;

[0043] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0045] The terms "first" and "second" in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. The term "multiple" in this application can mean at least two, for example, two, three, or more, and the embodiments of this application do not impose limitations.

[0046] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These embodiments should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that in the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0047] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0048] Figure 1This is a schematic diagram illustrating an application scenario applicable to the embodiments of this application. As shown in the figure, the application scenario mainly includes: terminal 10 and server 11. Terminal 10 and server 11 can interact via a communication network, which can employ communication methods including wireless communication and wired communication.

[0049] In some scenarios, terminal 10 can access the network and communicate with server 11 through cellular mobile communication technology. This cellular mobile communication technology may include 5th Generation Mobile Networks (5G) technology, and may also include future 6G technology.

[0050] In some embodiments, terminal 10 can access the network and communicate with server 11 via short-range wireless communication, which may include Wireless Fidelity (Wi-Fi) technology.

[0051] This application embodiment does not impose any limitation on the number of the above-mentioned devices, such as Figure 1 As shown, only terminal 10 and server 11 are described as examples. The following is a brief introduction to each of the above devices and their respective functions.

[0052] Terminal 10 is a device that can provide voice and / or data connectivity to users, including: handheld terminal devices with wireless connectivity, vehicle-mounted terminal devices, etc. For example, Terminal 10 includes, but is not limited to: mobile phones, tablets, laptops, PDAs, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in autonomous driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0053] Furthermore, an abnormal traffic detection client can be installed on terminal 10. This client can be software (e.g., an app, a browser, etc.), or a webpage, a mini-program, etc. In this embodiment, terminal 10 can use the aforementioned abnormal traffic detection client and can interact with server 11 to perform abnormal detection of various user requests on the website, thereby protecting the website from malicious attacks.

[0054] Furthermore, server 11 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0055] Furthermore, in this embodiment of the application, the server 11 may be equipped with an abnormal traffic detection device corresponding to the client. The abnormal traffic detection device is used to acquire N user requests within a set time window, and to segment the N user requests into words based on a vocabulary to obtain M user requests after word segmentation. Then, the similarity between any two user requests in the M user requests is calculated, and the M user requests are classified based on each similarity to obtain P user request sets. Finally, based on any one user request in each of the P user request sets, the abnormal detection results of the N user requests are determined, thereby detecting whether there is a malicious attack in the large number of user requests collected on the website.

[0056] To further illustrate the technical solutions provided in the embodiments of this application, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of this application provide method operation steps as shown in the following embodiments or drawings, the method may include more or fewer operation steps based on conventional or non-inventive methods. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application. In actual processing or when the device executes the method, it may be executed in the order shown in the embodiments or drawings, or in combination.

[0057] Figure 2 This is a flowchart illustrating an abnormal traffic detection method provided in an embodiment of this application. This process can be executed by an abnormal traffic detection device to improve the efficiency of abnormal traffic detection. Figure 2 As shown, the process includes the following steps:

[0058] 201: Obtain N user requests within a set time window, and segment each of the N user requests into words based on the vocabulary to obtain M segmented user requests.

[0059] In some embodiments, the vocabulary may be constructed using a Byte Pair Encoding (BPE) algorithm. The BPE algorithm is a word segmentation and encoding method for natural language processing. Its core is to construct a vocabulary where each character is treated as an independent word. This includes, but is not limited to, updating the vocabulary by iteratively merging frequently occurring character sequences in specific scenarios (such as high-concurrency web attack detection). First, the input training corpus is preprocessed, segmenting each word into a sequence based on its characters, and adding special symbols (such as...) at the beginning and end of each word. <w> and< / w> First, the boundaries of words are marked. Then, characters or subwords in the training corpus are used as the initial vocabulary in the word list. The frequency of occurrence of characters or subwords in the training corpus is counted and sorted according to frequency. Finally, among the frequency of character pairs obtained from the statistics, the most frequently occurring pair of adjacent characters or subwords is selected and merged. The selected character pairs are merged into a new character or subword, the newly formed subword is added to the word list, and the frequency statistics of characters or subwords are updated. As the merging operation proceeds, the word list will gradually expand and contain more independent characters or subwords, which will be convenient for subsequent word segmentation.

[0060] In some embodiments, segmenting N user requests into words based on a vocabulary to obtain M segmented user requests can be achieved by: replacing the interfering characters in each of the N user requests with the interfering characters based on a set character replacement rule to obtain N target user requests; wherein the interfering characters are used to represent characters that are not related to anomaly detection; and segmenting the N target user requests into words based on a vocabulary to obtain M segmented user requests.

[0061] Furthermore, character substitution rules include, but are not limited to, at least one of the following:

[0062] Replace Chinese characters in the user request with the first preset character, for example, replace Chinese characters with "CH";

[0063] Replace the email address in the user's request with a second preset character, for example, replace the email address with "MAIL";

[0064] Replace the numbers in the user's request with a third preset character, for example, replace the number string with "NUM";

[0065] Replace the domain name in the user's request with the fourth preset character, for example, replace the domain name with "U";

[0066] Merge multiple placeholders in a user request, for example, into two characters.

[0067] For example, take user requests q1: http: / / www.baidu.com / query? id=123&email=baidu@111.com&name=小小&number=1234567890 / and 1=1; user request q2: http: / / www.baidu.com / query? id=456&email=sina@126.com&number=3452341238 / and1=1'; user request q3: http: / / www.webai.com / and 1=1' as an example. First, perform replacement according to the character replacement rules shown above. After replacement, q1-->m1: [U] / query? id=[NUM]&email=[MAIL]&name=[CH]&number=[NUM] / and 1=1'; q2-->m2: [U] / query? id=[NUM]&email=[MAIL]&number=[NUM] / and 1=1'; q3-->m3: [U] / and 1=1'. Then, perform word segmentation on m1, m2, and m3 respectively based on the vocabulary. After word segmentation, q1-->d1: [[U], ' / ', 'query', '?', 'id', '=', '[NUM]', '&', 'email', '[MAIL]', '&', 'name', '[CH]', '&', 'number', '=', '[NUM]', ' / ', 'and 1=1', ']; q2-->d2: [[U], ' / ', 'query', '?', 'id', '=', '[NUM]', '&', 'email', '[MAIL]', '&', 'name', '[NUM]', ' / ', 'and1=1', ']; q3-->d3: [[U], ' / ', 'and 1=1', '].

[0068] In this step, the user request may refer to the Hypertext Transfer Protocol (HTTP), which is an application layer protocol used to transfer hypermedia documents. It consists of requests and responses, both of which are composed of messages, and the message includes a message header and a message body. The message header contains information such as the request / response line, message header fields, and line breaks, and the message body contains the actual request or response data (such as an HTML document).

[0069] In this step, by performing fuzzy processing on the interference characters that have nothing to do with anomaly detection and replacing them with abstract characters, and then using the BPE algorithm for word segmentation, both the deduplication ratio and the accuracy of the anomaly detection result can be improved.

[0070] It should be noted that the time window period can be flexibly set according to user needs, and this embodiment of the application does not impose specific restrictions here.

[0071] 202: Calculate the similarity between any two user requests among the above M user requests, and classify the M user requests based on each similarity to obtain a set of P user requests.

[0072] In some embodiments, the Locality Sensitive Hashing (LSH) algorithm can be used for similarity calculation. The LSH algorithm is an approximate nearest neighbor search method for large datasets. It uses a specific family of hash functions to increase the probability of similar elements being hashed into the same bucket. The LSH algorithm includes hash functions such as min-hash and p-stable LSH, suitable for different distance metrics. After indexing, it can efficiently perform online searches with a sublinear search time complexity, effectively reducing the amount of matching data and improving search efficiency.

[0073] In this step, when using the LSH algorithm to calculate the similarity between user requests, it can be done as follows: add each character of the first user request to the first matrix, and add each character of the second user request to the second matrix, and calculate the similarity between the first matrix and the second matrix; wherein, the first user request and the second user request are any two different user requests among the M user requests; if the similarity is greater than the similarity threshold, then the first user request and the second user request are classified into the same user request set; if the similarity is less than or equal to the similarity threshold, then the first user request and the second user request are classified into different user request sets, thereby classifying the M user requests to form P user request sets, where a user request set includes user requests of the same type, and P is an integer less than or equal to M.

[0074] For example, two minhash matrices can be created: one to add characters from the first user request (m1 as shown above), and the other to add characters from the second user request (m2 as shown above). Then, the similarity between the two minhash matrices is calculated. If the similarity is greater than a similarity threshold (e.g., 0.7), it indicates that the first user request and the second user request are similar, belong to the same category or are duplicate user requests, and can be placed in the same user request set. If the similarity is less than or equal to the similarity threshold, it indicates that the first user request and the second user request are not similar, do not belong to the same category or are different user requests, and can be placed in different user request sets.

[0075] 203: Based on any one of the user requests in the above P user request sets, determine the anomaly detection results for each of the N user requests.

[0076] In this step, if the detection engine detects abnormal traffic in any user request within the first user request set of P user request sets, then all user requests within the first user request set are determined to have abnormal traffic. For example, if a user request is randomly selected from the first user request set and input into the detection engine for abnormal traffic detection, and the result is determined to be abnormal traffic, then this result can be assigned to all requests within the first user request set. This eliminates the need to perform individual checks on each request of this type, significantly reducing the number of checks and improving detection efficiency.

[0077] In other embodiments, Long Short-Term Memory (LSTM) networks can be used for detection. The core of LSTM is a three-gate mechanism: a forget gate, an input gate, and an output gate. These gates control the flow of data in an adaptive manner, thereby capturing long-term dependent data and improving the accuracy of anomaly detection results.

[0078] In other embodiments, if the anomaly detection result of any user request in a certain user request set is determined to be a traffic anomaly, an alarm message can be output to indicate that all user requests included in the user request set have traffic anomalies, so that users can take timely measures to deal with them.

[0079] In this embodiment, firstly, N user requests within a set time window are obtained, and the N user requests are segmented into words based on a vocabulary to obtain M segmented user requests. Then, the similarity between any two user requests among the M user requests is calculated, and the M user requests are classified based on each similarity to obtain P sets of user requests. Finally, based on any one user request in each of the P sets of user requests, the anomaly detection result of each of the N user requests is determined. Compared with the prior art, it is not necessary to detect each user request one by one, which can reduce the number of detections, reduce resource consumption, and thus improve the efficiency of abnormal traffic detection.

[0080] Based on the above Figure 2 The method shown, Figure 3 This is a logical diagram illustrating a traffic anomaly detection method provided in an embodiment of this application.

[0081] Taking q1, q2, and q3 as examples, firstly, q1, q2, and q3 are blurred respectively, and the interfering characters in each of q1, q2, and q3 are replaced to obtain m1, m2, and m3; then, a word list is used to segment m1, m2, and m3 respectively to obtain d1, d2, and d3; next, the similarity between each pair of d1, d2, and d3 is calculated. If d1 and d2 meet the similarity threshold, then d1 and d2 are placed in the same user request set 1, while d3 does not meet the similarity threshold with either d1 or d2. If the required level is met, then d3 is placed in another user request set 2. Finally, a user request (such as d1) is randomly selected from user request set 1 and input into the detection engine, and d3 is selected from user request set 2 and input into the detection engine. The abnormal detection results of each are determined. If the abnormal detection result of d1 is determined to be traffic abnormal, then the abnormal detection result of d2 will also be assigned the value of traffic abnormal, and an alarm message can be output. If the abnormal detection result of d3 is determined to be traffic normal, then the corresponding detection can continue to be performed on another type of user request set.

[0082] Based on the same technical concept, this application also provides an abnormal traffic detection device, which can implement the abnormal traffic detection method flow described above in this application.

[0083] Figure 4 This is a schematic diagram of an abnormal flow detection device provided in an embodiment of this application. Figure 4 As shown, the device includes a word segmentation module 401, a similarity calculation module 402, and an anomaly detection module 403.

[0084] The word segmentation module 401 is used to acquire N user requests within a set time window, and to segment the N user requests into words based on a word list to obtain M user requests after word segmentation; wherein, the word list is used to record multiple characters of independent word segmentation, and N is equal to M, which is an integer greater than 0.

[0085] The similarity calculation module 402 is used to calculate the similarity between any two user requests among the M user requests, and classify the M user requests based on each similarity to obtain P user request sets; wherein, a user request set includes user requests of the same type, and P is an integer less than or equal to M.

[0086] The anomaly detection module 403 is used to determine the anomaly detection result of each of the N user requests based on any one of the user requests in the P user request sets.

[0087] In some embodiments, the word segmentation module 401 is specifically used for:

[0088] Based on the set character replacement rules, the interfering characters in each of the N user requests are replaced to obtain N target user requests; wherein, the interfering characters are used to represent characters that are not related to anomaly detection;

[0089] The N target user requests are segmented into words based on the vocabulary to obtain M segmented user requests.

[0090] In some embodiments, the similarity calculation module 402 is specifically used for:

[0091] Each character of the first user request is added to the first matrix, and each character of the second user request is added to the second matrix, and the similarity between the first matrix and the second matrix is ​​calculated; wherein, the first user request and the second user request are any two different user requests among the M user requests;

[0092] If the similarity is greater than the similarity threshold, then the first user request and the second user request are classified into the same set of user requests;

[0093] If the similarity is less than or equal to the similarity threshold, then the first user request and the second user request are classified into different user request sets.

[0094] In some embodiments, the anomaly detection module 403 is specifically used for:

[0095] If the detection engine detects that any user request in the first user request set of the P user request sets has abnormal traffic, then it is determined that the abnormal detection results of all user requests included in the first user request set are abnormal traffic.

[0096] It should be noted that the apparatus provided in this application embodiment can implement all the method steps in the above method embodiment and achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0097] Based on the same technical concept, this application also provides an electronic device that can realize the function of the aforementioned abnormal flow detection device.

[0098] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0099] At least one processor 501 and a memory 502 connected to at least one processor 501. In this embodiment, the specific connection medium between the processor 501 and the memory 502 is not limited. Figure 5The example shown is the connection between processor 501 and memory 502 via bus 500. Bus 500 is... Figure 5 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. The Bus 500 can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 5 The term 501 is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, the processor 501 can also be called a controller; there is no restriction on the name.

[0100] In this embodiment, memory 502 stores instructions executable by at least one processor 501. By executing the instructions stored in memory 502, at least one processor 501 can perform a data processing method as described above. Processor 501 can implement... Figure 5 The functions of each module in the device shown.

[0101] The processor 501 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 502 and calling data stored in memory 502, the processor can perform various functions and process data, thereby monitoring the device as a whole.

[0102] In this embodiment, processor 501 may include one or more processing units. Processor 501 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 501. In some embodiments, processor 501 and memory 502 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0103] Processor 501 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the abnormal traffic detection method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0104] Memory 502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 502 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 502 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 502 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0105] By designing and programming the processor 501, the code corresponding to the abnormal traffic detection method described in the foregoing embodiments can be embedded into the chip, thereby enabling the chip to execute the code during runtime. Figure 2 The illustrated embodiment provides a method for detecting abnormal traffic. How to design and program the processor 501 is a technique well-known to those skilled in the art and will not be described further here.

[0106] It should be noted that the electronic device provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0107] Based on the same technical concept, embodiments of this application provide a computer storage medium, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the abnormal traffic detection methods described above. Since the principle by which the above-described computer storage medium solves the problem is similar to that of an abnormal traffic detection method, the implementation of the above-described computer storage medium can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0108] In specific implementation, computer storage media can include: Universal Serial Bus Flash Drive (USB), portable hard drive, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0109] Based on the same technical concept, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the abnormal traffic detection methods described above. Since the principle of the above computer program product in solving the problem is similar to that of an abnormal traffic detection method, the implementation of the above computer program product can refer to the implementation of the method, and repeated details will not be described again.

[0110] Computer program products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0111] The methods in this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in this application are performed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, network equipment, user equipment, core network equipment, OAM, or other programmable devices.

[0112] The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; or an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0113] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0114] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that specifies the functions in one or more boxes. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including the instruction device, which is implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations of the present invention fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. An abnormal flow detection method, characterized in that, include: N user requests within a set time window are obtained, and the N user requests are segmented into words based on a vocabulary to obtain M user requests after word segmentation; wherein, the vocabulary is used to record multiple characters of independent word segmentation, the vocabulary is constructed using the byte pair encoding BPE algorithm, and N is equal to M, which are integers greater than 0; Calculate the similarity between any two user requests among the M user requests, and classify the M user requests based on each similarity to obtain P user request sets; wherein, a user request set includes user requests of the same class, and P is an integer less than or equal to M, and the similarity is calculated using the Locality Sensitive Hash (LSH) algorithm; Based on any one user request from each of the P user request sets, determine the anomaly detection results for each of the N user requests; The step of segmenting the N user requests based on a vocabulary to obtain M segmented user requests includes: Based on the set character replacement rules, the interfering characters in each of the N user requests are replaced to obtain N target user requests; wherein, the interfering characters are used to represent characters that are not related to anomaly detection; The N target user requests are segmented into words based on the vocabulary to obtain M segmented user requests.

2. The method as described in claim 1, characterized in that, The character replacement rules include at least one of the following: Replace the Chinese characters in the user's request with the first preset character; Replace the email address in the user's request with the second preset character; Replace the numeric string in the user's request with a third preset character; Replace the domain name in the user's request with the fourth preset character; Merge multiple placeholders in the user request.

3. The method as described in claim 1, characterized in that, The similarity between any two user requests from the M user requests is calculated, and the M user requests are classified based on each similarity to obtain a set of P user requests, including: Each character of the first user request is added to the first matrix, and each character of the second user request is added to the second matrix, and the similarity between the first matrix and the second matrix is ​​calculated; wherein, the first user request and the second user request are any two different user requests among the M user requests; If the similarity is greater than the similarity threshold, then the first user request and the second user request are classified into the same set of user requests; If the similarity is less than or equal to the similarity threshold, then the first user request and the second user request are classified into different user request sets.

4. The method according to any one of claims 1-3, characterized in that, The step of determining the anomaly detection results of each of the N user requests based on any one of the P user request sets includes: If the detection engine detects that any user request in the first user request set of the P user request sets has abnormal traffic, then it is determined that the abnormal detection results of all user requests included in the first user request set are abnormal traffic.

5. An abnormal flow detection device, characterized in that, include: The word segmentation module is used to acquire N user requests within a set time window, and to segment the N user requests into words based on a word list to obtain M user requests after word segmentation; wherein, the word list is used to record multiple characters of independent word segmentation, the word list is constructed using the byte pair encoding BPE algorithm, and N is equal to M, which are integers greater than 0; The similarity calculation module is used to calculate the similarity between any two user requests among the M user requests, and classify the M user requests based on each similarity to obtain P user request sets; wherein, a user request set includes user requests of the same class, and P is an integer less than or equal to M, and the similarity is calculated using the Locality Sensitive Hash (LSH) algorithm; The anomaly detection module is used to determine the anomaly detection results of each of the N user requests based on any one of the user requests in the P user request sets. Specifically, the word segmentation module is used for: Based on the set character replacement rules, the interfering characters in each of the N user requests are replaced to obtain N target user requests; wherein, the interfering characters are used to represent characters that are not related to anomaly detection; The N target user requests are segmented into words based on the vocabulary to obtain M segmented user requests.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a computer program stored in the memory, implements the method of any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-4.

Images