Abnormal traffic detection method, device, equipment and storage medium

Abstract

The application relates to the field of network security, and provides an abnormal traffic detection method, device, equipment and storage medium. The method utilizes the fluctuation characteristics of a cache hit rate curve of CDN resources in a domain preposition attack scene, combines traffic features affecting the cache hit rate in to-be-detected traffic of a source IP address with the cache hit rate of the source IP address in each sliding time window, predicts the probability of the source IP address initiating a domain preposition attack in a next sliding time window based on the association relationship between the change trend of a continuous multiple feature sequences of the source IP address in continuous multiple sliding time windows and the traffic type, and detects a batch of network traffics in a short time under the premise of meeting the requirements of low resource consumption and high precision detection, so that rapid positioning and accurate attack on abnormal IP addresses in the domain preposition attack scene are realized.

Description

Technical Field

[0001] The present application relates to the field of network security and provides an abnormal traffic detection device, equipment and storage medium. Background Art

[0002] With the advancement of information and communication technologies and the continuous increase in internet connection points, cybersecurity threats are also increasing. Domain fronting is a censorship circumvention technique that exploits the Hypertext Transfer Protocol over Secure Socket Layer (HTTPS) and Content Delivery Network (CDN) service features to conceal the true destination of communication.

[0003] The principle is to use different domain names at different communication layers. The attacker declares a legitimate domain name in the Server Name Indication (SNL) extension field of the Transport Layer Security (TLS) protocol that is visible to the detector to pass network censorship, and specifies the illegal domain name to be actually connected in the Host field of the HTTPS request header that is invisible to the detector to evade network censorship and connect to blocked illegal sites.

[0004] Currently, HTTPS traffic decryption equipment is deployed to collect server logs of various servers. A server log records multiple network flows generated when each client exchanges data with the server over a period of time. For each network flow, the following operations are performed: HTTPS traffic decrypts the HTTPS request header of a network flow to obtain the value of the Host field in the request header. When the value of the SNI extension field of the flow is not equal to the value of the Host field, the network flow is determined to be abnormal traffic that initiates a domain front-end attack.

[0005] However, when using the above method to detect massive network traffic in a network environment, a large amount of computing resources and bandwidth resources will be consumed. The above method is no longer applicable to the ever-growing network environment. Summary of the Invention

[0006] The embodiments of the present application provide an abnormal traffic detection device, apparatus, equipment and storage medium to solve the problem of consuming computing resources and bandwidth resources when detecting massive traffic.

[0007] In a first aspect, an embodiment of the present application provides a method for detecting abnormal traffic, including:

[0008] Collecting content distribution logs containing multiple to-be-detected flows, dividing the to-be-detected flows associated with the same source Internet Protocol IP address into a group, and obtaining at least one to-be-detected flow group;

[0009] For each traffic group to be detected, the following operations are performed respectively: based on a set sliding time window, a traffic group to be detected is divided into multiple continuous traffic sequences; each traffic sequence is subjected to feature extraction and preprocessing to obtain a feature sequence of a source IP address in each sliding time window, and a feature sequence includes the multi-dimensional traffic features and cache hit rate of the source IP address in a sliding time window; based on the correlation between the change trend of the multiple continuous feature sequences of the source IP address in multiple continuous sliding time windows and the traffic type, the probability of the source IP address launching a domain front attack in the next sliding time window is predicted, and when the predicted probability value exceeds the set threshold, the traffic to be detected associated with the source IP address is treated as abnormal traffic.

[0010] Optionally, feature extraction and preprocessing are performed on each traffic sequence to obtain a feature sequence of a source IP address in each sliding time window, wherein the following operations are performed for each traffic sequence:

[0011] Performing time coding and request coding on a plurality of to-be-detected flows included in the one flow sequence to obtain a plurality of time features and request features of the one source IP address within the one time window;

[0012] Obtaining a cache hit rate of the source IP address within the sliding time window based on cache hit statuses of the multiple traffic flows to be detected and the total number of requests initiated by the source IP address;

[0013] Normalizing the multiple time features, the request features, and the cache hit rate to obtain a feature sequence of the source IP address in the sliding time window.

[0014] Optionally, performing time coding and request coding on the multiple to-be-detected flows included in the one flow sequence to obtain multiple time features and request features of the one source IP address within the one time window includes:

[0015] Time-encoding the request timestamps of the plurality of to-be-detected flows in the flow sequence respectively to obtain a plurality of time features of the source IP address within the sliding time window;

[0016] Request encoding is performed on the multiple traffic flows to be detected to obtain a request feature of the source IP address within a sliding time window.

[0017] Optionally, the step of respectively time-coding the request timestamps of the plurality of to-be-detected flows in the flow sequence to obtain the plurality of time features of the source IP address within the sliding time window includes:

[0018] According to the time coding rule, the request timestamps of the multiple traffic flows to be detected in the traffic sequence are respectively encoded to obtain their respective original time features;

[0019] According to the tag encoding rule, the request timestamps of multiple to-be-detected times in the traffic sequence are encoded respectively to obtain respective additional time features.

[0020] Optionally, performing request encoding on the multiple traffic flows to be detected to obtain a request feature of the source IP address within a sliding time window includes:

[0021] Based on the to-be-detected traffic associated with the source IP address within the sliding time window, counting the total number of requests initiated by the source IP address within the sliding time window;

[0022] Based on the resource type of the requested resource of each to-be-detected traffic, counting the total number of resource types requested by the source IP address within the sliding time window;

[0023] Based on the response status codes of the traffic to be detected, counting the proportion of status codes of the source IP address within the sliding time window except for the response status codes indicating successful request processing;

[0024] The total number of requests, the total number of requested resource types, and the status code ratio of the source IP address within the sliding time window are used as request features of the source IP address within the sliding time window.

[0025] In a second aspect, an embodiment of the present application further provides an abnormal flow detection device, comprising:

[0026] a traffic collection unit, configured to collect content distribution logs containing a plurality of traffic flows to be detected, and to group the traffic flows to be detected associated with the same source Internet Protocol (IP) address into a group, thereby obtaining at least one traffic flow group to be detected;

[0027] The traffic analysis unit is used to perform the following operations for each traffic group to be detected: based on a set sliding time window, divide a traffic group to be detected into multiple continuous traffic sequences; perform feature extraction and preprocessing on each traffic sequence to obtain a feature sequence of a source IP address in each sliding time window, and a feature sequence includes the multi-dimensional traffic features and cache hit rate of the source IP address in a sliding time window; based on the correlation between the change trend of the multiple continuous feature sequences of the source IP address in multiple continuous sliding time windows and the traffic type, predict the probability of the source IP address launching a domain front attack in the next sliding time window, and when the predicted probability value exceeds a set threshold, treat the traffic to be detected associated with the source IP address as abnormal traffic.

[0028] Optionally, the traffic analysis unit performs the following operations on a traffic sequence:

[0029] Performing time coding and request coding on a plurality of to-be-detected flows included in the one flow sequence to obtain a plurality of time features and request features of the one source IP address within the one time window;

[0030] Obtaining a cache hit rate of the source IP address within the sliding time window based on cache hit statuses of the multiple traffic flows to be detected and the total number of requests initiated by the source IP address;

[0031] Normalizing the multiple time features, the request features, and the cache hit rate to obtain a feature sequence of the source IP address in the sliding time window.

[0032] Optionally, the traffic analysis unit is used to:

[0033] Time-encoding the request timestamps of the plurality of to-be-detected flows in the flow sequence respectively to obtain a plurality of time features of the source IP address within the sliding time window;

[0034] Request encoding is performed on the multiple traffic flows to be detected to obtain a request feature of the source IP address within a sliding time window.

[0035] Optionally, the traffic analysis unit is used to:

[0036] According to the time coding rule, the request timestamps of the multiple traffic flows to be detected in the traffic sequence are respectively encoded to obtain their respective original time features;

[0037] According to the tag encoding rule, the request timestamps of multiple to-be-detected times in the traffic sequence are encoded respectively to obtain respective additional time features.

[0038] Optionally, the traffic analysis unit is used to:

[0039] Based on the to-be-detected traffic associated with the source IP address within the sliding time window, counting the total number of requests initiated by the source IP address within the sliding time window;

[0040] Based on the resource type of the requested resource of each to-be-detected traffic, counting the total number of resource types requested by the source IP address within the sliding time window;

[0041] Based on the response status codes of the traffic to be detected, counting the proportion of status codes of the source IP address within the sliding time window except for the response status codes indicating successful request processing;

[0042] The total number of requests, the total number of requested resource types, and the status code ratio of the source IP address within the sliding time window are used as request features of the source IP address within the sliding time window.

[0043] In a third aspect, an embodiment of the present application further provides a computer device comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of any one of the above-mentioned abnormal traffic detection methods.

[0044] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium comprising a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of any one of the above-mentioned abnormal traffic detection methods.

[0045] The beneficial effects of this application are as follows:

[0046] An embodiment of the present application provides an abnormal traffic detection method, apparatus, device and storage medium, the method comprising: collecting content distribution logs containing multiple traffic to be detected, dividing the traffic to be detected associated with the same source Internet Protocol IP address into a group to obtain at least one traffic group to be detected; performing the following operations for each traffic group to be detected: based on a set sliding time window, dividing a traffic group to be detected into multiple continuous traffic sequences; performing feature extraction and preprocessing on each traffic sequence to obtain a feature sequence of a source IP address in each sliding time window, a feature sequence including multi-dimensional traffic features and cache hit rate of the source IP address in a sliding time window; then, based on the correlation between the change trend of multiple continuous feature sequences of a source IP address in multiple continuous sliding time windows and the traffic type, predicting the probability of the source IP address launching a domain front attack in the next sliding time window, and when the predicted probability value exceeds a set threshold, treating the traffic to be detected associated with the source IP address as abnormal traffic.

[0047] This application utilizes the characteristic that the cache hit rate curve of CDN resources fluctuates in the domain front attack scenario, combines the traffic characteristics that affect the cache hit rate in the traffic to be detected associated with a source IP address, and the cache hit rate of the source IP address in each sliding time window, and predicts the probability of the source IP address initiating a domain front attack in the next sliding time window based on the correlation between the change trend of multiple consecutive feature sequences of a source IP address in multiple consecutive sliding time windows and the traffic type. Compared with the detection method used in related technologies to compare whether the values ​​of the SNI extension field and the Host field in the network traffic are equal, the detection method provided by this application detects a batch of network traffic in a short period of time while taking into account the needs of low resource consumption and high-precision detection, thereby realizing the rapid positioning and precise attack of abnormal IP addresses in the domain front attack scenario.

[0048] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0050] Figure 1A A schematic diagram of a process for detecting abnormal traffic provided in an embodiment of the present application;

[0051] Figure 1B A logical diagram of a client accessing a server through DNS and CDN provided in an embodiment of the present application;

[0052] Figure 1C A logical diagram of the IP address corresponding to the DNS request domain name provided in the embodiment of the present application;

[0053] Figure 1D A logical diagram of dividing a traffic group to be detected based on a sliding time window provided in an embodiment of the present application;

[0054] Figure 1E A schematic diagram of a process for obtaining a characteristic sequence of a source IP address in a sliding time window provided in an embodiment of the present application;

[0055] Figure 1F Schematic diagram of cache hit rate curves in different scenarios provided by the embodiments of the present application;

[0056] Figure 1G A logical diagram of the LSTM network training provided in an embodiment of the present application;

[0057] Figure 1H Schematic diagram of the LSTM network structure provided in the embodiment of the present application;

[0058] Figure 2A A simplified flowchart of abnormal traffic detection in actual scenarios provided by an embodiment of the present application;

[0059] Figure 2B A complete flowchart of abnormal traffic detection in actual scenarios provided by the embodiment of the present application;

[0060] Figure 3 A schematic structural diagram of an abnormal flow detection device provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0062] Figure 5 This is a schematic diagram of the structure of a computing device in an embodiment of the present application. DETAILED DESCRIPTION

[0063] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of the technical solutions of this application, but not all of them. Based on the embodiments described in this application document, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the technical solutions of this application.

[0064] The following explains some of the terms used in the embodiments of the present application to facilitate understanding by those skilled in the art.

[0065] 1. Domain Fronting: This is a censorship circumvention technique that exploits the HTTPS protocol and CDN service features to conceal the true destination of communication. It works by using different domain names at different communication layers. Attackers declare a legitimate domain name in the TLS SNL extension, visible to detectors, to bypass censorship. However, they specify the actual illegitimate domain name in the Host field of the HTTPS request header, invisible to detectors, to evade censorship and connect to blocked, illegal sites.

[0066] 2. CDN: It is a distributed network architecture that enables users to access and obtain the required content faster by distributing content to server nodes deployed in various locations, thereby increasing content transmission speed, reducing network latency, alleviating server load, improving website reliability, and resisting attacks.

[0067] 3. SNI: It is an extension field of the TLS protocol, which is used to indicate the domain name of the server to which the client wants to connect when establishing an encrypted communication connection.

[0068] 4. Host field: In the HTTP protocol, when a client sends an HTTP request, it includes a Host field in the request header. This field is used to specify the server to which the request is to connect. The value of the Host field is usually the domain name of the server, such as "Host:example.com".

[0069] 5. Response status code: This is part of the HTTP response and is used to inform the client about the processing result of its request. The response status code consists of three digits and is divided into five categories, each of which represents a different response type.

[0070] Among them, 1xx means that the received request is being processed, 2xx means that the request has been successfully accepted, understood and processed by the server, 3xx means that the client needs to take further action to complete the request, 4xx means that the client's request is incorrect and the server cannot process it, and 5xx means that an error occurred while the server was processing the request.

[0071] 6. Long Short-Term Memory (LSTM) network: This is a time-recursive neural network designed to solve the problems of vanishing and exploding gradients during long sequence training. It is widely used in text translation, image analysis, document summarization, speech recognition, image recognition, music synthesis and other fields.

[0072] The following is a brief introduction to the design concept of the embodiment of this application:

[0073] With the advancement of information and communication technologies and the continuous increase in internet connection points, network security threats are also increasing. Domain fronting is a censorship circumvention technique that utilizes the HTTPS protocol and CDN service features to conceal the true destination of communication.

[0074] The principle is to use different domain names at different communication layers. The attacker declares a legitimate domain name in the SNL extension field of TLS that is visible to the detector to pass network censorship, and specifies the illegal domain name to be actually connected in the Host field of the HTTPS request header that is invisible to the detector to evade network censorship and connect to blocked illegal sites.

[0075] Currently, HTTPS traffic decryption equipment is deployed to collect server logs of various servers. A server log records multiple network flows generated when each client exchanges data with the server over a period of time. For each network flow, the following operations are performed: HTTPS traffic decrypts the HTTPS request header of a network flow to obtain the value of the Host field in the request header. When the value of the SNI extension field of the flow is not equal to the value of the Host field, the network flow is determined to be abnormal traffic that initiates a domain front-end attack.

[0076] However, when using the above method to detect massive network traffic in a network environment, a large amount of computing resources and bandwidth resources will be consumed. The above method is no longer applicable to the ever-growing network environment.

[0077] In view of this, an embodiment of the present application provides an abnormal traffic detection method, device, equipment and storage medium. The method specifically includes: collecting content distribution logs containing multiple traffic to be detected, dividing the traffic to be detected associated with the same source Internet Protocol IP address into a group, and obtaining at least one traffic group to be detected; for each traffic group to be detected, performing the following operations respectively: based on a set sliding time window, dividing a traffic group to be detected into multiple continuous traffic sequences; performing feature extraction and preprocessing on each traffic sequence respectively, obtaining a feature sequence of a source IP address in each sliding time window, a feature sequence containing the multi-dimensional traffic features and cache hit rate of the source IP address in a sliding time window; then, based on the correlation between the change trend of multiple continuous feature sequences of a source IP address in multiple continuous sliding time windows and the traffic type, predicting the probability of the source IP address launching a domain front attack in the next sliding time window, and when the predicted probability value exceeds the set threshold, the traffic to be detected associated with the source IP address is treated as abnormal traffic.

[0078] This application utilizes the characteristic that the cache hit rate curve of CDN resources fluctuates in the domain front attack scenario, combines the traffic characteristics that affect the cache hit rate in the traffic to be detected associated with a source IP address, and the cache hit rate of the source IP address in each sliding time window, and predicts the probability of the source IP address initiating a domain front attack in the next sliding time window based on the correlation between the change trend of multiple consecutive feature sequences of a source IP address in multiple consecutive sliding time windows and the traffic type. Compared with the detection method used in related technologies to compare whether the values ​​of the SNI extension field and the Host field in the network traffic are equal, the detection method provided by this application detects a batch of network traffic in a short period of time while taking into account the needs of low resource consumption and high-precision detection, thereby realizing the rapid positioning and precise attack of abnormal IP addresses in the domain front attack scenario.

[0079] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application can be combined with each other if there is no conflict.

[0080] Combine Figure 1A The flow chart shown specifically introduces how to use the abnormal traffic detection method provided in the embodiment of the present application to detect whether there is abnormal traffic in the network environment that initiates a domain front-end attack.

[0081] S101: Collect content distribution logs containing multiple traffic to be detected, divide the traffic to be detected associated with the same source Internet Protocol IP address into a group, and obtain at least one traffic group to be detected.

[0082] Combine Figure 1B The logical diagram shown first introduces the process of the client accessing the server through the Domain Name System (DNS) and CDN.

[0083] (1) User request: Since computer networks transmit data based on IP addresses rather than domain names, when a client (such as a browser) obtains the domain name entered by the user, it will first search the client cache, local hosts file, and DNS cache. If it is not found, it will trigger a DNS query request, and the local DNS server will search for the IP address corresponding to the domain name based on the request.

[0084] (2) DNS resolution: Figure 1C As shown, the local DNS server sequentially requests the root domain name server to obtain the IP address of the top-level domain name server, and then requests the top-level domain name server again to obtain the IP address of the authoritative domain name server. The authoritative DNS server returns one or more IP addresses mapped to the domain name. When using a CDN, the DNS also uses a technology called DNS load balancing to return the IP address of the server closest to the user.

[0085] (3) CDN edge node selection: The CDN network structure is usually composed of central nodes, regional nodes and edge nodes. In order to save CDN storage space while ensuring a certain hit rate, all content is stored in the central node, while regional nodes and edge nodes usually only store a part of popular content.

[0086] The client uses the IP address obtained from the DNS to establish a connection with the CDN's central node. The CDN service redirects the user's request to the optimal CDN edge node based on factors such as the user's geographic location and network conditions. The edge node then provides services to the user, ensuring a quick response to the user's request.

[0087] (4) Content transmission: The client uses the IP address obtained from the central node to establish a connection with the selected edge node. If the required content is already on the node, the edge node directly provides the content to the client, greatly reducing the delay in data transmission. If the required content is not on the edge node, the user's request is redirected to the regional node at the next level until it is traced back to the source server. The content is then pulled to the selected edge node and returned to the client, ensuring real-time interaction between the client and the server.

[0088] Through this series of steps, DNS helps the client find the server's address, while CDN further optimizes the content distribution process, improves access speed and content loading speed, and reduces the pressure on the source server.

[0089] When CDN provides content distribution services, it records multiple network flows generated by all client access requests within a time period in a content distribution log (i.e., CDN log), and then pushes the log file to the cloud storage service at specified time intervals.

[0090] Content distribution logs are collected regularly through the Application Programming Interface (API) provided by the cloud service provider. These log files record multiple network traffic records. A network is generated based on the request details at a CDN node. They contain data such as the request timestamp, source IP address, request method, request Uniform Resource Locator (URL), cache hit status, and response status code from the target server accessed by the source IP address. They are important for network monitoring, troubleshooting, security analysis, and optimizing network performance.

[0091] A traffic flow to be detected contains the following information:

[0092] (1) Request timestamp: records the specific time when the data packet is transmitted.

[0093] (2) Source IP address: The IP address of the client or terminal device that initiates the communication.

[0094] (3) Destination IP address: The IP address of the server or target device that receives the communication.

[0095] (4) Port number: Record the source port and destination port separately to identify the applications or services at both ends of the communication.

[0096] (5) Protocol type: indicates the protocol used for data transmission, such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), HTTP, HTTPS, etc.

[0097] (6) Packet size: the number of bytes in a single packet.

[0098] (7) Data direction: Indicates whether the data is flowing in (downloading) or out (uploading).

[0099] (8) TCP status: In a TCP connection, it includes flags such as Synchronize Sequence Numbers (SYN), Acknowledgement (ACK), and Finish (FIN), which indicate the status of the connection, such as establishment, confirmation, or termination.

[0100] (9) HTTP related information: request method, URL path, HTTP version, response status code, etc.

[0101] (10) Other metadata: This may also include underlying communication details such as TCP sequence numbers, acknowledgment numbers, and whether the packet is a new connection.

[0102] S102: For each traffic group to be detected, the following operations are performed respectively: based on the set sliding time window, a traffic group to be detected is divided into multiple continuous traffic sequences; each traffic sequence is subjected to feature extraction and preprocessing to obtain a feature sequence of a source IP address in each sliding time window, and a feature sequence includes the multi-dimensional traffic features and cache hit rate of the source IP address in a sliding time window; based on the correlation between the change trend of multiple consecutive feature sequences of a source IP address in multiple consecutive sliding time windows and the traffic type, the probability of the source IP address launching a domain front attack in the next sliding time window is predicted, and when the predicted probability value exceeds the set threshold, the traffic to be detected associated with the source IP address is treated as abnormal traffic.

[0103] Combine Figure 1D As shown in the logic diagram, the sliding time window moves along the set direction with a set step size, and the traffic to be detected with request timestamps in the same sliding time window in a traffic group to be detected is divided into a sequence, thereby obtaining multiple continuous traffic sequences.

[0104] This application introduces a sliding time window mechanism to dynamically detect the network behavior of the source IP address and construct a feature sequence of the source IP address in each sliding time window. A feature sequence contains the multi-dimensional traffic characteristics and cache hit rate of the source IP address in a sliding time window.

[0105] Among them, combined Figure 1E The following flow chart shows the following operations for one of the traffic sequences:

[0106] S1021: Time coding and request coding are performed on multiple to-be-detected flows contained in a flow sequence to obtain multiple time features and request features of a source IP address within a time window.

[0107] (1) Time-encode the request timestamps of multiple traffic flows to be detected in the traffic sequence respectively to obtain multiple time features of the source IP address within a sliding time window. This application provides two time encoding methods. One is to use a time encoding rule to encode the year, month, day, and hour of the request timestamp separately and convert the request timestamp into a digital code in a unified format. The other is to use a marking encoding rule to mark whether the request timestamp is a special time node such as a weekend or holiday.

[0108] Specifically, according to the time coding rule, the request timestamps of multiple to-be-detected flows in a flow sequence are encoded respectively to obtain their respective original time features.

[0109] The time coding rule uses the set year as the base number, and the difference between the request timestamp year and the set year is used as the request timestamp year code Y. A year has 12 months, and the month of the request timestamp is coded by month, resulting in the request timestamp month code M. A month has 31 days, and the date of the request timestamp is coded by day, resulting in the request timestamp date code D. A day has 24 hours, and the hour code T is the hour segment corresponding to the end time of the request timestamp, using a 2-hour base number.

[0110] For example, the request timestamp is "03:00 to 04:59 on January 2, 2000", and the original time feature obtained by encoding it is "Y00M01D02T03".

[0111] In addition to the original time features, the request timestamps of multiple detection times in a traffic sequence can also be encoded according to the marking coding rules, and whether the request timestamps are special time nodes such as weekends and holidays can be marked to obtain their respective additional time features.

[0112] For example, 1 indicates that the request timestamp is a weekend, 2 indicates that the request timestamp is a holiday, and 0 indicates that the request timestamp is a weekday or a non-holiday.

[0113] (2) Request encoding is performed on multiple traffic flows to be detected to obtain the request characteristics of the source IP address within a sliding time window.

[0114] This application constructs the request characteristics of a source IP address within a sliding time window from three aspects: total number of requests, resource diversity, and response status code.

[0115] Specifically, based on the to-be-detected traffic associated with a source IP address within a sliding time window, the total number of requests initiated by the source IP address within the sliding time window is counted.

[0116] Based on the resource type of the requested resource (i.e., the requested URL) of each traffic to be detected, the total number of resource types requested by the source IP address within a sliding time window is counted. Resource types include png, mp4, css, etc.

[0117] Based on the response status codes of each traffic flow to be tested, the percentage of response status codes other than those indicating successful request processing for that source IP address within a sliding time window is counted. In the HTTP protocol, 2xx indicates that the request was successfully accepted, understood, and processed by the server. Therefore, the percentage of the other four response status codes within this sliding time window is counted.

[0118] Finally, the total number of requests, the total number of requested resource types, and the status code ratio of a source IP address in a sliding time window are used as the request features of the source IP address in a sliding time window.

[0119] S1022: Based on the cache hit status of multiple traffic flows to be detected and the total number of requests initiated by the source IP address, obtain a cache hit rate of the source IP address within a sliding time window.

[0120] Based on the cache hits recorded in multiple to-be-detected flows belonging to the same source IP address, the total number of cache hits for the source IP address within a sliding time window is counted, as shown in Formula 1. Based on the total number of cache hits for the source IP address within a sliding time window and the total number of requests initiated by the source IP address, the cache hit rate of the source IP address within a sliding time window is calculated.

[0121]

[0122] S1023: Normalize multiple time features, request features, and cache hit rates to obtain a feature sequence of the source IP address in a sliding time window.

[0123] Normalization is an important step in data preprocessing, aiming to adjust the scale of the data so that it falls within a specific range for easier comparison and analysis. Common normalization methods include min-max scaling, z-score standardization, decimal scaling, max absolute scaling, Tan-H transformation, and logarithmic transformation. The appropriate normalization method can be selected based on the characteristics of the data and the requirements of subsequent analysis or modeling.

[0124] For example, feature extraction and preprocessing are performed on the multiple network flows shown in Table 1 to obtain the feature sequence shown in Table 2.

[0125] Table 1

[0126]

[0127] Table 2

[0128]

[0129] With time as the horizontal axis and the cache hit rate of CDN resources as the vertical axis, Figure 1F The figure shows the changes in the cache hit rate curve under different modes. In normal access scenarios, the cache hit rate curve of CDN resources generally remains stable at a high level, with periodic slight dips and recoveries, but overall, the fluctuation is small. In the domain fronting attack scenario, however, because the resource content requested by the client is rarely accessed by other clients, the CDN node may not store the relevant resource content, resulting in a low cache hit rate in the domain fronting attack scenario, which in turn causes large fluctuations in the cache hit rate curve in this scenario. Due to the sudden and targeted nature of domain fronting attacks, the cache hit rate curve may also form steep downward peaks or irregular peaks.

[0130] This application utilizes the characteristic that the cache hit rate curve of CDN resources fluctuates in the domain pre-attack scenario, combines the traffic characteristics that affect the cache hit rate in the traffic to be detected associated with a source IP address, and the cache hit rate of the source IP address in each sliding time window, and uses a time series analysis model based on a recurrent neural network (RNN) that can capture long-range dependencies in time series data to learn the correlation between the changing trend of multiple consecutive feature sequences of a source IP address in multiple consecutive sliding time windows and the traffic type, and then predicts the probability of the source IP address initiating a domain pre-attack in the next sliding time window based on the correlation. When the predicted probability value output by the model exceeds the set threshold, it means that the neural network predicts that the behavior pattern of the source IP address is highly suspected of a domain pre-attack, triggering an alarm and generating a domain pre-attack alarm, reminding management personnel to focus on monitoring the source IP address and its associated network traffic and access requests.

[0131] Time series analysis models used to resolve long-term dependencies include LSTM networks, gated recurrent units, Transformer networks, deep residual networks, etc. These neural networks enhance their ability to learn long-term dependencies on sequence data through network structures with different mechanisms. They are commonly used networks for tasks such as speech recognition, machine translation, and text generation.

[0132] like Figure 1GAs shown in the figure, using an LSTM network as an example, feature extraction and preprocessing are performed on the training sample set to obtain multiple time-continuous feature sequences. To ensure time continuity, these multiple time-continuous feature sequences are divided into training sequences, verification sequences, and test sequences. These three sequences are then input into the LSTM network in sequence. The training network learns the relationship between the cache content hit rate of the sample source IP address, the changing trend of multidimensional features within multiple consecutive sliding time windows, and traffic type. The LSTM network model parameters are then optimized to obtain a trained LSTM network.

[0133] like Figure 1H As shown, there are three gate structures inside LSTM, the forget gate, the input gate and the output gate. At time t, LSTM also has three inputs, the network input value X at time t t , the network output value h at time (t-1) t-1 and the cell state C at time (t-1) t-1 The forget gate controls the cell state C at time (t-1) t-1 The degree of forgetting determines the unit state C at time (t-1) t-1 How many cell states C can be retained until time t? t In the example, the input gate determines the network input value X at time t t How many cell states C can be retained until time t? t Due to the control of the forget gate, the cell state C at time t t The information from a long time ago can be stored in the memory, and due to the control of the input gate, it can prevent the current irrelevant content from entering the cell state C at time t t The output gate controls the cell state C at time t. t The degree of filtering determines the cell state C at time t t How many network output values ​​h are output to time t? t middle.

[0134] Combine Figures 2A-2B The schematic diagram shown gives the implementation process of applying the method of the application to a practical scenario.

[0135] S201: Regularly collect content distribution logs containing multiple traffic flows to be detected through an API provided by a cloud service provider, divide the traffic flows to be detected associated with the same source IP address into a group, and obtain at least one traffic flow group to be detected.

[0136] S202: For each traffic group to be detected, the following operations are performed respectively: based on the set sliding time window, a traffic group to be detected is divided into multiple continuous traffic sequences; each traffic sequence is subjected to feature extraction and normalization processing to obtain a feature sequence of a source IP address in each sliding time window. A feature sequence contains the multi-dimensional traffic features and cache hit rate of the source IP address in a sliding time window.

[0137] S203: Input multiple feature sequences associated with the same source IP address into an RNN-based time series analysis model to predict the probability of the source IP address launching a domain front-end attack within the next sliding time window; wherein, the time series analysis model is based on a training sample set, and the training model learns the cache content hit rate of the sample source IP address, the changing trend of the multi-dimensional features in multiple consecutive sliding time windows, and the correlation between the traffic type.

[0138] S204: Determine whether the predicted probability value exceeds a set threshold. If so, execute step 205; otherwise, execute step 206.

[0139] S205: The traffic to be detected associated with the source IP address is regarded as abnormal traffic, an alarm is triggered, and a domain front attack alarm is generated.

[0140] S206: The traffic to be detected associated with the source IP address is regarded as normal traffic.

[0141] S207: Determine whether all the traffic to be detected associated with each source IP address has been detected. If so, jump to step 201; otherwise, jump to step 203.

[0142] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a structural diagram of an abnormal flow detection device. Figure 4 As shown, the abnormal flow detection device 300 may include:

[0143] The traffic collection unit 301 is configured to collect content distribution logs containing multiple traffic flows to be detected, and to group the traffic flows to be detected associated with the same source Internet Protocol (IP) address into a group to obtain at least one traffic flow group to be detected;

[0144] The traffic analysis unit 302 is used to perform the following operations for each traffic group to be detected: based on a set sliding time window, divide a traffic group to be detected into multiple continuous traffic sequences; perform feature extraction and preprocessing on each traffic sequence to obtain a feature sequence of a source IP address in each sliding time window, and a feature sequence includes the multi-dimensional traffic features and cache hit rate of the source IP address in a sliding time window; based on the correlation between the change trend of the multiple continuous feature sequences of the source IP address in multiple continuous sliding time windows and the traffic type, predict the probability of the source IP address launching a domain front attack in the next sliding time window, and when the predicted probability value exceeds the set threshold, treat the traffic to be detected associated with the source IP address as abnormal traffic.

[0145] Optionally, the traffic analysis unit 302 performs the following operations on a traffic sequence:

[0146] Performing time coding and request coding on a plurality of to-be-detected flows included in the one flow sequence to obtain a plurality of time features and request features of the one source IP address within the one time window;

[0147] Obtaining a cache hit rate of the source IP address within the sliding time window based on cache hit statuses of the multiple traffic flows to be detected and the total number of requests initiated by the source IP address;

[0148] Normalizing the multiple time features, the request features, and the cache hit rate to obtain a feature sequence of the source IP address in the sliding time window.

[0149] Optionally, the traffic analysis unit 302 is configured to:

[0150] Time-encoding the request timestamps of the plurality of to-be-detected flows in the flow sequence respectively to obtain a plurality of time features of the source IP address within the sliding time window;

[0151] Request encoding is performed on the multiple traffic flows to be detected to obtain a request feature of the source IP address within a sliding time window.

[0152] Optionally, the traffic analysis unit 302 is configured to:

[0153] According to the time coding rule, the request timestamps of the multiple traffic flows to be detected in the traffic sequence are respectively encoded to obtain their respective original time features;

[0154] According to the tag encoding rule, the request timestamps of multiple to-be-detected times in the traffic sequence are encoded respectively to obtain respective additional time features.

[0155] Optionally, the traffic analysis unit 302 is configured to:

[0156] Based on the to-be-detected traffic associated with the source IP address within the sliding time window, counting the total number of requests initiated by the source IP address within the sliding time window;

[0157] Based on the resource type of the requested resource of each to-be-detected traffic, counting the total number of resource types requested by the source IP address within the sliding time window;

[0158] Based on the response status codes of the traffic to be detected, counting the proportion of status codes of the source IP address within the sliding time window except for the response status codes indicating successful request processing;

[0159] The total number of requests, the total number of requested resource types, and the status code ratio of the source IP address within the sliding time window are used as request features of the source IP address within the sliding time window.

[0160] After introducing the abnormal traffic detection method and apparatus according to an exemplary embodiment of the present application, a computer device according to another exemplary embodiment of the present application is introduced next.

[0161] Those skilled in the art will appreciate that various aspects of the present application can be implemented as systems, methods, or program products. Therefore, various aspects of the present application can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0162] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a computer device, see Figure 4 As shown, the computer device 400 may include at least a processor 401 and a memory 402. The memory 402 stores program codes, which, when executed by the processor 401, enable the processor 401 to perform any of the steps of the above-mentioned abnormal traffic detection method.

[0163] In some possible implementations, the computing device according to the present application may include at least one processor and at least one memory. The memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the abnormal traffic detection method according to various exemplary embodiments of the present application described above in this specification. For example, the processor may execute the following steps: Figure 1A Follow the steps shown in .

[0164] Refer to the following Figure 5 hereinafter, a computing device 500 according to this embodiment of the present application is described. Figure 5 The computing device 500 is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0165] like Figure 5 As shown, computing device 500 is implemented as a general-purpose computing device. Components of computing device 500 may include, but are not limited to, at least one processing unit 501, at least one storage unit 502, and a bus 503 connecting various system components (including storage unit 502 and processing unit 501).

[0166] Bus 503 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a processor or local bus using any of a variety of bus architectures.

[0167] The storage unit 502 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 5021 and / or a cache memory unit 5022 , and may further include a read-only memory (ROM) 5023 .

[0168] The storage unit 502 may also include a program / utility 5025 having a set (at least one) of program modules 5024, such program modules 5024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0169] Computing device 500 may also communicate with one or more external devices 504 (e.g., a keyboard, pointing device, etc.), one or more devices that enable a user to interact with computing device 500, and / or any device that enables computing device 500 to communicate with one or more other computing devices (e.g., a router, modem, etc.). Such communication may occur via input / output (I / O) interface 505. Furthermore, computing device 500 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via network adapter 506. As shown, network adapter 506 communicates with other modules of computing device 500 via bus 503. It should be understood that, although not shown, other hardware and / or software modules may be used in conjunction with computing device 500, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0170] Based on the same inventive concept as the above-mentioned method embodiment, various aspects of the abnormal traffic detection method provided by the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the abnormal traffic detection method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device can execute the following steps: Figure 1A Follow the steps shown in .

[0171] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0172] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0173] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for detecting abnormal traffic, characterized in that: include: Collecting content distribution logs containing multiple to-be-detected flows, dividing the to-be-detected flows associated with the same source Internet Protocol IP address into a group, and obtaining at least one to-be-detected flow group; For each traffic group to be detected, the following operations are performed: based on a set sliding time window, a traffic group to be detected is divided into multiple continuous traffic sequences; each traffic sequence is subjected to feature extraction and preprocessing to obtain a feature sequence of a source IP address in each sliding time window. A feature sequence includes the multi-dimensional traffic features and cache hit rate of the source IP address in a sliding time window; Based on the correlation between the changing trends of multiple consecutive feature sequences of the source IP address in multiple consecutive sliding time windows and the traffic type, the probability of the source IP address launching a domain front attack in the next sliding time window is predicted, and when the predicted probability value exceeds the set threshold, the traffic to be detected associated with the source IP address is treated as abnormal traffic.

2. The method according to claim 1, wherein The feature extraction and preprocessing are performed on each traffic sequence to obtain a feature sequence of a source IP address in each sliding time window, wherein the following operations are performed for each traffic sequence: Performing time coding and request coding on a plurality of to-be-detected flows included in the one flow sequence to obtain a plurality of time features and request features of the one source IP address within the one sliding time window; Obtaining a cache hit rate of the source IP address within the sliding time window based on cache hit statuses of the multiple traffic flows to be detected and the total number of requests initiated by the source IP address; Normalizing the multiple time features, the request features, and the cache hit rate to obtain a feature sequence of the source IP address in the sliding time window.

3. The method according to claim 2, wherein The performing time coding and request coding on the multiple to-be-detected flows included in the one flow sequence to obtain multiple time features and request features of the one source IP address within the one sliding time window includes: Time-encoding the request timestamps of the plurality of to-be-detected flows in the flow sequence respectively to obtain a plurality of time features of the source IP address within the sliding time window; Request encoding is performed on the multiple traffic flows to be detected to obtain a request feature of the source IP address within a sliding time window.

4. The method according to claim 3, wherein The step of respectively performing time encoding on the request timestamps of the plurality of to-be-detected flows in the flow sequence to obtain the plurality of time features of the source IP address within the sliding time window includes: According to the time coding rule, the request timestamps of the multiple traffic flows to be detected in the traffic sequence are respectively encoded to obtain their respective original time features; According to the tag encoding rule, the request timestamps of multiple to-be-detected times in the traffic sequence are encoded respectively to obtain respective additional time features.

5. The method according to claim 3, wherein The step of performing request encoding on the plurality of traffic flows to be detected to obtain a request feature of the source IP address within a sliding time window includes: Based on the to-be-detected traffic associated with the source IP address within the sliding time window, counting the total number of requests initiated by the source IP address within the sliding time window; Based on the resource type of the requested resource of each to-be-detected traffic, counting the total number of resource types requested by the source IP address within the sliding time window; Based on the response status codes of the traffic to be detected, counting the proportion of status codes of the source IP address within the sliding time window except for the response status codes indicating successful request processing; The total number of requests, the total number of requested resource types, and the status code ratio of the source IP address within the sliding time window are used as request features of the source IP address within the sliding time window.

6. An abnormal flow detection device, characterized in that: include: a traffic collection unit, configured to collect content distribution logs containing a plurality of traffic flows to be detected, and to group the traffic flows to be detected associated with the same source Internet Protocol (IP) address into a group, thereby obtaining at least one traffic flow group to be detected; The traffic analysis unit is configured to perform the following operations for each traffic group to be detected: based on a set sliding time window, divide a traffic group to be detected into multiple continuous traffic sequences; perform feature extraction and preprocessing on each traffic sequence to obtain a feature sequence of a source IP address in each sliding time window, wherein a feature sequence includes multi-dimensional traffic features and cache hit rate of the source IP address in a sliding time window; Based on the correlation between the changing trends of multiple consecutive feature sequences of the source IP address in multiple consecutive sliding time windows and the traffic type, the probability of the source IP address launching a domain front attack in the next sliding time window is predicted, and when the predicted probability value exceeds the set threshold, the traffic to be detected associated with the source IP address is treated as abnormal traffic.

7. The device according to claim 6, characterized in that The traffic analysis unit performs the following operations on a traffic sequence: Performing time coding and request coding on a plurality of to-be-detected flows included in the one flow sequence to obtain a plurality of time features and request features of the one source IP address within the one sliding time window; Obtaining a cache hit rate of the source IP address within the sliding time window based on cache hit statuses of the multiple traffic flows to be detected and the total number of requests initiated by the source IP address; Normalizing the multiple time features, the request features, and the cache hit rate to obtain a feature sequence of the source IP address in the sliding time window.

8. The device according to claim 7, wherein The flow analysis unit is used for: Time-encoding the request timestamps of the plurality of to-be-detected flows in the flow sequence respectively to obtain a plurality of time features of the source IP address within the sliding time window; Request encoding is performed on the multiple traffic flows to be detected to obtain a request feature of the source IP address within a sliding time window.

9. The device according to claim 8, wherein The flow analysis unit is used for: According to the time coding rule, the request timestamps of the multiple traffic flows to be detected in the traffic sequence are respectively encoded to obtain their respective original time features; According to the tag encoding rule, the request timestamps of multiple to-be-detected times in the traffic sequence are encoded respectively to obtain respective additional time features.

10. The device according to claim 8, wherein The flow analysis unit is used for: Based on the to-be-detected traffic associated with the source IP address within the sliding time window, counting the total number of requests initiated by the source IP address within the sliding time window; Based on the resource type of the requested resource of each to-be-detected traffic, counting the total number of resource types requested by the source IP address within the sliding time window; Based on the response status codes of the traffic to be detected, counting the proportion of status codes of the source IP address within the sliding time window except for the response status codes indicating successful request processing; The total number of requests, the total number of requested resource types, and the status code ratio of the source IP address within the sliding time window are used as request features of the source IP address within the sliding time window.

11. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that The method comprises a program code, and when the program code is run on a computer device, the program code is used to enable the computer device to execute the steps of the method according to any one of claims 1 to 5.

Images