Method and device for detecting attack of automatic network data acquirer

Abstract

One or more embodiments of the invention provide a network data automatic acquirer attack detection method and device, the method is applied to a server, and the server maintains an alarm threshold and a truncation threshold corresponding to a feature distance under at least two feature dimensions. Comprising the following steps: acquiring a first traffic sample set received by a server in a first time period and a second traffic sample set received by the server in a second time period; calculating at least two feature distances of the first traffic sample set and the second traffic sample set under at least two feature dimensions; under the condition that any one of the at least two feature distances exceeds the truncation threshold value corresponding to any one feature distance, re-determining the numerical value of any one feature distance as the truncation threshold value corresponding to any one feature distance; and determining a traffic anomaly index based on the at least two re-determined feature distances, and determining that the network data automatic acquirer attack exists in the first time period or the second time period under the condition that the traffic anomaly index exceeds an alarm threshold.

Description

technical field [0001] One or more embodiments of this specification relate to the field of Internet technologies, and in particular, to a method and apparatus for detecting an attack by an automatic network data acquirer. Background technique [0002] An automatic network data acquirer, also known as a web crawler, is a program or script that automatically captures Internet information according to certain rules. However, the proliferation of web crawlers will constitute a crawler attack, causing the server to bear a large number of access requests, depleting server resources, and hindering the access of normal users. Therefore, how to detect crawler attacks is a technical problem that needs to be solved urgently. [0003] On the one hand, the detection of traditional crawler attacks focuses on the characteristics of access traffic at the packet level or session level, such as detecting the access frequency of IP addresses, detecting the format of request headers, detecting...

AI Technical Summary

Problems solved by technology

However, the proliferation of web crawlers will constitute a crawler attack, causing the server to bear a large number of access requests, depleting server resources, and hindering normal user access. Therefore, how to detect crawler attacks is a technical problem that needs to be solved urgently

[0003] On the one hand, the detection of traditional crawler attacks focuses on the characteristics of access traffic at the packet level or session level, such as detecting the access frequency of IP addresses, detecting the format of request headers, detecting cookies (data stored on the user's local terminal), etc. However, these are rule detection methods that first summarize the static characteristics of crawler traffic, and then determine whether the specific traffic received conforms to these static characteristics. Although it is possible to identify whether a crawler attack has been encountered from the perspective of rules, due to the lack of an overall perspective and The static characteristics of crawler traffic are difficult to exhaust, so it is easy to cause missed detection

On the other hand, although it is also possible to detect whether a crawler attack may have been encountered by detecting traffic fluctuations, the traditional method of detecting traffic fluctuations cannot distinguish whether the traffic fluctuations are caused by crawler attacks or normal access, so the traffic fluctuations are simply The phenomenon is attributed to a crawler attack, which will easily lead to false detection

Images