Vertical malicious crawler traffic identification method based on deep learning

Abstract

The invention discloses a vertical malicious crawler traffic identification method based on deep learning, and belongs to the technical field of the Internet. According to the invention, deep learningis used for flow identification of website access behaviors; identity recognition is carried out on flow with malicious crawler behaviors through classification of access behavior characteristics, and the method comprises three parts of encoding website access flow into a three-dimensional vector, carrying out recognition training through a three-dimensional convolutional neural network, and finally establishing a classification recognition model for malicious crawler flow through optimization of network parameters. The website access flow is classified and identified through deep learning, the malicious crawler flow identification accuracy of the website is improved, and the website can deploy the corresponding security policy according to the identification result so as to improve the website performance and reduce the website redundant load.

Description

technical field [0001] The invention belongs to the technical field of computer network security, and in particular relates to a method for identifying malicious crawler traffic based on deep learning. [0002] technical background [0003] Crawlers are one of the most widely used technologies on the Internet today. They have been used in many fields such as finance, trade, and information technology. The preliminary research and data collection of many tasks are completed by crawler programs, and the crawled content is cleaned and processed. , the obtained data is extremely valuable. [0004] It is worth noting that in order to obtain the largest amount of data in the shortest time, some crawlers will use multi-threading, high concurrency, and even distributed technologies, which will greatly increase the pressure on the server. We classify the traffic generated by such crawlers as malicious crawler traffic. This kind of traffic brings enormous pressure on the server. In or...

AI Technical Summary

Problems solved by technology

[0007] The purpose of the present invention is to solve the problem of accurately identifying malicious crawler traffic in website access traffic, and proposes a malicious crawler traffic identification method based on deep learning, which combines deep learning with automatic learning features in the model building process The advantage is that the hidden layer is used to automatically extract signs from the sample data set without manual selection. Compared with the traditional traffic identification method based on machine learning, this method not only has a higher accuracy rate but also reduces the original construction cost. Difficulty

Images