Malicious web crawler recognition method and device

A malicious and crawler technology, applied in the Internet field, can solve problems such as poor accuracy of malicious crawlers on the Internet

Active Publication Date: 2015-03-04
BEIJING GRIDSUM TECH CO LTD
View PDF4 Cites 42 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The main purpose of the present invention is to provide a method and device for identifying malicious network reptiles to solve the problem of poor accuracy when identifying malicious network crawlers

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Malicious web crawler recognition method and device
  • Malicious web crawler recognition method and device
  • Malicious web crawler recognition method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other if there is no conflict. Hereinafter, the present invention will be described in detail with reference to the drawings and in conjunction with the embodiments.

[0024] In order to enable those skilled in the art to better understand the solution of the application, the technical solutions in the embodiments of the application will be clearly and completely described below in conjunction with the drawings in the embodiments of the application. Obviously, the described embodiments are only It is a part of the embodiments of this application, not all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the protection scope of this application.

[0025] It should be noted that the terms "first" and "second" in the description and cla...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a malicious web crawler recognition method and device. The method includes: acquiring to-be-detected web addresses; acquiring user access information corresponding to the to-be-detected web addresses; calculating target access rate according to the number of the to-be-detected web addresses with the corresponding user access information including target web terminal information and frequency of access to a target website through the to-be-detected web addresses within a preset time period; judging whether the target access rate is higher than a preset rate threshold or not; if yes, determining that the behaviors of access to the target website through the to-be-detected web addresses are malicious crawler access behaviors. Through the method and device, the problem of poor accuracy during recognition of malicious web crawlers is solved, further whether the behaviors of access to the target website through the to-be-detected web addresses are the malicious crawler access behaviors or not is determined under the condition that the target access rate is higher than the preset rate threshold, and accuracy in malicious web crawler recognition is improved.

Description

Technical field [0001] The present invention relates to the Internet field, in particular, to a method and device for identifying malicious network crawlers. Background technique [0002] A web crawler is a program that automatically obtains web content. For a website, a large number of malicious crawler requests will consume server performance, waste a lot of resources, and even cause server downtime. Therefore, it is necessary to ensure that users normally visit the website and avoid large-scale malicious crawlers from initiating visits to the website. [0003] The existing method of identifying malicious crawlers is to parse the log of the website server to find the network address that frequently visits the website from the log, filter the network address, and prohibit the network address from visiting the website again. But this method has a relatively high rate of manslaughter. Because a company or building usually has only one public network address, the network address r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/9566
Inventor 崔维福范浩文
Owner BEIJING GRIDSUM TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap