Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Crawler prevention system and method according to request data

A technology for requesting data and anti-crawling, applied in the field of data processing to achieve the effect of protecting interests

Pending Publication Date: 2020-06-05
珠海横琴电享科技有限公司
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The disadvantage is that malicious crawlers can use proxies and use different IPs for each request to bypass IP speed monitoring

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Crawler prevention system and method according to request data
  • Crawler prevention system and method according to request data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0020] The embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. Apparently, the described embodiments are only some of the embodiments of the present invention, not all of them.

[0021] In describing the present invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", " The orientation or position or sequence relationship indicated by "outer", "first", "second", etc. are based on the orientation or position or sequence relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or It should not be construed as limiting the invention by implying that a referenced device or element must have a particular orientation, be constructed, and operate in a particular orientation.

[0022] See attached figure 1 , the anti-crawler system according to th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a crawler prevention system and method according to request data. The system comprises a cookie encryption generation module, a cookie storage module and a comparative analysismodule; the cookie encryption generation module is used for generating cookie in an encryption manner according to a User-Agent, a current timestamp and a current IP address; the cookie storage module is used for storing cookie blacklist data; and the comparison and analysis module is used for comparing and analyzing the currently accessed cookie and cookie blacklist data so as to execute a reaction operation for the currently accessed cookie. According to the method and system, malicious crawlers can be effectively prevented from quickly acquiring a large amount of data by using proxy IPs, original data benefits of companies are protected, and the malicious crawlers are prevented from acquiring a large amount of enterprise original data and publishing the enterprise original data on other similar products; according to the method and system, the crawler can be recognized under the condition that the user does not log in, client login is not needed, the browsing intention of the client is reserved, and meanwhile the anti-crawler effect can be achieved.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to an anti-crawler system and method based on requested data. Background technique [0002] Now the original data in Internet products are easily obtained by other companies or individuals using malicious crawlers, which will damage the interests of the original products. This technology is used to solve the problem that most crawlers use proxy IPs to frequently access and obtain data. The existing closest technology related to the present invention is to identify malicious crawlers according to the user IP access frequency, and if the same IP has too many access times within a period of time, the IP will be added to the blacklist. The disadvantage is that malicious crawlers can use proxies and use different IPs for each request to bypass IP speed monitoring. Contents of the invention [0003] To address the above problems, the present invention provides an anti-crawler s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F21/56G06F16/951G06F21/60
CPCG06F21/56G06F16/951G06F21/60
Inventor 郑磊
Owner 珠海横琴电享科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com