Anti-crawler method for protecting website resource

A resource and website technology, applied in the Internet field, can solve the problems of crawler access efficiency, random access efficiency, etc., and achieve the effect of ensuring uniqueness and good anti-crawling effect

Inactive Publication Date: 2018-08-21
CHENGDU REMARK TECH CO LTD +1
View PDF5 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0011] Therefore, to develop an anti-crawling method that can effectively make up for the shortcomings of the existing anti-crawling technology, and make the access efficiency of crawlers lower than that of people’s random access through encryption, so as to ensure the uniqueness of data from another dimension is It is very necessary, and the invention also has important application prospects

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Anti-crawler method for protecting website resource

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0032] The present invention will be specifically introduced below in conjunction with the accompanying drawings and specific embodiments.

[0033] refer to figure 1 , the anti-climbing method of protecting website resource of the present invention, comprises the following steps:

[0034] S1. When the user first Http requests to access data, the website server returns a series of resource files, and then executes step S2; the resource files contain certain redundant data, making it difficult for crawlers to identify where valid codes are.

[0035] S2. For the second Http request to apply for a Token for accessing data, the website server returns the encrypted Token, and step S3 is executed;

[0036] S3, the user uses the code contained in the resource file returned by the website server in the step S1 to decrypt the Token returned by the website server in the step S2 and set the Token in the cookie, and then execute step S4;

[0037] S4. Use the decrypted Token to generate c...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an anti-crawler method for protecting a website resource. The method comprises the steps of when a first Http request of a user accesses a material, enabling a website server to return a series of resource files; when a second Http request applies to access a Token of the material, enabling the website server to return an encrypted Token; enabling the user to decrypt the Token by using a code included in the resource files returned by the website server and set the token into a Cookie; generating a code access material for sending a third Http request by using a decrypted Token; and enabling the website server to perform validity check on the Token and after the validity check is passed, return effective data of the access material to the user. According to the anti-crawler method, through an encryption manner, the access efficiency of a crawler is lower than random access efficiency of a person and the crawler is forced to abandon a capture behavior; and thus,the uniqueness of the data is guaranteed from another dimensionality; and a better anti-crawler effect can be achieved in combination with other solutions.

Description

technical field [0001] The invention relates to an anti-climbing method, in particular to an anti-climbing method for protecting website resources, and belongs to the technical field of the Internet. Background technique [0002] In order to ensure the uniqueness of the data on the website, webmasters do not want their data to be crawled by search engine crawlers, nor do they want their data to be grabbed by competitor crawlers for competitive product analysis. Therefore, I hope to have a technology that will not affect ordinary customers' access to the website, but also prevent my website resources from being seized by competitors on a large scale. [0003] The current common anti-crawler technologies are as follows: [0004] 1. Detect the access frequency of the IP address; [0005] 2. Detect whether user behavior is cyclical; [0006] 3. Detect whether the access request contains a legal browser header. [0007] These methods can deal with common crawler behaviors, bu...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/06H04L9/32H04L29/08
CPCH04L9/3213H04L9/3228H04L63/0428H04L63/1441H04L67/02
Inventor 吴杨
Owner CHENGDU REMARK TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap