Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method, server, client and system for preventing crawler crawling

A server-side, anti-climbing technology, applied in the field of data security, can solve the problems of shielding effect error, missed killing, manslaughter, etc., and achieve the effect of raising the threshold, preventing direct grabbing, and reducing risks

Active Publication Date: 2018-09-28
BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD +1
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, request shielding can only identify regular web crawlers (that is, crawlers that provide user agent information), and cannot be identified for many crawlers that simulate manual access
In addition, identification based on IP and other information may lead to accidental killing or missed killing, and there will be large errors in the shielding effect.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, server, client and system for preventing crawler crawling
  • Method, server, client and system for preventing crawler crawling
  • Method, server, client and system for preventing crawler crawling

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0070] Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of example embodiments to those skilled in the art. The drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted.

[0071] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of embodiments of the present disclosure. However, those skilled in ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses anti-crawling methods, a server, a client and a system. The method comprises the steps of segmenting original data so as to obtain a plurality of segmented data blocks, and storing position or value information of each segmented data block in a key / value form; randomly screening a plurality of positions according to the position or value information of the segmented data blocks, and recording screened key / value corresponding to the plurality of positions; processing the screened segmented data blocks of the plurality of positions so as to obtain confusing data; and splicing the plurality of recorded key / value so as to obtain a character string. According to the method, the original data is segmented, filled and spliced, the data is sent in a special format, and for the segmented data, the client covers the confusing data with a cascading style sheet (CSS) float layer, so that the user can see the original data. Through adoption of the method, crawling threshold is improved, the risk of important information acquisition is lowered, and the important information is prevented from being obtained directly or recognized by OCR.

Description

technical field [0001] The present disclosure generally relates to the technical field of data security, and in particular, relates to an anti-crawler crawling method, server, client and system. Background technique [0002] A web crawler (crawler for short) is a program for obtaining webpage content, and the crawler searches for webpages through link addresses of webpages. At present, the crawler technology is very mature. Through the set rules, the crawler can easily capture some important information in the source code of the page, such as product price, merchant phone number, product rating or key parameters of the product, etc. [0003] At present, there are generally two methods to prevent crawlers from crawling: image processing of important information and request blocking. Image processing is to replace the important information displayed in plain text in the source code with images for display, but image processing can only block the crawling of ordinary crawlers ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/06
CPCH04L63/1441H04L63/145
Inventor 吴凯王海旭
Owner BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com