Method and system for identifying and intercepting crawlers based on business data

A technology of business data and interception system, applied in the field of information security, can solve the problem of unable to stop the continuous acquisition of high-simulation crawlers, and achieve the effect of reducing the possibility of stealing the company's commercial pricing system

Active Publication Date: 2020-08-21
CTRIP TRAVEL NETWORK TECH SHANGHAI0
View PDF7 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The technical problem to be solved by the present invention is to provide a method and system for identifying and intercepting reptiles based on business data in order to overcome the defect that high-simulation reptiles cannot stop obtaining the core price data of online travel websites in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for identifying and intercepting crawlers based on business data
  • Method and system for identifying and intercepting crawlers based on business data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0035] The present invention is further illustrated below by means of examples, but the present invention is not limited to the scope of the examples.

[0036] like figure 1 As shown, the crawler identification and interception method based on business data of the present invention comprises the following steps:

[0037] Step 101, obtain the business data of the whole website in real time;

[0038] Among them, the specific business data can include login data, registration data, etc. Specifically, relevant business data can be obtained by burying the data on the front-end page of the website, and then perform structured data cleaning to extract relevant fields;

[0039] Step 102, obtain suspicious crawler data from business data, and determine the risk level of suspicious crawler data by aggregating crawler data in login information, registration frequency information, IP address information, device information, and access URL information

[0040] Step 103, establishing an I...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a crawler identification and interception method and system based on service data. The crawler identification and interception method and system based on service data comprises the following steps: S1, obtaining the service data of a whole website in real time; S2, obtaining suspicious crawler data from the service data; S3, establishing an IP white list information library, wherein effective IP addresses allowed to access the website are recorded in the IP white list information library; S4, judging whether the IP address of the suspicious crawler data is in the IP white list information library, and if not, executing S5; S5, sending the IP address of the suspicious crawler data to an anti-crawler database; S6, synchronously receiving the IP address of the suspicious crawler data by the anti-crawler database; and S7, judging whether the IP address of an online access crawler is in the anti-crawler database, and if so, intercepting the IP address of the online access crawler. By adoption of the crawler identification and interception method and system disclosed by the invention, the IP address of the crawler can be intercepted in real time, thereby reducing the possibility that commercial pricing systems of companies are stolen by the crawler on the service.

Description

technical field [0001] The invention relates to the technical field of information security, in particular to a method and system for identifying and intercepting crawlers based on business data. Background technique [0002] With the continuous development of Internet tourism, industry competition is inevitable. Here, the prices of various electronic travel platforms have become important data that every competitor wants to know. The existing technology usually uses cookies ( Website data), referer (website data), sessions (website data), and detect the openness of IP (interconnection protocol between networks) ports to confirm the crawler properties of the other party, and currently some online travel websites use pictures and pictures for digital prices Random numbers instead, but now that digital technology is becoming more and more developed, this still cannot stop high-simulation crawlers from continuously obtaining the core price data of the website. Contents of the...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/06
CPCH04L63/0236H04L63/105H04L63/205
Inventor 闵杰王润辉凌云任华炯
Owner CTRIP TRAVEL NETWORK TECH SHANGHAI0
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap