Method and system for identifying and intercepting crawlers based on business data

Abstract

The invention discloses a crawler identification and interception method and system based on service data. The crawler identification and interception method and system based on service data comprises the following steps: S1, obtaining the service data of a whole website in real time; S2, obtaining suspicious crawler data from the service data; S3, establishing an IP white list information library, wherein effective IP addresses allowed to access the website are recorded in the IP white list information library; S4, judging whether the IP address of the suspicious crawler data is in the IP white list information library, and if not, executing S5; S5, sending the IP address of the suspicious crawler data to an anti-crawler database; S6, synchronously receiving the IP address of the suspicious crawler data by the anti-crawler database; and S7, judging whether the IP address of an online access crawler is in the anti-crawler database, and if so, intercepting the IP address of the online access crawler. By adoption of the crawler identification and interception method and system disclosed by the invention, the IP address of the crawler can be intercepted in real time, thereby reducing the possibility that commercial pricing systems of companies are stolen by the crawler on the service.

Description

technical field [0001] The invention relates to the technical field of information security, in particular to a method and system for identifying and intercepting crawlers based on business data. Background technique [0002] With the continuous development of Internet tourism, industry competition is inevitable. Here, the prices of various electronic travel platforms have become important data that every competitor wants to know. The existing technology usually uses cookies ( Website data), referer (website data), sessions (website data), and detect the openness of IP (interconnection protocol between networks) ports to confirm the crawler properties of the other party, and currently some online travel websites use pictures and pictures for digital prices Random numbers instead, but now that digital technology is becoming more and more developed, this still cannot stop high-simulation crawlers from continuously obtaining the core price data of the website. Contents of the...

AI Technical Summary

Problems solved by technology

[0003] The technical problem to be solved by the present invention is to provide a method and system for identifying and intercepting reptiles based on business data in order to overcome the defect that high-simulation reptiles cannot stop obtaining the core price data of online travel websites in the prior art

Images