Anti-crawler key information protection method

Abstract

The invention discloses an anti-crawler key information protection method, and the method mainly comprises the following steps: dividing website display information into common information and key information, and generating a QR code containing a key information ID and an anti-crawler module URL entry address; when a user needs to inquire key information, enabling the user to scan or press the QRcode for a long time for recognition through a handheld terminal, enabling the anti-crawler module to collect a user ID for judgment and analysis: if the user ID is judged to be an abnormal user, blocking the subsequent access of the user ID; if the user is judged to be the normal user, associating the key information corresponding to the key information ID and automatically displaying the key information to the user at the handheld terminal. It can be seen from the steps that the website is protected at low cost and the key information is protected against illegal crawling simply and quickly, and meanwhile, the user experience can be greatly improved.

Description

technical field [0001] The invention relates to anti-crawler technology in Internet information security, in particular to a method for anti-crawler protection of key information. Background technique [0002] The latest Internet security company Imperva analyzed the website visits of 100,000 domain names around the world, saying that about 52% of Internet traffic worldwide comes from "robots". Since Imperva monitored network traffic in 2012, the network traffic generated by machines has surpassed the traffic generated by humans for five consecutive years. Only in 2015, the traffic generated by humans surpassed once by a slight advantage, but the data was quickly countered. That is to say, most of the existing website visitors are not human beings, but the execution of automated programs, which are referred to as crawlers in the industry. [0003] According to research statistics, about 20% of global Internet requests are generated by malicious crawlers. The proportion of ...

AI Technical Summary

Problems solved by technology

However, most crawlers will fake these UAs, so this method will often fail

[0015] 4. Limited Data Access

However, many websites are not applicable. When users need full data access, the experience of the website is not so good.

[0017] 5. Access frequency limitation

However, the possible execution efficiency of this method is low, which will reduce the execution efficiency of the overall website

[0035] By summarizing the above-mentioned existing mainstream anti-crawler technologies, we can see its disadvantages: first, anti-crawling is contradictory to user experience, anti-crawling technology makes crawlers more difficult to crawl, often resulting in poorer user experience; second, anti-crawling The crawling technology or strategy is not universal, and various existing strategies or technologies are superimposed and used. Personalization can enhance the anti-crawling effect, but it greatly increases the workload of website development and maintenance, and increases the cost; the third is that crawlers are combined with image recognition, etc. New technology, it is easy to bypass the anti-climbing strategy, making the anti-climbing technology invalid

Therefore, generally speaking, there is no technology with high anti-climbing strategy, low cost and good user experience in the current technology