A crawler method for solving reverse crawling of fonts

Abstract

The invention discloses a crawler method for solving the reverse crawling of fonts, which comprises the following steps: obtaining a self-defined graphic metadata database file of a data source website; marking a standard unique identification number on the graph metadata in the user-defined graph metadata database file; establishing the mapping relation table between the true character and the unique identification number of metadata standard; Establishing a mapping relationship table between pseudo words and a unique identification number of a graphic metadata standard; Establishing mappingtable between true and false characters; Inversely analizing. The invention has the advantages of high persistence, strong universality and the like in solving the reverse craweling of fonts, realizesthe flexible acquisition and update of font files, flexibly establishes the mapping relationship between true and false words, flexibly reversely analyzes false data, and guarantees the maximum dataaccuracy.

Description

technical field [0001] The invention relates to the field of network technology, in particular to a crawler method for solving font anti-crawling. Background technique [0002] The core problem to be solved in the crawler development process is to break through the anti-crawling technology of data sources (mainly websites). Common anti-crawling technologies include IP access restrictions, user login verification, dynamic loading of front-end data, etc. These anti-crawling technologies The technology has been around for a long time, and there are many current solutions. However, font anti-crawling, as a small anti-crawling technology before, has gradually become popular in large data source websites. This technology will cause the data obtained by crawlers to be lost. value, and there is no stable and reliable solution yet. [0003] Commonly used font files include files conforming to ttf, woff and other protocol specifications. Most of them are composed of a series of ASCII...

AI Technical Summary

Problems solved by technology

[0009] The most important thing is that if the data source website simply generates a custom font file, then by manually establishing a mapping between true and false characters, and then using it in the program to replace false characters, this method can directly solve font anti-climbing However, at present, some websites have adopted custom font generation that changes with the IP, and each IP custom font is regenerated multiple times a day, which is equivalent to the previously mentioned correspondence between A and B. It may be updated at any time and become A Corresponding to C, the establishment of a mapping relationship between A and B can only support the same IP to de-analyze fake data in a short period of time, and then cause data errors, which will be the most difficult problem

Images