Token acquisition method, data acquisition system, proxy server and storage medium

Abstract

The embodiment of the invention relates to the technical field of data acquisition, and discloses a token acquisition method, a data acquisition system, a proxy server and a storage medium. The methodis applied to a proxy server, a client communicates with at least one target server through the proxy server, the target server generates a token when the client requests service from the target server, the token is returned to the client through the proxy server, and the method comprises the steps that the token returned to the client by the target server is obtained through analysis; and sending the token to a crawler server, so that the crawler server crawls data from the corresponding target server according to the token. According to the embodiment of the invention, the token is acquiredthrough the proxy server, so that the token acquisition difficulty is greatly reduced, and data acquisition can be conveniently and effectively realized.

Description

technical field [0001] The invention relates to the technical field of data collection, in particular to a token acquisition method, a data collection system, a proxy server and a storage medium. Background technique [0002] Token (token) is a key used to identify the identity of the user to realize the communication between the customer service end and the server end. It is a string of strings generated by the server end as an identification of the client requesting service. When the user logs in for the first time, the server generates a token and returns the token to the client. In the future, the client only needs to carry the token to request data without carrying the user name and password again, thus playing a role of security and confidentiality. [0003] Crawler service is an important data collection method. With the advent of the era of big data, the application of crawler service will become more and more extensive, and it can provide various data intelligence f...

AI Technical Summary

Problems solved by technology

[0004] The inventor found that there are at least the following problems in related technologies: some special tokens cannot be obtained through simulated login with account passwords. It is not public, it is difficult to understand its encryption algorithm, and encryption generally requires the use of a key, so it is very difficult to generate the correct token, which greatly increases the difficulty of data crawling

Images