Internet mass data retrieval method and retrieval system

Abstract

The invention discloses an internet mass data retrieval method and system. The internet mass data retrieval system comprises a server and retrieval modules in communication connection with the server. The server is in communication connection with the user terminal; the method comprises the following steps: acquiring a retrieval condition input through a user terminal, processing according to the retrieval condition to obtain each piece of screening information, and sending a retrieval formula established according to each piece of screening information to a server; respectively substituting each piece of screening information and the corresponding retrieval formula into a screening result prediction model to determine the number of retrieval modules; according to the number of the retrieval modules, the retrieval tasks corresponding to the retrieval conditions are distributed to the retrieval modules; uploading the screening result of each retrieval module to a server; the screening results uploaded by all the retrieval modules are processed in the server according to the retrieval formula, and the retrieval results are issued round by round according to the importance of the screening results. According to the technical scheme, the task load of the retriever can be reduced.

Description

technical field [0001] The invention relates to the technical field of data retrieval, in particular to an Internet massive data retrieval method and an Internet massive data retrieval system. Background technique [0002] Internet data retrieval is an effective way of information collection. In the prior art, the basic working process of Internet data retrieval generally includes: discovering and collecting web page information in the Internet; According to the query keyword entered by the user, the server quickly checks out the document in the index library, and returns the query result to the user. [0003] Usually, one retrieval task is processed by one retriever. However, the retrieval workload corresponding to each retrieval task is not the same. Especially when the retrieval workload is heavy, one retriever performs task processing. The server brings heavy retrieval tasks, resulting in low retrieval efficiency, and may also cause errors in retrieval results due to ex...

AI Technical Summary

Problems solved by technology

[0003] Usually, one retrieval task is processed by one retriever. However, the retrieval workload corresponding to each retrieval task is not the same. Especially when the retrieval workload is heavy, one retriever performs task processing. The server brings heavy retrieval tasks, resulting in low retrieval efficiency, and may also cause errors in retrieval results due to excessive workload

Images