Domain-oriented network information search method

Abstract

The invention provides a domain-oriented network information search method. By the domain-oriented network information search method, data information can be unified on the same platform, information of multiple data sources can be searched, and various formats of data including structured, semi-structured and unstructured data are supported. The domain-oriented network information search method which is a network information acquisition method includes steps of enabling domain experts to specify domain website sets, submitting keywords according to domain characteristics and creating a domain keyword bank; compiling information acquisition strategies according to link and content analysis and then acquiring target web pages in domain websites; and extracting, filtering and classifying acquired information of the web pages, creating a database and storing the information according to an inverted index. The domain-oriented network information search method is specifically implemented by a web page acquisition spider module, a classifier training module and a data index module. The domain-oriented network information search method is high in adaptability and topic relevance, and a vertical search engine using the method as a core is high in recall ratio and precision ratio.

Description

technical field [0001] The invention is a field-oriented network information search method, and relates to relevant technologies such as the improvement of theme crawler collection strategies, webpage content extraction and classification, and the like. Background technique [0002] With the rapid growth of webpage information, the total number of webpages has exceeded 3.5 billion at present, and it is increasing at a rate of one million every day. This will cause the timeliness of webpage information indexed by general search engines to be poor, and it is difficult to meet the needs of different professional users. The rapid development of the Internet poses a huge challenge to the search of WEB information. So the vertical search engine facing the field came into being. [0003] The search engine based on topic web crawler (that is, the fourth generation search engine) has become a hot research direction of current search engine. Vertical search engines focus on a specif...

AI Technical Summary

Problems solved by technology

[0002] With the rapid growth of webpage information, the total number of webpages has exceeded 3.5 billion at present, and it is increasing at a rate of one million every day. This will cause the timeliness of webpage information indexed by general search engines to be poor, and it is difficult to meet the needs of different professional users. The rapid development of the Internet poses a huge challenge to the search of WEB information

Images