A uniform resource locator selecting method and a uniform resource locator selecting device

Abstract

The invention provides a uniform resource locator selecting method and a uniform resource locator selecting device. The method comprises the steps of according to one input keyword, acquiring at least one uniform resource locator related to the keyword; according the parameter information of each uniform resource locator in the at least one uniform resource locator and / or the weight ratio of the domain names used by the at least one uniform resource locator, statistically analyzing the correlation degree of each uniform resource locator and the keyword; according to the correlation degree of each uniform resource locator and the keyword, selecting a target uniform resource locator from the at least one uniform resource locator to use the target uniform resource locator as the search result of the uniform resource locator correlated with the keyword. According to the technical solution, the coverage rate and accuracy rate of acquiring uniform resource locators are effectively guaranteed, so that web page content with higher quality can be acquired according to the uniform resource locators to meet the requirement of public opinion analysis.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method for selecting a uniform resource locator and a device for selecting a uniform resource locator. Background technique [0002] At present, with the rapid development of the Internet, the amount of data in the Internet is increasing day by day, and the number of existing web pages has reached hundreds of billions. The rapid growth of data volume also puts forward higher requirements for the data acquisition ability of public opinion analysis. In related technologies, web crawler technology is usually used to obtain a Uniform Resource Locator (URL, Uniform Resource Locator) so as to obtain webpage content according to the Uniform Resource Locator. Oriented crawler technology. Targeted crawling technology, that is, only crawls certain specific uniform resource locators, which leads to targeted crawling technology that only obtains a certain range of uniform resource loca...

AI Technical Summary

Problems solved by technology

Targeted crawling technology, that is, only crawls certain specific uniform resource locators, which leads to targeted crawling technology that only obtains a certain range of uniform resource locators, resulting in the content of web pages obtained according to uniform resource locators. low coverage

The most typical non-directional crawler technology is the web crawler of the search engine, which basically traverses the entire Internet according to the depth or breadth of the search method. There will inevitably be various garbage, pornographic data, and unified resources that are useless for specific scenarios. Locator, the most important thing is that the accuracy of the webpage content obtained based on the uniform resource locator obtained by the non-directional crawler technology is not high, and many webpage contents are not required

Images