Webpage classification dictionary generation method and apparatus

Abstract

Embodiments of the invention disclose a webpage classification dictionary generation method and apparatus. The method comprises the steps of determining a sample uniform resource locator (URL) corresponding to a webpage classification sample of each category according to a predetermined webpage classification standard, and obtaining sample webpage contents corresponding to each sample URL; extracting sample text information in each piece of the sample webpage contents, performing word segmentation processing on the sample text information, and obtaining a corresponding sample word from the sample text information; and screening out a reverse word frequency value corresponding to the sample word from corresponding relationships between a plurality of pre-stored learning words and the reverse word frequency value, wherein the reverse word frequency value is a value determined according to the occurrence frequency of each learning word in corresponding learning text information; and storing the sample word and a weight value determined according to the corresponding reverse word frequency value in a webpage classification dictionary. Therefore, the webpage classification dictionary with higher accuracy is generated.

Description

technical field [0001] The invention relates to the technical field of Internet search, in particular to a method and device for generating a web page classification dictionary. Background technique [0002] The number of websites in the Internet is extremely large, and there are various types of websites, such as news, sports, shopping and so on. Faced with various websites, enterprises or organizations hope that internal staff only visit work-related websites. How to filter the websites that internal staff can access is a very urgent and important requirement. Then at this time, it is necessary to classify each website, and filter the website according to the category to which the website belongs, so as to filter out websites that are not allowed to be accessed. [0003] In the face of a large number of websites on the Internet, URLs (Uniform Resource Locators, Uniform Resource Locators) corresponding to the websites can be classified. When setting reasonable classificati...

AI Technical Summary

Problems solved by technology

[0004] In the prior art, when evaluating the importance of a sample word in a sample file, only the number of files in the sample file where the sample word is located is considered, and the accuracy of the determined importance of the sample word in the sample file (ie, the corresponding weight value) is not high. , resulting in low accuracy of the generated web page classification dictionary

Images