Method and device for segmentation on basis of webpage content classification

Abstract

The embodiment of the invention provides a method and a device for segmentation on the basis of webpage content classification. The method comprises the following steps of: extracting the text information of webpage contents in search resources; dividing the classes of the text information according to the classes of the webpage contents; segmenting the text information according to segmentation dictionaries corresponding to the classes of the text information. According to the embodiment of the invention, the classes of the text information of the webpage contents in the search resources are divided, and the text information is segmented on the basis of the segmentation dictionaries corresponding to the classes, so as to adapt to different classes of language characteristics better, meanwhile, the segmentation accuracy for different classes is also improved, and the optimal processing for local segmentation is realized; moreover, the improvement of the accuracy of segmentation is close to the intention of a user and improve the user experience, and then reduce the operations of re-input, search and the like of the user, and improve the simplicity of operation, meanwhile, the response of equipment on the operation of the user is reduced, and the consumption of the system resources of the equipment is reduced.

Description

Technical field [0001] The invention relates to the technical field of search, in particular to a method for word segmentation processing based on web content categories and a device for word segmentation processing based on web content categories. Background technique [0002] With the rapid development of the Internet, network applications tend to be diversified, and the amount of information on the Internet has increased dramatically. [0003] In various occasions, users often need to input key information to obtain related information. For example, enter keywords in a search engine to search for web information, enter keywords in a forum to search for posts, and so on. [0004] Word segmentation is the basis for information processing and information retrieval. All information processing and information retrieval work are performed after word segmentation. Therefore, the segmentation error will be added to the subsequent processing, and it is difficult to eliminate. Because of...

AI Technical Summary

Problems solved by technology

This method only needs to count the frequency of word groups in the corpus, but this method also has certain limitations. It will often extract some common word groups with high co-occurrence frequency but not words, such as "this" , "one", "some", "my", "many", etc., and the recognition accuracy of common words is poor, and the space-time overhead is large

[0006] On the one hand, the result of word segmentation is wrong, so that the related information obtained later is very different from the original expectation, the user experience is very poor, and the device system resources are wasted

On the other hand, users need to obtain relevant information, and they will input key information again to search, and the device needs to search, compare, and screen massive amounts of information again to obtain information related to search keywords, which not only makes user operations more cumbersome, but also consumes user time. , and will greatly increase the burden on the equipment and consume more equipment resources

Images