Chinese label extraction method for clustering search results of search engine

Abstract

The invention discloses a Chinese label extraction method for clustering search results of a search engine, which comprises the following steps of: S1, inputting search words by a user to form an input document; S2, selecting candidate words, and scoring all the candidate words; S3, judging whether unmarked candidate words exist, if not existing, skipping to a step S8; if existing, selecting a candidate word with highest score, expanding the selected candidate word into a set of ordered word sequences containing the word, and entering a step S4; S4, calculating the frequency of each ordered word sequence, and extracting the high-frequency word sequence; S5, scoring the high-frequency word sequence, and selecting a candidate word sequence; S6, judging whether the candidate word sequence is accepted as a label, if so, entering a step S7, otherwise, returning to the step S3; S7, performing clustering according to the generated label; and S8, completing the operation. The method can reduce noise labels, and the labels have better representativeness, simplicity and integrity.

Description

technical field [0001] The invention relates to the technical field of search engine-based search result clustering, in particular to a Chinese label extraction method for search engine search result clustering. Background technique [0002] Clustering the results returned by search engines is an important means to improve the service quality of search engines. It divides the webpages of the same subtopic into the same category, and describes each category with a label as a summary of the topic, which is convenient for users to quickly locate the webpage of the topic they are interested in. The research on clustering of search engine results is a hot and difficult point in modern search engine research. [0003] At present, the label generation methods for clustering the retrieval results of search engines can be divided into two categories: one is the method of clustering first and then extracting labels; the other is the method of extracting labels first and then dividing...

AI Technical Summary

Problems solved by technology

[0007] (1) Label noise problem: The clustering of retrieval results is generally based on titles and abstracts. However, titles and abstracts contain a large number of words that are not related to document content and topics, thus introducing a lot of noise in the process of label extraction.

However, the existing noise filtering technology mainly uses some simple methods such as removing html tags, removing meaningless symbols, removing stop words, etc., which cannot solve the noise problem well.

[0008] (2) The label does not have a good representativeness of the topic, and it is difficult to meet the query needs of users

[0009] (3) The problem of "incomplete" and too long labels

[0010] (4) The proportion of the number of clustered documents is too low

Since some clustering label extraction methods (such as latent semantic indexing methods) usually have a large computational cost, in order to ensure real-time online response, general search engine systems can only select the first few hundred web pages for clustering, so that the clustered has too few documents to more fully reflect the nature of the search results

Images