Multiple-document automatic abstracting method based on frequent itemset

Abstract

The invention discloses a multiple-document automatic abstracting method based on a frequent itemset. In the method, a frequent itemset excavating ideal in the association rules is introduced, and an associating method is utilized to excavate the frequent itemsets of effective itemset to serve as child themes; sentences are directly clustered to different child themes without carrying out sentence similarity computing; and multiple-document automatic abstracting is carried out on the basis of an SFI (sub-topics based on frequent item sets) method. In the method, the sentences are directly clustered to different child themes without carrying out sentence similarity computing, thus the method has the characteristics of high simplicity, high legibility, high practicability and the like.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a method for automatic summarization of multiple documents based on frequent itemsets. Background technique [0002] With the continuous development of the global information highway, especially the continuous popularization of Internet applications, a large number of electronic documents emerge every day, and are transmitted and exchanged on the Internet. Multi-document summarization is a text collection compression technology in which repeated information in multiple document collections under the same topic appears in the abstract at one time, and other topic-related information is sequentially extracted according to importance and compression ratio. At present, foreign research on multi-document summarization mainly focuses on the processing of English information. Commonly used multi-document summarization methods such as: based on the method of single-document summa...

AI Technical Summary

Problems solved by technology

However, the current semantic annotation theory is immature, so that relying solely on the semantic information of words cannot truly reflect the meaning expressed by the sentence, and the accuracy of the calculated sentence similarity is also questionable

[0006] The method of combining word form and word order: based on word form matching between words, semantic information is not considered, and the different influences of words of different parts of speech on sentences are not distinguished, and sentences with similar semantics often have lower similarity results unreasonable phenomenon

[0007] Dependency tree method: Sui Zhifang et al. proposed a sentence similarity calculation model based on the skeleton dependency tree based on the analysis of the syntactic structure of the sentence. Although it is a good calculation model in theory, its practical application is not strong.

[0008] Edit distance method: Che Wanxiang et al. applied the improved edit distance method to the calculation of Chinese sentence similarity, calculated the sentence similarity by calculating the edit distance of the words in the sentence, and added the semantic information of the vocabulary, although the result is better than purely based on semantics. The accuracy of the dictionary method is high, but the contribution of different words in the sentence to the whole is not consistent and phrase shifting often occurs in Chinese sentences. Inaccurate

[0009] Sentence similarity calculation is the most basic and critical step in multi-document subtopic division, and the research on Chinese similarity calculation is still in its infancy, and there are still great difficulties in using it for multi-document automatic summarization

Images