Topic extraction method based on news text

Abstract

The present invention relates to a topic extraction method based on a news text. The topic extraction method comprises the following operating steps: 1, setting a user defined dictionary, preprocessing the text and generating a word text, part of speech and participle series vectors; 2, carrying out weight calculation and ranking according to word information such as a word frequency, and extracting the words in higher weight as a subject term series of an article from news titles and contents; 3, acquiring a hotspot subject term series gathered by texts according to the weights of subject terms of all articles; 4, forming a subject term expression vector of a topic by the collection of the subject terms; 5, gathering topics by utilizing the inclusion relation of clue words in the titles, the subject terms and the contents of the articles as well as the subject term expression vector of the topic, finally, obtaining a plurality of hotspot topics. In the topic extraction method, the subject terms of the article are utilized to express the topic of the article and complete relevant article gathering, finally, two clue words are utilized to express the one current hotspot news topic.

Description

technical field [0001] The invention relates to the fields of natural language processing and artificial intelligence, in particular to a news text-based topic extraction method. Background technique [0002] With the popularization of the Internet, there are more and more ways for people to obtain information, and the Internet has gradually become the carrier of various information in society. Especially with the continuous development of China's economy, online news products have gradually become an important channel for people to obtain information, and more and more people obtain real-time news and related information through the Internet. Web text has become an important source of information for us, and a large number of news hotspots are generated every day. How to obtain current hot topics from massive news texts has become a necessary basic technology for news text processing. [0003] The general text clustering technology mostly adopts the automatic text clusteri...

AI Technical Summary

Problems solved by technology

These clustering algorithms have various shortcomings. The single-pass algorithm has a great correlation with the order of article input; the KNN algorithm has a high time complexity; and the k-means algorithm must determine the number of clusters in advance. number, but this is usually more difficult

Moreover, the clustering results obtained by these algorithms after the clustering is completed are often incomprehensible, and we cannot use specific words or content to express the clustering results

Images