A text aggregation method and system

Abstract

The invention provides a text aggregation method and a text aggregation system, which are used for aggregating long texts and short texts. The method comprises the following steps of clustering the long texts to obtain topics corresponding to the long texts, the long texts comprising titles; establishing a classification model, and obtaining an abstract of the long text and an entity set; establishing a first mapping set and a second mapping set by using the topic, the title, the abstract and the entity set; using the first mapping set and the second mapping set to train the classification model to obtain a trained classification model; obtaining an abstract of a long text to be measured, establishing a third mapping set by using the abstract of the long text to be measured and a short text to be measured, and obtaining a text aggregation result by using the third mapping set and the trained classification model. According to the method and the system, the entity characteristics of thelong text and the short text are utilized to screen out the short text containing the same entity as the long text, so that the calculation complexity is reduced, and the text aggregation efficiencyis ensured.

Description

technical field [0001] The present invention relates to the technical field of text clustering, and more specifically, to a text aggregation method and system. Background technique [0002] There are many sources of information in real life, including professional media websites, self-media platforms, and social media. It has become a development trend to integrate multiple semantically related information and short comments. For example, in scenarios such as displaying search results and presenting news information, most of them are displayed in the form of topics rather than single texts, which can integrate multiple news sources, reduce information redundancy, and provide users with richer information. [0003] In the process of integrating information and short comments, that is, in the process of integrating long texts and short texts, clustering algorithms are generally required. Clustering algorithms can aggregate and generate topics contained in news. Clustering alg...

AI Technical Summary

Problems solved by technology

Clustering algorithms can aggregate and generate topics contained in news. Clustering algorithms are usually only used in long texts, and the effect is usually not ideal when used in short texts, because long texts have relatively more vocabulary, which can provide a stable and rich Long text features are expressed; while short texts have less vocabulary and lack of context information, making it difficult to form valuable clusters

For short text integration, the general method is to use classification algorithms, but the classification algorithm requires manual preset topic types, but due to the multi-dimensionality of natural language data, it is easy to have the problem that short text content does not exactly match the preset types

Images