A method for extracting hotspots

Abstract

The application discloses a hotspot extraction method, comprising the following extraction steps: step S1, data analysis and extraction; step S2, fuzzy matching of keyword vectors; step S3, UIE model event address information extraction; step S4, similarity Bert model extracts a text vector, and the text vector is subjected to soft clustering; step S5, text vector clustering; step S6, ranking hotspot events to generate a daily report. Two-level aggregation is performed on the text, the text is first divided into large categories as hotspot events, and then similar events under each hotspot are further aggregated as subcategories. The rule matching and deep learning method are adopted, the latest UIE model is adopted for event extraction, event extraction is accurate, the text vector adopts the Bert model, and the representation ability is strong; the soft aggregation designed by the application is adopted for text aggregation, the aggregation speed is fast, and some samples without categories are allowed.

Description

Technical Field

[0001] This invention relates to the field of hotspot extraction technology, specifically a method for hotspot extraction. Background Technology

[0002] Traditional hotspot extraction systems mainly employ several methods: LDA topic modeling, bag-of-words model text clustering, and keyword extraction. LDA is an unsupervised probabilistic topic model that can model the implicit topics of documents. LDA can be divided into three levels: document, topic, and word. It obtains document-topic distributions through training on corpora and uncovers hidden topics within documents. However, when applying LDA to short text hotspot extraction, it often yields illogical topics. The bag-of-words model first calculates word frequencies in the text to obtain a bag-of-words vector, then clusters the text using k-means or hierarchical clustering methods, and finally extracts keywords from each cluster as the text's topic using TF-IDF. The text vector representation obtained by the bag-of-words model has many drawbacks, such as high dimensionality, lack of consideration for word order, and inability to represent relationships between words. Traditional clustering methods require each text to be clustered into a category, but in reality, many texts without topics do not belong to any category, thus failing to generate representative topics. Summary of the Invention

[0003] To overcome the shortcomings of existing technical solutions, this invention provides a hotspot extraction method that can effectively solve the problems raised in the background art.

[0004] The technical solution adopted by the present invention to solve its technical problem is: a method for extracting hot topics, including the following extraction steps: Step S1, analyze data and extract: analyze the data of the historical daily hot topics, reprocess the hot topic words, count the frequency of hot topic words and sort them, extract the high frequency hot topic words, and then use TextRank to extract keywords from the event titles under the high frequency hot topic words and perform manual screening;

[0005] Step S2, Fuzzy matching of keyword vectors: Train word2vec word vectors using massive hot topic event corpus, extract the word vectors of keywords under different hot topics and the word vectors after event title segmentation, calculate the cosine similarity between keyword word vectors and title segmentation word vectors to achieve fuzzy matching of event titles;

[0006] Step S3, UIE model event address information extraction: UIE model is a unified information extraction model. First, a certain amount of hot topic corpus data is labeled to mark the events and addresses in the text. Then, these data are used to fine-tune the UIE model to achieve convergence. Finally, the UIE model is used to extract the events and addresses in the title.

[0007] Step S4: The BERT similarity model extracts text vectors and performs soft clustering on the text vectors. The BERT model is pre-trained with a large amount of unlabeled text and then trained with a large number of similar text pairs. The vectors extracted by BERT can extract semantic information well. Similar text vectors are close in distance, while different text vectors are far apart. The text is first tokenized, with each Chinese character as a token. The token sequence is transformed into a matrix through the BERT embedding layer, and then passed through multiple layers of self-attention layers and feedforward neural networks to extract the text representation. The average of the extracted matrix in the sequence direction is the text vector. The cosine similarity function is used to calculate the similarity matrix of all text vectors, and the position pairs with values ​​greater than 0.9 are extracted. All position pairs are traversed, and position pairs with the same position are merged. When the number of clusters no longer changes, the clustering ends.

[0008] Step S5, Text Vector Clustering: Cluster the text vectors to obtain a set of similar events;

[0009] Step S6, Sort Hot Topics and Generate Daily Report: Sort the number of hot topics and take the top 6 to generate a daily report.

[0010] Furthermore, in step S3, the UIE model can extract relevant information from the text by specifying its schema. In some tasks, it can achieve a high F1 score by providing a small amount of data for fine-tuning.

[0011] Furthermore, in step S4, the vector of the address is extracted for complaints and emergencies, and the vector of the event is extracted for other categories.

[0012] Further, in step S5, when clustering the text vectors, BERT is used to extract the embeddings of all texts, and cosine similarity is used to calculate the similarity matrix of all embeddings. The similarity matrix is ​​a symmetric matrix with 1s on the diagonal. The lower half and diagonal of the matrix are set to 0, and positions in the matrix less than 0.9 are set to 0. All coordinate pairs greater than 0 in the similarity matrix are obtained. Coordinate pairs with the same position belong to the same category. All coordinate pairs are traversed and merged with each other. When merging can no longer be done, the aggregation result is obtained.

[0013] Compared with the prior art, the beneficial effects of the present invention are:

[0014] This invention performs two-level aggregation on text. First, it divides the text into large categories as hot events, and then further aggregates similar events under each hot event as subcategories. It employs a rule-matching and deep learning approach. Event extraction utilizes the state-of-the-art UIE model for accurate event extraction, and the text vectors use the BERT model for strong representation capabilities. Text aggregation employs a self-designed soft aggregation method, which is fast and allows for some uncategorized samples. Hotspot extraction is suitable, and event aggregation is accurate. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the extraction method of the present invention;

[0016] Figure 2 This is a schematic diagram of the vector clustering process in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figure 1-2 As shown, the present invention provides a method for hotspot extraction, including the following extraction steps:

[0019] Step S1, Analyze and extract data: Analyze the data of the historical daily hot topics, reprocess the hot topics, count the frequency of hot topics and sort them, extract the high-frequency hot topics, and then use TextRank to extract keywords from the event titles under the high-frequency hot topics and perform manual screening.

[0020] Step S2, Fuzzy matching of keyword vectors: Train word2vec word vectors using massive hot topic event corpora, extract the word vectors of keywords under different hot topics and the word vectors after event title segmentation, calculate the cosine similarity between keyword word vectors and title segmentation word vectors to achieve fuzzy matching of event titles.

[0021] Step S3, UIE model event address information extraction: The UIE model is a unified information extraction model. First, a certain amount of hot topic corpus data is labeled to mark the events and addresses in the text. Then, the UIE model is fine-tuned using this data to achieve convergence. Finally, the UIE model is used to extract the events and addresses in the title. The UIE model can extract the corresponding information in the text by specifying its schema. In some tasks, a small amount of data can be used for fine-tuning to achieve a high F1 score.

[0022] Step S4: The BERT similarity model extracts text vectors and performs soft clustering on the text vectors. The BERT model pre-processes the data using...

[0023] The system pre-trains on a massive amount of unlabeled text, then trains on text matching using a large number of similar text pairs. The vectors extracted by BERT effectively extract semantic information; similar text vectors are close in distance, while different text vectors are far apart. However, for complaint and emergency events, address vectors are extracted, while for other categories, event vectors are extracted. Therefore, the text is first tokenized, with each Chinese character as a token. The token sequence is transformed into a matrix through BERT's embedding layer, and then passed through multiple layers of self-attention layers and a feedforward neural network to extract the text representation. The average of the extracted matrix along the sequence direction is the text vector. The cosine similarity function is then used to calculate the similarity matrix of all text vectors, and the position pairs with values ​​greater than 0.9 are extracted. All position pairs are traversed, and position pairs with the same position are merged. The clustering ends when the number of clusters no longer changes.

[0024] Step S5, Text Vector Clustering: Cluster the text vectors to obtain a set of similar events. The text vector clustering adopts a soft aggregation method, which allows some samples to not belong to any category. Details are as follows: When clustering text vectors, BERT is used to extract the embeddings of all texts. The cosine similarity is used to calculate the similarity matrix of all embeddings. The similarity matrix is ​​a symmetric matrix with 1s on the diagonal. The lower half and diagonal of the matrix are set to 0, and the positions in the matrix less than 0.9 are set to 0. All coordinate pairs greater than 0 in the similarity matrix are obtained. Coordinate pairs with the same position belong to the same category. All coordinate pairs are traversed and merged with each other. When merging can no longer be done, the aggregation result is obtained.

[0025] Step S6, Sort Hot Topics and Generate Daily Report: Sort the number of hot topics and take the top 6 to generate a daily report.

[0026] Compared to traditional techniques, this solution performs two-level text aggregation. First, it divides the text into broad categories as hot events, and then further aggregates similar events under each hot event as subcategories. It employs a rule-matching and deep learning approach. Event extraction utilizes the state-of-the-art UIE model for accurate event extraction, and the text vectors use the BERT model for strong representation capabilities. Text aggregation employs a self-designed soft aggregation method, which is fast and allows for some uncategorized samples. Hotspot extraction is suitable, and event aggregation is accurate.

[0027] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for extracting hotspots, characterized in that, The extraction process includes the following steps: Step S1, analyze and extract data: analyze the data of the historical daily hot topics, reprocess the hot topics, count the frequency of hot topics and sort them, extract the high frequency hot topics, and then use TextRank to extract keywords from the event titles under the high frequency hot topics and perform manual screening. Step S2, Fuzzy matching of keyword vectors: Train word2vec word vectors using massive hot topic event corpus, extract the word vectors of keywords under different hot topics and the word vectors after event title segmentation, calculate the cosine similarity between keyword word vectors and title segmentation word vectors to achieve fuzzy matching of event titles; Step S3, UIE model event address information extraction: UIE model is a unified information extraction model. First, a certain amount of hot topic corpus data is labeled to mark the events and addresses in the text. Then, these data are used to fine-tune the UIE model to achieve convergence. Finally, the UIE model is used to extract the events and addresses in the title. Step S4: The BERT similarity model extracts text vectors and performs soft clustering on the text vectors. The BERT model is pre-trained with a large amount of unlabeled text and then trained with a large number of similar text pairs. The vectors extracted by BERT can extract semantic information well. Similar text vectors are close in distance, while different text vectors are far apart. The text is first tokenized, with each Chinese character as a token. The token sequence is transformed into a matrix through the BERT embedding layer, and then passed through multiple layers of self-attention layers and feedforward neural networks to extract the text representation. The average of the extracted matrix in the sequence direction is the text vector. The cosine similarity function is used to calculate the similarity matrix of all text vectors, and the position pairs with values ​​greater than 0.9 are extracted. All position pairs are traversed, and position pairs with the same position are merged. When the number of clusters no longer changes, the clustering ends. Step S5, Text Vector Clustering: Cluster the text vectors to obtain a set of similar events; Step S6, Sort Hot Topics and Generate Daily Report: Sort the number of hot topics and take the top 6 to generate a daily report.

2. The method for extracting hotspots according to claim 1, characterized in that, In step S3, the UIE model specifies its schema to extract relevant information from the text, providing a small amount of data fine-tuning for some tasks.

3. The method for extracting hotspots according to claim 1, characterized in that, In step S4, the vector of the address is extracted for complaints and emergencies, and the vector of the event is extracted for other categories.

4. The method for extracting hotspots according to claim 1, characterized in that, In step S5, when clustering text vectors, BERT is used to extract the embeddings of all texts, and cosine similarity is used to calculate the similarity matrix of all embeddings. The similarity matrix is ​​a symmetric matrix with 1s on the diagonal. The lower half and diagonal of the matrix are set to 0, and positions in the matrix less than 0.9 are set to 0. All coordinate pairs greater than 0 in the similarity matrix are obtained. Coordinate pairs with the same position belong to the same category. All coordinate pairs are traversed and merged. When merging can no longer be performed, the aggregation result is obtained.

Images