Network event theme mining method and system based on Wasserstein auto-encoder

Abstract

The invention belongs to but not limited to the technical field of topic mining, and discloses a Wasserstein auto-encoder-based network event topic mining method and system, a Wasserstein auto-encoder-based neural topic model EWTM is constructed, text data in an event is input, the text data is firstly subjected to data cleaning, word segmentation and stop word removal operations, and word bag representation is obtained; in each training period, carrying out encoding operation on the word bag representation to obtain a topic vector of the document; sampling from the hidden variables by using beta distribution to calculate MMD loss; calculating reconstruction loss, and finally obtaining total loss; carrying out back propagation, and carrying out parameter updating by using an optimizer so as to optimize the EWTM model; the process is continuously iterated until a specified training cycle number is reached. According to the method, the theme of the social network event can be automatically discovered and extracted, so that the core information of the event and the focus of user attention can be better understood.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the technical field of topic mining, and in particular relates to a network event topic mining method and system based on Wasserstein autoencoder. Background Art

[0002] The basic representation of events in social networks involves three key elements: the set of users U, the set of words W, and the set of blog posts D. In this representation, U represents the set of all users who have posted blog posts, W is the set of words that appear in all blog posts, and D is the set of all blog posts published by the user.

[0003] Event topic mining is to store the crawled social network data, extract and construct a blog post set D from it, obtain a word set W and a bag-of-words representation of all documents through a text preprocessing process, and then use the topic model for mining. The mining result is K topics, each topic consists of n topic words. In order to evaluate the mining effect, the topic coherence index Cv and the topic diversity index TD are used for evaluation. The flowchart of social network event topic mining is shown in the figure. Figure 1 shown.

[0004] In topic mining, the text clustering method first calculates the similarity between texts, classifies similar texts into the same set, and then extracts keywords from the set as subject words. The traditional topic model method trains the preprocessed text data to infer the distribution of topics behind the text. However, their grasp of context and semantics is not accurate enough, and the mined topics are not coherent enough.

[0005] The existing technology has the following technical problems and defects that need to be solved urgently in social network event topic mining:

[0006] 1. Insufficient context and semantic understanding: Traditional text clustering and topic modeling methods usually only consider shallow features such as word frequency when processing text, but fail to fully understand the context and semantic relationships in the text. This results in the lack of coherence in the extracted topic words, which cannot accurately reflect the core information of the event and the focus of user attention.

[0007] 2. Similarity calculation is not accurate enough: Text clustering methods rely on similarity calculation between texts to classify similar texts into the same set. However, most existing similarity calculation methods are based on shallow features, such as word frequency and TF-IDF, which fail to fully capture the semantic similarity between texts. This leads to inaccurate text clustering results and affects the extraction effect of topic words.

[0008] 3. Limited expressive power of topic models: Traditional topic models, such as LDA, can infer the distribution of topics behind the text, but their expressive power is limited and they cannot fully capture the complex semantic relationships and implicit topic structures in the text. This makes the model's topic mining effect less than ideal, and the topic coherence and diversity are low.

[0009] 4. Lack of effective topic evaluation indicators: The commonly used topic evaluation indicators in the existing technology, such as topic coherence (Cv) and topic diversity (TD), can evaluate the topic mining effect to a certain extent, but these indicators still have certain limitations. For example, topic coherence only focuses on the semantic consistency between topic words, but ignores the actual distribution of topic words in the document, and cannot comprehensively evaluate the topic mining effect.

[0010] 5. Inadequate processing of dynamics and timeliness: Event topics in social networks are often dynamic and time-sensitive. Existing technologies are insufficient in processing dynamically changing topics and timeliness information, making it difficult to update and capture the latest user focus in real time. This affects the accuracy and practicality of topic models in practical applications.

[0011] In summary, the problems that the existing technology urgently needs to solve in social network event topic mining are: a model is needed that can automatically discover and extract social network event topics, better understand the core information of the event and the focus of user attention, improve the coherence and diversity of the topic, and be able to handle dynamic changes and timeliness information. Summary of the invention

[0012] In view of the problems existing in the prior art, the present invention provides a network event topic mining method and system based on Wasserstein autoencoder.

[0013] The present invention is implemented as follows: a network event topic mining method based on Wasserstein autoencoder, constructing a neural topic model EWTM (Event-based WAE on TopicModel) based on Wasserstein autoencoder, using Beta distribution as prior distribution, combining reconstruction loss and Wasserstein distance loss, and automatically discovering and extracting topics of social network events;

[0014] The input of the model algorithm is the text data in the event. First, the text data is cleaned, segmented, and stop words are removed to obtain a bag-of-words representation. In each training cycle, the bag-of-words representation is encoded to obtain the topic vector of the document. The MMD loss is calculated by sampling from the latent variables using the Beta distribution. The reconstruction loss is then calculated, which measures the difference between the document generated by the model and the original document. Finally, the overall loss is obtained, which is a linear combination of the reconstruction loss and the MMD loss.

[0015] Finally, backpropagation is performed and the optimizer is used to update the parameters to optimize the EWTM model; this process is iterated until the specified number of training cycles is reached; this training process helps the model learn the topic information in the data and improve the coherence and diversity of the generated documents.

[0016] Furthermore, the EWTM model is implemented under the PyTorch framework, and its structure consists of two parts, namely Encoder and Decoder.

[0017] Furthermore, the encoder of the EWTM model consists of multiple linear layers and activation function layers, namely, a multi-layer perceptron MLP. First, the document W = {w1,w2,...,w m The encoder is mapped to an output layer with V dimensions. By looping through the layers of the encoder, a series of linear transformations are performed on the input, and the dropout operation is applied to obtain the latent variable Z. Then, the raw score of each unit in the output layer is converted into the topic vector θ∈S of the document w by applying the softmax function. V-1 The encoder approximately learns the topic probability distribution of document w, so that the approximated distribution Q(θ|w) is close to the true posterior distribution p(θ|w).

[0018] Furthermore, the main function of the decoder of the EWTM model is to map the topic representation θ in the latent space to the output layer, which produces the probability distribution of words. represents the probability of each word appearing in the vocabulary under a given topic; the calculation formula is as shown in (1),

[0019]

[0020] represents the probability estimate of the i-th word, h represents the output of the neuron, and the calculation formula is as follows (2),

[0021] h=W·θ+b (2)

[0022] W is the decoder weight matrix and b is the bias term.

[0023] Furthermore, the Beta distribution is used as the prior distribution of the latent variable Z, and then Z' = {z' i}~Beta(α,β), where α and β are the parameters of the Beta distribution, which affect the shape of the Beta distribution and thus adjust the prior information of the topic model.

[0024] Furthermore, the overall loss of the model consists of two parts: reconstruction loss L rec and MMD loss L MMD , the calculation formula is shown in (3):

[0025] L=L rec +λ·L MMD (3)

[0026] Among them, γ is a hyperparameter, which is mainly used to balance the weights so that features in different dimensions have different importance in subsequent calculations or applications;

[0027] L rec is the reconstruction loss of the EWTM model, which calculates the original document w and the reconstructed The negative cross entropy loss is used to measure the difference between two probability distributions, L rec The calculation of is as follows:

[0028]

[0029] L MMD The difference between two probability distributions is measured by calculating the distance between them. The Gaussian kernel function is used to calculate the distance. The calculation formula of the Gaussian kernel is as follows (5):

[0030]

[0031] The formula of MMD loss is as follows (6):

[0032]

[0033] The MMD distance uses n samples for unbiased estimation, where {z1,z2,...,z n The distribution Q in the latent space generated from the encoder θ , and {z′1,z′2,...,z′ n}From the true distribution P of the latent variables given the input document w θ Medium sampling.

[0034] Another object of the present invention is to provide a network event topic mining method based on Wasserstein autoencoder and a network event topic mining system based on Wasserstein autoencoder, comprising an encoder and a decoder; the input is text data in the event, and the output is a text vector, a topic category, and a topic word;

[0035] The encoder is composed of multiple linear layers and activation function layers, i.e., a multi-layer perceptron MLP;

[0036] The decoder is used to map the topic representation θ in the latent space to the output layer, which produces the probability distribution of words Represents the probability of each word in the vocabulary appearing under a given topic.

[0037] Another object of the present invention is to provide a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the network event topic mining method based on Wasserstein autoencoder.

[0038] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the network event topic mining method based on the Wasserstein autoencoder.

[0039] Another object of the present invention is to provide an information data processing terminal, which includes the network event topic mining system based on Wasserstein autoencoder.

[0040] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0041] First, the present invention proposes a neural topic model based on Wasserstein autoencoder, which uses Beta distribution as prior distribution and combines reconstruction loss and Wasserstein distance loss to automatically discover and extract topics of social network events, so as to better understand the core information of the events and the focus of users' attention.

[0042] The present invention evaluates the mining results of the model by using topic coherence and topic diversity indicators and compares them with other topic models. The results show that the EWTM model proposed in the present invention achieves better mining results under different topics.

[0043] Second, the present invention proposes a network event topic mining method based on Wasserstein autoencoder, and constructs a neural topic model EWTM (Event-based WAE on TopicModel) based on Wasserstein autoencoder. By adopting Beta distribution as the prior distribution, combined with reconstruction loss and Wasserstein distance loss, the topic of social network events can be automatically discovered and extracted. This method performs data cleaning, word segmentation, and stop word removal in text preprocessing, generates a bag-of-words representation, and encodes the bag-of-words representation in each training cycle to obtain the topic vector of the document, which greatly improves the accuracy and efficiency of topic mining.

[0044] The EWTM model further improves the effect of topic mining through the collaborative work of the encoder and decoder. The encoder consists of multiple linear layers and activation function layers, and uses a multi-layer perceptron MLP to map documents to latent variable space, thereby approximating the topic probability distribution of the learning document, making the approximated distribution close to the true posterior distribution. The decoder maps the topic representation in the latent space to the output layer, generates the probability distribution of words, and represents the probability of each word in the vocabulary under a given topic, ensuring the coherence and diversity of the generated documents.

[0045] By using the Beta distribution as the prior distribution of the latent variable, sampling from it to calculate the MMD loss, and combining it with the reconstruction loss to form the overall loss of the model. This method balances the difference between the reconstructed document and the original document, while measuring their difference by calculating the distance between the two probability distributions through the Gaussian kernel function, which significantly improves the topic coherence and diversity of the model. The optimization process of the model updates parameters through backpropagation and optimizer, ensuring the efficient implementation of the EWTM model in the PyTorch framework.

[0046] The present invention effectively solves the problems of the prior art in terms of insufficient context and semantic understanding, inaccurate similarity calculation, and limited expressiveness of topic models. By constructing an EWTM model based on the Wasserstein autoencoder, the consistency and diversity of topic mining are improved, and accurate discovery and extraction of social network event topics are achieved. This method has significant technical progress in text clustering and topic models, and provides strong technical support for understanding the core information of social network events and the focus of user attention.

[0047] Third, the expected benefits and commercial value of the technical solution of the present invention after transformation are as follows: Accurate topic mining capabilities: Through the Wasserstein autoencoder technology, it is possible to more accurately mine and understand the topics and related content of network events, and improve the efficiency of information integration and analysis. Improve content recommendation and search engines: The capabilities of search engines and content recommendation systems are optimized, allowing users to find the required information more quickly and accurately, improving user experience and satisfaction. Enhance information security and monitoring capabilities: Conduct more in-depth and comprehensive monitoring and analysis of network events, and improve the identification and response speed of potential threats and abnormal events. Support decision-making and predictive analysis: Based on accurate topic mining and analysis, provide enterprises and government departments with more accurate data support for decision-making and future trend prediction.

[0048] Fourth, the present invention proposes a document topic analysis method based on the EWTM model, which achieves more efficient and accurate topic modeling by optimizing the encoder and decoder structures. In the encoder part, a multi-layer perceptron (MLP) structure is used to map the document to a high-dimensional output layer, and hidden variables are generated through linear transformation and dropout operations. The scores of the output layer are converted into the topic vectors of the document through a function, so that the encoder can approximately learn the topic probability distribution of the document, which is close to the true posterior distribution. This method overcomes the limitations of traditional topic models in processing high-dimensional data and significantly improves the performance of the model.

[0049] The main function of the decoder part is to map the topic representation in the latent space to the output layer, generate the probability distribution of words, and represent the probability of each word in the vocabulary under a given topic. By optimizing the weight matrix and bias term of the decoder, the probability estimate of the word is accurately calculated. Using the Beta distribution as the prior distribution of the latent variable, the parameters sampled from it can adjust the prior information of the topic model, making the model more adaptable and robust when processing different types of documents. This decoder design makes the topic model more flexible and accurate in the process of generating text.

[0050] The overall loss of the model consists of two parts: reconstruction loss and MMD loss. The reconstruction loss calculates the negative cross entropy difference between the original document and the reconstructed document to measure the difference between the two probability distributions; the MMD loss calculates the distance between the two probability distributions, uses a Gaussian kernel function to measure the difference, and uses samples for unbiased estimation. This dual loss design ensures that the model can maintain a faithful reconstruction of the original information of the document during the learning process, and can effectively capture the distribution characteristics in the latent space, improving the overall performance and generalization ability of the topic model.

[0051] The present invention has achieved significant technical progress in many aspects. First, by improving the encoder and decoder structure, the accuracy and efficiency of topic modeling are improved; second, by using Beta distribution as a prior distribution, the adaptability of the model to different types of documents is enhanced; third, by introducing reconstruction loss and MMD loss, the model is ensured to capture the integrity of document information and the potential distribution during the learning process; finally, by using Gaussian kernel function and unbiased estimation method, the probability distribution difference is accurately measured, and the generalization performance of the model is improved. Overall, the present invention provides an efficient, accurate and adaptable solution for document topic analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the event topic mining process provided by the present invention;

[0053] Figure 2 It is a structural diagram of an EWTM model provided by an embodiment of the present invention;

[0054] Figure 3 is a microblog preprocessing flow chart provided by an embodiment of the present invention;

[0055] Figure 4 1 is a graph showing the results of a comparison experiment of different topic models provided by an embodiment of the present invention, (a) a graph showing the comparison results of topic coherence Cv, and (b) a graph showing the comparison results of topic diversity TD;

[0056] Figure 5 This is a comprehensive comparison result diagram of the event topic mining experiment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] ###Example 1: News document topic analysis

[0059] Background: A news media company needs to perform topic analysis on a large number of newly released news documents every day in order to classify and recommend them to different reader groups.

[0060] 1) Data collection: Collect news documents published over a period of time and read the genome-wide association study (GWAS) data of these documents as the basic data for exposure factors and outcome variables.

[0061] 2) Data preprocessing: Use dedicated data processing software to preprocess the collected news document data, including denoising, data format conversion and quality control, to ensure data consistency and integrity.

[0062] 3) Select instrumental variables: Select appropriate single nucleotide polymorphisms (SNPs) as instrumental variables in the GWAS data of news documents, and use PLINK software to perform linkage disequilibrium (LD) screening, remove instrumental variables with linkage disequilibrium and weak instrumental variables, and ensure that the selected SNPs are independent of each other.

[0063] 4) Remove confounding factors: Remove confounding factors from the preprocessed data to reduce noise interference.

[0064] 5) Topic modeling: Through the encoder of the EWTM model, the news document is mapped to a high-dimensional output layer, linear transformation and dropout operations are performed to generate latent variables, and the scores of the output layer are converted into the topic vectors of the document by applying the function.

[0065] 6) Topic representation and word probability distribution: The decoder of the EWTM model is used to map the topic representation in the latent space to the output layer to generate the probability distribution of words, which represents the probability of each word in the vocabulary under a given topic.

[0066] 7) Loss calculation: Calculate the reconstruction loss and MMD loss, use Gaussian kernel function and unbiased estimation method to measure the probability distribution difference, and optimize the model parameters.

[0067] 8) Topic analysis results: The final topic vectors and word probability distribution results are applied to the news classification and recommendation system to achieve accurate content recommendations for different reader groups.

[0068] News media companies can efficiently conduct topic analysis and classification on news documents published daily, provide more accurate content recommendation services, and improve user satisfaction and platform traffic.

[0069] ###Example 2: Academic Literature Topic Analysis

[0070] A scientific research institution needs to conduct subject analysis on a large amount of academic literature so that researchers can quickly find research results in related fields.

[0071] 1) Data collection: Collect academic literature in this field and read the genome-wide association study (GWAS) data from these literature as the basic data for exposure factors and outcome variables.

[0072] 2) Data preprocessing: Use dedicated data processing software to preprocess the collected academic literature data, including denoising, data format conversion and quality control, to ensure data consistency and integrity.

[0073] 3) Selection of instrumental variables: Select appropriate single nucleotide polymorphisms (SNPs) as instrumental variables in the GWAS data of academic literature, and use PLINK software to perform linkage disequilibrium (LD) screening, remove instrumental variables with linkage disequilibrium and weak instrumental variables, and ensure that the selected SNPs are independent of each other.

[0074] 4) Remove confounding factors: Remove confounding factors from the preprocessed data to reduce noise interference.

[0075] 5) Topic Modeling: Through the encoder of the EWTM model, the academic literature is mapped to a high-dimensional output layer, linear transformation and dropout operations are performed to generate latent variables, and the scores of the output layer are converted into the topic vectors of the documents by applying functions.

[0076] 6) Topic representation and word probability distribution: The decoder of the EWTM model is used to map the topic representation in the latent space to the output layer to generate the probability distribution of words, which represents the probability of each word in the vocabulary under a given topic.

[0077] 7) Loss calculation: Calculate the reconstruction loss and MMD loss, use Gaussian kernel function and unbiased estimation method to measure the probability distribution difference, and optimize the model parameters.

[0078] 8) Topic analysis results: The final topic vectors and word probability distribution results are applied to the academic document classification system to achieve rapid search and analysis of research results in related fields.

[0079] Scientific research institutions can efficiently conduct subject analysis and classification of large amounts of academic literature, helping researchers quickly find research results in related fields and improving scientific research efficiency and the utilization rate of results.

[0080] The network event topic mining method based on Wasserstein autoencoder provided in the embodiment of the present invention constructs a neural topic model EWTM (Event-based WAE on Topic Model) based on Wasserstein autoencoder, takes Beta distribution as prior distribution, combines reconstruction loss and Wasserstein distance loss, and automatically discovers and extracts topics of social network events;

[0081] The input of the model algorithm is the text data in the event. First, the text data is cleaned, segmented, and stop words are removed to obtain a bag-of-words representation. In each training cycle, the bag-of-words representation is encoded to obtain the topic vector of the document. The MMD loss is calculated by sampling from the latent variables using the Beta distribution. The reconstruction loss is then calculated, which measures the difference between the document generated by the model and the original document. Finally, the overall loss is obtained, which is a linear combination of the reconstruction loss and the MMD loss.

[0082] Finally, backpropagation is performed and the optimizer is used to update the parameters to optimize the EWTM model; this process is iterated until the specified number of training cycles is reached; this training process helps the model learn the topic information in the data and improve the coherence and diversity of the generated documents.

[0083]

[0084]

[0085] The embodiment of the present invention constructs an EWTM (Event-based WAE on Topic Model) model to mine the topics of social network events. The EWTM model is implemented in the PyTorch framework and its structure includes two parts, namely Encoder and Decoder. Figure 2 shown.

[0086] The encoder of the EWTM model consists of multiple linear layers and activation function layers, namely, a multi-layer perceptron MLP. First, the document W = {w1,w2,...,w m The encoder is mapped to an output layer with V dimensions. By looping through the layers of the encoder, a series of linear transformations are performed on the input, and the dropout operation is applied to obtain the latent variable Z. Then, the raw score of each unit in the output layer is converted into the topic vector θ∈S of the document w by applying the softmax function. V-1 The encoder approximately learns the topic probability distribution of document w, so that the approximated distribution Q(θ|w) is close to the true posterior distribution p(θ|w).

[0087] The main function of the decoder of the EWTM model is to map the topic representation θ in the latent space to the output layer, which produces the probability distribution of words represents the probability of each word appearing in the vocabulary under a given topic; the calculation formula is as shown in (1),

[0088]

[0089] represents the probability estimate of the i-th word, h represents the output of the neuron, and the calculation formula is as follows (2),

[0090] h=W·θ+b (2)

[0091] W is the decoder weight matrix and b is the bias term.

[0092] Considering that the context information of microblog text is limited and the number of words is small, it presents a relatively discrete characteristic, and Beta distribution is more suitable for this characteristic. Therefore, Beta distribution is used as the prior distribution of the latent variable Z, and then Z' = {z′ i}~Beta(α,β), where α and β are the parameters of the Beta distribution, which affect the shape of the Beta distribution and thus adjust the prior information of the topic model.

[0093] The overall loss of the model consists of two parts: reconstruction loss L rec and MMD loss L MMD , the calculation formula is shown in (3):

[0094] L=L rec +λ·L MMD (3)

[0095] Among them, γ is a hyperparameter, which is mainly used to balance the weights so that features in different dimensions have different importance in subsequent calculations or applications;

[0096] L recis the reconstruction loss of the EWTM model, which calculates the original document w and the reconstructed The negative cross entropy loss is used to measure the difference between two probability distributions, L rec The calculation of is as follows:

[0097]

[0098] L MMD The difference between two probability distributions is measured by calculating the distance between them. The Gaussian kernel function is used to calculate the distance. The calculation formula of the Gaussian kernel is as follows (5):

[0099]

[0100] The formula of MMD loss is as follows (6):

[0101]

[0102] Specifically, the MMD distance uses n samples for unbiased estimation, where {z1,z2,...,z n The distribution Q in the latent space generated from the encoder θ , and {z′1,z′2,...,z′ n}From the true distribution P of the latent variables given the input document w θ Medium sampling.

[0103] The embodiment of the present invention provides a network event topic mining method based on Wasserstein autoencoder and a network event topic mining system based on Wasserstein autoencoder, comprising an encoder and a decoder; the input is text data in the event, and the output is text vector, topic category, and topic word;

[0104] The encoder is composed of multiple linear layers and activation function layers, i.e., a multi-layer perceptron MLP;

[0105] The decoder is used to map the topic representation θ in the latent space to the output layer, which produces the probability distribution of words Represents the probability of each word in the vocabulary appearing under a given topic.

[0106] The experimental data set of the embodiment of the present invention is collected from the Weibo platform by web crawler technology. It is then preprocessed, including steps such as text cleaning, word segmentation, and stop word removal. The comparison models mainly include traditional probabilistic topic models and topic models based on deep learning. The evaluation indicators use topic coherence and topic diversity, which are of great reference significance for the quality assessment of topic models. Finally, the comparison experimental results and mining results are analyzed, and the results show that the EWTM model achieves better mining results under different topics.

[0107] 1.1 Experimental data sources

[0108] This paper uses Weibo API and Selenium automation framework to crawl Weibo data related to various events from 2020 to 2021. During this period, many major events that have attracted widespread attention have occurred around the world, which have attracted widespread attention from users and provided rich data resources for the experiment.

[0109] Each crawled microblog was stored in JSON format, and a total of 70,373 microblogs were selected for the experiment by extracting the text information of each microblog. The event topic mining model proposed in the present invention is used to mine and analyze the event topics and related features that have aroused discussion and attention during this period, providing useful data support and analysis results for further understanding the focus of public attention and social opinion.

[0110] 1.2 Data Preprocessing

[0111] Since the original data contains a lot of noise and irrelevant text, such as repeated text, too short text, meaningless symbols, etc., which is not conducive to the development of experiments, the microblog text data collected from the social network platform is preprocessed. The process mainly includes data cleaning, word segmentation, stop word removal, and building a bag-of-words model. The overall processing flow is as follows Figure 3 shown.

[0112] (1) Data cleaning

[0113] Duplicate texts can interfere with data analysis and modeling, leading to biased and inaccurate results. Considering that users may post multiple identical content, the present invention uses the fuzzywuzzy library to calculate the similarity of each text and removes texts with a similarity greater than 90%. Secondly, Weibo text data usually contains a large amount of irrelevant information, which interferes with subsequent research and analysis and reduces data quality. Bai Fumei uses the emoji library to delete emoticons and uses the re regular expression to remove irrelevant text information such as links and special symbols. This can make text data cleaner and easier to process, helping to improve the efficiency and accuracy of subsequent analysis tasks.

[0114] (2) Text segmentation

[0115] Word segmentation is a fundamental task in natural language processing. Its main goal is to split a text into the smallest units with semantic meaning, such as words or phrases. Chinese words are not separated by symbols, and the combination of Chinese characters in a sentence also varies with the context. To solve the problem of Chinese text word segmentation, this invention uses the Hanlp word segmenter to segment microblog text data. The Hanlp word segmenter is a popular and efficient Chinese word segmentation tool, and its dictionary covers various common words and phrases, which can divide words relatively accurately.

[0116] (3) Removing stop words

[0117] Removing stop words is to remove common words that frequently appear in the text but contribute little to the semantic analysis of the text. For Chinese stop words, such as "的" (de), "是" (shi), "在" (zai), etc. During the process of removing stop words, a stop word list is usually used, which contains the words to be removed. This invention uses the Harbin Institute of Technology Chinese stop word list, which contains common and meaningless words in Chinese text processing. Since the microblogs posted by users contain English content, English stop words are also added to the stop word list, such as "the", "and", "is", etc. The stop word list is not fixed and needs to add stop words according to the requirements of specific tasks to remove those words that are meaningless or unimportant in the current context. The word segmentation results of Hanlp are shown in Table 1:

[0118] Table 1 Hanlp word segmentation results

[0119]

[0120]

[0121] (4) Building a dictionary

[0122] A dictionary usually refers to a data structure that contains a vocabulary list, where each word is associated with a unique identifier. This identifier is usually an integer, used to represent and operate words in a computer program. At the same time, the dictionary also contains the document frequency information of each word, that is, the number of documents in which the word appears in the corpus. After building the dictionary, filter the dictionary according to a given threshold to remove words whose document frequencies are outside this threshold. This helps to identify topic-related words in the text, thereby improving the accuracy of topic mining.

[0123] 1.3 Comparative models

[0124] To prove the effectiveness of the neural topic model proposed in this invention, this invention selects four representative topic models for comparison, namely the traditional probabilistic topic model LDA, and the deep learning-based BATM, GSM, and WLDA models, as follows:

[0125] LDA (Latent Dirichlet Allocation): The LDA model is a probabilistic topic model used to discover potential topic structures in a document collection. It assumes that each document is a mixture of multiple topics, and each topic is represented by a distribution of a set of words. Through Bayesian inference, it can reveal the hidden topic information in the document collection and model and analyze the topic distribution of text data.

[0126] BATM (Bidirectional Adversarial Topic model): The BATM model combines the ideas of topic model and adversarial network, aiming to learn the shared topic distribution and word translation relationship in text data. During training, the topic model is used to model the text data in the source language and the target language, and then the adversarial network is used to achieve the bidirectional correspondence of topic distribution and the association of word translation between the two languages, making it play an important role in cross-language tasks.

[0127] ·GSM (Gaussian Softmax distribution): The GSM model is a topic model based on a variational inference method of a neural network. The core of this method is to use a parameterized distribution to represent the topic in a document and allow training through back propagation. The model combines a generative topic model (GTM) and a variational autoencoder (VAE) to capture the deep semantic structure of a document.

[0128] WLDA (Wasserstein Latent Dirichlet Allocation): The WLDA model is a neural network topic model based on WAE. It uses Dirichlet distribution as the prior distribution and uses the information diffusion kernel function to calculate the Wasserstein distance when performing distribution matching.

[0129] EWTM (Event-based Wasserstein AutoEncoder on Topic Model): The EWTM model is a neural topic model based on the Wasserstein autoencoder constructed by the present invention, which uses the Beta distribution as the prior distribution and the Gaussian kernel function to calculate the MMD loss.

[0130] 1.4 Evaluation indicators

[0131] Since the topic model proposed in this invention performs an unsupervised task and the experimental purpose is to mine the topic of the event, indicators such as accuracy and recall are not applicable. Therefore, topic coherence Cv and topic diversity TD are used to measure the topic mining results.

[0132] (1) Thematic coherence

[0133] Topic coherence measures the correlation between words contained in a topic to determine whether the topic has clear semantic content, thereby evaluating the coherence and consistency of the mined topic. It uses the Boolean sliding window method to perform one-set segmentation on the topic words, that is, to divide the topic words into a set, and then uses cosine similarity and normalized pointwise mutual information (NPMI) to indirectly measure the coherence of the topic words. The larger the Cv index of a topic, the more coherent the topic is. The formula is shown in (7):

[0134]

[0135] Where t is the topic, n is the number of topic words in t, S cos is the cosine similarity score. First, for word w in t i , calculate its v in all topics NPMI (w i ) vector, which represents the NPMI value between each word and all other words, and calculates the NPMI vector mean v of all words in t NPMI (w 1:n ), the final Cv score is v NPMI (w i ) and v NPMI (w 1:n ) is the average of the cosine similarities between .

[0136] The following gives NPMI, v NPMI (w i ) and v NPMI (w 1:n ) is calculated as:

[0137]

[0138] v NPMI (w i )={NPMI(w i ,w j )} j＝1,...,n (9)

[0139]

[0140] The ε in the above formula (3-8) ensures that the denominator ≠ 0.

[0141] (2) Thematic Diversity TD

[0142] Topic diversity is an indicator used to measure the diversity of topics covered in a text corpus. The topic diversity index can understand the uniformity of the topic distribution of a text collection, and whether there is repetition or concentration on a specific topic. The calculation formula is shown in (11):

[0143]

[0144] In T different topics, each topic has L keywords. cnt(l,t) indicates the number of times the lth keyword in the tth topic appears in the keyword set of all topics. The higher the TD value, the greater the difference between topics, that is, the higher the topic diversity. The range is (0,1]. When DT(T) is equal to 1, it means that each keyword in the T topics is different.

[0145] 1.4.5 Experimental parameter setting

[0146] The parameters of the EWTM model mainly include the maximum and minimum document frequencies of the retained vocabulary, the number of iterations, the batch size, the number of topics, the number of subject words, etc.

[0147] The present invention refers to the parameter settings of other neural topic models, sets the minimum document frequency of the retained vocabulary to 10, that is, it appears in at least 10 documents, the maximum frequency to 0.3, the number of topics to 10, 15, 20, 25, 30, 15 topic words per topic, the batch size to 256, and the number of iterations to 500. The initial learning rate is set to 0.001, and the weight coefficient λ ranges from 0.1 to 1.

[0148] A computer device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the network event topic mining method based on Wasserstein autoencoder.

[0149] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the network event topic mining method based on Wasserstein autoencoder.

[0150] An information data processing terminal comprises the network event topic mining system based on Wasserstein autoencoder.

[0151] To ensure fair experimentation, all algorithms used the same dataset before training. The comparison results of Cv and TD indicators mined under different topic models are shown in the following table. Figure 4As shown in the figure, it can be seen that the EWTM model has achieved good results in microblog event topic mining. Under all the number of topics, the topic coherence of the EWTM model is the highest. When the number of topics is set to 10, 15, and 20, the Cv index and TD index of the EWTM model are higher than those of the BATM model and the WLDA model.

[0152] Table 2 shows the event topic mining scores of different topic models. It can be seen from Table 2 that under different numbers of topics, LDA's TD index has achieved high scores, that is, basically no repeated topics have been mined, but its Cv index is the lowest among all models. This is because LDA is a probability graph-based model that attempts to identify potential topics from documents and assigns a topic to each word in the document. It assumes that the vocabulary in the document is disordered, and the words of the same topic are scattered in different positions without considering their relative positions in the text. This causes the LDA model to include a variety of unrelated words when generating topics, thereby increasing the diversity of topics. This also causes it to be unable to capture the contextual information of the context, and the mined topics are not coherent enough.

[0153] In comparison, topic models based on neural networks can better capture nonlinear relationships in texts by introducing nonlinear activation functions and deep structures, which enables them to model more complex semantic relationships between words and texts, thereby improving the ability to mine coherent topics.

[0154] The GSM model is based on the variational autoencoder architecture and focuses on modeling the structured information of text data. When faced with short microblog texts that usually contain a lot of noise or minor information, this redundant information is processed by increasing the number of topics. Therefore, as the number of topics increases, the GSM model tends to generate diverse but relatively less coherent topics.

[0155] Table 2. Results of topic mining experiments

[0156]

[0157] Since the number of events in the dataset is limited, but as the number of topics increases, each topic contains only a few relevant words, resulting in topic fragmentation. This makes each topic not rich enough and difficult to provide meaningful topic words or distinctions. Therefore, when the number of topics in the EWTM model exceeds 20, the topic coherence and topic diversity indicators decrease.

[0158] Generally speaking, higher coherence means that the words within the same topic are highly related, forming an organized and consistent semantic structure, which makes the topic easier to interpret and understand. However, in order to achieve high coherence, the model tends to make the words within each topic similar, which will lead to reduced topic diversity. In order to more comprehensively evaluate the performance of the topic model, considering the trade-off between coherence and diversity, the comprehensive score is obtained by weighted summing Cv and TD, with a weight coefficient of 0.5. The comprehensive scores of different topic models are shown in Table 3.

[0159] Table 3 Comprehensive scores of event topic mining experiments

[0160]

[0161]

[0162] When the number of topics is less than 20, the EWTM model achieves the highest comprehensive score on the experimental dataset. As the number of topics increases, it is slightly lower than the LDA and GSM models. Considering that the task is unlabeled, some models generate topics containing more irrelevant words when the number of topics increases, which is inconsistent with human judgment. As a result, the topic diversity index score is increased, and the comprehensive score also increases accordingly.

[0163] The comprehensive comparison results of different topic models are as follows: Figure 5 As shown, overall, the EWTM model works best when the number of topics is 10. Table 4 shows the mining results when the number of topics is 10.

[0164] The rows in the results represent topics, and each topic consists of 15 keywords. For the convenience of display, only the first 7 are listed. Each topic is related to a social network event. For example, the keywords "Brexit", "Bill", "Agreement", "Approval" and "UK" in the first row are related to the "Brexit" event; the keywords "Death", "Kobe", "NBA", "Early Morning" and "Helicopter" in the second row are related to the "Death of Kobe" event; the keywords "Takeoff", "Realization", "Chang'e", "Moon" and "First Time" in the third row are related to the "Chang'e 5 Lunar Exploration Satellite" event.

[0165] Table 4 Event topic mining results

[0166]

[0167] This paper mainly studies the topic mining method of social network events. First, the neural topic model EWTM based on Wasserstein autoencoder is introduced in detail; then a large amount of Weibo data is collected through web crawler technology, and the data is preprocessed to facilitate the input of the model; then the topic coherence and topic diversity indicators are used to evaluate the mining results of the model and compared with other topic models. The results show that the model proposed in this chapter has achieved good results in topic mining, and finally the mining results are displayed.

[0168] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0169] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A network event topic mining method based on Wasserstein autoencoder, characterized in that: Construct a neural topic model EWTM based on Wasserstein autoencoder, which uses Beta distribution as prior distribution and combines reconstruction loss and Wasserstein distance loss to automatically discover and extract topics of social network events; The input of the model algorithm is the text data in the event. First, the text data is cleaned, segmented, and stop words are removed to obtain the bag-of-words representation. In each training cycle, the bag-of-words representation is encoded to obtain the topic vector of the document. The MMD loss is calculated by sampling from the latent variables using the Beta distribution. The reconstruction loss is then calculated. Finally, the overall loss is obtained, which is a linear combination of the reconstruction loss and the MMD loss. Finally, backpropagation is performed and the optimizer is used to update the parameters to optimize the EWTM model; this process is iterated until the specified number of training cycles is reached.

2. The network event topic mining method based on Wasserstein autoencoder according to claim 1, characterized in that: The EWTM model is implemented under the PyTorch framework, and its structure consists of two parts: Encoder and Decoder.

3. The network event topic mining method based on Wasserstein autoencoder according to claim 2, characterized in that: The encoder of the EWTM model consists of multiple linear layers and activation function layers, namely, a multi-layer perceptron MLP. First, the document W = {w1,w2,...,w m The encoder is mapped to an output layer with V dimensions. By looping through the layers of the encoder, a series of linear transformations are performed on the input, and the dropout operation is applied to obtain the latent variable Z. Then, the raw score of each unit in the output layer is converted into the topic vector θ∈S of the document w by applying the softmax function. V-1 The encoder approximately learns the topic probability distribution of document w, so that the approximated distribution Q(θ|w) is close to the true posterior distribution p(θ|w).

4. The network event topic mining method based on Wasserstein autoencoder according to claim 2, characterized in that: The main function of the decoder of the EWTM model is to map the topic representation θ in the latent space to the output layer, which produces the probability distribution of words Represents the probability of each word in the vocabulary appearing under a given topic; The calculation formula is as shown in (1): represents the probability estimate of the i-th word, h represents the output of the neuron, and the calculation formula is as follows (2), h=W·θ+b (2) W is the decoder weight matrix and b is the bias term.

5. The network event topic mining method based on Wasserstein autoencoder according to claim 1, characterized in that: Beta distribution is used as the prior distribution of the latent variable Z, and then sampling is performed to obtain Z' = {z i ′}~Beta(α,β), where α and β are the parameters of the Beta distribution, which affect the shape of the Beta distribution and thus adjust the prior information of the topic model.

6. The network event topic mining method based on Wasserstein autoencoder according to claim 1, characterized in that: The overall loss of the model consists of two parts: reconstruction loss L rec and MMD loss L MMD , the calculation formula is shown in (3): L=L rec +λ·L MMD (3) Among them, γ is a hyperparameter, which is mainly used to balance the weights so that features in different dimensions have different importance in subsequent calculations or applications; L rec is the reconstruction loss of the EWTM model, which calculates the original document w and the reconstructed The negative cross entropy loss is used to measure the difference between two probability distributions, L rec The calculation of is as follows: L MMD The difference between two probability distributions is measured by calculating the distance between them. The Gaussian kernel function is used to calculate the distance. The calculation formula of the Gaussian kernel is as follows (5): The formula of MMD loss is as follows (6): The MMD distance uses n samples for unbiased estimation, where {z1,z2,...,z n The distribution Q in the latent space generated from the encoder θ In the sampling, {z1′,z′2,...,z′ n }From the true distribution P of the latent variables given the input document w θ Medium sampling.

7. A network event topic mining system based on Wasserstein autoencoder according to any one of claims 1 to 6, characterized in that: Includes encoder and decoder; input is text data in the event, output is text vector, topic category, topic word; The encoder is composed of multiple linear layers and activation function layers, i.e., a multi-layer perceptron MLP; The decoder is used to map the topic representation θ in the latent space to the output layer, which produces the probability distribution of words Represents the probability of each word in the vocabulary appearing under a given topic.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the network event topic mining method based on Wasserstein autoencoder as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor executes the steps of the network event topic mining method based on Wasserstein autoencoder as described in any one of claims 1 to 6.

10. An information data processing terminal, characterized in that: The information data processing terminal includes the network event topic mining system based on Wasserstein autoencoder as described in claim 7.

Images