Interpretability-Enhanced Neural Topic Modeling Method, System and Device Based on Enhanced Network
By combining Gaussian encoding network and logical Sty coding network and introducing pre-trained word embeddings into Gaussian decoding network, the problem of insufficient training efficiency and interpretability of neural topic models is solved, and efficient topic extraction and feature sparse problem is achieved.
Patent Information
- Application Number
- CN202310153401.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-23
- Publication Date
- 2025-07-22
- Estimated Expiration
- 2043-02-23
AI Technical Summary
The existing neural theme models have shortcomings in training efficiency and interpretability, especially in short text sparse features, making it difficult to effectively explore implicit features.
The enhanced network method is adopted to combine the Gaussian coding network with the logical Sty coding network, enrich context information through pre-trained word embedding, integrate topic distribution, and improve the interpretability of the model.
It improves the training efficiency and interpretability of neural topic modeling, alleviates the problem of sparse features of short texts, and improves the quality and interpretability of topic extraction.
Smart Images
Figure CN116306584B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of text data mining, and relates to a neural topic modeling method, system and device, specifically to an interpretable neural topic modeling method, system and device based on an enhanced network. Background Art
[0002] With the popularization of the Internet, the information from social media has exploded, and the demand for automatically discovering specific information from massive text data has become particularly urgent. In order to accurately and efficiently obtain the content that users are interested in, text content platforms must first organize, explore and understand the data in a certain logic. The topic is such a widely used information organization logic, which has the advantage of being easy for humans to understand.
[0003] The topic model is an unsupervised learning method for discovering the latent semantic structure of text, and can be applied to various tasks in natural language processing, such as text classification, sentiment analysis and recommendation systems. According to its specific implementation methods, it can be divided into traditional topic models and neural topic models. The parameters of traditional topic models can be estimated by variational inference or Gibbs sampling, but when the model structure changes, both methods require complex mathematical derivations and the scalability of the model is poor. The neural topic model based on variational autoencoders can adjust parameters through backpropagation and has high training efficiency, but its interpretability is lower than that of traditional topic models, and there are feature sparsity problems when applied to short texts. Summary of the Invention
[0004] The purpose of the present invention is to propose an interpretable neural topic modeling method, system and device based on an enhanced network, and use the enhanced network and pre-trained word embeddings to improve the interpretability of the model.
[0005] The technical solution adopted by the method of the present invention is: an interpretable neural topic modeling method based on an enhanced network, including the following steps:
[0006] Step 1: Perform data preprocessing on the pre-analyzed document;
[0007] Step 2: Input the preprocessed text data into the Gaussian encoding network, use the word frequency vector of the document as the input of the encoder and map it to the Gaussian distribution space to obtain a topic distribution;
[0008] Step 3: Input the preprocessed text data into the logistic encoding network, use the word frequency vector of the document as the input of the encoder and map it to the logistic normal distribution space to obtain a topic distribution;
[0009] Step 4: Combine the topic distributions obtained by the two encoding networks through an enhanced fusion method to obtain an enhanced topic distribution;
[0010] Step 5: In the Gaussian decoding network, introduce the additional semantic knowledge of each word into the decoder through pre-trained word embeddings to enrich the context information and obtain the topic-word matrix;
[0011] Step 6: In the Gaussian decoding network, combine the enhanced topic distribution in Step 4 and the topic-word matrix in Step 5 to obtain the conditional probability of the words in the document and reconstruct the document.
[0012] The technical solution adopted by the system of the present invention is: An interpretable neural topic modeling system based on an enhanced network, including the following modules:
[0013] Module 1: Perform data preprocessing on the pre-analyzed document;
[0014] Module 2: Input the preprocessed text data into the Gaussian encoding network, that is, use the word frequency vector of the document as the input of the encoder and map it to the Gaussian distribution space to obtain a topic distribution;
[0015] Module 3: Input the preprocessed text data into the logistic encoding network, that is, use the word frequency vector of the document as the input of the encoder and map it to the logistic normal distribution space to obtain a topic distribution;
[0016] Module 4: Combine the topic distributions obtained by the two encoding networks through an enhanced fusion method to obtain an enhanced topic distribution;
[0017] Module 5: In the Gaussian decoding network, introduce the additional semantic knowledge of each word into the decoder through pre-trained word embeddings to enrich the context information and obtain the topic-word matrix;
[0018] Module 6: In the Gaussian decoding network, combine the enhanced topic distribution in Step 4 and the topic-word matrix in Step 5 to obtain the conditional probability of the words in the document and reconstruct the document.
[0019] Further, in the data preprocessing of Step 1, after reading the data set, removing stop words and tokenizing, a corpus is obtained, the words that have appeared in the corpus are counted, a vocabulary is constructed, and each document is processed into a bag-of-words vector representation for subsequent use.
[0020] Further, the specific process of Step 2 is as follows:
[0021] Taking the preprocessed document vector x d as the input, passing through a multi-layer perceptron network to obtain the encoded vector π d , and obtaining μ d , Σ d through linear transformation, and the two are used to parameterize the Gaussian distribution N(μ d , ∑d ), where MLP is a multi - layer perceptron network, and l1, l2 are linear transformations;
[0022] π d = MLP(x d )
[0023] μ d = l1(π d )
[0024] logσ d = l2(π d )
[0025]
[0026] Then, the topic distribution θ' is obtained through the following process:
[0027] ε d ~N(0, I 2 )
[0028] h d = μ d + ε d * σ d
[0029] θ' d = soft max(W θ ·h d + b θ )
[0030] The above formula defines a multivariate normal distribution, q(θ' d |x d ) = N(μ d , Σ d ) as an approximate distribution of the posterior distribution p(θ d ).
[0031] Furthermore, the logistic encoding network described in step 3 uses the strategy of Laplace approximation to approximately simulate the Dirichlet prior using the log - normal distribution. Its network structure includes four layers: a V - dimensional document representation layer, an H - dimensional semantic extraction layer, a K - dimensional re - parameterization layer, and a K - dimensional topic distribution layer;
[0032] First, the encoding network maps x d to the H - dimensional semantic representation space using the following transformation, where, is the weight matrix of the semantic extraction layer, b d is the bias term of the semantic extraction layer, and a d is the semantic representation vector of x d :
[0033] a d= softplus(W d ·x d + b d )
[0034] To further infer the topic distribution of document x d According to the idea of Laplace approximation, a multivariate Gaussian distribution is constructed. The encoding network uses the following non-linear transformation to map a d to two parameters μ and σ of the multivariate Gaussian distribution 2 , where W μ and W σ are reparameterized weight matrices, b μ and b σ are the corresponding bias terms, BN(·) is the Batch Normalization layer, and the outputs μ and σ 2 are the mean and the diagonal elements of the covariance matrix of the posterior Gaussian distribution corresponding to x d :
[0035] μ = BN(W μ ·a d + b μ )
[0036] σ 2 = exp(BN(W σ ·a d + b σ ))
[0037] Finally, through the Laplace approximation module, the encoding network infers the corresponding topic distribution θ” for x d by the following formula:
[0038] θ” = Laplace_Approximation(μ, σ)
[0039] The posterior distribution q(θ”|x d ) obtained through the above transformation approximately follows the Dirichlet distribution.
[0040] Furthermore, the enhancement network described in step 4 obtains the Gaussian network topic distribution θ' through step 2 and the logistic network topic distribution θ” through step 3. The similarity between the two distributions is measured by the following two different metrics:
[0041] o1 = cos(θ', θ”)
[0042]
[0043] where o1 is measured based on cosine similarity and o2 is measured based on KL divergence.
[0044] Construct an enhanced network based on the similarity of two distributions. The final fusion calculation formula for the enhanced distribution θ is as follows:
[0045]
[0046] Furthermore, the Gaussian decoding network that introduces word embeddings described in step 5:
[0047] First, use the pre-trained word vectors to represent the words in the vocabulary as a word vector matrix where r represents the dimension of the word embedding, that is, the embedding vector of the i-th word is represented as WE i ;
[0048] To further obtain a topic-word matrix with rich context, the decoding network uses a multivariate Gaussian distribution N(μ K , ∑ K ) to model the k-th topic in the embedding space. Through the following calculations, a topic-word matrix that introduces word embeddings is obtained, where μ k and ∑ k are the topic centroid and topic concentration in the embedding space respectively:
[0049]
[0050]
[0051] Furthermore, in step 6, reconstruct the document, combine the enhanced topic distribution and the topic-word matrix that introduces word embeddings,
[0052] to obtain the conditional probability of the words in the document for topic extraction:
[0053] p(w d,i |θ d ) = ∑ k θ d,k ·TW (k,i)
[0054] Based on the same inventive concept, the present invention also designs an interpretable neural topic model structure based on an enhanced network, including: a preprocessing module that performs data preprocessing on the pre-analyzed document;
[0055] The first topic distribution module inputs the preprocessed text data into a Gaussian encoding network, that is, uses the word frequency vector of the document as the input of the encoder and maps it to the Gaussian distribution space to obtain a topic distribution;
[0056] The second topic distribution module inputs the preprocessed text data into a logistic encoding network, that is, uses the word frequency vector of the document as the input of the encoder and maps it to the logistic normal distribution space to obtain a topic distribution;
[0057] A fusion module that combines the topic distributions obtained from two encoding networks through a weighted fusion method to obtain an enhanced topic distribution;
[0058] A decoding module that introduces additional semantic knowledge of each word into the decoder through pre-trained word embeddings to enrich the context information and obtain a topic-word matrix; in the Gaussian decoding network, combines the enhanced topic distribution in step 4 and the topic-word matrix in step 5 to obtain the conditional probability of the words in the document and reconstruct the document.
[0059] Based on the same inventive concept, the present invention also designs an electronic device, including:
[0060] One or more processors;
[0061] A storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement an interpretable neural topic modeling method based on an enhanced network.
[0062] Based on the same inventive concept, the present invention also designs a computer-readable medium having a computer program stored thereon, and when the program is executed by a processor, it implements an interpretable neural topic modeling method based on an enhanced network.
[0063] The advantages of the present invention are as follows:
[0064] (1) The network model trained on the entire training set can achieve remarkable results in text classification.
[0065] (2) The network model trained on the entire training set can achieve remarkable results in topic extraction.
[0066] (3) Through the enhanced network, the implicit features in short texts can be explored, and the interpretability of topic extraction can be improved.
[0067] (4) By introducing pre-trained word embeddings into the Gaussian decoding network, integrating single-word semantic information, alleviating the feature sparsity problem of short texts, and improving the interpretability of the model.
[0068] This method uses a neural topic model based on a variational autoencoder, which can adjust parameters through backpropagation, has high training efficiency, low cost, strong generalization ability, and good scalability. In addition, based on the Gaussian encoding network, a logistic encoding network is used for topic modeling to simulate the Dirichlet prior of traditional topic models, and the two are fused through an enhanced network to explore the implicit features in short texts and improve the quality of topic extraction; and pre-trained word embeddings are introduced to enrich the context semantic information and integrate additional semantic information in the decoding network, which can effectively improve the interpretability of the model and alleviate the feature sparsity problem. Brief Description of the Drawings
[0069] Figure 1 is a flowchart of the method according to an embodiment of the present invention;
[0070] Figure 2 is a model structure diagram according to an embodiment of the present invention;
[0071] Figure 3 is a Gaussian coding network structure diagram in an embodiment of the present invention;
[0072] Figure 4 is a logistic coding network structure diagram in an embodiment of the present invention;
[0073] Figure 5 is an enhanced network structure diagram in an embodiment of the present invention;
[0074] Figure 6 is a decoding network structure diagram with word embedding introduced in an embodiment of the present invention. Detailed Description of the Invention
[0075] To facilitate the understanding and implementation of the present invention by those of ordinary skill in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not intended to limit the present invention.
[0076] As shown in the attached Figure 1 、 Figure 2 figures, a method for interpretable neural topic modeling based on an enhanced network provided by the present invention includes the following steps:
[0077] Step 1: Perform data preprocessing on the pre-analyzed document;
[0078] In this embodiment, the specific implementation of Step 1 includes the following sub-steps:
[0079] Step 1.1: Obtain a public data set containing d documents;
[0080] Step 1.2: Read the data set, remove stop words and perform word segmentation to obtain a corpus D;
[0081] Step 1.3: Count the words that have appeared in the corpus D and construct a vocabulary W, where V is the size of the vocabulary;
[0082] W = {w1, w2,..., w V}
[0083] Step 1.4: Process each document into a bag-of-words vector x d , where x d,i represents the frequency of occurrence of the i-th word in document d;
[0084] xd = [x d,1 , x d,2 ,..., x d,V
[0085] Step 2: Input the preprocessed text data into the Gaussian encoding network, that is, use the term frequency vector of the document as the input of the encoder and map it to the Gaussian distribution space to obtain a topic distribution;
[0086] As shown in the Gaussian encoding network structure diagram, using the preprocessed document vector x Figure 3 as the input, through a multi-layer perceptron network, the encoded vector π d is obtained, and μ d is obtained through a linear transformation, and ∑ d . Both are used to parameterize the Gaussian distribution N(μ d , ∑ d ), where MLP is the multi-layer perceptron network, and l1, l2 are linear transformations; d )
[0087] π d = MLP(x d )
[0088] μ d = l1(π d )
[0089] logσ d = l2(π d )
[0090]
[0091] Then, the topic distribution θ' is obtained through the following process:
[0092] ε d ~ N(0, I 2 )
[0093] h d = μ d + ε d * σ d
[0094] θ' d = soft max(W θ · h d + b θ )
[0095] The above formula defines a normal distribution and uses the reparameterization trick for sampling, that is, using the random variable ε d to represent randomness, which follows a standard normal distribution, and then transforming this random variable to allow optimization of the parameters through backpropagation. q(θ'd |x d ) = N(μ d , ∑ d ) as the approximate distribution of the posterior distribution p(θ d ), where W θ is the weight and b θ is the bias;
[0096] Step 3: Input the preprocessed text data into the logistic encoding network, that is, use the word frequency vector of the document as the input of the encoder and map it to the logistic normal distribution space to obtain a topic distribution;
[0097] As Figure 4 , the structure diagram of the logistic encoding network in this embodiment. The logistic encoding network uses the strategy of Laplace approximation to approximately simulate the Dirichlet prior in the traditional topic model with the logistic normal distribution. Its network structure includes four layers (V-dimensional document representation layer, H-dimensional semantic extraction layer, K-dimensional reparameterization layer, and K-dimensional topic distribution layer);
[0098] First, the encoding network uses the following transformation to map x d to the H-dimensional semantic representation space, where is the weight matrix of the semantic extraction layer, b d is the bias term of the semantic extraction layer, and a d is the semantic representation vector of x d :
[0099] a d = softplus(W d ·x d + b d )
[0100] To further infer the topic distribution of document x d , according to the idea of Laplace approximation, a multivariate Gaussian distribution is constructed. The encoding network uses the following non-linear transformation to map a d to the two parameters μ and σ 2 of the multivariate Gaussian distribution, where W μ and W σ are the reparameterized weight matrices, b μ and b σ are the corresponding bias terms, and BN(·) is the batch normalization layer, and the outputs μ and σ 2 are the mean and the diagonal elements of the covariance matrix of the posterior Gaussian distribution corresponding to x d :
[0101] μ = BN(W μ ·a d + b μ )
[0102] σ 2 = exp(BN(W σ ·a d +b σ ))
[0103] Finally, through the Laplace approximation module, the encoding network infers the corresponding topic distribution θ" for x d using the following formula:
[0104] θ" = Laplace_Approximation(μ, σ)
[0105] The posterior distribution q(θ"|x d ) obtained through the above transformation approximately follows the Dirichlet distribution;
[0106] Step 4: Combine the topic distributions obtained from the two encoding networks through an enhancement fusion method to obtain an enhanced topic distribution; the enhancement methods include, for example, a confidence-based fusion method: set a confidence function to fuse the two networks. Feature fusion methods: early fusion and late fusion. Early fusion first fuses features from multiple layers and then trains a predictor on the fused features; late fusion improves the detection performance by combining detection results from different layers.
[0107] As shown in the attached Figure 5 , the enhanced network structure diagram of this example. The Gaussian network topic distribution θ' is obtained through Step 2, and the logistic network topic distribution θ" is obtained through Step 3. The similarity (correlation) between the two distributions is measured by the following two different metrics:
[0108] o1 = cos(θ', θ")
[0109]
[0110] where o1 is measured based on cosine similarity, and o2 is measured based on KL divergence. KL divergence, also known as relative entropy, describes the difference or similarity between two probability distributions and is a metric used to measure the similarity between two probability distributions.
[0111] D KL is the KL divergence value between θ' and θ", that is, the similarity measurement result of the two topic distributions.
[0112] Construct an enhanced network based on the similarity (correlation) between the two distributions. The final fusion calculation formula for the enhanced distribution θ is as follows:
[0113]
[0114] Step 5: In the Gaussian decoding network, introduce the additional semantic knowledge of each word into the decoder through pre-trained word embeddings to enrich the context information and obtain the topic-word matrix;
[0115] As shown in the Figure 6 accompanying figure, the structure diagram of the Gaussian decoding network with word embeddings introduced in this example. In the Gaussian decoding network, pre-trained word vectors are embedded to introduce additional semantic information, enrich the context, and obtain the topic-word matrix;
[0116] First, use the pre-trained word vectors to represent the words in the vocabulary as a word vector matrix where r represents the dimension of the word embedding, that is, the embedding vector of the i-th word is represented as WE i ;
[0117] To further obtain the topic-word matrix with rich context, the decoding network uses the multivariate Gaussian distribution N(μ K , ∑ K ) to model the k-th topic in the embedding space. Through the following calculations, obtain the topic-word matrix with word embeddings introduced, where μ k and ∑ k are the topic centroid and topic concentration in the embedding space respectively:
[0118]
[0119]
[0120] Step 6: In the Gaussian decoding network, combine the enhanced topic distribution in Step 4 and the topic-word matrix with word embeddings introduced in Step 5 to obtain the conditional probability of the words in the document and reconstruct the document.
[0121] As shown in Figure 6 the figure, for the decoding network structure diagram in this embodiment, the decoding network aims to reconstruct the bag-of-words vector representation of the original document with θ as the input.
[0122] In this embodiment, by combining the pre-trained word embeddings to enrich the context information, the conditional probability that the i-th word in document d belongs to k topics is obtained through the following formula:
[0123] p(w d,i |θ d ) = ∑ k θ d,k ·TW (k,i)
[0124] According to the properties of the Gaussian distribution, words closer to the center of the theme have a higher probability, and the distances between these words are relatively small and their semantics are similar, indicating that they are more likely to reflect the same theme. Therefore, enriching the context information through pre-trained word embeddings can further alleviate the problem of feature sparsity and thus improve the interpretability of the model.
[0125] Based on the same inventive concept, the present invention also designs a highly interpretable neural theme model structure based on an enhanced network, including: a preprocessing module for performing data preprocessing on the pre-analyzed document;
[0126] A first theme distribution module that inputs the preprocessed text data into a Gaussian encoding network, that is, uses the word frequency vector of the document as the input of the encoder and maps it to the Gaussian distribution space to obtain a theme distribution;
[0127] A second theme distribution module that inputs the preprocessed text data into a logistic encoding network, that is, uses the word frequency vector of the document as the input of the encoder and maps it to the logistic normal distribution space to obtain a theme distribution;
[0128] A fusion module that combines the theme distributions obtained by the two encoding networks through a weighted fusion method to obtain an enhanced theme distribution;
[0129] A decoding module that introduces additional semantic knowledge of each word into the decoder through pre-trained word embeddings to enrich the context information and obtain a theme-word matrix; in the Gaussian decoding network, combining the enhanced theme distribution in step 4 and the theme-word matrix in step 5 to obtain the conditional probability of the words in the document and reconstruct the document.
[0130] Based on the same inventive concept, the present invention also designs an electronic device, including:
[0131] One or more processors;
[0132] A storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement a highly interpretable neural theme modeling method based on an enhanced network.
[0133] Based on the same inventive concept, the present invention also designs a computer-readable medium, on which a computer program is stored, and when the program is executed by a processor, it implements a highly interpretable neural theme modeling method based on an enhanced network.
[0134] The method of the present invention is to perform semantic mining on massive text data. First, text data is obtained and preprocessed. After removing stop words and tokenizing the data, a vocabulary is constructed, and the text data is represented by a bag-of-words vector. The preprocessed data is respectively input into a Gaussian encoding network and a logistic encoding network to obtain two topic distributions. An enhancement network is constructed to merge the two topic distributions to obtain a feature-enhanced topic distribution. Then, in the Gaussian decoding network, pre-trained word vectors are embedded, and additional semantic information is introduced to enrich the context to obtain a topic-word matrix. Finally, by merging the topic distribution and the pre-trained word embedding, the conditional probability of words in the document is obtained, and the document is reconstructed, and then topic extraction is performed. The present invention can achieve topic extraction of massive text data and obtain topics with high quality and strong interpretability. The present invention adopts the method of an enhancement network, combines the Gaussian encoding network with the logistic encoding network, explores the implicit features in short texts, and obtains an enhanced topic distribution. By adopting the method of introducing pre-trained word embeddings in the Gaussian decoder, the context information is enriched, and additional semantic knowledge is fused, which can alleviate the problem of feature sparsity and improve the interpretability of the model.
[0135] It should be understood that the above description of the preferred embodiment is relatively detailed, and thus it should not be considered as limiting the protection scope of the invention patent of the present invention. Under the inspiration of the present invention, those of ordinary skill in the art can also make substitutions or modifications without departing from the scope protected by the claims of the present invention, and all fall within the protection scope of the present invention. The scope of protection requested by the present invention shall be subject to the appended claims.
Claims
1. An interpretable neural topic modeling method based on an enhanced network, characterized in that It includes the following steps: Step 1: Perform data preprocessing on the pre-analyzed document; Step 2: Input the preprocessed text data into the Gaussian encoding network, that is, use the word frequency vector of the document as the input of the encoder and map it to the Gaussian distribution space to obtain a topic distribution; Step 3: Input the preprocessed text data into the Logistic encoding network, that is, use the word frequency vector of the document as the input of the encoder and map it to the Logistic normal distribution space to obtain a topic distribution; Step 4: Combine the topic distributions obtained by the two encoding networks through an enhanced fusion method to obtain an enhanced topic distribution; Step 5: Use the Gaussian decoding network for decoding, introduce the additional semantic knowledge of each word into the decoder through pre-trained word embeddings to enrich the context information, and obtain a topic-word matrix; Step 6: In the Gaussian decoding network, combine the enhanced topic distribution in Step 4 and the topic-word matrix in Step 5 to obtain the conditional probability of the words in the document and reconstruct the document.
2. The method for interpretable neural topic modeling based on an enhanced network according to claim 1, wherein: In the data preprocessing in Step 1, after reading the data set, removing stop words and tokenizing, a corpus is obtained, the words that have appeared in the corpus are counted, a vocabulary is constructed, and each document is processed into a bag-of-words vector representation for subsequent use.
3. The method for interpretable neural topic modeling based on an enhanced network according to claim 1, characterized in that: The specific process of Step 2 is as follows: Using the preprocessed document vector as the input, passing through a multi-layer perceptron network to obtain the encoded vector , and obtaining and through linear transformation, where both are used to parameterize the Gaussian distribution , where MLP is the multi-layer perceptron network, and are linear transformations; Then, the topic distribution is obtained through the following process : The above formula defines a multivariate normal distribution, as an approximate distribution for the posterior distribution where is the weight, and is the bias.
4. The method for interpretable neural topic modeling based on an enhanced network according to claim 1, wherein: The Logistic encoding network described in Step 3 uses the strategy of Laplace approximation to approximately simulate the Dirichlet prior using the Logistic normal distribution. Its network structure includes four layers, a V-dimensional document representation layer, an H-dimensional semantic extraction layer, a K-dimensional reparameterization layer, and a K-dimensional topic distribution layer; First, the encoding network uses the following transformation to map to an H-dimensional semantic representation space, where is the weight matrix of the semantic extraction layer, is the bias term of the semantic extraction layer, is 's semantic representation vector: For further inferring the topic distribution of the document According to the idea of Laplace approximation, a multivariate Gaussian distribution is constructed. The encoding network uses the following non-linear transformation to map to two parameters of the multivariate Gaussian distribution and , where and are the weight matrices for reparameterization, and are the corresponding bias terms, BN(·) is the batch normalization layer, and the outputs and are the diagonal elements of the mean and covariance matrix of the corresponding posterior Gaussian distribution: Finally, through the Laplace approximation module, the encoding network infers the corresponding topic distribution through the following formula for and : The posterior distribution obtained through the above conversion will approximately follow a Dirichlet distribution.
5. The method for interpretable neural topic modeling based on an enhanced network according to claim 1, wherein: The enhanced fusion method described in step 4 obtains the Gaussian network topic distribution through step 2 and obtains the logistic network topic distribution through step 3 The similarity between the two distributions is measured by the following two different metrics: Among them, Measured based on cosine similarity, Measured based on KL divergence, which is also known as relative entropy and describes the difference or similarity between two probability distributions. It is a metric used to measure the similarity between two probability distributions, Is And The KL divergence value of, that is, the similarity measurement result of the two topic distributions; Construct an enhanced network based on the similarity of two distributions, and finally the fusion calculation formula of the enhanced distribution is as follows: 。 6. The method for interpretable neural topic modeling based on an enhanced network according to claim 1, wherein: The Gaussian decoding network described in Step 5: First, use the pre-trained word vectors to represent the words in the vocabulary as a word vector matrix , where represents the dimension of the word embedding, that is, the embedding vector of the i-th word is represented as ; To further obtain a theme-word matrix with rich context, the decoding network uses a multivariate Gaussian distribution , models the k-th theme in the embedding space, and obtains a theme-word matrix introducing word embeddings through the following calculations, where and are the theme centroid and theme concentration of the embedding space respectively: 。 7. The method for interpretable neural topic modeling based on an enhanced network according to claim 1, wherein: Step 6 reconstructs the document, combines the enhanced topic distribution and the topic-word matrix with introduced word embeddings, and obtains the conditional probability of the words in the document for topic extraction: 。 8. An interpretable neural topic model system based on an enhanced network, characterized in that, It includes: A preprocessing module that performs data preprocessing on the pre-analyzed document; A first topic distribution module that inputs the preprocessed text data into the Gaussian encoding network, that is, uses the word frequency vector of the document as the input of the encoder and maps it to the Gaussian distribution space to obtain a topic distribution; A second topic distribution module that inputs the preprocessed text data into the Logistic encoding network, that is, uses the word frequency vector of the document as the input of the encoder and maps it to the Logistic normal distribution space to obtain a topic distribution; A fusion module that combines the topic distributions obtained by the two encoding networks through a weighted fusion method to obtain an enhanced topic distribution; A decoding module that introduces the additional semantic knowledge of each word into the decoder through pre-trained word embeddings to enrich the context information and obtain a topic-word matrix; In the Gaussian decoding network, combine the enhanced topic distribution in Step 4 and the topic-word matrix in Step 5 to obtain the conditional probability of the words in the document and reconstruct the document.
9. An electronic device, characterized in that, It includes: One or more processors; A storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the method according to any one of claims 1 to 7.
10. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the method according to any one of claims 1-7.