A seed topic modeling method based on word vector semantic guidance

Abstract

The application relates to a seed topic modeling method based on word vector semantic guidance, and belongs to the technical field of natural language processing and text mining. The method acquires a target corpus, a word vector model and a seed word set corresponding to each topic, calculates semantic similarity between each word in a word table and each seed word set, constructs a semantic similarity field through threshold screening, introduces a semantic enhancement factor controlled by a similarity enhancement coefficient on the basis of a Gibbs sampling conditional probability, forms an enhanced sampling distribution, performs exploration sampling or uses sampling according to an exploration probability, and iteratively updates topic assignment counts, estimates topic-word distribution and document-topic distribution after sampling is completed, and outputs topic representative words and document topic compositions. The application can expand the seed semantic guidance range, realize continuous adjustment of guidance intensity, reduce the risk of sampling chain locking, and improve the semantic consistency and operation stability of topic modeling.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and text mining technology, specifically involving a seed topic modeling method based on word vector semantic guidance. In particular, it involves a method that constructs a semantic similarity field corresponding to seed words and introduces semantic similarity into the folded Gibbs sampling process. It also combines exploratory sampling and utilization sampling to achieve topic discovery in large-scale document sets. This method can be applied to scenarios such as news aggregation, digital library document organization, and customer feedback text analysis. Background Technology

[0002] As the volume of text data generated by information systems such as news platforms, digital libraries, and enterprise customer feedback platforms continues to expand, relying on manual reading and classification of each document is insufficient to meet the needs of large-scale document organization, content discovery, and semantic analysis. Topic models can automatically discover potential topics from a large number of documents and describe each topic with a few representative words, enabling users to quickly grasp the content composition of the document collection.

[0003] Latent Dirichlet Allocation (LDA) is a commonly used topic modeling method. LDA models can represent a document as a probability mixture of multiple topics and each topic as a probability distribution of multiple words. During model inference, the folded Gibbs sampling method can be used to gradually estimate the document-topic distribution and topic-word distribution by continuously adjusting the topic assignment results of each lexical unit in the corpus.

[0004] Traditional LDA models primarily rely on the co-occurrence statistics of words to identify topics, classifying them based on the common occurrence of words in a document. However, this approach cannot directly utilize the semantic information inherent in the words themselves. Therefore, in some corpora, while topics obtained through traditional LDA models may exhibit certain statistical regularities, the semantic correlation between words within the same topic may be low. This leads to discrepancies between the obtained topic structure and the semantic categories understood by humans, impacting the processing performance of subsequent tasks such as text classification and clustering.

[0005] To improve the controllability of topic models, existing technologies have also proposed seed-guided topic models. For example, seed-guided LDA allows users to pre-define several seed words for different topics and adjust the model's prior parameters to make the specified seed words more likely to be assigned to the corresponding topics. This approach can introduce human prior knowledge into the topic modeling process to a certain extent.

[0006] However, existing seed guidance methods typically focus primarily on the pre-specified seed word itself. For other words with similar semantics but not directly designated as seed words, it's difficult to obtain corresponding semantic guidance. For example, when the seed word is "encryption," related words like "decryption" and "ciphertext" cannot automatically receive guidance based solely on this semantic association, resulting in a limited vocabulary coverage for seed guidance. Existing SeededLDA seed guidance only applies to the seed word itself, with fixed guidance strength and a lack of mechanisms for handling suboptimal seed words.

[0007] On the other hand, in the folded Gibbs sampling process, the topic assignment result of the current lexical depends on the assignment count formed by previous sampling. When incorrect topic assignments are formed in the early stage of sampling due to inaccurate seed word selection or strong randomness of the initial count, these incorrect topic assignments will further affect the subsequent sampling probability, causing errors to accumulate and reinforce themselves, which may lead to the sampling process being fixed in an undesirable topic structure. This invention refers to this phenomenon as sampling chain locking.

[0008] Therefore, how to extend the seed word guidance range to words that are semantically related to the seed word while retaining the relatively transparent and efficient characteristics of the folded Gibbs sampling calculation process, so that the seed guidance intensity can be continuously adjusted, and at the same time retain the ability to correct undesirable seed anchors and initial misassignments during the sampling process, is a problem that existing seed-guided topic modeling techniques need to solve. Summary of the Invention

[0009] In view of this, the purpose of this invention is to provide a seed topic modeling method based on word vector semantic guidance, so as to solve the problems of limited seed guidance coverage, difficulty in flexibly adjusting semantic guidance intensity, and sampling chain locking caused by imperfect seed anchor points in existing seed topic modeling methods.

[0010] This invention utilizes word vectors to represent the semantic relationships between words, establishes a semantic similarity field covering the vocabulary based on the seed word set corresponding to each topic, and introduces semantic similarity into the conditional probability of folded Gibbs sampling as a multiplicative enhancement factor. Furthermore, by controlling the switching between exploratory sampling and utilization sampling through exploration probability, a random exploration path is always maintained during topic sampling. This allows the use of seed semantic information to guide topic formation while preserving the competition and adjustment capabilities between topics.

[0011] To achieve the above objectives, this invention provides a seed topic modeling method based on word vector semantic guidance, comprising the following steps.

[0012] S1. Obtain the target corpus and word vector model, and determine the total number of topics to be constructed. Obtain the seed word set corresponding to each topic, and set the prior parameters and similarity enhancement coefficient of the implicit Dirichlet assignment model. Similarity threshold and exploration probability p .

[0013] S2, for each word in the vocabulary of the target corpus. Determine the words respectively The semantic similarity between the seed word set corresponding to each topic and the similarity threshold is calculated. Semantic similarity is filtered to obtain a semantic similarity field covering the vocabulary and each topic, and words are determined based on the filtered semantic similarity. The corresponding set of candidate topics.

[0014] S3 involves folded Gibbs sampling of lexical units in the target corpus. Based on the conditional probabilities of the latent Dirichlet assignment model, the sampling is performed according to the semantic similarity between the current lexical unit and each topic, as well as the similarity enhancement coefficient. Determine the semantic enhancement factor and use it to multiplicatively adjust the conditional probability to obtain the enhanced sampling distribution of the current lexical corresponding to each topic.

[0015] S4, for the current word element, when its corresponding candidate topic set is not empty, explore the probability. p Perform exploratory sampling, randomly select a topic from the candidate topic set, and assign a probability of 1- p The process involves performing sampling and selecting topics according to the augmented sampling distribution. When the corresponding candidate topic set is empty, the process continues to select topics according to the augmented sampling distribution, update the topic assignment count based on the selected topics, and iteratively perform topic sampling on the lexical units in the target corpus.

[0016] S5. After completing topic sampling, estimate the topic-word distribution and document-topic distribution based on topic assignment counts, and output the representative words of each topic and the topic composition of each document based on the topic-word distribution and document-topic distribution.

[0017] Furthermore, in S1, the word vector model can be a publicly released pre-trained word vector model, or it can be trained using the target corpus. Regarding the... k Each theme sets a seed keyword set ,in, , K The total number of topics to be built.

[0018] The prior parameters of the implicit Dirichlet allocation model include and ; and Used to control the smoothness of topic distribution; This is the similarity enhancement coefficient, used to control the strength of semantic guidance. This is a similarity threshold used to determine whether a word and its corresponding topic form a semantic association; p The probability of exploration is used to control the probability of performing exploratory sampling during topic sampling.

[0019] Furthermore, in S2, when the word w Belongs to the k Seed keyword set for each topic At that time, the words w With seed word set The semantic similarity between them is set as follows:

[0020] When words w Not part of the seed word set At that time, calculate the words separately. w Word vectors and seed word set The cosine similarity between the word vectors of each seed word is calculated, and the maximum value is determined from the multiple cosine similarity results.

[0021] in, Indicator w With the k Seed keyword set for each topic The semantic similarity between them; s represents the seed word set. Seed words in; Indicator w The corresponding word vectors; This represents the word vector corresponding to the seed word s; Word vectors With word vectors Cosine similarity between them.

[0022] When the semantic similarity is obtained Less than the similarity threshold When the corresponding semantic similarity is set to 0, it indicates that the word... w With the k The topics do not constitute a semantic association that meets the similarity threshold requirement.

[0023] By performing the above calculations on each word in the vocabulary and all topics, a semantic similarity field covering the entire vocabulary and all topics is obtained, which extends the guiding information of the seed word from the seed word itself to other words that are semantically related to the seed word.

[0024] Furthermore, words wCorresponding candidate topic set Represented as:

[0025] Candidate topic set Including words w There exists a similarity threshold. The required semantic relationships among the various topics. When When empty, it indicates the current word. w There are no similarity thresholds between it and the seed word set of any topic. The required semantic association.

[0026] Furthermore, in S3, for the lexical units in the target corpus Let the word element be The corresponding vocabulary index is t And word elements Belongs to document m Then the word element Assigned to the k The enhanced conditional probabilities of each topic are:

[0027] in, Indicates the current word element The corresponding topic assignment results; Indicates exclusion of the current word element The subsequent topic assignment results; This represents the word data in the target corpus; and These are the prior parameters of the implicit Dirichlet assignment model; The similarity enhancement coefficient; Indicates the word corresponding to the current lexical unit. w With the k Seed word set for each topic Semantic similarity between them.

[0028] Indicates exclusion of the current word element After that, the document m Zhong was assigned to the first k The number of lexical units for each topic; Indicates exclusion of the current word element Then, the word with the thesaurus index t is assigned to the t-th word. k Number of times for each topic; Indicates exclusion of the current word element Later, he was assigned to the first k The total number of terms in each topic; V Indicates the size of the vocabulary corresponding to the target corpus; superscript This indicates that the current word is excluded when calculating the corresponding statistic. .

[0029] in, It is a semantic enhancement factor.

[0030] When words w With the k As the semantic similarity between the seed word sets corresponding to each topic increases, the semantic enhancement factor also increases, thereby increasing the likelihood of the current word being assigned to the next topic. k The relative probability of each topic. When When the semantic enhancement factor is equal to 1, it will not produce additional semantic bias on the original conditional probability of the implicit Dirichlet allocation model.

[0031] By changing the similarity enhancement coefficient λ, the influence of word vector semantic information on the topic sampling process can be continuously adjusted, while retaining the original probability terms formed by the co-occurrence relationships of words in the target corpus. Therefore, semantic information does not replace the original corpus co-occurrence information, but rather performs a multiplicative adjustment based on the original sampling distribution.

[0032] Furthermore, in S4, for the current word to be sampled, the candidate topic set is first determined based on the word w corresponding to the current word. .

[0033] When candidate topic set When not empty, based on the exploration probability p Determine whether the current sampling process should perform exploratory sampling or exploitation sampling.

[0034] When performing exploratory sampling, from the candidate topic set A topic is randomly selected uniformly from the included topics, and the selected topic is used as the topic assignment result for the current word unit.

[0035] When performing sampling, multinomial sampling is performed based on the enhanced sampling distribution of each topic corresponding to the current lexical determined by S3, and the topic assignment result of the current lexical is determined based on the sampling result.

[0036] The probability of performing exploratory sampling is p The execution probability of sampling is 1- p .

[0037] When candidate topic set When empty, topic sampling is performed directly according to the enhanced sampling distribution obtained in S3. Since the current word... wThe semantic similarity between the seed word sets corresponding to all topics is set to 0. Therefore, the semantic enhancement factor for each topic is 1, and the corresponding enhancement sampling distribution degenerates into an LDA sampling distribution that does not contain semantic enhancement.

[0038] After selecting a topic for the current lexical unit, the corresponding topic assignment count is updated based on the selected topic, and topic sampling continues for other lexical units. Through multiple rounds of iterative updates, stable topic assignment results are gradually obtained.

[0039] The aforementioned exploratory sampling ensures that a random exploration path exists throughout the sampling process. In cases where manually determined seed words are biased, exploratory sampling maintains a competitive relationship among multiple semantically related topics, reducing the likelihood of a single seed anchor continuously controlling the topic assignment results for related lemmas.

[0040] Especially in the early stages of sampling, since topic assignment counting still has a strong randomness, when undesirable seed words or initial misassignments affect subsequent counting, exploratory sampling can continuously provide the possibility of different topics being selected, thereby weakening the sampling chain lock caused by the continuous self-reinforcement of misassignments.

[0041] Compared to the approach of running a standard LDA model for several rounds before initiating seed guidance at a preset time, this invention follows the exploration probability throughout the entire topic sampling process. p The exploration branch is retained, so there is no need to predetermine the timing of semantic guidance switching for different target corpora.

[0042] Furthermore, in S5, after completing the predetermined rounds of topic sampling, the topic-word distribution is estimated based on the obtained topic assignment counts.

[0043] No. k The first topic in the thesaurus t The topic-word distribution of each word is as follows:

[0044] in, Indicates the first k The thesaurus under the topic number t The probability of each word; The term index is indicated as t The word was assigned to the first k Number of times for each topic; Indicates being assigned to the k The total number of terms in each topic; These are the corresponding prior parameters; V This refers to the size of the vocabulary.

[0045] Estimating document-topic distribution based on topic assignment count:

[0046] in, Document m Belongs to the k The proportion of each theme; Document m Zhong was assigned to the first k The number of lexical units for each topic; Document m The total number of lexical units in the text; These are the corresponding prior parameters; K The total number of topics.

[0047] Based on the topic-word distribution, a list of representative words for each topic can be determined according to the probability of each word under the corresponding topic; based on the document-topic distribution, the proportion of different documents corresponding to each topic can be determined, and the topic composition of each document can be formed.

[0048] In a preferred embodiment, Set to 0.1, Set to 0.01, similarity enhancement coefficient The similarity threshold is set to 0.5–2.0, more preferably 1.0–2.0; Set to 0.4-0.5; exploration probability p The value is set to 0 to 0.2, more preferably 0.1; the number of seed words for each topic is set to 1 to 8, more preferably 3.

[0049] Compared with the prior art, the present invention has at least the following beneficial effects.

[0050] (1) The semantic similarity between each word in the vocabulary and the seed word set corresponding to different topics is determined by word vectors, and the similarity threshold is used for screening to form a semantic similarity field covering the vocabulary. As a result, the guiding role of seed words is no longer limited to the seed words given by humans directly, but can be propagated to other words that have semantic relationships with the seed words, which is conducive to improving the consistency between the obtained topic structure and the artificial semantic category.

[0051] (2) This invention introduces the following on the basis of the traditional folded Gibbs sampling conditional probability. The multiplicative semantic enhancement factor, while preserving the original word co-occurrence evidence in the target corpus, adjusts the topic sampling probability using word vector semantic information. By changing the similarity enhancement coefficient λ, the influence of semantic information on topic sampling can be continuously adjusted, thus overcoming the problem of the difficulty in adjusting the guidance strength of traditional seed-guided methods.

[0052] (3) Using the probability of exploration pA hybrid sampling approach combining controlled exploratory sampling and exploitation sampling preserves random exploration paths throughout the topic sampling process, eliminating the need for manual determination of the transition point from regular sampling to seed-guided sampling. Exploratory sampling maintains competition among multiple related topics and reduces the probability of accumulating and self-reinforcing misassignments in the early stages of sampling, thereby improving the operational stability of the topic model.

[0053] (4) The semantic enhancement of the present invention is achieved by multiplicative enhancement factor on the basis of the original folded Gibbs sampling probability, without changing the basic inference framework of the original topic model. Therefore, it can maintain a low additional computational cost while introducing semantic guidance capability.

[0054] (5) Experimental results show that on the three corpora of 20 Newsgroups, Reuters-21578 and Web of Science, the present invention achieves good overall results in classification, clustering and topic consistency evaluation, and the fluctuation of multiple independent runs is lower than that of the ordinary LDA model.

[0055] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the overall process of the seed topic modeling method based on word vector semantic guidance of the present invention; Figure 2 This is a schematic diagram of the decision-making process for the hybrid sampling process of the present invention; Figure 3 A comparison chart showing the classification accuracy of this invention and the comparison method on a 20 Newsgroups corpus; Figure 4 A comparison chart of the classification accuracy of this invention and the comparison method on the Reuters-21578 corpus; Figure 5 This is a comparison chart showing the classification accuracy of this invention and the comparison method on the Web of Science corpus. Detailed Implementation

[0057] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0058] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0059] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0060] like Figure 1 As shown, this embodiment provides a seed topic modeling method based on word vector semantic guidance. It constructs a topic model using a target corpus, a word vector model, and seed word sets tailored to different topics. The entire process includes system initialization, semantic similarity field calculation, augmented sampling probability calculation, hybrid sampling, and parameter estimation and output.

[0061] First, execute S1 to initialize the topic modeling process.

[0062] Obtain the target corpus to be processed and build a vocabulary based on the target corpus. Obtain a word vector model that can represent the semantic information of each word in the vocabulary. The word vector model can be a pre-trained word vector model that has already been trained, or it can be trained based on the target corpus.

[0063] Determine the total number of topics to be built K For each topic, a corresponding set of seed words is manually assigned. Each seed word set contains one or more seed words that can represent the semantic content of the corresponding topic.

[0064] For example, for a target corpus with a clear category structure, representative words can be selected as seed words for each topic based on the desired topic categories. Seed words are used to provide a small amount of artificial semantic information for the topic modeling process, rather than directly limiting which words can be included in the final topic.

[0065] Simultaneously set the prior parameters of the LDA model. , Similarity enhancement coefficient Similarity threshold And the exploration probability p.

[0066] Then, S2 is executed to calculate the semantic similarity field.

[0067] For any word in the vocabulary w Calculate words in sequence w and K Semantic similarity between seed word sets corresponding to each topic.

[0068] If the current word w It belongs to the seed word set Then the current word w With the k The semantic similarity between topics is set to 1.

[0069] If the current word w Not a seed word set If the seed word is in the middle, then calculate the current word. w Word vectors and seed word set The cosine similarity between the word vectors of each seed word is calculated, and the highest cosine similarity is selected as the current word. w With the k The semantic similarity between the seed word sets corresponding to each topic, i.e.:

[0070] For example, when the seed word of a topic is a word that can represent a specific semantic category, words that are closer to the seed word in the word vector space can obtain higher semantic similarity, while words with greater semantic differences can obtain lower semantic similarity.

[0071] After obtaining the semantic similarity, compare the semantic similarity with a preset similarity threshold. Compare. If Then the corresponding Set to 0.

[0072] Therefore, not all topics with non-zero original cosine similarity will enter the subsequent semantic guidance process; only topics with semantic similarity reaching the threshold will be used as the current word. w Semantic related topics.

[0073] After performing the above processing on all words and topics in the vocabulary, a semantic similarity field is formed. This semantic similarity field essentially describes the degree of semantic association between each word in the vocabulary and each topic.

[0074] Further, based on the semantic similarity field, a set of candidate topics corresponding to each word is determined:

[0075] For the same word w its candidate topic set It can contain one topic, multiple topics, or be empty.

[0076] When the candidate topic set contains multiple topics, it indicates that the current word has semantic associations with the seed words corresponding to multiple topics that meet the threshold requirements. This invention does not force the word to be fixed to a certain topic during the semantic similarity field calculation stage, but continues to maintain the competitive relationship between multiple topics during the subsequent hybrid sampling process.

[0077] Then S3 is executed to enhance the sampling probability of the folded Gibbs sampling.

[0078] Topic sampling is performed on each lexical unit in the target corpus. For those belonging to documents... m The thesaurus index is t The current word element First, exclude the original topic assignment of the term from the current statistical count. Then, calculate its assignment to the first topic, then the next topic, and so on. K The conditional probability of each topic.

[0079] Regarding the first k For each topic, its enhanced conditional probability is determined according to the following formula:

[0080] In the formula This is used to reflect the topic sampling probability formed by the traditional LDA model based on corpus co-occurrence information, while This is used to adjust the sampling probability of the above topics based on the semantic relationship of word vectors.

[0081] When the current word w With the k When topics have high semantic similarity, the semantic enhancement factor corresponding to that topic increases, making the current word more likely to be assigned to that topic in subsequent sampling.

[0082] Meanwhile, since the semantic enhancement factor is a multiplicative adjustment based on the original LDA sampling probability, the word co-occurrence information contained in the target corpus itself still participates in the topic sampling process and will not be completely replaced by the seed semantic information.

[0083] For example, when As the number of seeds increases, the influence of seed semantic information on sampling probability strengthens under the same semantic similarity conditions; when... When the value is reduced, the skew effect of semantic information on the sampling probability weakens, thereby enabling continuous adjustment of the semantic guidance strength.

[0084] Then S4 is executed to perform mixed sampling.

[0085] like Figure 2 As shown, for the word corresponding to the current lexical unit w First, determine the set of candidate topics. Is it empty?

[0086] When candidate topic set When not empty, based on the exploration probability p Determine whether the current sampling will execute an exploration branch or an exploit branch.

[0087] With probability p When executing an exploration branch, from the set of candidate topics A topic is randomly selected uniformly as the topic assignment result for the current lexical unit.

[0088] With probability 1- p When executing the branch, multinomial sampling is performed according to the enhanced sampling distribution obtained in S3, and the topic corresponding to the current lexical is determined based on the sampling results.

[0089] When candidate topic set When empty, multinomial sampling is performed directly according to the enhanced sampling distribution obtained in S3. At this point, for all topics... The corresponding semantic enhancement factors are all equal to 1, so the sampling probability is consistent with the corresponding LDA sampling probability.

[0090] After sampling the current lexical unit, the topic assignment count is updated based on the newly determined topic, and the next lexical unit is processed. Once all lexical units in a round have been reassigned to topics, the next round of sampling begins, and this process continues until the set sampling process is completed.

[0091] The exploration branch and the exploitation branch serve different functions. The exploitation branch makes full use of the semantic similarity field and the co-occurrence count of the current corpus, making words with high semantic similarity to a seed word of a certain topic more likely to enter that topic. The exploration branch, on the other hand, allows the current word to maintain the possibility of reselection among multiple topics with semantic relevance.

[0092] In the early stages of sampling, the current topic count may contain significant randomness. If relying entirely on the current count and a fixed seed anchor, a single undesirable topic assignment can negatively impact subsequent sampling, gradually intensifying as the topic count is updated. By consistently retaining a probability of [value missing] in each round of sampling... p The exploration branches allow for continuous adjustment of topic assignment, thereby reducing the probability of misassignment forming a self-reinforcing closed loop.

[0093] Execute S5 after all sampling is completed.

[0094] Based on the final topic assignment count, according to:

[0095] Obtain the topic-word distribution.

[0096] according to:

[0097] Obtain the document-topic distribution.

[0098] For each topic, it can be arranged as follows: The words in the vocabulary are sorted from largest to smallest, and words with higher probabilities are selected as representative words for the corresponding topics. For each document, it can be based on... Determine the proportion of each topic in the document to obtain the topic composition of the corresponding document.

[0099] Example 1 This embodiment uses a 20 Newsgroups corpus to verify the topic modeling effect of the present invention.

[0100] The dataset used was the official ByDate version of 20 Newsgroups corpus, with 11,314 documents in the training set and 7,532 documents in the test set, covering a total of 20 newsgroup categories.

[0101] For each category, specify 3 seed keywords and set the total number of topics. K =20, =0.1, =0.01, =2.0, =0.4, p=0. The word vectors are 100-dimensional Word2Vec word vectors trained on the target corpus.

[0102] This invention is compared with seven other methods: LDA, SeededLDA, NMF, BERTopic, ETM, ProdLDA, and CTM. Document-topic distribution is used as a feature for logistic regression classification.

[0103] In the experimental results, the classification accuracy of this invention was 0.5587, macro F1 was 0.5373, and clustering NMI was 0.4382; LDA was 0.4886, 0.4614, and 0.3933; NMF was 0.4743, 0.4160, and 0.3737; SeededLDA was 0.5265, 0.5048, and 0.4218; BERTopic was 0.3257, 0.2774, and 0.4151; ETM was 0.5684, 0.5410, and 0.1315; ProdLDA was 0.5590, 0.5307, and 0.1329; and CTM was 0.5068, 0.4818, and 0.3999.

[0104] When calculating topic consistency using 120,000 Wikipedia documents as external references, the present invention yielded an NPMI of 0.212 and a PMI of +0.658. In this embodiment, both the clustering NMI and the external topic consistency evaluation of the present invention achieved good results.

[0105] Figure 3 The comparison results of the classification accuracy of the present invention and various comparative methods are shown in this embodiment.

[0106] Example 2 This embodiment uses the Reuters-21578 corpus.

[0107] The Reuters-21578 ModApte dataset was used to partition the R8 subset, with the training set containing 5627 documents and the test set containing 2210 documents, covering a total of 8 financial news categories.

[0108] Specify 3 seed words for each category and set them. K =8、 =0.1、 =0.01、 =2.0、 =0.4、 p =0.2, the word vectors are 100-dimensional Word2Vec word vectors obtained by training the target corpus.

[0109] In the experimental results, the classification accuracy of this invention was 0.9189, macro F1 was 0.8022, and clustering NMI was 0.5936; LDA was 0.8607, 0.6013, and 0.4975; NMF was 0.6930, 0.1977, and 0.2236; SeededLDA was 0.8800, 0.6755, and 0.5321; BERTopic was 0.7398, 0.4613, and 0.5070; ETM was 0.8176, 0.5816, and 0.2473; ProdLDA was 0.8515, 0.6309, and 0.2529; and CTM was 0.8236, 0.5052, and 0.4283.

[0110] In the Wikipedia external reference consistency evaluation, the NPMI obtained by this invention is 0.223, and the PMI is +1.098.

[0111] Figure 4 The comparison results of the classification accuracy of each method in this embodiment are shown.

[0112] Example 3 This embodiment uses the Web of Science corpus.

[0113] The Web of Science corpus used contains 32,890 documents in the training set and 7,047 documents in the test set, covering a total of 7 subject categories.

[0114] Specify 3 seed words for each category and set them. K =7、 =0.1、 =0.01、 =1.0、 =0.4、 p =0.2, and a word vector model trained based on the target corpus is used.

[0115] In the experimental results, the classification accuracy of this invention was 0.6642, macro F1 was 0.6363, and clustering NMI was 0.4195; LDA was 0.6113, 0.5292, and 0.3887; NMF was 0.6065, 0.5095, and 0.3278; SeededLDA was 0.6407, 0.5738, and 0.3967; BERTopic was 0.5075, 0.3481, and 0.3804; ETM was 0.6499, 0.6089, and 0.1554; ProdLDA was 0.6272, 0.5754, and 0.1933; and CTM was 0.5826, 0.4875, and 0.3281.

[0116] In the Wikipedia external reference consistency evaluation, the NPMI of this invention is 0.278 and the PMI is +1.584.

[0117] Figure 5 The comparison results of the classification accuracy of each method in this embodiment are shown. In this embodiment, the classification accuracy, macro F1, and clustering NMI of the present invention are all higher than those of the corresponding comparison methods listed.

[0118] Example 4 This example is used to illustrate the similarity enhancement coefficient. Similarity threshold and exploration probability p Impact on model processing performance.

[0119] For: , as well as Perform a parameter scan.

[0120] Experimental results show that, on the three corpora tested, The model performance remained relatively stable overall within the range of 0.5 to 4. =0.4 achieved good results on all three corpora. In the 20 Newsgroups corpus, p When the value is 0.1, a peak value of 0.561 is obtained; in the Reuters and Web of Science corpora, when p Once a certain range is reached, a relatively wide performance platform is formed.

[0121] Furthermore, each topic can surpass the corresponding unguided baseline with only 1 seed word, and most of the performance gains can be obtained with 3 seed words per topic, and these gains can be maintained up to 8 seed words per topic.

[0122] In the delayed guidance comparison experiment, the method of performing 50 rounds of unguided LDA sampling before enabling seed guidance resulted in a score that was about 6 to 8 percentage points lower than that of full mixed sampling on both test corpora.

[0123] In the seed word contamination experiment, after replacing the third seed word of each category with an irrelevant word in the corpus, the accuracy of the present invention decreased by about 2 to 6 percentage points. This indicates that the semantic similarity field will spread the semantic information pointed to by the seed word to other related words. Therefore, it is advisable to confirm the representativeness of the seed word in actual use.

[0124] When the number of topics K is overestimated to twice the actual number of topics, the results of this invention on three corpora are still better than the LDA model under the same parameter configuration.

[0125] System Implementation In one implementation, the above method can also be performed by a seed topic modeling system guided by word vector semantics.

[0126] The seed topic modeling system includes an initialization module, a semantic similarity field calculation module, an enhanced sampling probability calculation module, a hybrid sampling module, and a parameter estimation and output module.

[0127] The initialization module is used to execute S1, obtain the target corpus and word vector model, and determine the total number of topics. K Obtain the seed keyword set for each topic and set it. , , , and p .

[0128] The semantic similarity field calculation module is used to execute S2, calculate the semantic similarity between words in the vocabulary and the seed word sets of each topic, form a semantic similarity field, and determine the corresponding candidate topic set.

[0129] The enhanced sampling probability calculation module is used to execute S3, which determines the enhanced sampling distribution based on the LDA conditional probability and the semantic enhancement factor.

[0130] The hybrid sampling module is used to perform S4, based on the candidate topic set and the exploration probability. p Choose to explore sampling or utilize sampling, and update the topic assignment count.

[0131] The parameter estimation and output module is used to execute S5, calculate the topic-word distribution and document-topic distribution based on the topic assignment count obtained from sampling, and output the representative words of each topic and the topic composition of each document.

[0132] The system implementation method described above uses the same topic modeling processing logic as the method implementation method described above, and each module can perform the corresponding function according to the corresponding steps in the method described above.

[0133] Implementation methods of electronic devices and storage media In one implementation, the seed topic modeling method based on word vector semantic guidance described above can be executed by an electronic device. The electronic device includes a processor and a memory, the memory storing a computer program. When the computer program is executed by the processor, the processor performs the seed topic modeling method described in S1 to S5 above.

[0134] In another embodiment, a computer-readable storage medium is provided, wherein a computer program is stored in the computer program, which, when executed by a processor, implements the above-described seed topic modeling method based on word vector semantic guidance.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A seed topic modeling method based on word vector semantic guidance, characterized in that, Performed by a computer, including the following steps: S1. Obtain the target corpus and word vector model, and determine the total number of topics to be constructed. Obtain the seed word set corresponding to each topic, and set the prior parameters and similarity enhancement coefficient of the implicit Dirichlet assignment model. Similarity threshold And the exploration probability p; S2, for each word in the vocabulary of the target corpus Determine the words respectively The semantic similarity between the seed word set corresponding to each topic and the similarity threshold. The semantic similarity is filtered to obtain a semantic similarity field covering the vocabulary and each topic, and the word is determined based on the filtered semantic similarity. The corresponding set of candidate topics; S3, perform folded Gibbs sampling on the lexical units in the target corpus. Based on the conditional probability corresponding to the latent Dirichlet allocation model, determine the semantic enhancement factor according to the semantic similarity between the word corresponding to the current lexical unit and each topic and the similarity enhancement coefficient λ. Then, use the semantic enhancement factor to multiplicatively adjust the conditional probability to obtain the enhanced sampling distribution of the current lexical unit corresponding to each topic. S4, for the current word element, when its corresponding candidate topic set is not empty, use the exploration probability... p Perform exploratory sampling, randomly select a topic from the candidate topic set, and select the topic with a probability of 1- p Perform sampling and select topics according to the enhanced sampling distribution; when the corresponding candidate topic set is empty, select topics according to the enhanced sampling distribution, update the topic assignment count according to the selected topics, and iteratively perform topic sampling on the lexical units in the target corpus; S5. After completing the topic sampling, estimate the topic-word distribution and document-topic distribution based on the topic assignment count, and output the representative words of each topic and the topic composition of each document based on the topic-word distribution and the document-topic distribution.

2. The seed topic modeling method based on word vector semantic guidance according to claim 1, characterized in that, In S2, for the first k Seed keyword set for each topic The word With the seed word set semantic similarity between Determined in the following manner: when hour, ； when hour, ； in, k Index for the topic, , s For seed word set Seed words in For words w The corresponding word vectors, Seed word s The corresponding word vectors, Represents the word vector With the word vector Cosine similarity between them; When the semantic similarity Less than the similarity threshold At that time, the semantic similarity Set to 0.

3. The seed topic modeling method based on word vector semantic guidance according to claim 2, characterized in that, The word w Corresponding candidate topic set for: ； in, Used to characterize the word w There exist similarity thresholds that satisfy the above criteria. A collection of topics with semantic relationships.

4. The seed topic modeling method based on word vector semantic guidance according to claim 1, characterized in that, In S3, the word elements Assigned to the k The enhanced conditional probability of each topic is determined by the following formula: ； in, Indicates the current word element The topic assignment results Indicates excluding the current word element The topic assignment results of the remaining lexical units, This represents the word metadata in the target corpus. and Assign prior parameters to the implicit Dirichlet model. The similarity enhancement coefficient is... The word corresponding to the current lexical unit w With the k Seed keyword set for each topic Semantic similarity between them Indicates exclusion of the current word element Post-document m Zhong was assigned to the first k The number of lexical units for each topic Indicates exclusion of the current word element The post-vocabulary index is t The word was assigned to the first k Number of times for each topic Indicates exclusion of the current word element Later assigned to the k The total number of terms for each topic V Indicates the size of the vocabulary.

5. The seed topic modeling method based on word vector semantic guidance according to claim 1, characterized in that, In S5, the first step is estimated according to the following formula. k The topic corresponds to the first t The topic-word distribution of each word: Estimate the document using the following formula. m Corresponding to the k Documents by topic - Topic distribution: in, Indicates that the vocabulary index is t The word was assigned to the first k Number of times for each topic Indicates being assigned to the k The total number of terms for each topic Document m Zhong was assigned to the first k The number of lexical units for each topic Document m The total number of lexical units in the text V Indicates vocabulary size, K Indicates the total number of topics.

6. The seed topic modeling method based on word vector semantic guidance according to claim 1, characterized in that, The word vector model is either a pre-trained word vector model or a word vector model trained based on the target corpus.

7. The seed topic modeling method based on word vector semantic guidance according to claim 1, characterized in that, The prior parameters The prior parameter is 0.

1. The similarity enhancement coefficient is 0.

01. The similarity threshold is between 0.5 and 2.

0. The exploration probability is 0.4 to 0.

5. p The value ranges from 0 to 0.2, and the seed word set for each topic includes 1 to 8 seed words.

8. A seed topic modeling system based on word vector semantic guidance, characterized in that, include: The initialization module is used to acquire the target corpus and word vector model, and determine the total number of topics to be constructed. K Obtain the seed word set corresponding to each topic, and set the prior parameters and similarity enhancement coefficient of the implicit Dirichlet assignment model. Similarity threshold and exploration probability p ; The semantic similarity field calculation module is used to determine the semantic similarity between each word in the vocabulary of the target corpus and the seed word set corresponding to each topic, and to calculate the similarity based on the similarity threshold. The semantic similarity is filtered to obtain a semantic similarity field, and the candidate topic set corresponding to the word is determined based on the filtered semantic similarity. The enhanced sampling probability calculation module is used to perform folded Gibbs sampling on lexical units in the target corpus, and to calculate the probability based on the semantic similarity between the word corresponding to the current lexical unit and each topic, as well as the similarity enhancement coefficient. Determine the semantic enhancement factor, and use the semantic enhancement factor to multiplicatively adjust the conditional probability corresponding to the latent Dirichlet assignment model to obtain the enhanced sampling distribution; The hybrid sampling module is used to sample topics with the aforementioned exploration probability when the candidate topic set corresponding to the current lexical unit is not empty. p A topic is randomly selected from the candidate topic set with a probability of 1- p Select a topic according to the enhanced sampling distribution, and select a topic according to the enhanced sampling distribution when the candidate topic set is empty. Update the topic assignment count according to the selected topic and iteratively perform topic sampling. The parameter estimation and output module is used to estimate the topic-word distribution and document-topic distribution based on the topic assignment count after the topic sampling is completed, and output the representative words of each topic and the topic composition of each document.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the seed topic modeling method based on word vector semantic guidance as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the seed topic modeling method based on word vector semantic guidance as described in any one of claims 1 to 7.

Images