Semantic enhanced topic clustering data analysis method
Through the semantic enhanced theme clustering data analysis method, the clustering theme model and various technical means are used to solve the problem of traditional theme models ignoring semantic relationships, achieving more efficient and accurate theme extraction, and improving the quality of the theme model and efficiency.
Patent Information
- Application Number
- CN202411129078.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-29
AI Technical Summary
The existing traditional theme models rely on bag-of-word models, ignoring the complex semantic relationships contained in the text, resulting in poor theme extraction results, and the clustering theme models are too dependent on word frequency characteristics, making it difficult to achieve high theme quality.
Using semantic enhanced theme clustering data analysis method, the clustering theme model is constructed, document embedding, feature dimensionality reduction and cluster analysis are carried out, and global word frequency information, intercluster word frequency information, word semantic information and document semantic information are counted, using SBERT, PCA, HDBSCAN and other technologies improve the accuracy and efficiency of the theme model.
It improves the accuracy and efficiency of the theme model, enhances the comprehensiveness and accuracy of weight allocation in the theme model, and can better capture text features and improve the quality of theme representation.
Smart Images

Figure CN119046704A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a semantically enhanced topic clustering data analysis method. Background Art
[0002] With the rapid growth of text data, manual analysis of text topics has become increasingly difficult. In order to automatically reveal potential topics in text, researchers have proposed topic models, which are powerful unsupervised topic analysis tools. Topic models automatically discover different topics from a large collection of text by analyzing text information. Each topic represents an interpretable semantic concept and is characterized by a set of topic descriptors. Due to its high efficiency and interpretability, topic models have been widely used in many fields, such as document retrieval, content recommendation, opinion and event mining, trend analysis, etc. Currently popular topic models can be divided into three categories: traditional algebraic and probabilistic models, neural topic models (NTMs) combined with neural network structures, and clustering topic models.
[0003] However, most of the traditional topic models rely on the bag-of-words model, which treats words as independent features and ignores the complex semantic relationships contained in the text, resulting in poor topic extraction. The clustering topic model clusters the pre-trained word vectors at the word level, rearranges the word weights based on the word frequency features, and selects topic descriptors, but the quality of the topics obtained in this way depends largely on the word frequency features and can hardly reach the performance of the most basic LDA. Therefore, it is very necessary to design a semantically enhanced topic clustering data analysis method. Summary of the invention
[0004] The purpose of the present invention is to provide a semantically enhanced topic clustering data analysis method to improve the accuracy and efficiency of building a topic model and to improve the comprehensiveness and precision of weight allocation in the topic model.
[0005] To achieve the above object, the present invention provides the following solutions:
[0006] A semantically enhanced topic clustering data analysis method comprises the following steps:
[0007] Construct a clustering topic model and obtain topic clusters through the clustering topic model;
[0008] Perform topic characterization on the topic clusters to obtain characterization results; topic characterization includes: statistical global word frequency information, statistical cluster word frequency information, statistical word semantic information and statistical document semantic information;
[0009] The topic content of the topic cluster is obtained based on the characterization results.
[0010] Optionally, specific steps of constructing a clustering topic model include: document embedding, feature dimension reduction, and cluster analysis.
[0011] Optionally, the document embedding adopts the SBERT framework; the SBERT framework is used to generate sentence embeddings; and the sentence embeddings are used to calculate sentence similarity and perform information retrieval.
[0012] Optionally, the feature dimensionality reduction adopts a PCA algorithm, a t-SNE algorithm or a UMAP algorithm; the feature dimensionality reduction is used to reduce high-dimensional data to low-dimensional data.
[0013] Optionally, the cluster analysis adopts a HDBSCAN algorithm or a K-means algorithm.
[0014] Optionally, the global word frequency information is counted using a TF-IDF method; the calculation formula of the TF-IDF method is: Among them, t is a word, d is a document, and f is t,d is the number of times a word appears in a document, ∑ t' f t',d is the total number of times all words appear in the document, TF(t,d) is the term frequency matrix, IDF(t,D) is the inverse document frequency, D is the document set, n is the number of documents, and {d∈D|t∈d} is the number of documents containing the word in the document set.
[0015] Optionally, the calculation formula for statistical cluster word frequency information is: Among them, |K| is the number of clusters, |t∈K| is the number of clusters containing words, IDFi is the inverse document frequency between clusters, and TFIDF is the product of word frequency and inverse document frequency.
[0016] Optionally, a specific method for counting word semantic information is: respectively replace the word frequency, inverse document frequency and IDFi in the TF-IDF method to obtain the t-TFIDF-IDFi method, and capture text features of the topic cluster through the t-TFIDF-IDFi method.
[0017] Optionally, replace the term frequency, inverse document frequency and IDFi in the TF-IDF method respectively. The specific formulas after replacement are: sTF = TF × M s ; Where D is the set of documents, n is the number of documents, sTF is the term frequency after replacement, TF is the term frequency, M s is the word similarity matrix, sIDF is the inverse document frequency after replacement, t i is a term, d is a document, μ(t i,d) is the average similarity between the term and all terms in the document, sIDFi is the IDFi after replacement, and k is the document formed by concatenating all documents in the topic cluster.
[0018] Optionally, the specific formula for counting document semantic information is: td-TFIDF-IDFi=t-TFIDF-IDFi⊙Docs_Weight; wherein, td-TFIDF-IDFi is the final weight strategy value, t-TFIDF-IDFi is a variant of TFIDF and IDFi, ⊙ is the Hadamard product, and Docs_Weight is the document semantic weight.
[0019] According to the specific embodiment provided by the present invention, the present invention discloses the following technical effects: the semantically enhanced topic clustering data analysis method provided by the present invention comprises: constructing a clustering topic model, and obtaining a topic cluster through the clustering topic model; performing topic characterization on the topic cluster to obtain a characterization result; the topic characterization comprises: statistical global word frequency information, statistical inter-cluster word frequency information, statistical word semantic information and statistical document semantic information; and obtaining the topic content of the topic cluster according to the characterization result. The method improves the accuracy and efficiency of constructing the topic model, and also improves the comprehensiveness and precision of weight allocation in the topic model. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0021] Figure 1 This is a flow chart of the semantically enhanced topic clustering data analysis method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0023] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0024] like Figure 1As shown, an embodiment of the present invention provides a semantically enhanced topic clustering data analysis method, comprising the following steps:
[0025] Step 100: Construct a clustering topic model, and obtain topic clusters through the clustering topic model.
[0026] Specifically, the clustering topic model is called the ERCcR model, and the specific steps of constructing the clustering topic model include: document embedding, feature dimensionality reduction and clustering analysis.
[0027] More specifically, document embedding converts natural language text into comparable vector form through the SBERT framework; the SBERT framework generates sentence embeddings that can be used for sentence similarity calculation and information retrieval tasks by fine-tuning the twin network structure; this step significantly improves the task quality and ensures accurate expression of semantics.
[0028] Feature dimensionality reduction techniques include PCA algorithm, t-SNE algorithm and UMAP algorithm; among them, PCA is famous for its low computational complexity and easy interpretability, but it is not suitable for processing nonlinear structures; although t-SNE can capture nonlinear structures and maintain local neighborhood relationships, its computational cost is high and it is sensitive to parameter selection; UMAP provides higher computational efficiency while maintaining local and global structures, although its parameter setting is more complex and less interpretable. This embodiment enables the clustering topic model to flexibly select appropriate dimensionality reduction techniques based on the characteristics of the data and the analysis objectives.
[0029] In the process of building a clustering topic model, cluster analysis generally uses the HDBSCAN algorithm. Among them, HDBSCAN is a density-based clustering algorithm that can automatically identify the optimal number of clusters (number of topics), so that HDBSCAN performs well when processing clusters of different shapes and noise data, thereby improving the accuracy and robustness of the clustering results. However, in certain specific cases, such as comparative experiments or specific application scenarios, it may be necessary to discover a specific number of topics, which makes HDBSCAN unable to meet the needs, because the number of topics it automatically determines may not be consistent with expectations, thereby affecting the consistency of the topics. In this embodiment, the HDBSCAN algorithm is replaced with the K-means algorithm under certain circumstances. K-means is an intuitive and fast clustering algorithm that can discover topics composed of similar documents according to a pre-set number of topics, and has high clarity and controllability. This method of flexibly selecting algorithms according to different situations ensures the fairness of the experiment and the comparability of the results.
[0030] Step 200: Perform topic characterization on the topic cluster to obtain characterization results; the topic characterization includes: statistical global word frequency information, statistical inter-cluster word frequency information, statistical word semantic information and statistical document semantic information.
[0031] Specifically, topic representation is to determine the contribution of words in the topic to the topic on the premise of obtaining a clustering topic model, and find the most important words in the topic by sorting the contribution, and use them as topic descriptors to represent the topic. Here, D is defined as the set of all documents {d1, d2, …, d n}, where d i is the i-th document, n is the number of documents in the collection, V is the set of all words in the document collection D {t1, t2, ..., t v}, where t v are words of different vocabulary lengths, v is the vocabulary length, and K is the set of clusters, i.e., topic clusters.
[0032] More specifically, the TF-IDF method is used to count the global word frequency information. TF-IDF combines word frequency (TF) and inverse document frequency (IDF). The calculation formula of the TF-IDF method is:
[0033]
[0034] Among them, t is a word, d is a document, and f is t,d is the number of times a word appears in a document, ∑ t' f t',d is the total number of times all words appear in the document, TF(t,d) is the term frequency matrix, IDF(t,D) is the inverse document frequency, D is the document set, and {d∈D|t∈d} is the number of documents in the document set that contain the word. Among them, term frequency is used to measure the frequency of a word appearing in a document; inverse document frequency is used to measure the prevalence of a word in the entire document set. The more frequently a word appears, the lower its IDF value.
[0035] The specific steps of counting the word frequency information between clusters are: extending the inverse document frequency as the penalty item of word frequency to the inverse topic cluster frequency, and adding the inverse topic cluster frequency as a penalty item to the weight strategy to form a more complete word frequency counting method, namely TFIDF-IDFi. The specific calculation formula is:
[0036]
[0037] Among them, |K| is the number of clusters, |t∈K| is the number of clusters containing words, IDFi is the inverse document frequency between clusters, and TFIDF is the product of word frequency and inverse document frequency, which is used to indicate the importance of a word in a document.
[0038] The method of statistical word semantic information is to use word semantic embedding to enhance the expression ability of TFIDF-IDFi. The specific implementation steps are:
[0039] Construct a word similarity matrix and multiply it with the word frequency matrix to introduce semantics into the word frequency information. The specific calculation formula is:
[0040] sTF=TF×M s ;
[0041] Among them, sTF is the term frequency matrix after replacement, TF is the term frequency matrix, M s M is a v×v word similarity matrix. s The SBERT framework embeds words and calculates cosine similarity, which captures the pairwise semantic similarity between words.
[0042] Replace the inverse document frequency with a variant of the intra-document semantic similarity; the specific calculation formula is:
[0043]
[0044] Where D is the document set, n is the number of documents, sIDF is the inverse document frequency after replacement, and t i is a term, d is a document, μ(t i ,d) is the average similarity between the term and all terms in the document. If μ(t i ,d) is higher, the lower the semantic contribution of the word to the entire sentence.
[0045] Replace IDFi; the specific calculation formula is:
[0046]
[0047] Among them, sIDFi is the replaced IDFi, and k is the document concatenated from all documents in the topic cluster. By replacing the parts of TFIDF-IDFi, a new method of integrating word semantics is obtained, which can better capture text features and improve the quality of topic representation. The new method is called t-TFIDF-IDFi.
[0048] The specific steps for counting document semantic information are as follows:
[0049] Perform nested clustering on all documents within each identified topic to obtain multiple nested cluster centers, and assign higher weights to words close to the nested cluster centers as weighted items in the final weight strategy;
[0050] Specifically, the nested cluster center set is defined as C = {c1, c2, ..., c m}, where c m is the center point of the mth fine-grained cluster, and the document embedding in the i-th cluster is defined as E i ={e i1 ,e i2 ,…,eig} Among them, e ig is the g-th embedding vector in the i-th fine-grained cluster, and the calculation formula for the document semantic weight in the topic is:
[0051]
[0052] Among them, f t,C is the number of times the word appears in the topic, f t,e is the number of times the word appears in the document whose document embedding is e, sim() is the similarity calculation function, e and c are both vectors, and Docs_Weight is the semantic weight of the document in the topic.
[0053] Combining t-TFIDF-IDFi with the document semantic weight, we get the final weight strategy, which is calculated as follows:
[0054] td-TFIDF-IDFi=t-TFIDF-IDFi⊙Docs_Weight;
[0055] Among them, td-TFIDF-IDFi is the final weight strategy value, t-TFIDF-IDFi is a variant of TFIDF and IDFi, and ⊙ is the Hadamard product.
[0056] Step 300: Obtain the topic content of the topic cluster according to the characterization result.
[0057] The specific examples provided in this embodiment are as follows:
[0058] This experiment is conducted on four datasets of different lengths, including two long text news datasets (20Newsgroups and BBC News) and two short text paper datasets (M10 and DBLP). These datasets have been preprocessed to ensure that they can run properly on traditional models and ensure the fairness of the experiment.
[0059] Furthermore, the preprocessing process is: remove punctuation marks in the data set to ensure the purity of the text; perform word form restoration to unify the basic form of words; remove stop words to eliminate common words that have no practical contribution to text analysis; filter out words whose frequency of occurrence is lower than or higher than the threshold frequency according to the frequency of occurrence of words in the corpus; remove documents whose length is less than the minimum document length to ensure the quality and consistency of the analysis data. The preprocessing process parameters are shown in Table 1:
[0060] Table 1 Dataset preprocessing parameters
[0061]
[0062] Among them, D represents the number of documents in the dataset, V represents the length of the dataset vocabulary, L represents the number of true labels in the dataset, avgL is the average length of all documents, tf is the threshold frequency, and minL is the minimum document length.
[0063] The models used in the experiment include: NMF model, LDA model, ProdLDA, CombinedTM, ZeroShotTM, BERTopic, CETopic and ERCcR model.
[0064] The experiment used the Normalized Point Mutual Information (NPMI) and Topic Coherence (Cv) method to measure the consistency of the topic. The calculation formula of NPMI is:
[0065]
[0066] Among them, P(w i ,w j ) is the probability of appearing in the same window p, P(w i ) and P(w j ) are word w i With the word w j The probability of appearing alone in the window, NPMI(w i ,w j ) is the normalized point mutual information value.
[0067] Furthermore, for the preprocessed dataset, the model was experimented with 5, 10, 15, 20, 25 and 30 topics. The consistency and diversity scores of the experimental model on the long text dataset and the short text dataset are shown in Table 2 and Table 3 respectively:
[0068] Table 2 Experimental model long text dataset score table
[0069]
[0070] Table 3 Experimental model short text dataset score table
[0071]
[0072]
[0073] The data in bold and underlined are the best results, and the data in bold only are the suboptimal results.
[0074] Furthermore, qualitative experiments were conducted on the preprocessed datasets using TFIDF-IDFi, t-TFIDF-IDFi, and td-TFIDF-IDFi. The scores on the long text dataset and the short text dataset are shown in Table 4 and Table 5, respectively:
[0075] Table 4 Comparison of scores of different methods on long text dataset
[0076]
[0077] Table 5 Comparison of scores of different methods on short text dataset
[0078]
[0079] The data in bold and underlined are the best results, and the data in bold only are the suboptimal results.
[0080] The beneficial effects of the present invention are as follows:
[0081] 1) It can better capture text features and improve the quality of topic representation;
[0082] 2) It improves the importance and comprehensiveness of evaluation words in specific topics, thereby improving the accuracy and depth of topic clustering analysis;
[0083] 3) The accuracy and efficiency of building topic models are improved; the comprehensiveness and accuracy of weight distribution in topic models are also improved.
[0084] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0085] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A semantically enhanced topic clustering data analysis method, characterized in that: The steps include: Constructing a clustering topic model, and obtaining topic clusters through the clustering topic model; Performing topic characterization on the topic cluster to obtain a characterization result; the topic characterization includes: statistical global word frequency information, statistical inter-cluster word frequency information, statistical word semantic information and statistical document semantic information; The topic content of the topic cluster is obtained according to the characterization result.
2. The semantically enhanced topic clustering data analysis method according to claim 1, characterized in that: The specific steps of building a clustering topic model include: document embedding, feature dimension reduction and cluster analysis.
3. The semantically enhanced topic clustering data analysis method according to claim 2, characterized in that: The document embedding adopts the SBERT framework; the SBERT framework is used to generate sentence embedding; the sentence embedding is used to calculate sentence similarity and perform information retrieval.
4. The semantically enhanced topic clustering data analysis method according to claim 2, characterized in that: The feature dimensionality reduction adopts PCA algorithm, t-SNE algorithm or UMAP algorithm; the feature dimensionality reduction is used to reduce high-dimensional data to low-dimensional data.
5. The semantically enhanced topic clustering data analysis method according to claim 2, characterized in that: The cluster analysis adopts HDBSCAN algorithm or K-means algorithm.
6. The semantically enhanced topic clustering data analysis method according to claim 1, characterized in that: The TF-IDF method is used to count the global word frequency information; the calculation formula of the TF-IDF method is: Among them, t is a word, d is a document, and f is t,d is the number of times a word appears in a document, ∑ t' f t',d is the total number of times all words appear in the document, TF(t,d) is the term frequency matrix, IDF(t,D) is the inverse document frequency, D is the document set, n is the number of documents, and {d∈D|t∈d} is the number of documents containing the word in the document set.
7. The semantically enhanced topic clustering data analysis method according to claim 6, characterized in that: The calculation formula for statistical cluster word frequency information is: Among them, |K| is the number of clusters, |t∈K| is the number of clusters containing words, IDFi is the inverse document frequency between clusters, and TFIDF is the product of word frequency and inverse document frequency.
8. The semantically enhanced topic clustering data analysis method according to claim 7, characterized in that: The specific method of counting word semantic information is: respectively replacing the word frequency, inverse document frequency and IDFi in the TF-IDF method to obtain the t-TFIDF-IDFi method, and capturing the text features of the topic cluster through the t-TFIDF-IDFi method.
9. The semantically enhanced topic clustering data analysis method according to claim 8, characterized in that: The term frequency, inverse document frequency and IDFi in the TF-IDF method are replaced respectively, and the specific formulas after replacement are: sTF = TF × M s ; Where D is the set of documents, n is the number of documents, sTF is the term frequency after replacement, TF is the term frequency, M s is the word similarity matrix, sIDF is the inverse document frequency after replacement, t i is a term, d is a document, μ(t i ,d) is the average similarity between the term and all terms in the document, sIDFi is the IDFi after replacement, and k is the document formed by concatenating all documents in the topic cluster.
10. The semantically enhanced topic clustering data analysis method according to claim 8, characterized in that: The specific formula for counting document semantic information is: td-TFIDF-IDFi=t-TFIDF-IDFi⊙Docs_Weight; among them, td-TFIDF-IDFi is the final weight strategy value, t-TFIDF-IDFi is a variant of TFIDF and IDFi, ⊙ is the Hadamard product, and Docs_Weight is the document semantic weight.
