A keyword-related topic analysis method based on local word embedding technology

Abstract

The present invention discloses a keyword-related topic analysis method based on local word embedding technology. When a user enters a keyword, the method uses global word embedding technology to analyze the keyword in text big data, sorts the cosine similarity between the vocabulary and the keyword in the data, and selects the subject words. The subject dictionary is constructed based on the subject words, and the text big database is segmented by time, author, article type, etc. The local word embedding analysis is performed on the keywords and the subject dictionary within the segment, and the local cosine similarity in each segment is used to obtain the theme of each segment. Then, the theme change of the keyword in the text big database can be analyzed. The present invention can be widely used in the fields of social science, public opinion monitoring, historical document analysis, etc., and provides an efficient and accurate theme dynamic analysis tool for text mining.

Description

Technical Field

[0001] The present invention belongs to the technical field of text data analysis, and specifically relates to a keyword-related topic analysis method based on local word embedding technology. Background Art

[0002] In the field of social science research, efficient retrieval and semantic analysis of text big data have become important research needs, especially for keyword topic analysis, which can help researchers quickly grasp the core issues of texts, discover potential research hotspots, and track the evolving trends of topics. However, traditional topic analysis methods often face problems such as low computational efficiency, insufficient semantic interpretation, and lack of dynamic analysis capabilities when processing large-scale and complex text data, making it difficult to meet the needs of social science research for in-depth text mining. Current mainstream topic analysis technologies mainly include LDA (Latent Dirichlet Allocation) based on probabilistic models, NMF (Non-negative Matrix Factorization) based on matrix decomposition, LSA (Latent Semantic Analysis), statistical factor analysis methods, and word embedding technologies based on deep learning. However, these methods each have significant limitations and cannot achieve a balance between efficiency, interpretability, and dynamic analysis capabilities.

[0003] LDA, a classic generative probabilistic model, assumes that documents are generated from a mixture of multiple topics and uses term distributions to describe topics. Its advantage lies in its strong interpretability and ability to intuitively present the association between topics and words. However, LDA's limitation lies in its assumption that topics are static and mixed, making it ineffective in capturing the complex semantic structure of text. This is particularly true when the text contains polysemous words or when topics change dynamically over time. NMF uses non-negative matrix factorization to decompose the "document-term matrix" into two low-dimensional matrices: "document-topic" and "topic-term" matrices, thereby extracting the latent topic structure. While NMF is computationally more efficient than LDA, due to its lack of probabilistic foundations, its topic interpretation is weak and it still faces computational resource constraints when processing very large or long-span text data. LSA, also based on matrix factorization, uses singular value decomposition (SVD) to reduce dimensionality and extract latent semantic features of text. However, it also lacks probabilistic interpretation and can lose important semantic information during dimensionality reduction, limiting the accuracy of topic analysis.

[0004] Factor analysis (see Kostov, Ronald, and Joel Bullock, "Factor Matrix Text Filtering and Clustering," Journal of the American Society for Information Science and Technology, Vol. 56, No. 9, 2005: pp. 946-968) is a statistical technique designed to extract latent factors from observed variables to explain the correlations between them. Its advantages lie in its ability to reduce data complexity through dimensionality reduction and its ability to model errors. However, this method places high demands on the linear relationships in the data, and its computational complexity increases dramatically with data size. Furthermore, it lacks specific optimization for text data, making it difficult to adapt to the needs of large-scale text analysis.

[0005] Word embedding technology (see Chinese patent application CN118468848A: Text Sentiment Analysis Method and System Integrating Long Short-Term Memory Networks and Word Embeddings) can capture the global semantic associations of words by mapping them into a high-dimensional vector space. However, traditional word embedding methods often ignore the local context of words, the author's attitude, and the dynamic characteristics of word meaning evolution over time, making it difficult for their analysis results to truly reflect the semantic complexity of the text. Summary of the Invention

[0006] In order to solve the problems faced by traditional topic modeling technology when processing large-scale text data, such as low computational efficiency, insufficient semantic interpretation ability, and lack of dynamic analysis ability, the present invention provides a keyword-related topic analysis method based on local word embedding technology. Through segmented local word embedding analysis, combined with global and local word vector modeling, efficient keyword topic mining and dynamic evolution analysis are achieved.

[0007] A keyword-related topic analysis method based on local word embedding technology includes the following steps:

[0008] (1) Clean the text big data to obtain the word segmentation library of the text big data;

[0009] (2) The user inputs the keywords to be analyzed;

[0010] (3) Perform global word embedding analysis on the word segmentation database, extract several words closest to the keywords from the word segmentation database based on the global word embedding, classify these words and form a thematic dictionary;

[0011] (4) Segmenting text big data;

[0012] (5) Perform local word embedding analysis in each segment, extract the topic closest to the keyword from the topic dictionary based on the local word embedding, and use the topic name as the topic name of the segment;

[0013] (6) Sort out the topic names of each segment, summarize the theme change patterns of keywords between each segment, and obtain the theme judgment results of keywords in each segment.

[0014] Furthermore, the text big data includes a large number of historical articles from newspapers, book series, and web pages, and these articles have diverse content from different periods and regions.

[0015] Furthermore, the specific implementation method of step (1) is as follows: first, the articles in the text big data are sorted, arranged in chronological order and classified by region; then, useless words in the articles are deleted, and only meaningful texts are retained, and then the word segmentation package in the R language is called, and the word segmentation package is used to perform word segmentation operations on the articles in the text big data to obtain the word segmentation library of the text big data.

[0016] Furthermore, the specific implementation of step (3) is as follows:

[0017] 3.1 Using text big data to train word embedding neural network models;

[0018] 3.2 Input the words in the word library one by one into the trained model to obtain the corresponding word vectors;

[0019] 3.3 Calculate the cosine similarity between each word in the word segmentation database and the keyword based on the word vector, and extract the top N words with the largest cosine similarity from the word segmentation database, where N is a natural number greater than 1;

[0020] 3.4 Divide the extracted words into several category groups, each category group corresponds to a topic, thus forming a topic dictionary consisting of multiple topics.

[0021] Furthermore, in step 3.4, the extracted words are firstly classified by subject using factor analysis method to complete the rough classification; then, professionals adjust the rough classification results according to the needs, and define a subject name for each category group.

[0022] Furthermore, in step (4), selection is made according to user needs, and the entire text big data is segmented into several components according to time, region or author.

[0023] Furthermore, the specific implementation of step (5) is as follows:

[0024] 5.1 For any segment, use the segment data to train a word embedding neural network model;

[0025] 5.2 Input the words in the subject dictionary into the trained model one by one to obtain the corresponding word vectors;

[0026] 5.3 For any topic, calculate the cosine similarity between each word in the category group corresponding to the topic and the keyword based on the word vector, and take the average value as the similarity between the topic and the keyword;

[0027] 5.4 According to step 5.3, traverse all topics in the topic dictionary, select the topic with the greatest similarity, and use this topic name as the topic name of the segment.

[0028] Furthermore, the word embedding neural network model adopts Skip-gram (skip-gram model).

[0029] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the above-mentioned keyword-related topic analysis method based on local word embedding technology.

[0030] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned keyword-related topic analysis method based on local word embedding technology.

[0031] The method of the present invention can not only efficiently process large-scale text data, but also dynamically track the temporal correlation between keywords and subject dictionaries, support user-defined subject words, and observe the dynamic changes of word meanings between paragraphs. Its core innovations include: first, the present invention adopts a segmented local word embedding analysis strategy to divide the text into several paragraphs for independent calculation, significantly reducing memory usage and computational burden, making large-scale text analysis possible; second, the present invention abandons the inefficient practice of traditional global word embedding to analyze all words, and only performs global embedding modeling on the keywords input by the user, ensuring that the generated topics are highly correlated with the keywords, while further optimizing the computational efficiency; in addition, the present invention constructs a subject dictionary through global word embedding technology, and adopts a coarse and fine two-step screening mechanism to extract the subject words with the highest correlation with the keywords, thereby enhancing the accuracy of subject analysis; finally, the present invention uses local word embedding technology to quantify the correlation between keywords and subject dictionaries in each paragraph, realizes paragraph-level topic identification and cross-paragraph topic evolution analysis, thereby revealing the dynamic change law of text semantics.

[0032] Compared with existing technologies, the method of the present invention has significant advantages in terms of computational efficiency, semantic interpretation ability and dynamic analysis. Its segmented processing strategy effectively solves the problems of high memory usage and long computational time of traditional methods, making it capable of analyzing ultra-large-scale text data. At the same time, the introduction of local word embedding technology enables the contextual sensitivity and temporal changes of vocabulary to be captured, thereby improving the accuracy and depth of topic analysis. In addition, the flexibility of user-defined keywords further enhances the applicability of the method of the present invention, and it can be widely used in social sciences, public opinion analysis, historical document research and other fields. In the future, the method of the present invention can be further optimized by introducing more advanced embedding representation technology and dynamic modeling frameworks to meet more complex text semantic analysis needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the flow of the keyword-related topic analysis method of the present invention.

[0034] Figure 2 Schematic diagram of the data cleaning process in the method of the present invention.

[0035] Figure 3 A schematic diagram of the process of constructing a subject dictionary according to the method of the present invention.

[0036] Figure 4 Schematic diagram of the process of extracting relevant topics in segments by the method of the present invention.

[0037] Figure 5 Schematic diagram of keywords, keyword cosine correlation ranking analyzed after global word embedding, and topic dictionary construction results in an embodiment of the present invention.

[0038] Figure 6 Schematic diagram of a curve showing the change in similarity between keywords and topics over time according to the embodiment of the present invention.

[0039] Figure 7 Schematic diagram of topic determination results of keywords in various segments in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to describe the present invention more specifically, the technical solution of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] like Figure 1 As shown, the keyword-related topic analysis method based on local word embedding technology of the present invention includes the following steps:

[0042] Step S1: Clean the text database to obtain word segmentation data.

[0043] Large text databases include newspapers, book series, internet web pages and other data information, which usually contain multiple articles. These articles can contain diverse content from different periods and regions.

[0044] like Figure 2 As shown, first, the articles in the text big data database are sorted. The sorting method can be adjusted according to user needs, including chronological arrangement, regional arrangement, etc.; then, useless words in the text are deleted, and only meaningful text is retained; then, a word segmentation package is created, and the word segments are cut to obtain the word segmentation vocabulary of the text big data.

[0045] Step S2: Wait for the user to input the keyword words to be analyzed.

[0046] The keyword vocabulary input by the user can be a single word or a word vector composed of multiple words.

[0047] Step S3: Perform global word embedding analysis on the cleaned text big data word segmentation, calculate the cosine similarity between the keyword vocabulary and other words in the text big data word segmentation library, determine the N words closest to the keyword through cosine similarity, classify these N words, and form a topic dictionary.

[0048] like Figure 3 As shown in the figure, we first perform global word embedding analysis on the word segmentation of text big data. The word embedding technology is based on the Skip-gram model. Its core idea is to learn the distributed representation of words through neural networks, that is, word vectors, so that words with similar semantics are close in the vector space. It is iteratively optimized through the following log-likelihood function:

[0049]

[0050] in: T is the total number of words in the corpus, c is the context window size, Negative sampling or hierarchical Softmax can be used for calculation.

[0051] After the word embedding model is trained, each word w Corresponding to a high-dimensional vector v w , two words w 1 and w The semantic similarity of 2 can be measured by cosine similarity:

[0052]

[0053] in: represents the vector dot product, and are the norms of the vectors respectively. The cosine similarity range should be [-1, 1]. The closer to 1, the more similar the semantics.

[0054] Then, the cosine similarity results between the keyword and each word in the word segmentation library are sorted, and according to the user settings, the N words in the word segmentation library that are most relevant to the keyword are screened out; these N words that are most similar to the keyword are subject-classified through factor analysis, and the subject classification result is a coarse classification.

[0055] After the rough classification is completed, professional personnel can be asked to make fine classification adjustments to the classification results according to needs. Each classification group forms a theme, and finally a theme dictionary consisting of multiple themes is formed.

[0056] Step S4: Segment the big data text.

[0057] The method of segmenting big data text can be selected according to user needs. For example, the complete text database can be divided into several parts according to time, region, author, etc. to facilitate data processing.

[0058] Step S5: Perform local word embedding analysis in each segment, calculate the cosine similarity between the keyword vocabulary and the subject words in the subject dictionary in each segment, and screen out the subject word with the highest cosine similarity between the keywords in each paragraph by analyzing the cosine similarity between the keywords and each subject word. The subject word is then identified as the subject of the segment.

[0059] like Figure 4 As shown in the figure, first, a word embedding algorithm is analyzed in the segment, and the segmented data is used to train the Skip-gram model. Then, the words in the topic dictionary are input into the trained model one by one to obtain the corresponding word vectors. For any topic, the cosine similarity between each word and the keyword in the category group corresponding to the topic is calculated based on the word vector, and the average value is taken as the similarity between the topic and the keyword. Finally, all topics in the topic dictionary are traversed, the topic with the largest similarity is taken, and the topic name is used as the topic name of the segment.

[0060] Step S6: sort out the topics in each segment, summarize the theme change rules of the keywords between segments, and obtain the theme determination results of the keywords in each segment.

[0061] According to step S5, the segmented data of each end is analyzed to obtain the cosine similarity between the keywords and the topics in each segment. The segment number is used as the horizontal axis and the cosine similarity is used as the vertical axis to draw the following diagram: Figure 6 The theme change curve shown in FIG. 1 can be used to determine the theme in each segment and the changing pattern of each theme in the segment by observing the curve.

[0062] In this embodiment, we analyze the Chinese newspaper text data spanning 72 years through the above steps, and use this embodiment to analyze the theme changes of the specific keyword "Chinese" during these 72 years. Figure 5 The data processing result of step S3 is shown. When the user enters the keyword "Chinese", this embodiment will perform word embedding analysis on the global level, sort the keywords and subject words according to the cosine correlation, obtain the 20 words with the highest correlation, and classify these 20 words to form a subject dictionary with 3 topics.

[0063] Figure 6 The figure shows the changes in the themes of the keywords in this embodiment in 15 era segments spanning 72 years. It can be seen that the present invention can analyze long-term text big data and detect the changes in the themes of keywords over time. Figure 7 The topic categories of keywords in each era segment are shown. The text database is divided into 15 segments. Figure 6 The comparison results of the cosine similarity of each segment are taken, and the topic with the highest cosine similarity in each segment is taken as the topic determination result of the keyword in the segment.

[0064] The above description of the embodiments is intended to facilitate understanding and application of the present invention by those skilled in the art. It is apparent that those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without requiring creative effort. Therefore, the present invention is not limited to the above embodiments. Any improvements or modifications made by those skilled in the art based on the disclosure of the present invention should fall within the scope of protection of the present invention.

Claims

1. A keyword-related topic analysis method based on local word embedding technology, comprising the following steps: (1) Clean the text big data to obtain the word segmentation library of the text big data; (2) The user inputs the keywords to be analyzed; (3) Perform global word embedding analysis on the word segmentation database, extract several words closest to the keywords from the word segmentation database based on the global word embedding, classify these words and form a thematic dictionary; (4) Segmenting text big data; (5) Perform local word embedding analysis in each segment, extract the topic closest to the keyword from the topic dictionary based on the local word embedding, and use the topic name as the topic name of the segment; (6) Sort out the topic names of each segment, summarize the theme change patterns of keywords between each segment, and obtain the theme determination results of keywords in each segment.

2. The keyword-related topic analysis method based on local word embedding technology according to claim 1, characterized in that: The text big data includes a large number of historical articles from newspapers, book series, and web pages, and these articles have diverse content from different periods and regions.

3. The keyword-related topic analysis method based on local word embedding technology according to claim 1, characterized in that: The specific implementation method of step (1) is as follows: first, the articles in the text big data are sorted, arranged in chronological order and classified by region; then, useless words in the articles are deleted, and only meaningful texts are retained, and then the word segmentation package in the R language is called, and the word segmentation package is used to perform word segmentation operations on the articles in the text big data to obtain the word segmentation library of the text big data.

4. The keyword-related topic analysis method based on local word embedding technology according to claim 1, characterized in that: The specific implementation of step (3) is as follows: 3.1 Using text big data to train word embedding neural network models; 3.2 Input the words in the word library one by one into the trained model to obtain the corresponding word vectors; 3.3 Calculate the cosine similarity between each word in the word segmentation database and the keyword based on the word vector, and extract the top N words with the largest cosine similarity from the word segmentation database, where N is a natural number greater than 1; 3.4 Divide the extracted words into several category groups, each of which corresponds to a topic, thus forming a topic dictionary consisting of multiple topics.

5. The keyword-related topic analysis method based on local word embedding technology according to claim 4, characterized in that: In step 3.4, the extracted words are first classified by subject using the factor analysis method to complete the rough classification; then, professionals adjust the rough classification results according to needs and define a subject name for each category group.

6. The keyword-related topic analysis method based on local word embedding technology according to claim 1, characterized in that: In the step (4), selection is made according to user needs, and the entire text big data is segmented into several components according to time, region or author.

7. The keyword-related topic analysis method based on local word embedding technology according to claim 4, characterized in that: The specific implementation of step (5) is as follows: 5.1 For any segment, use the segment data to train the word embedding neural network model; 5.2 Input the words in the subject dictionary into the trained model one by one to obtain the corresponding word vectors; 5.3 For any topic, calculate the cosine similarity between each word in the category group corresponding to the topic and the keyword based on the word vector, and take the average value as the similarity between the topic and the keyword; 5.4 According to step 5.3, traverse all topics in the topic dictionary, select the topic with the greatest similarity, and use this topic name as the topic name of the segment.

8. The keyword-related topic analysis method based on local word embedding technology according to claim 4 or 7, characterized in that: The word embedding neural network model adopts Skip-gram.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: The processor is used to execute the computer program to implement a keyword-related topic analysis method based on local word embedding technology as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements a keyword-related topic analysis method based on local word embedding technology as described in any one of claims 1 to 8.

Images