LLM-based rare word theme mining system

Abstract

The invention discloses a rare word theme mining system based on LLM, and relates to the technical field of text analysis and information extraction, and the technical scheme is characterized in that the rare word theme mining system comprises a term extraction module, a semantic space positioning module and a theme distribution module; the invention aims to combine the advantages of a seed-guided topic discovery method and LLM to more accurately identify and mine specific topics related to rare words, the system overcomes the limitation of processing the rare words in the prior art, the ability to understand the meanings of the rare words is improved through an innovative method, the efficient topic discovery process is realized, and the method is suitable for large-scale popularization and application. New breakthrough and innovative application are brought to the fields of text analysis and information extraction, and users are assisted to better understand and mine hidden values in text data.

Description

Technical Field

[0001] The present invention relates to the technical field of text analysis and information extraction, and more specifically, to a rare word topic mining system based on LLM. Background Art

[0002] In the field of text analysis and information extraction, topic discovery has always been an important research direction. Although traditional topic discovery methods, such as Latent Dirichlet Allocation (LDA), have achieved good results, they often fail to accurately capture specific topics that users really care about, especially when dealing with rare words. Rare words include niche words and emerging words, which pose challenges to topic discovery due to their low frequency or recent emergence in the corpus. However, these rare words often contain rich information and unique value, which are of great significance for in-depth understanding of text content and discovery of hidden topics.

[0003] In recent years, with the rapid development of large language models (LLMs), their outstanding performance in natural language processing tasks has attracted widespread attention. LLMs have powerful text generation and understanding capabilities, can capture richer semantic information, and provide new ideas for processing rare words. Therefore, applying LLMs to seed-guided topic discovery, especially topic mining for rare words, has broad research prospects and practical application value. In terms of seed-guided topic discovery, existing technologies mainly guide the process of topic discovery by introducing seed words provided by users. These methods extract relevant topic terms based on the semantic dimensions of seed words, thereby ensuring that the discovered topics are more in line with the specific needs of users. However, when dealing with rare words, these methods are often affected by high-frequency words in the corpus, making it difficult to accurately capture topics related to rare words.

[0004] At present, topic discovery methods based on LLMs are gradually being explored. The powerful text generation and understanding capabilities of LLMs enable them to better handle low-frequency and emerging vocabulary. By using LLMs to generate and parse text data, these methods are expected to more accurately capture topic information related to rare words. Currently, topic discovery methods based on LLMs still face some challenges when dealing with rare words. Although LLMs can generate texts containing rare words, how to accurately understand the semantics of these rare words remains a problem.

[0005] In order to overcome the above challenges, the present invention provides a rare word topic mining system based on LLM, which aims to discover rare words that users are interested in.

[0006] The purpose of the present invention is to provide a rare word topic mining system based on LLM to solve the problems such as the limitations of understanding and processing rare words in the prior art.

[0007] The above technical objectives of the present invention are achieved through the following technical solutions: a rare word topic mining system based on LLM, including a term extraction module, a semantic space positioning module and a topic assignment module;

[0008] The term extraction module uses a large language model LLM to extract all terms in the corpus, generates a set of terms based on the input document, and then traverses the entire corpus to obtain a complete term set T;

[0009] After extracting all terms, the semantic space positioning module analyzes each term separately. i ,The analysis of a term is divided into two parts according to whether it is an emerging word, and detects whether a term is an emerging word by querying the LLM and judging it based on the confidence of its output;

[0010] The topic assignment module searches for seed topics in the current semantic context, completes topic assignment, and obtains the topic corresponding to the seed topic s i The term set T i .

[0011] The present invention is further configured such that: the semantic space positioning module adopts a context-guided semantic positioning strategy and a retrieval-enhanced semantic positioning strategy.

[0012] The present invention is further configured as follows: the retrieval-enhanced semantic localization strategy comprises the following steps:

[0013] S1: LLM tries to infer the term t from the context i The general meaning of

[0014] S2: Search and term t i The latest relevant Internet information, denoted as R(t i );

[0015] S3: Select and M i The most similar information i , calculate the Mi and the R(t i ), the calculation formula is as follows:

[0016] r i =argmax(Sim-sbert(Mi, R(t i )));

[0017] S4: Check the retrieved knowledge i Does it match the contextual meaning? If so, update the LLM's understanding of the emerging term.

[0018] The present invention is further configured such that: the context-guided semantic localization strategy inputs multiple contexts containing rare words into the LLM to help it accurately determine the semantic space of the term according to the given context.

[0019] The present invention is further configured as follows: the topic allocation module uses LLM as the term set T i Generate a corresponding explanation, according to which the explanation is extracted from the term set T i Select the top K relevant terms in .

[0020] The present invention is further configured that: the LLM selects the first K relevant terms from the term set Ti according to the interpretation, which can be formalized as the following formula:

[0021] ex i , topic i =LLM(x i , prompt3.4);

[0022] Among them, topic i The term t i The seed theme, while ex i is the term t within the subject i The formula for obtaining the top k terms of topic i is as follows:

[0023] topK i =LLM(T i , K, prompt3.5).

[0024] By adopting the above technical solution, the present invention realizes the whole process of extracting rare words from the corpus, accurately locating their semantic space and intelligently assigning topics through the collaborative work of three core modules; firstly, the term extraction module uses the ability of LLM to generate a set of terms according to the input document, and traverses the entire corpus to obtain a complete term set T, so that the context understanding ability of LLM can be used to identify rare terms that may be missed by traditional statistical methods, thereby laying a solid foundation for subsequent analysis;

[0025] Secondly, the semantic space positioning module analyzes the extracted terms separately and adopts different strategies according to whether the terms are emerging words. For niche terms, a context-guided semantic positioning strategy is adopted. By inputting multiple contexts containing rare words, LLM can accurately determine the semantic space of the terms. For emerging words, a retrieval-enhanced semantic positioning strategy is adopted. By retrieving the latest information on the Internet and calculating the similarity with the general meaning, the most relevant information is selected to update LLM's understanding. This not only improves the recognition and understanding of emerging terms, but also reduces the confusion between terms in different fields.

[0026] Finally, the topic assignment module queries the seed topic in the current semantic context, completes the topic assignment, and obtains the topic corresponding to the seed topic s i The term set T i This module uses LLM as the terminology set T i Generate corresponding explanations and use these explanations to get the i The top K relevant terms are selected from the dataset, which not only improves the accuracy of topic assignment but also enhances the relevance and practicality of the results.

[0027] In summary, the present invention has the following beneficial effects:

[0028] 1. By utilizing the advanced contextual understanding capabilities of LLM, the present invention can more accurately identify and extract rare terms from large amounts of text, including those that may be overlooked by traditional statistical methods; through the retrieval-enhanced semantic localization strategy, the present invention enables the system to adapt to the development of language, effectively understand and process emerging vocabulary, and ensure timely capture and accurate interpretation of the latest terms; the topic assignment module of the present invention selects the terms most relevant to the topic through the interpretations generated by LLM, thereby improving the accuracy and relevance of topic assignment and making the results more in line with user needs;

[0029] 2. The present invention reduces manual intervention through automated term extraction and semantic positioning processes, improves the efficiency of processing large amounts of corpus, and makes rare word topic mining faster and more efficient; the system can adapt to different fields and ever-changing language environments, has good flexibility and adaptability, and can cope with various complex corpus analysis tasks; the present invention adopts context guidance and retrieval enhancement strategies, so that the system can reduce confusion and misjudgment of niche terms and emerging vocabulary, and improve the accuracy of semantic positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a structural schematic diagram of embodiment 1 of the present invention;

[0031] Figure 2 This is a schematic diagram of the process in Example 1 of the present invention;

[0032] Figure 3 is the statistical information of the NYT, Yelp, Soccer and NBA data sets in Example 2 of the present invention;

[0033] Figure 4 is the statistical information of the first five terms retrieved by different algorithms in Example 2 of the present invention;

[0034] Figure 5 is a comparison chart of ablation analysis performance in Example 2 of the present invention;

[0035] Figure 6is a performance comparison chart of four data sets under different confidence thresholds in Example 2 of the present invention;

[0036] Figure 7 It is a flowchart of Prompt3.1 to Prompt3.5 used in the rare word topic mining system based on LLM in Example 1 of the present invention;

[0037] Figure 8 It is the LLM prompt based on term interpretation in Example 2 of the present invention. DETAILED DESCRIPTION

[0038] The following is combined with Figure 1-8 The present invention is described in further detail.

[0039] Example 1: A rare word topic mining system based on LLM, such as Figure 1 As shown, it includes a term extraction module, a semantic space positioning module and a topic assignment module;

[0040] Term extraction module: The term extraction module uses a large language model LLM to extract all terms in the corpus, generates a set of terms based on the input document, and then traverses the entire corpus to obtain the complete term set T;

[0041] After extracting all the term sets T from the corpus, the present invention analyzes each term t separately i , the analysis of a term is divided into two parts according to whether it is an emerging word. Since the large language model (LLM) may produce hallucinations when encountering unfamiliar information, we detect these hallucinations to determine whether a term is an emerging word. The present invention uses Prompt3.1 to query LLM to determine whether LLM is familiar with the term, and determines whether the term is an emerging word based on the confidence of the LLM output.

[0042] Semantic space positioning module: The semantic space positioning module of the present invention mainly adopts a semantic positioning strategy based on context guidance and a semantic positioning strategy enhanced by retrieval;

[0043] Context-guided semantic localization strategy: Even if LLM has a high confidence level for the term, the term t i It may also be a niche word known to LLM. Due to the professionalism of such terms, LLM may confuse them with terms from other fields when there is insufficient domain-specific information. Therefore, a lot of domain background knowledge is needed to guide LLM to capture the semantics of a specific field. To this end, Figure 1 As shown in the purple box, the present invention adopts a context-guided semantic space positioning strategy, which inputs multiple contexts containing rare words into the LLM to help it accurately determine the semantic space of the term according to the given context.

[0044] Retrieval-enhanced semantic localization strategy: If the output probability is below a threshold, then term t i Considered as emerging words in social networks, large language models lack relevant knowledge and can only infer approximate semantics even with the context of emerging terms. Therefore, the present invention retrieves the latest Internet information to enhance the understanding of large models, such as Figure 1 As shown in the red box, the present invention first uses prompt3.2 to infer the general meaning in the context. Then, the latest information corresponding to the term is retrieved, and SentenceBERT is used to calculate the similarity between this information and the general meaning, and the most relevant information is selected to filter out the noise.

[0045] Let the term t i The general meaning is M i , the corresponding search result is R(t i ), the formula for filtering out the most relevant information using general meaning is as follows:

[0046] r i =argmax(Sim-sbert(M i , R(t i )));

[0047] Finally, the present invention uses prompt3.3 to check whether the retrieved knowledge matches the contextual meaning, thereby updating LLM's understanding of emerging terms.

[0048] The above process can be formalized as follows: Assume that the rare word t is being analyzed i The exact semantics of x i We first use prompt3.1 to query each term to obtain y = LLM (t i , prompt3.1) and the confidence level of the LLM for the answer. Subsequently, we determine whether the term is new based on the output probability value of y (i.e., the confidence level), which leads to two situations, which can be formally expressed as follows:

[0049]

[0050] Among them, pro(y) represents the probability value of the output tag y, θ represents the threshold used to distinguish new words from known words based on the probability output by LLM, M represents the approximate meaning of the term inferred by LLM based on the context in the corpus, and R(t i ) represents the term t i Search results on the Internet.

[0051] Topic assignment module: After the present invention accurately locates the exact semantic space of rare words through multiple rounds of historical dialogues, the topic assignment module of the present invention can query the seed topic in the current semantic context, complete the topic assignment, and obtain the corresponding seed topic s i The term set T i .

[0052] In addition, to identify the top K most relevant terms under each seed topic, we use a large language model to generate corresponding explanations for these topics. Then, LLM selects the top K relevant terms from the term set Ti according to the explanations, which can be formalized as the following formula:

[0053] ex i , topic i =LLM(x i , prompt3.4);

[0054] Among them, topic i The term t i The seed theme, while ex i is the term t within the subject i The formula for obtaining the top k terms of topic i is as follows:

[0055] topK i =LLM(T i , K, prompt3.5).

[0056] The five key prompts prompt3.1-prompt3.5 are as follows Figure 7 shown.

[0057] Verification Example 2:

[0058] Experimental setting dataset:

[0059] During the experimental verification phase, the present invention conducted experiments on two classic public datasets: NYT (New York Times news dataset) and Yelp (restaurant review dataset from Yelp); the present invention also adopted two specially collected new datasets, namely Soccer and NBA, which are designed to capture rare and professional terms frequently used in specific sports communities.

[0060] Soccer dataset: The Soccer dataset contains comments from football fans in the Reddit community r / soccer. The time span of comment collection is from January 10, 2024 to February 10, 2024. These comments contain a wealth of unique terms related to football culture. Six well-known football clubs are selected as seed words, and the system of the present invention is used to mine rare topic terms related to these clubs.

[0061] NBA dataset: The NBA dataset comes from the Reddit community r / nba, and contains comments from basketball fans in the same time period. The time range of the comments is also from January 10, 2024 to February 10, 2024. The comments contain professional terms related to NBA culture and use eight famous NBA team names as seed words to explore obscure topic terms related to these teams.

[0062] Dataset division and statistics: In order to evaluate the performance of the system, all data sets are divided into 80% training data and 20% test data. Specific data set statistics include the number of terms, the number of comments, etc. Figure 3 shown.

[0063] This invention aims to verify the effectiveness and accuracy of the system in dealing with rare and specialized terms in specific fields. By analyzing comments in football and basketball communities, the system is able to mine rare subject terms related to specific clubs or teams, thereby demonstrating its application potential and practical value in specific fields.

[0064] Baseline Model:

[0065] When evaluating the rare word topic mining system of the present invention, the present invention sets three types of baseline models for comparison to ensure the comprehensiveness and accuracy of the evaluation. These baseline models include target topic models, seed-guided topic models, and topic discovery methods based on large language models (LLMs).

[0066] The target topic models include SeededLDA and FoTo. SeededLDA uses seed words to generate topics and documents containing these words, while FoTo is the latest model designed specifically for social short texts and can mine specific topics therein.

[0067] Seed-guided topic models include CatE and SeedTopicMine, which learn term embedding and category discrimination through seed word guidance to select terms that distinguish different categories. Seed-guided topic models can jointly learn term embedding and specificity in the input corpus, thereby improving the accuracy of topic mining.

[0068] Based on the topic discovery system of LLMs, this paper selects GPT4 as one of the baselines. GPT4 adds multiple seed topics and selects terms related to these seed words from the document. For fair comparison, GPT4 also uses a large model to select the top K most relevant terms based on the interpretation of the terms.

[0069] By setting these baseline models, we can comprehensively evaluate the performance of the system proposed in this invention in mining rare word topics. These baseline models cover different topic mining methods, thus ensuring the comprehensiveness and accuracy of the evaluation. By comparing with these baseline models, the advantages and innovations of the system of this invention can be more clearly demonstrated.

[0070] Evaluation Protocol:

[0071] 1. Assessment of thematic coherence

[0072] When evaluating the rare word topic mining system of the present invention, a comprehensive set of evaluation protocols is adopted to ensure the accuracy and reliability of the evaluation results. Since traditional topic coherence metrics (such as NPMI) are not suitable for seed-guided discovery containing many rare terms, we introduce two new metrics: Intrusion Detection Task (IDT) and Coherence Rating Task (CRT), which use large language models (LLMs) to evaluate topic coherence and are more relevant to human judgment.

[0073] Intrusion Detection Task (IDT): IDT constructs an intrusion vocabulary W containing words from different topics. i , and then randomly select a i Select 5 words from the first 10 words of W and add one from W i The invasion word w k , forming a detection list L k . Using large language models to detect intrusion words w k , which can effectively evaluate the consistency of vocabulary in the topic and the recognition ability of foreign vocabulary, such as Figure 8 As shown, the specific calculation method of the intrusion detection task (IDT) is as follows:

[0074] f(w k )=LLM(L k ,Prompt IDT )

[0075]

[0076] Among them, 1(w k =f(w k )) is an indicator function, indicating that the large language model has detected the intrusion word as w k . Coherence Rating Task (CRT): CRT is performed by i Randomly select 5 words from the evaluation topic T 5i, and create three evaluation topics for each topic (("1" = not very related, "2" = moderately related, "3" = very related). These evaluation topics are then evaluated using a large language model. This method can evaluate the coherence and integrity of the vocabulary in the topic. The formula for calculating the CRT metric is as follows:

[0077]

[0078] We use the first 10 words per topic as input to the large language model for evaluation. For the words generated by other baseline models, due to the lack of explanations, we use the large language model to add standard explanations to them. Specifically, the large language model generates corresponding explanations based on the topics it determines. The final result is the average of the results obtained by running the evaluation metrics 10 times.

[0079] 2. Vocabulary Accuracy Assessment

[0080] In addition to the topic coherence metric, we also used MACC, a relevance metric used in previous work to determine the relevance between words and topics. The specific formula is as follows:

[0081]

[0082] where t i，j Represents the subject T i The jth most relevant word in i，j ∈si) is an indicator function used to determine t i，j Is it s i (ie t i，j Is it related to seeds? i Since this evaluation relies on manual judgment, we invited five annotators to perform independent evaluations. The reported MACC@K score reflects the average MACC@K of these five annotators. A high level of inter-annotator agreement was observed, with Fleiss' kappa scores of 0.894, 0.870, 0.821, and 0.856 for the NYT, Yelp, Soccer, and NBA datasets, respectively. As mentioned earlier, K=10 was set in our experiments, so we report the MACC@10 scores.

[0083] By introducing two new evaluation methods, IDT and CRT, and using MACC as the correlation index, the present invention can more accurately evaluate the performance of the system of the present invention in mining rare word topics and ensure the reliability and effectiveness of the evaluation results.

[0084] Quantitative analysis of overall performance:

[0085] Figure 4 The comparison results show the performance advantages of the proposed rare word topic mining model (RWTM) and GPT-4 based on large language model (LLM) on four different datasets, which cover a variety of fields, making the comparison results widely representative and convincing. In the intrusion detection task (IDT), both RWTM and GPT-4 showed excellent performance, significantly higher than other seed-guided models and target topic models, which shows that these two LLM-based methods have obvious advantages in identifying and excluding foreign words that are not related to the topic; in the coherence rating task (CRT), RWTM and GPT-4 also performed well, further confirming their ability to construct coherent and relevant topics; in terms of vocabulary accuracy, the MACC scores of RWTM and GPT-4 are also significantly higher than other traditional methods, reflecting their high accuracy in accurately identifying words related to specific topics.

[0086] The main reason why RWTM and GPT-4 can significantly outperform other methods is that they do not rely on the traditional word co-occurrence mechanism. Instead, they use the powerful semantic understanding ability of LLM to truly understand the relevance between words and seed topics, rather than simply based on the frequency of word occurrence; Figure 4 The results clearly show that the LLM-based method has significant advantages in rare word topic mining, forming an insurmountable methodological barrier. These results not only verify the effectiveness of RWTM, but also demonstrate the wide application potential of LLM in the field of natural language processing.

[0087] Qualitative analysis of overall performance:

[0088] The present invention compares the traditional method with the rare word topic mining system based on LLM through overall performance qualitative analysis. Figure 5 As shown in the figure, in this analysis, we randomly selected two seed words from the Soccer and NBA datasets and showed the top 5 words retrieved by each method, which are mainly reflected in the following points:

[0089] SeededLDA performance: SeededLDA often retrieves irrelevant or overly general words. For example, in the Soccer dataset, when the seed word is "Manchester United FC", the retrieved word "GOALS" lacks uniqueness and does not belong to the category specific to Manchester United, which shows that SeededLDA has obvious deficiencies in topic focus and vocabulary selection.

[0090] Performance of CatE and SeedTopicMine: Although some of the words found by CatE and SeedTopicMine are accurate, there are still a large number of errors in both categories. These models are able to identify relevant words in some cases, but the overall accuracy is not as good as the LLM-based rare word topic mining system; CatE relies on word co-occurrence within the corpus, while SeedTopicMine utilizes pre-training knowledge, which may cause the words they retrieve to include content related to other football clubs rather than words uniquely related to the "Manchester United" club.

[0091] In contrast, the rare word topic mining system based on LLM provided by the present invention is significantly better than the traditional method in terms of accuracy, can better understand the semantics of seed words, and retrieve words that are highly relevant to specific topics. The present invention not only improves the uniqueness of words, but also ensures the relevance of retrieval results. Through qualitative analysis, the rare word topic mining system based on LLM provided by the present invention performs well in mining rare words and topics, and can effectively avoid the problems of generalization and irrelevant words commonly found in traditional methods.

[0092] Overall performance ablation analysis:

[0093] Since the MACC@10 metric is not sufficient to distinguish the differences between GPT-4 and our model and the changes between other cut models, we introduce a new metric. Specifically, we calculate the proportion of correctly recognized words in each topic category to quantify the performance gap. The calculation formula of the overall evaluation metric ACC is as follows:

[0094]

[0095] In order to observe the performance difference of seed-guided topic discovery task in extreme cases, the present invention divides the large corpus into smaller subsets so that the relevant terms of each seed topic become scarce. The present invention divides the large corpus into smaller subsets so that the terms corresponding to each seed topic are particularly scarce. Assuming that the data is divided into N equal parts, the calculation formula of this extreme indicator MACCmini is as follows:

[0096]

[0097] Where (MACC@10)i represents the MACC@10 of the dataset in the i-th small corpus. The corpus used is 5,000 reviews of the Soccer and NBA datasets, and each batch contains 100 reviews, so N=50.

[0098] To verify the performance of each part of the model, four variants are as follows:

[0099] GPT4: It can be regarded as a variant that does not consider the context in which the terms are located, and directly allows the large language model to extract terms related to various topics from the corpus.

[0100] Terms: Use a statistically based phrase mining algorithm to capture the comprehensive vocabulary within the corpus, rather than relying on a large language model, while keeping other modules unchanged.

[0101] RAG: Eliminating the external knowledge retrieval process.

[0102] Filtering: When retrieving external knowledge, the internal term meaning is not used to re-rank the search results, but the best search results are directly selected.

[0103] Observation results such as Figure 5 As shown, in the evaluation of the RWTM model and its variants, we found that RWTM outperformed the four abridged versions in all four evaluation aspects. This significant advantage shows that combining context understanding, external knowledge retrieval, and knowledge from a large language model (LLM) is crucial for mining rare terms from specific topics. In addition, compared to GPT4, RWTM is able to avoid the hallucination problem that may be caused by direct term extraction, as the latter may incorrectly include or fail to extract novel terms without context constraints. For example, GPT4 may misclassify the abbreviation "ETH" or classify emerging terms such as "Onana" into irrelevant categories.

[0104] Furthermore, our analysis of the Terms variants reveals the limitations of relying solely on statistics-based phrase mining algorithms for term extraction. This approach often results in the generation of a large number of meaningless terms such as “this” and “uncalled”, which in turn produces a significant drop in accuracy (ACC) metrics. This is because large language models may incorrectly assign these terms to specific topics in the absence of valid contextual information. Therefore, the RWTM model demonstrates its superior performance in the rare word topic mining task through its comprehensive approach, which leverages the capabilities of contextual information, external knowledge retrieval, and LLM, and highlights the importance of adopting such a comprehensive approach.

[0105] Overall performance parameter analysis:

[0106] A core feature of the RWTM model is to use the confidence threshold of the large language model (LLM) to classify rare words. To further analyze the impact of this confidence threshold on model performance, we conducted experiments on four different datasets to explore the accuracy (ACC) at different threshold levels.

[0107] The experimental results are as follows Figure 6As shown, the optimal threshold is not 1, which indicates that retrieving external knowledge for all terms may lead to suboptimal results. This is because searching for some terms may introduce wrong information, thus affecting the accuracy of term interpretation. For example, searching for "ETH" may bring noise information related to Ethereum or ETH Zurich, while the correct semantics of these terms need to rely on contextual information in the corpus to infer.

[0108] For the Yelp and NYT datasets, the optimal confidence threshold is significantly higher than that for the Soccer and NBA datasets. This difference suggests that the two classic datasets, Yelp and NYT, rely more on context-guided semantic localization rather than retrieving new knowledge. For these classic datasets, the semantics of many terms have been learned by large language models, and over-retrieval may introduce noise and affect the performance of the model.

[0109] The above experimental results emphasize the importance of setting a reasonable confidence threshold when applying the RWTM model. By carefully selecting the threshold, the model can more effectively balance internal knowledge and external retrieval, thereby improving the classification accuracy of rare words. At the same time, this also shows the impact of the characteristics of different datasets on the model parameter settings, suggesting that in practical applications, we need to adjust the model configuration according to the characteristics of the dataset to achieve the best mining effect.

[0110] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A rare word topic mining system based on LLM, characterized by: It includes term extraction module, semantic space positioning module and topic assignment module; The term extraction module uses a large language model LLM to extract all terms in the corpus, generates a set of terms based on the input document, and then traverses the entire corpus to obtain a complete term set T; The semantic space positioning module, after extracting all the terms, analyzes each term ti separately, dividing its analysis into two parts according to whether the term is an emerging word, and detects whether the term is an emerging word by querying the LLM and judging it according to the confidence of its output; The topic assignment module searches for seed topics in the current semantic context, completes topic assignment, and obtains the topic corresponding to the seed topic s i The term set T i .

2. The rare word topic mining system based on LLM according to claim 1 is characterized by: The semantic space positioning module adopts a context-guided semantic positioning strategy and a retrieval-enhanced semantic positioning strategy.

3. The rare word topic mining system based on LLM according to claim 2 is characterized by: The context-guided semantic localization strategy inputs multiple contexts containing rare words into the LLM to help it accurately determine the semantic space of a term based on a given context.

4. The rare word topic mining system based on LLM according to claim 2 is characterized by: The retrieval-enhanced semantic localization strategy includes the following steps: S1: LLM tries to infer the term t from the context i The general meaning of S2: Search and term t i The latest relevant Internet information, denoted as R(t i ); S3: Select and M i The most similar information i , calculate the similarity between Mi and R(ti), the calculation formula is as follows: <h2 style=";text-align:left;direction:ltr">r<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> = argmax(Sim-sbert(Mi) + R(t)<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> ))); S4: Check the retrieved knowledge i Does it match the contextual meaning? If so, update the LLM's understanding of the emerging term.

5. The rare word topic mining system based on LLM according to claim 1 is characterized by: The topic assignment module uses LLM to assign terminology to the term set T i Generate a corresponding explanation, according to which the explanation is extracted from the term set T i Select the top K relevant terms in .

6. The rare word topic mining system based on LLM according to claim 5 is characterized by: The LLM is based on the interpretation from the terminology set T i Selecting the top K relevant terms can be formalized as the following formula: ex i ,topic i =LLM(x i ,prompt3.4); Among them, topic i The term t i The seed theme, while ex i is the term t within the subject i The formula for obtaining the top k terms of topic i is as follows: topK i =LLM(T i ,K,prompt3.5)。

Images