A hotspot extraction method, storage medium and server
By pre-training the corpus data and adjusting the twin network parameters, combining clustering algorithms and semantic scores and keyword scores, the problem of unsatisfactory hot spot extraction effect in the existing technology is solved, and higher accuracy and clustering are achieved Class effects.
Patent Information
- Application Number
- CN202010950134.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-10
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-09-10
AI Technical Summary
In the existing automatic hotspot extraction method, the clustering effect is limited, resulting in the returned topn hotspots often have the same meaning category, and the extraction effect is not ideal.
By pre-training the corpus data of the hot spots to be extracted, the roberta model in professional fields is obtained, the twin network training samples are constructed, the model parameters are adjusted, and the hot spots are extracted by combining clustering algorithms and semantic scores and keyword scores.
It improves the accuracy and clustering effect of hot spot extraction, reduces the unk situation of words, optimizes the similarity distance, makes similar text more compact, and dissimilar text more scattered, improving the accuracy of hot spots.
Smart Images

Figure CN112131463B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a hotspot extraction method, a storage medium and a server. Background Art
[0002] In today's era, with the development of network information technology, text data is growing rapidly. Using computers to automatically process information and timely and accurately mine hot topics from this information is of great significance for understanding the latest public opinion hotspots, studying sudden hotspots, and grasping the trends of future hot topics. The current hot spot automatic extraction method generally uses a clustering algorithm to obtain the corresponding categories, and then extracts keywords from each category as the final hot spot result. Limited by the clustering effect, the topn hot spots returned often have categories with the same meaning, and the effect of hot spot extraction is not ideal. Summary of the invention
[0003] The present invention provides a hotspot extraction method, a storage medium and a server, which are used to improve the accuracy of hotspot extraction.
[0004] In a first aspect, the present invention provides a hotspot extraction method, the hotspot extraction method comprising:
[0005] Obtaining corpus data of hot spots to be extracted, wherein the corpus data includes multiple texts;
[0006] Pre-train the Roberta model in the general field based on the corpus data to obtain the Roberta model in the professional field;
[0007] According to the Roberta model in the professional field, the feature vector of each text in multiple texts is extracted;
[0008] Construct training samples of the twin network based on the feature vector of each text in multiple texts;
[0009] According to the training samples, the parameters of the Roberta model in the professional field are adjusted through the twin network to obtain the target Roberta model;
[0010] According to the target Roberta model, extract the feature vector of each text in multiple texts;
[0011] Clustering algorithms are used to cluster multiple texts of corpus data to obtain multiple categories;
[0012] Extract hot spots of corpus data according to the target roberta model and multiple categories.
[0013] In the above scheme, by pre-training the roberta model in the general field with the corpus data of the hot spots to be extracted, the roberta model can better capture the unique information in the corpus data of the hot spots to be extracted. And pre-training can reduce the unk (unknown) situation of the characters in the corpus data of the hot spots to be extracted, laying the foundation for the subsequent more accurate extraction of the hot spots of the corpus data. In addition, by adjusting the parameters of the roberta model in the form of a twin network, the similarity distance can be directly optimized, so that similar texts are more compact in distance and dissimilar texts are more dispersed in distance, thereby improving the accuracy of subsequent clustering and improving the accuracy of subsequent extraction of hot spots in the corpus data.
[0014] In a specific implementation, according to the roberta model in a professional field, the feature vector of each text in multiple texts is extracted specifically as follows: taking the last set number of layers of the roberta model in the professional field, averaging the feature vectors of each word in each layer of the set number of layers; adding the feature vectors of the set number of layers, taking the average value, and obtaining the feature vector of each text. By fusing the feature vectors of the last few layers, the feature representation of the acquired text can be improved, so that the feature vector of the text has complete semantic information, and the effect of subsequent clustering and the accuracy of hotspot extraction are improved.
[0015] In a specific embodiment, constructing a training sample of a twin network based on the feature vector of each text in a plurality of texts includes: forming a plurality of text pairs with each text in the plurality of texts and each other text except the text, each text pair including two texts; calculating the similarity between the feature vectors of the two texts in each text pair; sorting the plurality of text pairs according to the similarity from high to low; selecting a text pair whose similarity is in the first proportion value as a positive sample of the training sample; and selecting a text pair whose similarity is in the second proportion value as a negative sample of the training sample.
[0016] In a specific implementation, calculating the similarity between the feature vectors of two texts in each text pair specifically includes: calculating the cosine similarity between the feature vectors of the two texts in each text pair.
[0017] In a specific implementation, according to the training samples, the parameters of the Roberta model in the professional field are adjusted by means of a twin network, and the target Roberta model is obtained, including: constructing two Roberta models with shared parameters according to the Roberta model in the professional field; inputting the two texts in each text pair of the positive sample and the negative sample of the training sample into the two Roberta models with shared parameters, and outputting the feature vectors of the two texts; calculating the cosine similarity between the feature vectors of the two texts; and optimizing the parameters of the two Roberta models with shared parameters by means of a contrastive loss function to obtain the target Roberta model. In order to adjust the parameters of the Roberta model in the professional field.
[0018] In a specific implementation, the contrastive loss function is:
[0019]
[0020] Where y represents the label value 1 of the positive sample or the label value 0 of the negative sample;
[0021] d represents the cosine distance between two texts in a text pair of positive or negative samples;
[0022] Margin represents the cosine distance interval set to fine-tune the parameters of the roberta model in the professional field.
[0023] In a specific implementation, the clustering algorithm is a density-based DBSCAN clustering algorithm. It is not necessary to determine the number of categories, and the categories can be merged in subsequent processing operations to improve the clustering effect.
[0024] In a specific implementation, a clustering algorithm is used to cluster multiple texts of the corpus data to obtain multiple categories:
[0025] Step 1: Select the N_Sample parameter of the DBSCAN clustering algorithm as 1 to cluster multiple texts of the corpus data;
[0026] Step 2: Delete the outliers to obtain clustering results with multiple different categories;
[0027] Step 3: Calculate the average of the feature vectors of the text in each category of the clustering result to obtain the centroid of each category of the clustering result;
[0028] Step 4: Calculate the cosine similarity between the centroids of any two categories in different categories of the clustering results; if the cosine similarity between the centroids of two categories is greater than the set threshold, the two categories are merged;
[0029] Repeat steps 3 to 4 above until the cosine similarity between the centroids of any two categories is no greater than the set threshold, and output multiple categories to reduce the similarity between different categories and improve the clustering effect.
[0030] In a specific implementation, the hot spots of the corpus data are extracted according to the target roberta model and multiple categories: the text with the highest semantic score and keyword score in each category of the multiple categories is extracted as the hot spots of the corpus data according to the method of combining semantic score and keyword score. Using text instead of keywords for display can more intuitively express the meaning of each category.
[0031] In a specific implementation, according to a method combining semantic scores and keyword scores, extracting the text with the highest semantic scores and keyword scores in each of the multiple categories as the hot spots of the corpus data is specifically as follows:
[0032] The feature vectors of the texts contained in each category are averaged to obtain the centroid of each category;
[0033] Calculate the cosine similarity between the centroid and each text in the category to obtain the semantic score of each text in the category;
[0034] Perform word segmentation on the text contained in each category;
[0035] Extract keywords for each category using the tf-idf method;
[0036] Sort by importance and select the top n keywords;
[0037] The keyword quantity feature is obtained according to the number of keywords contained in each text in the category;
[0038] Divide the keyword quantity feature by n to get the keyword score of each text in the category;
[0039] Select the text with the highest sum of semantic score and keyword score in each category as the template of that category;
[0040] Sort the templates of multiple categories according to the number of text items contained in each category;
[0041] The templates corresponding to the first h categories are selected as the hot spots of the corpus data.
[0042] By selecting templates in categories that rank among the top few in terms of the number of texts contained in the categories as hot templates, the accuracy of template extraction can be improved.
[0043] In a second aspect, the present invention further provides a storage medium storing a computer program, which, when executed on a computer, enables the computer to execute any one of the above-mentioned hotspot extraction methods.
[0044] In a third aspect, the present invention further provides a server, comprising a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute any of the above-mentioned hotspot extraction methods by calling the computer program stored in the memory. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 A flowchart of a hotspot extraction method provided by an embodiment of the present invention;
[0046] Figure 2 A schematic diagram of adjusting the parameters of the Roberta model in a professional field by means of a twin network according to training samples provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0048] In order to facilitate understanding of the hotspot extraction method provided by the embodiment of the present invention, the application scenario of the hotspot extraction method provided by the embodiment of the present invention is first described below. The hotspot extraction method is used to extract hotspots in a corpus data. The hotspot extraction method is described in detail below with reference to the accompanying drawings.
[0049] refer to Figure 1 , the hotspot extraction method provided by the embodiment of the present invention includes:
[0050] S10: Obtaining corpus data of hot spots to be extracted, wherein the corpus data includes multiple texts;
[0051] S20: pre-train the roberta model in the general field according to the corpus data to obtain the roberta model in the professional field;
[0052] S30: extracting a feature vector of each text in the multiple texts according to the Roberta model in the professional field;
[0053] S40: constructing a training sample of the twin network according to the feature vector of each text in the multiple texts;
[0054] S50: According to the training samples, the parameters of the Roberta model in the professional field are adjusted by means of the twin network to obtain the target Roberta model;
[0055] S60: extracting a feature vector of each text in the multiple texts according to the target Roberta model;
[0056] S70: clustering multiple texts of the corpus data using a clustering algorithm to obtain multiple categories;
[0057] S80: Extract hot spots of corpus data according to the target Roberta model and multiple categories.
[0058] In the above scheme, by pre-training the roberta model in the general field using the corpus data of the hot spots to be extracted, the roberta model can better capture the unique information in the corpus data of the hot spots to be extracted. And pre-training can reduce the unk (unknown) situation of the characters in the corpus data of the hot spots to be extracted, laying the foundation for the subsequent more accurate extraction of the hot spots of the corpus data. In addition, by adjusting the parameters of the roberta model in the form of a twin network, the similarity distance can be directly optimized, so that similar texts are more compact in distance and dissimilar texts are more dispersed in distance, thereby improving the accuracy of subsequent clustering and improving the accuracy of subsequent extraction of hot spots in the corpus data. The following is a detailed introduction to each of the above steps in conjunction with the accompanying drawings.
[0059] First, the corpus data of the hot spots to be extracted is obtained, wherein the corpus data includes multiple texts. The corpus data can be a sports news, financial news, military news, social news, entertainment news, historical news, etc.
[0060] Next, the roberta model in the general field is pre-trained according to the corpus data to obtain the roberta model in the professional field. Among them, the roberta model in the general field can be a general roberta model trained based on databases such as Chinese Wikipedia, Baidu Encyclopedia, Sina and Weibo. The specific method of pre-training the roberta model in the general field using the corpus data of the hot spots to be extracted to obtain the roberta model in the professional field is the pre-training method in the prior art. The general roberta model will be cut at the character level according to the vocab.txt file. In the corpus data of the hot spots to be extracted, if there are words that are not included in vocab.txt, they will be added to vocab.txt, and the embedding layer of the general roberta model will be expanded according to the length of vocab.txt.
[0061] Next, according to the roberta model in the professional field, extract the feature vector of each text in the multiple texts. When specifically extracting, you can first take the last set number of layers of the roberta model in the professional field, and average the feature vectors of each word in each layer of the set number of layers; then add the feature vectors of the set number of layers, take the average value, and get the feature vector of each text. By fusing the feature vectors of the last few layers, the feature representation of the acquired text can be improved, so that the feature vector of the text has complete semantic information, and the effect of subsequent clustering and the accuracy of hotspot extraction are improved. When specifically determining the number of the last set number of layers, the last set number of layers can be the last 2 layers, the last 3 layers, the last 4 layers, the last 5 layers, etc. The following formula can be used for calculation:
[0062]
[0063] Where V represents the output feature vector;
[0064] i represents a layer of the Robert model in the above professional field, and its value range is from 1 to the number of layers set at the end, which is 4 here;
[0065] j represents a character input to the roberta model. The value range of j is 1 to m, where m represents the length of the text.
[0066] W represents the feature vector of a character input to the roberta model.
[0067] The roberta model is a 12-layer transformer model. The feature vector of each word extracted contains semantic information. The feature vector of the sentence (i.e. each text) is obtained by averaging, and the semantic information of the sentence can be obtained. The features of the last few layers are slightly different. By fusing the features of the last few layers, the obtained feature representation can be improved, so that it has complete semantic information, and the effect of subsequent clustering and the accuracy of hotspot extraction can be improved.
[0068] Next, according to the feature vector of each text in the multiple texts, the training sample of the twin network is constructed. When constructing the training sample, each text in the multiple texts can be first combined with each other text except the text to form multiple text pairs, each text pair includes two texts; then, the similarity between the feature vectors of the two texts in each text pair is calculated; then, the multiple text pairs are sorted from high to low according to the similarity; then, the text pairs with the similarity in the first ratio value are selected as the positive samples of the training samples; then, the text pairs with the similarity in the second ratio value are selected as the negative samples of the training samples. Among them, the first ratio value can be 5%, 10%, 15%, 20%, etc. The second ratio value can be 5%, 10%, 15%, 20%, etc. And the first ratio value and the second ratio value can be equal or unequal. By constructing positive and negative samples, the accuracy of the parameters of the subsequent fine-tuning of the roberta model is improved.
[0069] In addition, when calculating the similarity between the feature vectors of the two texts in each text pair, the cosine similarity between the feature vectors of the two texts in each text pair can be calculated, or the similarity between the feature vectors of the two texts expressed by using the Euclidean distance can be calculated; or the similarity between the feature vectors of the two texts expressed by using the Manhattan distance can be calculated.
[0070] Next, according to the training samples, the parameters of the professional field Roberta model are adjusted through the twin network to obtain the target Roberta model. Figure 2 When adjusting the parameters of the roberta model in a professional field, two roberta models with shared parameters can be constructed based on the roberta model in the professional field; then, the two texts in each text pair of the positive sample and the negative sample of the training sample are respectively input into the two roberta models with shared parameters, and the feature vectors of the two texts are output; then, the cosine similarity between the feature vectors of the two texts is calculated; then, the parameters of the two roberta models with shared parameters are optimized through the contrastive loss function to obtain the target roberta model. By adopting the above method, it is convenient to adjust the parameters of the roberta model in the professional field.
[0071] When specifically determining the above contrastive loss, the contrastive loss function can be:
[0072]
[0073] Where y represents the label value 1 of the positive sample or the label value 0 of the negative sample;
[0074] d represents the cosine distance between two texts in a text pair of positive or negative samples;
[0075] Margin indicates the cosine distance interval, and can be set to margin = 0.75, 0.80, 0.85, 0.90, etc. to fine-tune the parameters of the roberta model in the professional field. The learning rate of the contrastive loss function can be set to 5e-6 to fine-tune the parameters of the roberta model in the professional field.
[0076] Next, according to the target Roberta model, the feature vector of each text in the multiple texts is extracted. Since the target Roberta model has adjusted the parameters of the Roberta model through the twin network, it can directly optimize the similarity distance, making similar texts more compact in distance and dissimilar texts more dispersed in distance, thereby improving the accuracy of subsequent clustering and the accuracy of subsequent extraction of hot spots in the corpus data.
[0077] Next, a clustering algorithm is used to cluster multiple texts of the corpus data to obtain multiple categories. The clustering algorithm may be a density-based DBSCAN clustering algorithm. Since the DBSCAN clustering algorithm does not need to determine the number of categories during the clustering process, the categories can be merged in subsequent processing operations to improve the clustering effect.
[0078] Specifically, clustering algorithms are used to cluster multiple texts of corpus data, and multiple categories can be obtained:
[0079] Step 1: Select the N_Sample parameter of the DBSCAN clustering algorithm as 1 to cluster multiple texts of the corpus data;
[0080] Step 2: Delete the outliers to obtain clustering results with multiple different categories;
[0081] Step 3: Calculate the average of the feature vectors of the text in each category of the clustering result to obtain the centroid of each category of the clustering result;
[0082] Step 4: Calculate the cosine similarity between the centroids of any two categories in different categories of the clustering results; if the cosine similarity between the centroids of two categories is greater than the set threshold, the two categories are merged;
[0083] Repeat steps 3 to 4 above until the cosine similarity between the centroids of any two categories is no greater than the set threshold, and output multiple categories to reduce the similarity between different categories and improve the clustering effect.
[0084] Next, the hot spots of the corpus data are extracted according to the target roberta model and multiple categories. When extracting hot spots, the text with the highest semantic score and keyword score in each category of multiple categories can be extracted as the hot spots of the corpus data based on the method of combining semantic score and keyword score. This can reduce the neglect of semantic features of text in the process of hot spot extraction and improve the accuracy of hot spot extraction. And using text instead of keywords for display can more intuitively express the meaning of each category.
[0085] Specifically, a method combining semantic scores and keyword scores is adopted to extract the text with the highest semantic scores and keyword scores in each category of multiple categories as the hot spots of the corpus data. The following method can be adopted: first, the feature vectors of the texts contained in each category are averaged to obtain the centroid of each category; then, the cosine similarity between the centroid and each text in the category is calculated to obtain the semantic score of each text in the category; then, the text contained in each category is segmented; then, the keywords of each category are extracted by the tf-idf method; then, the keywords with the top n importance are selected according to the order of importance; then, the keyword quantity feature is obtained according to the number of keywords contained in each text in the category; then, the keyword quantity feature is divided by n to obtain the keyword score of each text in the category; then, the text with the highest sum of the semantic score and the keyword score in each category is selected as the template of the category; then, the templates of multiple categories are sorted according to the number of texts contained in each category; then, the templates corresponding to the categories with the top h texts are selected as the hot spots of the corpus data. The existing method of displaying the top n hot spots by keywords cannot intuitively understand the meaning of the category. If the keyword extraction is inaccurate, the display effect will be greatly affected. In the present invention, the templates in the categories with the largest number of texts in the categories are selected as hot templates, thereby improving the accuracy of the extracted templates.
[0086] Among them, in the above ranking by importance, the number of "n" in selecting the top n keywords in importance can be 3, 4, 5, 6, etc. The accuracy of the extracted template can be improved by combining semantic and keyword evaluation indicators.
[0087] The specific method of sorting the templates of multiple categories according to the number of texts contained in each category is as follows: the more texts a category contains, the higher the templates in the category are ranked; the fewer texts a category contains, the lower the templates in the category are ranked. The "h" in the above selection of the templates corresponding to the first h categories containing the number of texts can be 2, 3, 4, 5, 6, etc.
[0088] By pre-training the roberta model in the general field with the corpus data of the hot spots to be extracted, the roberta model can better capture the unique information in the corpus data of the hot spots to be extracted. And pre-training can reduce the unk (unknown) situation of the characters in the corpus data of the hot spots to be extracted, laying the foundation for more accurate subsequent extraction of hot spots in the corpus data. In addition, by adjusting the parameters of the roberta model in the form of a twin network, the similarity distance can be directly optimized, making similar texts more compact in distance and dissimilar texts more dispersed in distance, thereby improving the accuracy of subsequent clustering and improving the accuracy of subsequent extraction of hot spots in the corpus data.
[0089] In addition, an embodiment of the present invention further provides a storage medium, in which a computer program is stored, and when the computer program is run on a computer, the computer executes any of the above hotspot extraction methods. The effects thereof are described above and will not be repeated here.
[0090] In addition, an embodiment of the present invention further provides a server, the server comprising a processor and a memory, the memory storing a computer program, and the processor calling the computer program stored in the memory to execute any of the above hotspot extraction methods. The effects thereof are referred to the above description and will not be repeated here.
[0091] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A hotspot extraction method, It is characterized in that include: Acquire corpus data of hot spots to be extracted, wherein the corpus data includes multiple texts; Pre-training a Roberta model in a general field according to the corpus data to obtain a Roberta model in a professional field; Extracting a feature vector of each of the plurality of texts according to the Roberta model of the professional field; Constructing a training sample of a twin network according to a feature vector of each of the multiple texts; According to the training samples, adjusting the parameters of the Roberta model in the professional field by means of a twin network to obtain a target Roberta model; Extracting a feature vector of each of the plurality of texts according to the target Roberta model; Clustering the multiple texts of the corpus data using a clustering algorithm to obtain multiple categories; Extracting hot spots of the corpus data according to the target Roberta model and the multiple categories; The feature vector of each of the multiple texts is extracted according to the Roberta model in the professional field as follows: Taking the last set number of layers of the roberta model in the professional field, and averaging the feature vectors of each word in each layer of the set number of layers; The feature vectors of the set number of layers are added together, and their average value is taken to obtain the feature vector of each text.
2. The hotspot extraction method according to claim 1, It is characterized in that The step of constructing a training sample of a twin network according to the feature vector of each of the plurality of texts comprises: Combining each of the plurality of texts with each of the other texts except the text into a plurality of text pairs, each text pair including two texts; Calculate the similarity between the feature vectors of the two texts in each text pair; Sorting the multiple text pairs according to similarity from high to low; Selecting a text pair whose similarity is in the first ratio value as a positive sample of the training sample; A text pair whose similarity is at the second lowest ratio value is selected as a negative sample of the training sample.
3. The hotspot extraction method according to claim 2, It is characterized in that The specific method of calculating the similarity between the feature vectors of two texts in each text pair is as follows: Calculate the cosine similarity between the feature vectors of the two texts in each text pair.
4. The hotspot extraction method according to claim 3, It is characterized in that The step of adjusting the parameters of the Roberta model in the professional field by means of a twin network according to the training sample to obtain the target Roberta model comprises: According to the roberta model of the professional field, two roberta models with shared parameters are constructed; Inputting two texts in each text pair of the positive sample and the negative sample of the training sample into the two roberta models with shared parameters respectively, and outputting feature vectors of the two texts; Calculate the cosine similarity between the feature vectors of the two texts; The parameters of the two shared-parameter Roberta models are optimized through the contrastive loss function to obtain the target Roberta model.
5. The hotspot extraction method according to claim 4, It is characterized in that The contrastive loss function is: Wherein, y represents the label value 1 of the positive sample, or the label value 0 of the negative sample; d represents the cosine distance between two texts in the text pair of the positive sample or negative sample; Margin indicates the set cosine distance interval.
6. The hotspot extraction method according to claim 1, It is characterized in that The clustering algorithm used is the density-based DBSCAN clustering algorithm.
7. The hotspot extraction method according to claim 6, It is characterized in that The clustering algorithm is used to cluster the multiple texts of the corpus data to obtain multiple categories: Step 1: Select the N_Sample parameter of the DBSCAN clustering algorithm to be 1, and cluster multiple texts of the corpus data; Step 2: Delete the outliers to obtain clustering results with multiple different categories; Step 3: averaging the feature vectors of the texts in each category of the clustering result to obtain the centroid of each category of the clustering result; Step 4: Calculate the cosine similarity between the centroids of any two categories in different categories of the clustering result; if the cosine similarity between the centroids of two categories is greater than a set threshold, merge the two categories; Repeat the above steps 3 to 4 until the cosine similarity between the centroids of any two categories is no greater than the set threshold, and output the multiple categories.
8. The hotspot extraction method according to claim 1, It is characterized in that The hot spots of extracting the corpus data according to the target roberta model and the multiple categories are specifically: According to a method combining semantic scores and keyword scores, texts with the highest semantic scores and keyword scores in each of the multiple categories are extracted as hotspots of the corpus data.
9. The hotspot extraction method according to claim 8, It is characterized in that The method of combining semantic scores and keyword scores to extract the text with the highest semantic scores and keyword scores in each of the multiple categories as the hot spots of the corpus data is specifically as follows: The feature vectors of the texts contained in each category are averaged to obtain the centroid of each category; Calculate the cosine similarity between the centroid and each text in the category to obtain a semantic score for each text in the category; Perform word segmentation on the text contained in each category; Extract keywords for each category using the tf-idf method; Sort by importance and select the top n keywords; The keyword quantity feature is obtained according to the number of keywords contained in each text in the category; Divide the keyword quantity feature by n to obtain the keyword score of each text in the category; Select the text with the highest sum of semantic score and keyword score in each category as the template of that category; Sorting the templates of the multiple categories according to the number of text items contained in each category; The templates corresponding to the first h categories are selected as the hot spots of the corpus data.
10. A storage medium, It is characterized in that The storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the hotspot extraction method according to any one of claims 1 to 9.
11. A server, It is characterized in that It comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the hotspot extraction method as claimed in any one of claims 1 to 9 by calling the computer program stored in the memory.
