Short article automatic extraction classification and keyword method and device using the method

Abstract

The invention discloses an automatic text extraction, classification and keyword method, comprising: inputting a plurality of texts, each of which has at least one short sentence; preprocessing the plurality of texts to generate a plurality of preprocessed texts; dividing the plurality of preprocessed texts into a plurality of categories of different topics according to a text topic analysis model, and outputting a plurality of preprocessed texts and a plurality of corresponding category labels, each of which has a plurality of corresponding candidate keywords; determining the keywords corresponding to each of the plurality of categories according to the plurality of preprocessed texts, the plurality of category labels corresponding to the plurality of preprocessed texts, an open source word vector pre-training set, the plurality of categories and the plurality of corresponding candidate keywords and a matching algorithm; performing the matching algorithm to extract the corresponding keywords again for the merged categories for updating; and establishing a word vector text classifier according to the plurality of categories and the keywords corresponding to each of the plurality of categories to achieve the purpose of automatic text extraction and classification.

Description

Technical Field

[0001] The present invention relates to a short article automatic extraction classification and keyword method, and in particular to a short article automatic extraction classification and keyword method that can perform word vector classification training through keywords in a small sample size. Background Art

[0002] The current training data sets for short article classification models are mostly news databases, and most of them are used for news collection or online public opinion analysis. Their characteristics include sufficient text quantity, consistent article themes, and clear syntactic structure. Therefore, they are suitable for classification using existing training word vectors and classifier methods.

[0003] However, when general enterprises process customer service cases (such as emails or message questions), it is not easy to execute general clustering and classification natural language processing (NLP) models due to the small number of cases (such as between 100,000 and 1,000,000). The existing technology of training word vectors and classifiers for classification is only applicable to large sample numbers. In addition, the classifiers trained by machine learning in the existing technology are more time-consuming and resource-intensive, and cannot be fine-tuned according to needs after being established. If keywords such as product names or function names need to be added to meet business needs, they can only be retrained and lack flexibility.

[0004] In view of this, the prior art needs to be improved. Summary of the invention

[0005] Therefore, the main purpose of the short article automatic extraction classification and keyword method and device of the present invention is to select the best candidate keywords as the keywords corresponding to a specific category by using a suitable algorithm, so that the group of keywords can represent the key points of most short articles from the perspective of word vector analysis, and can perform word vector classification through keywords with a small sample size.

[0006] Another main purpose of the short article automatic extraction classification and keyword method and device of the present invention is to solve the problem of poor analysis efficiency of word vector training in small sample numbers in the prior art by introducing open source word vector pre-training word sets.

[0007] Another main purpose of the short article automatic extraction classification and keyword method and device of the present invention is to have a keyword extraction module that can re-extract and update the keywords of the merged categories, so as to correct the problem that the category keywords of the prior art are not representative enough.

[0008] Another main purpose of the short article automatic extraction classification and keyword method and device of the present invention is that compared with the prior art, when performing classification, the present invention does not require training word vectors and training classifiers through machine learning, thereby achieving the purpose of saving system resources.

[0009] Another main purpose of the short article automatic extraction classification and keyword method and device of the present invention is to use an open source word vector pre-training set and a suitable algorithm to extract the best multiple groups of keywords and combine them with the word vector similarity classification module for classification. When keyword adjustments are needed in the future (for example, adding product names or function names to meet business needs), there is no need to re-train, thereby achieving a more agile and flexible purpose.

[0010] The present invention discloses a method for automatic extraction, classification and keywording of short articles, comprising: inputting a plurality of short articles, each of which has at least one short sentence; performing a preprocessing on the plurality of short articles to generate a plurality of preprocessed short articles; according to a text topic analysis model, the plurality of preprocessed short articles are divided into a plurality of categories of different topics, and the plurality of preprocessed short articles and a plurality of corresponding category labels are output, and each of the plurality of categories has a plurality of corresponding candidate keywords; according to the plurality of preprocessed short articles, the plurality of category labels corresponding to the plurality of preprocessed short articles, an open source word vector pre-training set, the plurality of categories and the plurality of corresponding candidate keywords and a matching algorithm, the keywords corresponding to each category of the plurality of categories are determined; the matching algorithm is then performed to extract the corresponding keywords again for the merged categories for updating; and according to the plurality of categories and the keywords corresponding to each category of the plurality of categories, a word vector short article classifier is established to achieve the purpose of automatic extraction and classification of short articles. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 Schematic diagram of a short article automatic extraction, classification and keyword device according to an embodiment of the present invention.

[0012] Figures 2 to 5 For the embodiment of the present invention Figure 1 The schematic diagram of the operation of the short article automatic extraction classification and keyword device is shown.

[0013] Figure 6 Schematic diagram of the process of automatic extraction, classification and keyword analysis of short articles in accordance with the first embodiment of the present invention.

[0014] The reference numerals are described as follows:

[0015] 10: Automatic extraction and classification of short articles and keyword device

[0016] 102: Pre-processing module

[0017] 104: Short article topic classification module

[0018] 106: Keyword Extraction Module

[0019] 108: Keyword Merging Module

[0020] 110: Similarity classification module

[0021] 12: Input device

[0022] 14: Output device

[0023] T1~Tm:Short text

[0024] T1'~Tm': Preprocessing short text

[0025] C1~Cn:Category

[0026] TI1~TIm: Category label

[0027] KW1~KWn:Keywords

[0028] 60: Process

[0029] 600~612: Steps DETAILED DESCRIPTION

[0030] Please refer to Figure 1 , Figure 1 FIG. 1 is a schematic diagram of a short article automatic extraction, classification and keyword device 10 according to an embodiment of the present invention. Figure 1As shown, the short article automatic extraction classification and keyword device 10 includes a pre-processing module 102, a short article topic classification module 104, a keyword extraction module 106, a keyword merging module 108 and a similarity classification module 110. In short, an input device 12 inputs the short articles T1-Tm to the pre-processing module 102, and the pre-processing module 102 performs pre-processing on the short articles T1-Tm, such as word segmentation, stop words, capitalization, part-of-speech restoration, and verb form processing, to generate pre-processed short articles T1'-Tm'; the short article topic classification module 104 can classify the pre-processed short articles T1'-Tm' into categories C1-Cn of different topics according to a text topic analysis model, and output the pre-processed short articles T1'-Tm' and corresponding category labels TI1-Tim; the keyword extraction module 106 can classify the pre-processed short articles T1'-Tm' into categories C1-Cn of different topics according to a text topic analysis model, and output the pre-processed short articles T1'-Tm' and corresponding category labels TI1-Tim according to the pre-processing module 102; the keyword extraction module 106 can classify the pre-processed short articles T1'-Tm' into categories C1-Cn of different topics according to the pre-processing module 104; and output the pre-processed short articles T1'-Tm' and corresponding category labels TI1-Tim according to the pre-processing module 102. Pre-processing short texts T1'-Tm', category labels TI1-TIm, an open source word vector pre-training set and a matching algorithm, extract the corresponding keywords KW1-KWn of categories C1-Cn; the keyword merging module 108 determines the vector distance between keywords KW1-KWn, and merges the categories whose vector distances of the corresponding keywords in categories C1-Cn are less than a threshold; the keyword extraction module 106 re-extracts keywords from the merged categories in categories C1-Cn to update the problem that the keywords of the merged categories are not representative. Finally, the similarity classification module 110 can calculate the distance based on the keywords KW1-KWn and their word vectors, establish a word vector short text classifier and provide it to an output device 14, so as to improve the keyword and classification recognition accuracy of subsequent short texts with a small number of samples (such as calculating which group of keywords in the subsequent specific short text and keywords KW1-KWn have the highest similarity, and then judging that the specific short text belongs to the corresponding category). In this way, the present invention can be more flexible in classifying word vectors using keywords in a small sample size, and does not require training of word vectors and classifiers, thereby saving resources.

[0031] For more information, see Figures 2 to 5 , Figures 2 to 5 FIG. 1 is a schematic diagram of the operation of the apparatus 10 for automatically extracting, classifying and keywording short articles according to an embodiment of the present invention. Figure 2 As shown, after the pre-processing module 102 performs word segmentation, stop word processing, capitalization processing, part-of-speech restoration processing, verb form processing, etc., the short texts T1-T6 can be generated with only important words and suitable for processing and provided to the pre-processed short text topic classification module 104. The pre-processing is well known to those skilled in the art and will not be described in detail here. Figure 3As shown, the essay topic classification module 104 classifies the pre-processed essays T1'-T6' into categories C1-C4, and determines the corresponding category labels TI1-TI6 as categories C1, C2, C2, C3, C1, and C4, respectively. The essays can be classified into main categories and essays with similar themes can be placed in the same category. Then, the keyword extraction module 106 extracts the corresponding keywords KW1-KW4 of categories C1-C4 to represent each category. Then, as Figure 4 As shown, the keyword merging module 108 determines that the vector distance between keywords KW2 and KW3 is less than the threshold and merges category C3 into category C2 to merge the close categories. Then, the keyword extraction module 106 extracts keywords from the merged category C2, that is, Figure 5 The updated keyword KW2 is shown (the original category label TI4 is also updated to category C2).

[0032] In detail, the text topic analysis model of the short article topic classification module 104 may include at least one of a Gibbs Sampling Dirichlet Multinomial Mixture Model (GSDMM), a Dirichlet Mixture Model (DMM), a Generalized Polya Urn-Dirichlet Multinomial Mixture (GPU-DMM), a Latent Dirichlet Allocation (LDA), a Topic Representative Term Discovery (TRTD), a Biterm Topic Model (BTM), a Latent Semantic Indexing (LSI) and a Latent Semantic Analysis (LSA). For example, GSDMM first randomly groups the pre-processed short texts T1'-Tm', and then moves the short texts until the short texts in each category C1-Cn of different topics are similar in nature. GSDMM and other methods are well known in the art and will not be described here for brevity.

[0033] On the other hand, the open source word vector pre-training set in the keyword extraction module 106 may include at least one of the pre-training sets wiki-news-300d-1M.vec, wiki-news-300d-1M-subword, crawl-300d-2M.vec and crawl-300d-2M-subword. For a small sample size, word vector training cannot have a good effect. Therefore, by introducing an open source word vector pre-training word set, the analysis efficiency is improved. The open source word vector pre-training set is also well known in the art and will not be repeated here for the sake of brevity.

[0034] In this case, the keyword extraction module 106 executes a matching algorithm to determine the keyword corresponding to a specific category according to the sum of multiple similarities of multiple groups of candidate keywords corresponding to a specific category in categories C1~Cn or at least one of multiple percentages of passages that pass the threshold. In detail, the keyword extraction module 106 calculates the similarity of each group of candidate keywords with all pre-processed passages in the specific category based on the open source word vector pre-training set, and then sums the similarities or sets a threshold (such as 0.3) and calculates the percentage of passages that pass the threshold (such as the number of passages with a similarity greater than 0.3 is 70, and the number of passages in the specific category is 100, then the percentage of passages that pass the threshold is 0.7), and then selects the best candidate keyword with the highest sum of similarities or the highest percentage of passages that pass the threshold in the specific category as the keyword corresponding to the specific category. The similarity between the keyword and the short text is calculated by mapping them into a vector space, and then calculating the difference between different vectors to calculate the similarity. For example, ["hello", "world"] / ["hi", "world"] can be calculated to have a similarity of 0.76. Similarity calculation is also well known in the art and will not be described here for brevity.

[0035] It is worth noting that the above embodiment mainly performs a matching algorithm to select the candidate keyword with the highest similarity sum or the highest passing threshold text ratio in a specific category as the keyword corresponding to the specific category, to extract the corresponding keywords KW1~KWn of categories C1~Cn, and then establish a word vector text classifier to classify subsequent texts. Those skilled in the art can make modifications or changes accordingly, but are not limited to this. For example, after the first extraction of the corresponding keywords KW1~KWn of categories C1~Cn, the above embodiment will merge the categories whose vector distances of the corresponding keywords in categories C1~Cn are less than a threshold to correct the problem of poor representativeness, but in other embodiments, it is not necessary to merge. In addition, the above-mentioned embodiment uses the corresponding keywords KW1~KWn of categories C1~Cn to establish a word vector short text classifier to classify subsequent short texts. Therefore, in practical applications, the similarity classification module 110 can also adjust the categories C1~Cn and keywords KW1~KWn according to needs to establish a word vector short text classifier (such as adding or reducing categories), which can have better flexibility.

[0036] On the other hand, the above-mentioned embodiment uses a best candidate keyword with the highest similarity sum or the highest passing threshold short text ratio among multiple groups of candidate keywords as the keyword corresponding to the specific category. In one embodiment, the multiple groups of candidate keywords can be generated according to a genetic algorithm, and when the genetic algorithm executes a specified number of cycles, or when a highest similarity sum or a highest passing threshold short text ratio of a best candidate keyword is greater than a matching threshold, the matching algorithm uses the best candidate keyword as the keyword corresponding to the specific category.

[0037] For example, a genetic algorithm can encode multiple words in a specific category to generate multiple groups of parent candidate keywords, such as selecting 3 words from 5 words as a group of parent candidate keywords, and then generating 10-20 groups of parent candidate keywords. Then, the matching algorithm calculates the sum of similarities of the 10-20 groups of candidate keywords or the proportion of short texts that pass the threshold, and retains the candidate keywords whose sum of similarities or the proportion of short texts that pass the threshold is greater than a specific value (such as the proportion of short texts that pass the threshold is greater than 0.8). Then, the genetic algorithm mates the reserved candidate keywords. For example, the presence or absence of 5 words can be represented by binary codes as shown in the following table. In the two groups of reserved candidate keywords, one word originally exists in one group and not in the other. After mating, it becomes both groups (the first word), neither group (the third word), or the one that originally exists becomes not and the one that originally did not exist becomes existing (the fourth word):

[0038] Originally reserved candidate keywords Retain candidate keywords after mating 1 0 1 0 1 1 0 0 1 1 0 1 0 1 1 1 1 0 0 1

[0039] In addition, the genetic algorithm can also perform mutations to randomly change the presence or absence of specific words in the reserved candidate keywords with a certain probability (set to a low probability), such as the reserved candidate keywords that originally did not have the third word suddenly have the third word. In this case, the genetic algorithm can mate or mutate the reserved candidate keywords to form a loop to generate new candidate keywords, and then the matching algorithm calculates the similarity sum or the threshold passage ratio of the new candidate keywords. In this way, when the genetic algorithm executes a specified number of loops, or the highest similarity sum or the highest threshold passage ratio of a best candidate keyword is greater than a matching threshold, the matching algorithm uses the best candidate keyword as the keyword corresponding to the specific category and ends the loop of the genetic algorithm.

[0040] Therefore, the operation of the short article automatic extraction, classification and keyword device 10 can be summarized as a short article automatic extraction, classification and keyword process 60, such as Figure 6 As shown, it includes the following steps:

[0041] Step 600: Start.

[0042] Step 602: Input multiple short texts.

[0043] Step 604: pre-process the plurality of short texts to generate a plurality of pre-processed short texts.

[0044] Step 606: Classify the plurality of pre-processed short articles into a plurality of categories of different topics according to a text topic analysis model, and output the plurality of pre-processed short articles and a plurality of corresponding category labels.

[0045] Step 608: Extract corresponding groups of keywords of the multiple categories according to the multiple pre-processed short texts, the multiple category labels, an open source word vector pre-training set, and a matching algorithm.

[0046] Step 610: Establish a word vector short text classifier according to the multiple categories and the multiple groups of keywords.

[0047] Step 612: End.

[0048] The detailed operation of the short article automatic extraction classification and keyword process 60 can refer to the relevant content of the short article automatic extraction classification and keyword device 10, which will not be repeated here for the sake of brevity.

[0049] In addition, the short article automatic extraction classification and keyword device 10 may include a processing device and a storage unit. The processing device may be a microprocessor or an application-specific integrated circuit (ASIC). The storage unit may be any data storage device, which is used to store a program code, and the program code is read and executed by the processing device to perform the functions of the pre-processing module 102, the short article theme classification module 104, the keyword extraction module 106, the keyword merging module 108 and the similarity classification module 110, thereby completing each step of the short article automatic extraction classification and keyword process 60. The storage unit may be a subscriber identity module (SIM), a read-only memory (ROM), a random-access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., but is not limited thereto.

[0050] In summary, the present invention can execute a suitable algorithm to utilize an open source word vector pre-training set, extract keywords corresponding to each category in a small sample size, and then establish a word vector short article classifier based on the keywords to classify subsequent short articles. It is more flexible and does not require training word vectors and classifiers to save resources.

[0051] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the claims of the present invention should fall within the scope of the present invention.

Claims

1. A method for automatic extraction, classification and keywording of short articles, comprising: Input a plurality of short texts, wherein each of the plurality of short texts has at least one short sentence; Performing a preprocessing on the plurality of short texts to generate a plurality of preprocessed short texts; According to a text topic analysis model, the plurality of pre-processed short articles are divided into a plurality of categories of different topics, and a plurality of category labels corresponding to the plurality of pre-processed short articles are output, and each of the plurality of categories has a plurality of corresponding candidate keywords; According to the plurality of pre-processed short articles, the plurality of category labels corresponding to the plurality of pre-processed short articles, an open source word vector pre-training set, the plurality of categories and the plurality of candidate keywords corresponding to each of the plurality of categories, and a matching algorithm, the similarity between each group of candidate keywords corresponding to each category and all pre-processed short articles in the category is calculated, and then the candidate keyword with at least one of the highest sum of similarities or the highest proportion of short articles passing the threshold among the plurality of candidate keywords corresponding to each category is used as the keyword corresponding to the category; Then, the matching algorithm is performed to determine the vector distance between the keywords corresponding to each of the multiple categories, the categories whose vector distance is less than a threshold are merged, and the corresponding keywords are extracted again for the merged categories to be updated; as well as A word vector short article classifier is established according to the multiple categories and the keywords corresponding to each of the multiple categories, so as to achieve the purpose of automatic extraction and classification of short articles.

2. The method for automatic extraction, classification and keywording of short articles as described in claim 1, wherein the preprocessing comprises at least one of a word segmentation process, a stop word process, a case processing, a part-of-speech restoration process, and a verb form process.

3. The method for automatic extraction, classification and keywording of short articles as described in claim 1, wherein the text topic analysis model includes at least one of a Gibbs sampling algorithm of a Dirichlet polynomial mixture model, a Dirichlet polynomial mixture model, a Dirichlet polynomial mixture model improved by GPU, a latent Dirichlet distribution model, a topic representative word discovery, a two-word topic model, a latent semantic index and a latent semantic analysis; the open source word vector pre-training set includes at least one of the pre-training sets wiki-news-300d-1M.vec, wiki-news-300d-1M-subword, crawl-300d-2M.vec and crawl-300d-2M-subword.

4. The method for automatically extracting, categorizing and keywording short articles as claimed in claim 1, further comprising: The matching algorithm selects a best candidate keyword with a highest similarity sum or a highest passing threshold short text ratio in a specific category as the keyword corresponding to the specific category.

5. The method for automatically extracting, categorizing and keywording short articles as claimed in claim 1, further comprising: generating a plurality of candidate keywords according to a genetic algorithm; and When the genetic algorithm executes a specified number of cycles, or a maximum similarity sum or a maximum passing threshold text ratio of a best candidate keyword is greater than a matching threshold, the matching algorithm uses the best candidate keyword as the keyword corresponding to the specific category.

6. The method for automatic text extraction, classification and keywording as claimed in claim 5, wherein the step of generating the plurality of candidate keywords according to the genetic algorithm comprises: A plurality of words in the specific category are encoded to generate a plurality of groups of parent candidate keywords in the plurality of groups of candidate keywords.

7. The method for automatically extracting, categorizing and keywording short articles as claimed in claim 6, further comprising: A plurality of retained candidate keywords in the plurality of groups of parent candidate keywords whose similarities or ratios of short texts passing a threshold are greater than a specific value are mated or mutated to generate a plurality of groups of new candidate keywords in the plurality of groups of candidate keywords.

8. A short article automatic extraction classification and keyword system, comprising: a processor for executing a program; and A storage unit is coupled to the processor and is used to store the program; wherein the program is used to instruct the processor to execute the short text automatic extraction, classification and keyword method as described in claims 1 to 7.

Images