Special domain keyword extraction method in combination with semantic information

Abstract

The invention discloses a special field keyword extraction method in combination with semantic information, and belongs to the field of corpus recognition processing. Comprises: an expert constructs a domain dictionary; automatically discovering new words from the domain literature through a new word discovery algorithm, and inputting the new words into the domain dictionary after manual confirmation so as to continuously update the domain dictionary; loading the domain dictionary to a word segmentation device; a keyword is calculated in the target corpus through a TextRank algorithm, and a weight value of the keyword is obtained; replacing a keyword part in an original sentence of the target corpus with MASK to form an MASK sentence; vectors of the MASK sentence and an original sentence are extracted through a BERT algorithm, and the similarity value of the MASK sentence and the original sentence is calculated through the extracted vectors; and carrying out weighted calculation on the weight calculated through the TexkRank in the step (4) and the sentence similarity value extracted in the step (6) to obtain a final weight value. According to the method, the keywords of the scene in the special field can be accurately extracted, and data support is provided for upper-layer applications such as document association analysis, retrieval and situation display.

Description

Technical Field

[0001] The present invention relates to the technical field of corpus recognition and processing, and in particular to a method for extracting keywords in a specific field combined with semantic information. Background Art

[0002] Currently, the information obtained is based solely on form submissions, which prevents in-depth exploration of document content and lacks sufficient input for true data mining. Keyword extraction from text is necessary to support higher-level applications such as document association analysis, retrieval, and situational awareness. Currently, accurate keyword extraction is not possible for specialized domains, and there is a need for keyword extraction technology that supports these scenarios. Summary of the Invention

[0003] The present invention provides a method for extracting domain-specific keywords in combination with semantic information, in order to solve the above-mentioned problems in the prior art.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] The present invention provides a method for extracting domain-specific keywords in combination with semantic information, comprising:

[0006] (1) Experts build domain dictionaries;

[0007] (2) Automatically discover new words from domain literature through new word discovery algorithms, manually confirm them and enter them into the domain dictionary to continuously update the domain dictionary;

[0008] (3) Load the domain dictionary into the word segmenter;

[0009] (4) Calculate keywords in the target corpus using the TextRank algorithm and obtain the keyword weights;

[0010] (5) Replace the key words in the original sentences of the target corpus with MASK to form MASK sentences;

[0011] (6) Extract the vectors of the MASK sentence and the original sentence using the BERT algorithm, and calculate the similarity value between the MASK sentence and the original sentence using the extracted vectors;

[0012] (7) The weight calculated by TexkRank in step (4) and the sentence similarity value extracted in step (6) are weighted to obtain the final weight value.

[0013] On this basis, the present invention can also be improved as follows: the method for extracting keywords in a specific field in combination with semantic information further includes:

[0014] (8) Repeat the above steps for multiple keywords to obtain the final weight values of all keywords, and sort these keywords according to the weight values of all keywords to obtain a word key ranking table.

[0015] On this basis, the present invention can also be improved as follows: Step (2) specifically includes:

[0016] Preprocess the target corpus to remove interference;

[0017] Calculating mutual information on the preprocessed target corpus to obtain a set of words, and grouping the words in the set that are greater than a first threshold into candidate bigrams;

[0018] Calculate the left information entropy of the word in the candidate bigram. If the left information entropy of the word is less than the second threshold, perform multiple left expansions on the word with a step size of 1 each time to obtain the left-expanded word WORD. N , where N = 1, 2, ... n, after each left expansion, the left information entropy of the left expanded word is calculated and judged. When the left expansion length is greater than the third threshold and the left information entropy is less than the second threshold, WORD n It is not a new word, and WORD n-1 Add to the set whose left information entropy meets the conditions;

[0019] Calculate the right information entropy of the words in the set whose left information entropy meets the conditions. If the right information entropy of the word is less than the second threshold, perform multiple right expansions on the word with a step size of 1 each time to obtain the right-expanded word WORD Y , where Y = 1, 2, ... y, after each right extension, the right information entropy of the right-extended word is calculated and judged. When the right extension length is greater than the third threshold and the right information entropy is less than the second threshold, WORD y It is not a new word, and WORD y-1 Added to the domain dictionary.

[0020] On this basis, the present invention can also be improved as follows: Step (4) specifically includes:

[0021] Preprocess the target corpus to remove interference;

[0022] Slide the window over the preprocessed target text to construct nodes and edges, where the words are nodes and the connections between words that appear in the window constitute edges.

[0023] According to the constructed nodes and edges, the weight of each node is calculated through iteration until convergence, and the weight value of each node is obtained.

[0024] On this basis, the present invention can also be improved as follows: preprocessing includes word segmentation, part-of-speech tagging and removal of stop words.

[0025] The beneficial effects provided by the present invention are:

[0026] The present invention proposes a method for extracting keywords in a specific domain by combining semantic information, which can accurately extract keywords in specific domain scenarios and provide data support for upper-level applications such as document association analysis, retrieval and situation display.

[0027] Advantages of additional aspects of the present invention will be given in part in the following description and in part will become obvious from the following description or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention.

[0029] Figure 2 Schematic diagram of the new word discovery algorithm flow in an embodiment of the present invention.

[0030] Figure 3 Schematic diagram of the BERT algorithm principle. DETAILED DESCRIPTION

[0031] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0032] Keyword extraction is a basic research direction in the field of natural language processing, and there are many algorithms and mature open source tools. Keyword extraction is calculated based on word segmentation. Chinese keyword extraction is different from English keyword extraction and is more difficult, especially in professional field scenarios. The current word segmentation algorithm cannot segment professional vocabulary. Therefore, the present invention assists in constructing a dictionary for professional field scenarios through a new word discovery algorithm. Common Chinese keyword extraction methods include the TF-IDF algorithm based on word frequency and the TextRank algorithm based on graph sorting algorithm. However, some important keywords may appear less frequently, but they are indeed important keywords from a semantic point of view. The present invention first uses the TextRank algorithm to preliminarily screen out keywords, then masks the keywords of the original sentence to form a series of new sentences, calculates the similarity between the new sentences and the original sentences, and sorts them according to the similarity value. If the similarity is higher than the threshold, it proves that the masked words are not important. If the similarity is lower than the threshold, it proves that the masked words are important. Finally, the weight and similarity value are weighted by the weight calculation formula to obtain the final weight. The formula is as follows:

[0033] W=αW TextRank +βlog a s

[0034] Where a=-10, s is the similarity calculated between the original sentence and the masked sentence.

[0035] New word discovery is achieved based on mutual information and left-right entropy. Mutual information is used to determine the degree of cohesion between words, while left-right entropy is used to determine the degree of freedom between words.

[0036] Cohesion is used to measure the correlation between two words, whether they often appear together to form new words. The formula for calculating 2-gram cohesion is as follows:

[0037]

[0038] PMI 值 The higher it is, the higher the cohesion between the two words is, and the greater the possibility that the two words can form one word.

[0039] Entropy is a measure of information. The greater the entropy, the greater the uncertainty, and the smaller the entropy, the smaller the uncertainty. The entropy calculation formula is as follows:

[0040] H(x)=-Σ x∈X P(x)logP(x)

[0041] The calculation formula of left entropy is as follows:

[0042] H Left (LeftW)=-∑ Left∈A P(LeftW)logP(LeftW)

[0043] The calculation formula for right entropy is as follows:

[0044] H Right (WRight)=--∑ Right∈A P(WRight)logP(WRight)

[0045] Then, the degree of freedom is calculated as follows:

[0046] free=min(H Left , H Right )

[0047] The TextRank algorithm is adapted from the PageRank algorithm, the core web page ranking algorithm used by Google Search, and uses a graph model to calculate keywords in text. The PageRank algorithm views the entire Internet as a directed graph, with web pages as nodes and links between web pages as edges. The importance of a page is determined by the links between web pages. The formula for calculating web page importance is as follows:

[0048]

[0049] Among them, V represents the web page, S represents the score of each web page, the larger the S is, the more important the web page is, d is the damping coefficient, which is generally 0.85, In(V i ) indicates the web page set pointing to web page i, Out(V j ) indicates that web page j points to a collection of web pages.

[0050] The TextRank algorithm slides a fixed window over the segmented text. The words in each window form an edge based on the co-occurrence relationship, eventually forming a graph. The words in the document are regarded as nodes, and the co-occurrence relationship between words is regarded as an edge. The weight value is obtained through iterative calculation, and the topN words with the highest weight are selected as keywords.

[0051] like Figure 3 The figure below shows the schematic diagram of the BERT algorithm principle.

[0052] The BERT model is mainly used to judge the semantic importance. BERT is a pre-trained language model released by Google. Its main structure is the Encoder part of the Transformer, and word vector representation is achieved through the MASK mechanism and Next Sentence Prediction methods.

[0053] The BERT model's input is the sum of Token Embeddings, Segment Embeddings, and Position Embeddings. The MASK mechanism randomly masks 15% of tokens during pre-training. Of these masked tokens, 10% are replaced with other words, 10% remain unchanged, and 80% are replaced with [MASK]. The model then predicts the masked tokens, enabling bidirectional training of the model. The NSP (Next Sentence Prediction) mechanism, during BERT training, takes paired sentences as input and predicts whether the second sentence in the pair follows the first. During training, 50% of the input consists of pairs of sentences where the second sentence follows the first in the original document. For the other 50% of the input, a sentence is randomly selected from the corpus as the second sentence.

[0054] During the training process, the loss functions of these two strategies are optimized to complete the pre-trained model. After fine-tuning, the trained model can be used for downstream tasks such as text classification, named entity recognition, and question-answering. The present invention uses the pre-trained model to obtain sentence vectors and achieve semantic representation of sentences.

[0055] The following are specific examples.

[0056] like Figure 1 FIG. 1 is a flow chart of an embodiment of the present invention.

[0057] The present invention provides a method for extracting domain-specific keywords in combination with semantic information, comprising:

[0058] (1) Experts build domain dictionaries;

[0059] (2) Automatically discover new words from domain literature through new word discovery algorithms, manually confirm them and enter them into the domain dictionary to continuously update the domain dictionary;

[0060] (3) Load the domain dictionary into the word segmenter;

[0061] (4) Calculate keywords in the target corpus using the TextRank algorithm and obtain the keyword weights;

[0062] (5) Replace the key words in the original sentences of the target corpus with MASK to form MASK sentences;

[0063] (6) Extract the vectors of the MASK sentence and the original sentence using the BERT algorithm, and calculate the similarity value between the MASK sentence and the original sentence using the extracted vectors;

[0064] (7) The weight calculated by TexkRank in step (4) and the sentence similarity value extracted in step (6) are weighted to obtain the final weight value.

[0065] Optionally, in some possible implementations, the method for extracting domain-specific keywords in combination with semantic information further includes:

[0066] (8) Repeat the above steps for multiple keywords to obtain the final weight values of all keywords, and sort these keywords according to the weight values of all keywords to obtain a word key ranking table.

[0067] like Figure 2 FIG. 1 is a flow chart of a new word discovery algorithm according to an embodiment of the present invention. Optionally, in some possible implementations, step (2) specifically includes:

[0068] Preprocess the target corpus to remove interference;

[0069] Calculating mutual information on the preprocessed target corpus to obtain a set of words, and grouping the words in the set that are greater than a first threshold into candidate bigrams;

[0070] Calculate the left information entropy of the word in the candidate bigram. If the left information entropy of the word is less than the second threshold, perform multiple left expansions on the word, each with a step size of 1, to obtain the left-expanded word WORD(N), where N=1, 2, ...n. Calculate the left information entropy of the left-expanded word after each left expansion and judge. When the length of the left expansion is greater than the third threshold and the left information entropy is less than the second threshold, WORD(n) is not a new word, and add WORD(n-1) to the set whose left information entropy meets the conditions.

[0071] Calculate the right information entropy of the words in the set whose left information entropy meets the conditions. If the right information entropy of the word is less than the second threshold, perform multiple right extensions on the word, each with a step size of 1, to obtain the right-extended word WORD(Y), where Y=1, 2, ...y. After each right extension, calculate the right information entropy of the right-extended word and judge. When the length of the right extension is greater than the third threshold and the right information entropy is less than the second threshold, WORD(y) is not a new word, and WORD(y-1) is added to the domain dictionary.

[0072] Optionally, in some possible implementations, step (4) specifically includes:

[0073] Preprocess the target corpus to remove interference;

[0074] Slide the window over the preprocessed target text to construct nodes and edges, where the words are nodes and the connections between words that appear in the window constitute edges.

[0075] According to the constructed nodes and edges, the weight of each node is calculated through iteration until convergence, and the weight value of each node is obtained.

[0076] Optionally, in some possible implementations, preprocessing includes word segmentation, part-of-speech tagging, and stop word removal.

[0077] The solution provided by the present invention has the following effects:

[0078] The present invention proposes a method for extracting keywords in a specific domain by combining semantic information, which can accurately extract keywords in specific domain scenarios and provide data support for upper-level applications such as document association analysis, retrieval and situation display.

[0079] Optionally, in some possible implementations, all or part of the above implementations may be included.

[0080] It should be understood that in the description of this specification, reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this description, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and some features of different embodiments or examples without contradiction.

[0081] Of course, without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these changes and modifications should all fall within the scope of protection of the claims of the present invention.

Claims

1. A method for extracting keywords in a specific field by combining semantic information, characterized in that: include: (1) Experts build domain dictionaries; (2) Automatically discover new words from domain literature through new word discovery algorithms, manually confirm them and enter them into the domain dictionary to continuously update the domain dictionary; (3) Load the domain dictionary into the word segmenter; (4) Calculate keywords in the target corpus using the TextRank algorithm and obtain the keyword weights; (5) Replace the key words in the original sentences of the target corpus with MASK to form MASK sentences; (6) Extract the vectors of the MASK sentence and the original sentence using the BERT algorithm, and calculate the similarity value between the MASK sentence and the original sentence using the extracted vectors; (7) The weight calculated by TexkRank in step (4) and the sentence similarity value extracted in step (6) are weighted to obtain the final weight value.

2. The method for extracting domain-specific keywords in combination with semantic information according to claim 1, characterized in that: The method for extracting domain-specific keywords in combination with semantic information further includes: (8) Repeat the above steps for multiple keywords to obtain the final weight values of all keywords, and sort these keywords according to the weight values of all keywords to obtain a word key ranking table.

3. The method for extracting domain-specific keywords in combination with semantic information according to claim 2, characterized in that: Step (2) specifically includes: Preprocess the target corpus to remove interference; Calculating mutual information on the preprocessed target corpus to obtain a set of words, and grouping the words in the set that are greater than a first threshold into candidate bigrams; Calculate the left information entropy of the word in the candidate bigram. If the left information entropy of the word is less than the second threshold, perform multiple left extensions on the word, each with a step size of 1, to obtain the left-extended word WORIDN, where N = 1, 2, ... n. After each left extension, calculate the left information entropy of the left-extended word and judge. When the left extension length is greater than the third threshold and the left information entropy is less than the second threshold, WORD n It is not a new word, and WORD n-1 Add to the set whose left information entropy meets the conditions; Calculate the right information entropy of the words in the set whose left information entropy meets the conditions. If the right information entropy of the word is less than the second threshold, perform multiple right expansions on the word with a step size of 1 each time to obtain the right-expanded word WORD Y , where Y = 1, 2, ... y, and after each right extension, the right information entropy of the right-extended word is calculated and judged. When the right extension length is greater than the third threshold and the right information entropy is less than the second threshold, WORDy is not a new word, and WORDy is converted to y-1 Added to the domain dictionary.

4. The method for extracting domain-specific keywords in combination with semantic information according to claim 3, characterized in that: Step (4) specifically includes: Preprocess the target corpus to remove interference; Slide the window over the preprocessed target text to construct nodes and edges, where the words are nodes and the connections between words that appear in the window constitute edges. According to the constructed nodes and edges, the weight of each node is calculated through iteration until convergence, and the weight value of each node is obtained.

5. The method for extracting domain-specific keywords in combination with semantic information according to claim 4, characterized in that: Preprocessing includes word segmentation, part-of-speech tagging, and stop word removal.

Images