Keyword extraction method for large-scale English patent text
By combining the unsupervised keyword extraction method with the TF-IDF algorithm and the large language model, the problem of keyword extraction in the prior art cannot be performed in combination with context semantics, and efficient and accurate keyword extraction of large-scale English patent texts is achieved.
Patent Information
- Application Number
- CN202411536569.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-02-25
AI Technical Summary
The existing English patent text keyword extraction method only considers the word frequency or co-occurrence relationship of words, and cannot perform keyword extraction based on the context semantics of the text, resulting in poor results.
Unsupervised keyword extraction method combining TF-IDF algorithm and large language model is adopted, and the patent text is preprocessed to generate a vocabulary collection of words and binaries, and the keyword candidate set is initially screened, and the final keyword collection is determined through the large language model optimization screening, combining context semantics.
It effectively improves the efficiency and accuracy of large-scale English patent text keyword extraction, solves the invalid vocabulary problem caused by word frequency statistics in traditional methods, and improves the accuracy of the definition of polysemous words.
Smart Images
Figure CN119514533A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of text information processing, and in particular to a keyword extraction method for large-scale English patent texts. Background Art
[0002] With the rapid development of scientific and technological innovation, patents have become an important means to protect inventions and promote technological exchanges. In-depth analysis of patent texts can uncover valuable technical intelligence and provide strong support for corporate R&D and intellectual property protection.
[0003] Existing patent keyword extraction methods are mainly divided into unsupervised methods and supervised methods. Unsupervised methods refer to methods that do not rely on pre-annotated labels to extract keywords, while supervised methods refer to methods that extract keywords based on annotated training data sets. Among them, supervised methods usually require a lot of human resources for keyword annotation, especially in the context of the current surge in patent information. In order to respond to patent infringement lawsuits or apply for patents, professionals often need to mine hundreds or even thousands of patent texts, and even need to analyze data mining tasks of up to tens of thousands of patent texts. In these tasks facing large-scale patent texts, the difficulty of manual keyword annotation can be imagined.
[0004] Therefore, in the absence of patent text keyword annotation resources, unsupervised methods are more feasible to extract keywords from a large number of patent texts. However, many existing unsupervised methods only consider word frequency or word co-occurrence relationships, and cannot combine the contextual semantics of the text for keyword extraction, and their results are often unsatisfactory. Summary of the invention
[0005] In view of the shortcomings of the prior art, the present invention provides a keyword extraction method for large-scale English patent texts, which solves the problem that the keyword extraction method for English patent texts only considers word frequency or word co-occurrence relationship and cannot extract keywords in combination with the contextual semantics of the text.
[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a keyword extraction method for large-scale English patent texts, comprising the following steps:
[0007] S01. Input large-scale English patent texts;
[0008] S02. Preprocessing a large amount of English patent text to obtain a vocabulary set including words and bigrams;
[0009] S03. Perform preliminary keyword screening on the preprocessed patent text based on the TF-IDF algorithm to generate a keyword candidate set for each text;
[0010] S04. Call the large language model to optimize and screen the keyword candidate set, and determine the final keyword set for each text based on the contextual semantics of the English patent text.
[0011] Preferably, the large-scale English patent text in step S01 contains more than 100,000 English patent texts, and the texts come from public data of patent offices or patent websites. The public data are stored in the form of structured files, and the structured files contain patent titles and abstracts. The non-structured text composed of patent titles and abstracts is the large-scale English patent text.
[0012] Preferably, the patent text preprocessing process in step S02 includes word segmentation of the patent text, removal of stop words and removal of punctuation marks.
[0013] Preferably, the vocabulary set in step S02 includes a word set and a bigram set of the original patent text, and a bigram in the bigram set is a combination of any two words in the text.
[0014] Preferably, the TFIDF in step S03 is used for word frequency and inverse document statistics, and the top N keyword candidates with TFIDF scores are obtained by sorting the TF-IDF values. The keyword candidates and bigrams are then converted to all lowercase, duplicate bigrams are removed, and the keyword candidates already included in the bigrams are removed to obtain the keyword candidate set.
[0015] Preferably, the calculation formula of the TF-IDF algorithm in step S03 is:
[0016] TF-IDF i,j =tf i，j ×idf i，j (1)
[0017] Among them, tf i，j Indicates word frequency, idf i，j Represents the inverse document frequency, and the calculation formulas are as follows:
[0018]
[0019] In the above formula, tf i，j Indicates the frequency of term i appearing in document j, n i，j Indicates the number of times term i appears in document j, n k,j represents the number of times term k appears in document j, ∑ k n k,j Represents the sum of the occurrences of all terms in document j.
[0020]
[0021] In the above formula, D is the total number of all documents; |{j:t i ∈d}| is the entire document set d, containing the term t j The number of documents.
[0022] The higher the TF-IDF value, the more frequently the word appears in the document, and the fewer times it appears in other documents, thus better highlighting the characteristics or key content of the document.
[0023] Preferably, the large language model used in step S04 is a natural language processing model gpt-4omini based on deep learning. The natural language processing model further filters the keyword candidate set according to the requirements put forward by the user and the context content, excludes meaningless words screened out by the word frequency-based method, and generates a bigram set as candidate keywords during the patent text preprocessing stage and while collecting word keywords; then, the keyword candidate set is screened out by the TF-IDF algorithm by combining the vocabulary set and the bigram set, and then the large language model is called for further optimization, which can effectively exclude irrelevant words and solve the problem of polysemous interpretation of English words.
[0024] Preferably, the final keyword set in step S04 includes all words that are derived from the keyword candidate set generated in step S03.
[0025] Preferably, the input of step S04 is a patent text in a large-scale English patent text, and a keyword candidate set corresponding to the current text.
[0026] Preferably, the output of step S04 is a final keyword set for each text in the large-scale English patent text.
[0027] Beneficial Effects
[0028] The present invention provides a keyword extraction method for large-scale English patent texts. Compared with the prior art, it has the following beneficial effects:
[0029] 1. In the present invention, by combining the unsupervised keyword extraction method of the TF-IDF algorithm and the large language model, a bigram set is generated as candidate keywords during the patent text preprocessing stage and while collecting word keywords; then, the keyword candidate set is screened out by the TF-IDF algorithm in combination with the vocabulary set and the bigram set, and then the large language model is called for further optimization, which can effectively exclude irrelevant words and solve the problem of polysemous interpretation of English words, thereby effectively improving the efficiency and accuracy of keyword extraction of large-scale English patent texts. This method is suitable for batch operations of more than 100,000 patent texts, which can ensure the relevance and accuracy of keyword extraction, and provide stronger support and convenience for patent information retrieval.
[0030] 2. In the present invention, by combining a large language model and a TF-IDF algorithm to process large-scale English patent texts, the efficiency and accuracy of keyword extraction can be effectively improved. The method first inputs a large-scale English patent text; secondly, the large-scale English patent text is preprocessed to obtain a vocabulary set containing words and bigrams; then, the preprocessed patent text is preliminarily screened for keywords based on the TFIDF algorithm to generate a keyword candidate set for each text; finally, the large language model is called to optimize and screen the keyword candidate set, and the final keyword set for each text is determined in combination with the contextual semantics of the English patent text. This method can process up to hundreds of thousands of patent texts, solves the problem of invalid vocabulary caused by word frequency statistics in traditional methods, and improves the accuracy of the interpretation of polysemous words. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 A flowchart of a keyword extraction method for large-scale English patent texts proposed by the present invention;
[0032] Figure 2 This is an example diagram of the text processing process of a keyword extraction method for large-scale English patent texts proposed by the present invention;
[0033] Figure 3 A graph showing the keyword extraction results of a keyword extraction method for large-scale English patent texts proposed in the present invention using TF-IDF alone and gpt-40mini alone;
[0034] Figure 4 This is an example diagram of a scoring list of one of the texts in the subjective evaluation phase of a keyword extraction method for large-scale English patent texts proposed by the present invention;
[0035] Figure 5 This is an example diagram of a score summary table of one of the texts in the subjective evaluation phase of the keyword extraction method for large-scale English patent texts proposed in the present invention. DETAILED DESCRIPTION
[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0037] See also Figure 1-Figure 5 The present invention provides two technical solutions, which specifically include the following embodiments:
[0038] Embodiment 1:
[0039] A keyword extraction method for large-scale English patent texts includes the following steps:
[0040] S01. Inputting a large-scale English patent text. The large-scale English patent text in step S01 contains more than 100,000 English patent texts. The texts come from public data of patent offices or patent websites. The public data is stored in the form of structured files. The structured files contain patent titles and abstracts. The non-structured text composed of patent titles and abstracts is the large-scale English patent text.
[0041] S02, preprocessing a large amount of English patent text to obtain a vocabulary set including words and bigrams. The patent text preprocessing process in step S02 includes word segmentation, stop word removal, and punctuation removal of the patent text;
[0042] The input of the preprocessing process is the unstructured large-scale patent text obtained in step S01, and the output is as follows:
[0043] The output file includes a total vocabulary set file integrated from the vocabulary sets of all texts, and the total vocabulary set is stored in a structured file for the TF-IDF algorithm in step S03 to calculate the inverse document frequency IDF; in addition, the output includes the same number of vocabulary set files as the patent text, that is, each patent text generates a structured vocabulary set file separately for the TF-IDF algorithm in step S03 to calculate the word frequency TF and the word frequency inverse document frequency TF-IDF; the vocabulary set in step S02 includes the word set and bigram set of the original patent text, and the bigram in the bigram set is a combination of any two words in the text. In this way, the capture of implicit semantic relationships in patent texts can be enhanced and the coverage of keyword extraction can be expanded. This method not only improves the recognition ability of complex technical terms and concepts, but also reduces the dependence on word order, thereby effectively extracting keyword combinations with more semantic value and improving the flexibility and accuracy of keyword extraction;
[0044] S03, perform preliminary keyword screening on the preprocessed patent text based on the TF-IDF algorithm to generate a keyword candidate set for each text. The TF-IDF in step S03 is used for word frequency and inverse document statistics. By sorting the TF-IDF values, the top N keyword candidates with TF-IDF scores are obtained. The keyword candidates and bigrams are then converted to all lowercase, duplicate bigrams are removed, and keyword candidates already included in bigrams are removed to obtain a keyword candidate set. The calculation formula of the TF-IDF algorithm is:
[0045] TF-IDF i,j =tf i，j ×idf i，j (1)
[0046] Among them, tf i，j Indicates word frequency, idf i，j Represents the inverse document frequency, and the calculation formulas are as follows:
[0047]
[0048] In the above formula,
[0049] tf i，j Indicates the frequency of term i appearing in document j, n i，j Indicates the number of times term i appears in document j, n k,j Expressive words
[0050]
[0051] In the above formula, D is the total number of all documents; |{j:t i ∈d}| is the entire document set d, containing the term t j The number of documents.
[0052] The higher the TF-IDF value, the more frequently the word appears in the document, and the fewer times it appears in other documents, thus better highlighting the characteristics or key content of the document.
[0053] S04. Call the large language model to optimize and screen the keyword candidate set, and determine the final keyword set for each text based on the contextual semantics of the English patent text.
[0054] By combining the TF-IDF algorithm and the unsupervised keyword extraction method of the large language model, the efficiency and accuracy of keyword extraction from large-scale English patent texts can be effectively improved. During the patent text preprocessing stage and while collecting word keywords, a set of bigrams is generated as candidate keywords. Subsequently, the vocabulary set and the bigram set are combined to filter out the keyword candidate set through the TF-IDF algorithm, and then the large language model is called for further optimization, which can effectively exclude irrelevant words and solve the problem of polysemous interpretation in English.
[0055] Embodiment 2:
[0056] On the basis of Example 1, the large language model used in step S04 is a natural language processing model gpt-40mini based on deep learning. The natural language processing model further filters the keyword candidate set according to the requirements put forward by the user and the context content, and excludes meaningless words screened out by the word frequency-based method. The final keyword set in step S04 contains all the words derived from the keyword candidate set generated in step S03. The input of step S04 is the patent text in the large-scale patent text and the keyword candidate set corresponding to the current text. The output of step S04 is the final keyword set of each text in the large-scale English patent text.
[0057] In this embodiment, this method is tested. Since this method is an unsupervised method, there is no comparison experiment of precision, recall rate, and F1 score. The experiment is as follows:
[0058] The English texts of 154,170 utility model patents were obtained as large-scale patent text inputs, and this method, which combines TF-IDF with the large language model gpt-4omini, was used to extract text keywords. When using gpt-4omini to optimize and screen candidate keywords, the prompt words used were: "Based on the provided text, refine and sort the following keywords from most important to least important."; max_token was set to:
[0059] len(text)*0.75+len(keywords)*2+100
[0060] Among them, len(text) is the current text length calculated using Python's len function, len(keywords) is the length of the current text candidate keyword list calculated using Python's len function, and 100 is the buffer size.
[0061] In addition, two other methods are used to extract keywords and compared with the method in this paper.
[0062] The other two comparison methods are the method using TFIDF alone and the method using the large language model gpt-4omini alone.
[0063] When using gpt-4omin alone, the prompt is: "Extract 30 keywords from the following text, keywords on1y included tuples and words, sort them from most important to the least important, if a keyword is included in tuples, remove it: {text)"; the max_tokens value is set to:
[0064] len(text)*0.75+200
[0065] Among them, len(text) is the current text length calculated by using Python's len function, and 200 is the preset buffer size (since 30 keywords need to be extracted, the buffer size here refers to the buffer size len(keywords)*2 set when using the method in this article. Assuming that each keyword is a tuple, then len(keywords)*2 is 30*2*2=120, and an additional 80 is left, and the final buffer is 200).
[0066] The results of extracting keywords from the same text using the above three methods are as follows Figure 3 As shown, the displayed results only include the top ten keywords for each method.
[0067] In order to evaluate the above results, this experiment introduced a subjective evaluation method to score the keywords extracted by the three methods.
[0068] In the subjective evaluation phase, ten paragraphs of text were selected and keywords were extracted using three methods. Three evaluators conducted subjective evaluations on the keywords extracted by different methods. The evaluation method was set as follows:
[0069] The evaluation score of each keyword is an integer from 1 to 5, and the meaning of each score is as follows:
[0070] 5 points: Very relevant and can directly restore the main content of the text.
[0071] 4 points: Relatively relevant, able to infer the core part of the text, but the restored content is not complete.
[0072] 3 points: Moderately relevant, but the correlation is weak. It is difficult to fully restore the specific content through phrases.
[0073] 2 points: The amount of information is limited. Although there is a certain relevance, it is almost impossible to restore the text content.
[0074] 1 point: The amount of information is very small and no specific content can be restored from the word or phrase.
[0075] A high keyword score indicates that the evaluator believes that the keyword can fully express the meaning of the text; a low keyword score indicates that the evaluator believes that the keyword cannot fully express the meaning of the text.
[0076] In order to minimize the influence of subjective judgment, only text and keywords are listed in the scoring table. All keywords extracted by the three methods are shuffled and arranged in one column. Only the top 10 keywords are scored for each method, and overlapping keywords of multiple methods are scored only once.
[0077] For each of the three methods above, the final score is the average of all the keyword scores of the method, rounded to two decimal places. Figure 4 As shown, the score statistics represent e.g. Figure 5 shown.
[0078] The final scores are:
[0079] The patented method is 3.95, the method using TF-IDF alone is 3.47, and the method using gpt-4omini alone is 3.63. It can be seen that the patented method, that is, the method combining TF-IDF and gpt-4omini, has received a higher evaluation in the subjective evaluation link than the two methods alone.
[0080] Based on the test results of the above embodiments, the following conclusions are drawn:
[0081] Through this method, large-scale English patent texts are first input and preprocessed to form a rich vocabulary set containing words and bigrams; then, the TF-IDF algorithm is used to perform preliminary keyword screening on the processed text, and a keyword candidate set is constructed for each text; finally, the candidate set is finely optimized by calling a large language model, and the final keyword set for each text is accurately determined in combination with the contextual semantics of the text.
[0082] The present invention can efficiently process hundreds of thousands of English patent texts, effectively overcomes the problem of invalid vocabulary caused by word frequency statistics in traditional methods, and significantly improves the accuracy of polysemous word interpretation, thereby achieving significant progress in the field of keyword extraction.
[0083] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the application should be included in the protection scope of the present application.
Claims
1. A keyword extraction method for large-scale English patent texts, characterized by: The following steps are involved: S01. Input large-scale English patent texts; S02. Preprocessing a large amount of English patent text to obtain a vocabulary set including words and bigrams; S03. Perform preliminary keyword screening on the preprocessed patent text based on the TF-IDF algorithm to generate a keyword candidate set for each text; S04. Call the large language model to optimize and screen the keyword candidate set, and determine the final keyword set for each text based on the contextual semantics of the English patent text.
2. The keyword extraction method for large-scale English patent text according to claim 1, characterized in that: The large-scale English patent text in step S01 contains more than 100,000 English patent texts, which are from public data of patent offices or patent websites. The public data is stored in the form of structured files, which contain patent titles and abstracts. The non-structured text composed of patent titles and abstracts is the large-scale English patent text.
3. The keyword extraction method for large-scale English patent text according to claim 1 is characterized in that: The patent text preprocessing process in step S02 includes word segmentation of the patent text, removal of stop words and removal of punctuation marks.
4. The keyword extraction method for large-scale English patent text according to claim 1, characterized in that: The vocabulary set in step S02 includes a word set and a bigram set of the original patent text, and a bigram in the bigram set is a combination of any two words in the text.
5. The keyword extraction method for large-scale English patent text according to claim 1 is characterized in that: The TF-IDF in step S03 is used for word frequency and inverse document statistics. By sorting the TFIDF values, the top N keyword candidates with TF-IDF scores are obtained, and then the keyword candidates and bigrams are converted to all lowercase, duplicate bigrams are removed, and the keyword candidates already included in the bigrams are removed to obtain the keyword candidate set.
6. The keyword extraction method for large-scale English patent text according to claim 1, characterized in that: The calculation formula of the TF-IDF algorithm in step S03 is: TF-IDF i,j =tf i，j ×idf i，j (1) Among them, tf i，j Indicates word frequency, idf i，j Represents the inverse document frequency, and the calculation formulas are as follows: In the above formula, tf i，j Indicates the frequency of term i appearing in document j, n i，j Indicates the number of times term i appears in document j, n k,j represents the number of times term k appears in document j, ∑ k n k,j Represents the sum of the occurrences of all terms in document j. In the above formula, D is the total number of all documents; |{j:t i ∈d}| is the entire document set d, containing the term t j The number of documents.
7. The keyword extraction method for large-scale English patent text according to claim 1 is characterized in that: The large language model used in step S04 is a natural language processing model gpt-4omini based on deep learning. The natural language processing model further filters the keyword candidate set according to the requirements put forward by the user and the context content, and excludes meaningless words screened out by the word frequency-based method.
8. The keyword extraction method for large-scale English patent text according to claim 1, characterized in that: The final keyword set in step S04 includes all words that are derived from the keyword candidate set generated in step S03.
9. The keyword extraction method for large-scale English patent text according to claim 1, characterized in that: The input of step S04 is the patent text in the large-scale English patent text and the keyword candidate set corresponding to the current text.
10. The keyword extraction method for large-scale English patent text according to claim 1, characterized in that: The output of step S04 is the final keyword set of each text in the large-scale English patent text.