Single English patent text keyword extraction method based on fine tuning BERT model

Abstract

The invention discloses a single English patent text keyword extraction method based on a fine-tuning BERT model, the method utilizes a large-scale English patent text and keywords thereof to perform fine-tuning on the BERT model, so that the BERT model can accurately identify the keywords in a single English patent text, and the specific steps are as follows: inputting the large-scale English patent text and the keywords thereof; preprocessing and marking each patent text; performing fine tuning on the BERT model by using the mark data, so that the BERT model can identify keywords through an NER task; inputting a single English patent text, preprocessing the single English patent text, performing an NER task by using the fine-tuned BERT model, and generating category judgment of each word; according to the method, the powerful semantic comprehension ability of the BERT model and the accurate recognition ability of the NER task are combined, the accuracy of extracting the keywords of the single English patent text is improved, and patent information retrieval and analysis can be effectively supported.

Description

Technical Field

[0001] The present invention relates to the technical field of text information processing, and specifically to a single English patent text keyword extraction method based on a fine-tuned BERT model. Background Art

[0002] With the acceleration of global technological innovation, patent texts have become an important carrier of technological information, containing more than 90% of the world's technological information. According to the "Intellectual Property Facts and Figures 2023" released by the World Intellectual Property Organization (WIPO), as of 2022, the number of valid patents in the world has reached 17.3 million. Faced with such a large amount of patent texts, quickly and accurately extracting key information has become an important task. Keyword extraction, as the basis of information retrieval and text analysis, can help researchers quickly locate and understand the core content of the patent.

[0003] Existing keyword extraction methods are mainly divided into two categories: unsupervised methods and supervised methods. Unsupervised methods do not rely on pre-labeled tags and usually extract keywords based on word frequency, co-occurrence relationship or text structure. However, such methods cannot combine contextual semantics, resulting in inaccurate extraction results and difficulty in meeting the needs of high-quality information retrieval. In contrast, supervised methods rely on labeled training data sets and use machine learning models for keyword recognition, which can better combine contextual semantics and thus extract more accurate keywords. In recent years, with the development of deep learning technology, especially the emergence of the BERT model, more powerful technical support has been provided for supervised methods. The BERT model can learn rich language features and contextual information by pre-training a large amount of text data. By fine-tuning the BERT model, it can be adapted to specific tasks, such as named entity recognition (NER), thereby achieving accurate keyword extraction.

[0004] In this context, the present invention proposes a keyword extraction method for a single English patent text based on a fine-tuned BERT model, aiming to improve the accuracy of keyword extraction by combining deep learning technology with large-scale annotated data. This method makes full use of the powerful semantic understanding ability of the BERT model, accurately identifies keywords through the NER task, and through training with large-scale annotated data, the model can learn complex semantic patterns, so that it can still accurately extract keywords when processing unseen patent texts. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a single English patent text keyword extraction method based on a fine-tuned BERT model, which has the following advantages: the embodiment of the present invention significantly improves the accuracy and efficiency of keyword extraction by fine-tuning the BERT model and combining large-scale annotated data. By utilizing the powerful semantic understanding ability of the BERT model, the present invention can better combine contextual semantics and accurately identify keywords, especially showing wide applicability when processing English patent texts of different lengths. In addition, the present invention selects BERT-for-patents released by Google as a pre-training model, which further optimizes the processing of complex semantics and professional terms in patent texts, improves the overall performance, and provides more powerful support for patent information retrieval and analysis.

[0006] Technical Solution

[0007] To achieve the above object, the present invention provides the following technical solution: a method for extracting keywords from a single English patent text based on a fine-tuned BERT model, comprising the following steps:

[0008] S1. Input a large-scale English patent text and its keywords, wherein the large-scale English patent text contains more than 100,000 patent texts, and the text is stored in the form of a structured file, including patent titles, abstracts and their keywords;

[0009] S2. Preprocess each patent text, generate a vocabulary set, and mark the vocabulary set according to keywords;

[0010] S3. Fine-tune the pre-trained BERT model using labeled data. By adding a fully connected layer to the BERT model, the model is trained using labeled data to complete keyword recognition for the named entity recognition (NER) task.

[0011] S4. Input a single English patent text, and use the fine-tuned BERT model for NER tasks after preprocessing to generate category judgments for each word. The final keyword set is determined through post-processing steps.

[0012] Preferably, the preprocessing process in step S2 includes: segmenting the patent text, removing stop words and punctuation marks, and generating the following vector:

[0013] a) Character ordinal vector: convert the word into the corresponding numerical ordinal in the BERT-for-patents dictionary;

[0014] b) Filling token vector: mark valid word positions and filling positions;

[0015] c) Semantic vector: mark the semantic features of text;

[0016] d) Label vector: Label keywords as “I” and non-keywords as “O”.

[0017] Preferably, the pre-trained BERT model used in step S3 is the BERT-for-patents model released by Google, whose input text length is limited to 512 words, and outputs the NER classification result through a fully connected layer.

[0018] Preferably, the post-processing step of step S4 includes:

[0019] a) de-duplication the words judged as keywords;

[0020] b) Merge keywords that appear repeatedly or in different forms;

[0021] c) Generate the final keyword set.

[0022] Preferably, the large-scale English patent text in step S1 is derived from public data of a patent office or a patent website, and the keywords are pre-extracted by a method combining TF-IDF with a large language model.

[0023] Preferably, during the fine-tuning process of step S3, the annotated data includes a training set of 140,613 patent texts and a test set of 13,199 patent texts, and the model performance is evaluated by precision, recall and F1 score.

[0024] Preferably, in step S4, the fine-tuned BERT model can process English patent texts of different lengths and output a category judgment sequence for each word.

[0025] Preferably, the post-processing step further includes performing a semantic consistency check on the merged keyword set to eliminate redundancy and ambiguity.

[0026] Preferably, when the input text length of the BERT-for-patents model exceeds 512 words, it is truncated or segmented to adapt to the model input limitation.

[0027] Preferably, the final keyword set is used to support patent information retrieval, technology trend analysis or patent text abstract generation.

[0028] Beneficial Effects

[0029] Compared with the prior art, the present invention provides a single English patent text keyword extraction method based on a fine-tuned BERT model, which has the following beneficial effects:

[0030] 1. This single English patent text keyword extraction method based on the fine-tuned BERT model can deeply understand the contextual semantics of patent texts and accurately identify professional terms and technical keywords by fine-tuning the BERT-for-patents model and combining it with the named entity recognition (NER) task. Experimental data show that compared with traditional methods such as KeyBERT, YAKE and RAKE, the F1 score of this scheme is significantly improved, verifying its advantages in semantic relevance and recognition accuracy.

[0031] 2. This single English patent text keyword extraction method based on the fine-tuned BERT model is compatible with input texts of different lengths (texts with more than 512 words can be processed in segments) through preprocessing (such as truncation and segmentation) and sliding window mechanism, targeting the characteristics of long texts and dense professional terms in patent texts, ensuring effective parsing of complex semantic structures.

[0032] 3. This single English patent text keyword extraction method based on the fine-tuned BERT model adopts automated pre-processing (word segmentation, stop word removal, vector generation) and post-processing (keyword deduplication, normalization and merging, semantic verification) processes to reduce manual intervention and improve the overall processing speed. Combined with large-scale annotated data (140,000+ training sets), it has the ability to quickly adapt to new patent texts.

[0033] 4. This single English patent text keyword extraction method based on the fine-tuned BERT model introduces semantic consistency verification in the post-processing step. Through context association analysis, domain knowledge base filtering and manual review, redundant and ambiguous keywords are eliminated to ensure the technical relevance of the output results. The final keyword set can be directly applied to patent information retrieval systems, technology trend analysis platforms or automated summary generation tools to improve the efficiency of patent analysis and decision-making support capabilities.

[0034] 5. This single English patent text keyword extraction method based on the fine-tuned BERT model can output the extracted keywords through an API interface or visualization tool, and can be seamlessly integrated into the intelligent retrieval system, technology map construction and patent database management platform, providing efficient technical intelligence services for scientific research institutions, enterprises and patent examination departments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flowchart of a single English patent text keyword extraction method based on a fine-tuned BERT model proposed in the present invention;

[0036] Figure 2 This is an example of the text processing process of a single English patent text keyword extraction method based on a fine-tuned BERT model proposed in the present invention;

[0037] Figure 3A schematic diagram of the architecture of a BERT model obtained during the fine-tuning process of a single English patent text keyword extraction method based on a fine-tuned BERT model proposed in the present invention;

[0038] Figure 4 This is a table showing the precision, recall, and F1 score of a comparative experiment of a single English patent text keyword extraction method based on a fine-tuned BERT model proposed in the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] See also Figure 1-4 , a single English patent text keyword extraction method based on a fine-tuned BERT model, comprising the following steps:

[0041] S1. Input large-scale English patent texts and their keywords: There are more than 100,000 large-scale English patent texts, which are stored in the form of structured files. The files cover patent titles, abstracts and their corresponding keywords. These texts are derived from public data of patent offices or patent websites, providing a rich data foundation for subsequent model training. The purpose of the input is to obtain a large amount of patent texts and annotation information for training models to learn the relationship pattern between patent texts and keywords.

[0042] S2. Preprocess each patent text, generate a vocabulary set, and mark the vocabulary set according to keywords: During preprocessing, first segment the patent text to divide the continuous text into single words; then remove stop words, such as "the" and "and", which are of little significance for keyword extraction; at the same time, remove punctuation marks to make the text more concise, and then mark the vocabulary set according to keywords. The marking method is related to the BERT model used later, and generate labeled data for model training, so that the model can learn the position and characteristics of keywords in the text.

[0043] S3. Use labeled data to fine-tune the pre-trained BERT model. By adding a fully connected layer to the BERT model, the labeled data is used to train the model to complete keyword recognition for the named entity recognition (NER) task: Select a suitable pre-trained BERT model (such as the BERT-for-patents model released by Google), and add a fully connected layer to it. The fully connected layer is responsible for mapping the features extracted by the BERT model to the classification space of the NER task. Use the previously generated labeled data to train the model after adding the fully connected layer, so that the model learns the annotation rules and has the ability to recognize keywords in patent texts.

[0044] S4. Input a single English patent text, perform the NER task using the fine-tuned BERT model after preprocessing, generate a category judgment for each word, and determine the final keyword set through post-processing steps: Input a single English patent text with keywords to be extracted into the system, first perform the same preprocessing operation as in S2, and then use the fine-tuned BERT model to perform the NER task. The model generates a category judgment for each word. The judgment value is "I" indicating that the word is a keyword, and "O" indicating that it is not a keyword. Finally, through the post-processing steps, the words judged as keywords are deduplicated, merged, and other operations are performed to determine the final keyword set.

[0045] In the above embodiment, as a preferred solution, the preprocessing process in step S2 includes: segmenting the patent text, removing stop words and punctuation marks, and generating the following vectors:

[0046] a) Character sequence vector: Each word in the patent text is converted into a corresponding digital sequence in the BERT-for-patents dictionary. If the number of words in the text is less than 512, the excess positions are filled with specific symbols; if the number of words in the text is greater than 512, it can be truncated or segmented according to the model input requirements before conversion. This processing is to convert the text information into a digital form that the model can understand, so as to facilitate the model's subsequent calculations.

[0047] b) Padding marker vector: used to mark the positions of valid words and padding positions in the text. Positions less than or equal to the actual number of words in the text are marked with specific symbols to indicate that these are real words. Positions exceeding the number of words in the text and less than 512 are filled with different specific symbols to enable the model to distinguish between valid words and padding content.

[0048] c) Semantic vector: Marks the semantic features of the text. Positions less than or equal to 512 are marked with specific symbols. These semantic features are generated by the BERT-for-patents model and help the model understand the semantic information of the text and better identify keywords.

[0049] d) Mark vector: Artificially apply labels, mark keywords as "I" and non-keywords as "O". If a keyword appears repeatedly in the text, it is marked only once. For an English patent text containing n words, n labels need to be applied to provide clear labeling information for model training.

[0050] In the above embodiment, as a preferred solution, the pre-trained BERT model used in step S3 is the BERT-for-patents model released by Google, whose input text length is limited to 512 words, and outputs the NER classification result through the fully connected layer. The model has been pre-trained with a large amount of text and has a strong semantic understanding ability, which is suitable for processing patent texts. The input text length is limited to 512 words. When the input text exceeds this length, it needs to be truncated or segmented. During the fine-tuning process, a fully connected layer is added to the BERT-for-patents model. The fine-tuned model converts the features extracted by the BERT model into the classification results of the NER task through the fully connected layer, that is, it determines whether each word is a keyword.

[0051] In the above embodiment, as a preferred solution, the post-processing step of step S4 includes:

[0052] a) De-duplicate words that are judged to be keywords: Since the model may repeatedly recognize certain words during the keyword recognition process, de-duplicate processing can remove these repeated keywords to ensure the simplicity of the keyword set.

[0053] b) Merge keywords that appear repeatedly or in different forms: Some keywords may appear in different forms in the text, such as singular and plural forms, different tenses of verbs, etc. Merging will unify these different forms of keywords into one, making the keyword set more standardized.

[0054] c) Generate the final keyword set: After deduplication and merging, the obtained keywords are aggregated to generate the final keyword set, which will be used for subsequent patent information retrieval, analysis and other tasks.

[0055] In the above embodiment, as a preferred solution, the large-scale English patent text in step S1 is derived from the public data of the patent office or patent website, and the keywords are pre-extracted by combining TF-IDF with a large language model. The public data of the patent office or patent website is authoritative and extensive, and can provide rich and diverse patent texts for model training. The keywords are pre-extracted by combining TF-IDF with a large language model. TF-IDF can measure the importance of words in the text, and the large language model can more accurately identify keywords in combination with contextual semantics. The combination of the two improves the accuracy of initial keyword extraction and provides more reliable annotation data for subsequent model training.

[0056] In the above embodiment, as a preferred solution, during the fine-tuning process of step S3, the annotated data includes a training set of 140,613 patent texts and a test set of 13,199 patent texts. The model performance is evaluated by precision, recall and F1 score. The use of a large-scale training set (140,613 patent texts) allows the model to learn rich patent text features and keyword patterns, thereby improving the generalization ability of the model. The test set (13,199 patent texts) is used to evaluate the performance of the model on unseen data. The precision measures the proportion of keywords recognized by the model that are truly correct; the recall measures the proportion of actual keywords correctly recognized by the model; the F1 score comprehensively considers the precision and recall, evaluates the model performance more comprehensively, and helps determine the optimal training state and parameter settings of the model.

[0057] In the above embodiment, as a preferred solution, in step S4, the fine-tuned BERT model can process English patent texts of different lengths and output a category judgment sequence for each word. In actual applications, the lengths of patent texts vary. The fine-tuned BERT model can process English patent texts of different lengths through preprocessing (such as truncation and segmentation) and its own structural design. For the input text, the model processes each word and outputs a sequence of n judgment results (n is the number of words in the text). Each judgment result indicates whether the word is a keyword, which facilitates the subsequent post-processing step to determine the keyword set.

[0058] In the above embodiment, as a preferred solution, the post-processing step further includes performing a semantic consistency check on the merged keyword set to eliminate redundancy and ambiguity. After completing keyword deduplication and merging, a semantic consistency check is performed to check whether there is redundant information (such as keywords with similar meanings) and ambiguity (such as unclear understanding caused by multiple meanings of a word) in the keyword set by using context association analysis, domain knowledge base filtering and possible manual review. This ensures that the final keyword set is more semantically accurate and relevant, improves the quality of keywords, and provides more reliable support for patent information retrieval and analysis.

[0059] In the above embodiment, as a preferred solution, when the input text length of the BERT-for-patents model exceeds 512 words, truncation or segmentation is performed to adapt to the model input limitation. Since the input text length of the BERT-for-patents model is limited to 512 words, when encountering a patent text exceeding this length, truncation processing can be used to directly intercept the first 512 words; or segmentation processing can be used to divide the long text into multiple fragments of 512 words or less, and input them into the model for processing respectively. This can ensure that the model can process long texts normally and will not cause errors or be unable to process due to input length problems.

[0060] In the above embodiment, as a preferred solution, the final keyword set is used to support patent information retrieval, technology trend analysis or patent text abstract generation. In patent information retrieval, the keyword set can be used as a search condition to help quickly locate relevant patents; in terms of technology trend analysis, by analyzing a large number of patent text keywords, the development trend of a specific technology field can be discovered; when generating a patent text abstract, the keyword set can provide key information for generating the abstract, so that the abstract can more accurately reflect the core content of the patent, thereby improving the efficiency of patent analysis and decision-making support capabilities.

[0061] In summary, the single English patent text keyword extraction method based on the fine-tuned BERT model is as follows: Figure 1 As shown, the following steps are included:

[0062] S1. Input large-scale English patent text and its keywords:

[0063] Large-scale English patent texts contain more than 100,000 English patent texts. The texts come from public data from patent offices or patent websites. Patent public data is stored in the form of structured files. The structured files contain patent titles and abstracts. The non-structured texts composed of patent titles and abstracts are large-scale patent texts.

[0064] The input of step S1 is a structured patent document; the output is a structured file with the same number of texts, and the file contains structured text consisting of the patent title, abstract and its keywords.

[0065] S2. Preprocess each patent text to generate a vocabulary set, and mark the vocabulary set according to keywords. The format of the result of the marking process is related to the BERT model selected in step S3. When the model selected in step S3 is BERT-for-patents, the preprocessing process in step S2 needs to output the following four vectors:

[0066] a) Character sequence vector: generated by BERT-for-patent, each word is converted into a corresponding numeric sequence in the BERT-for-patents dictionary, and the positions exceeding the number of text words and less than 512 are filled with specific symbols;

[0067] b) Filled token vector: Generated by BERT-for-patent, positions less than or equal to the number of words in the text are marked with specific symbols; positions exceeding the number of words in the text and less than 512 are filled with different specific symbols;

[0068] c) Semantic vector: generated by BERT-for-patent, positions less than or equal to 512 are marked with specific symbols;

[0069] d) Marking vector: artificially applied, keywords are marked as "I" and non-keywords are marked as "O". If a keyword appears repeatedly in the text, it will not be marked repeatedly, but only once. For an English patent text containing n words, n marks need to be applied.

[0070] S3. Fine-tune the pre-trained BERT model using labeled data to enable it to recognize keywords through the NER task:

[0071] In the fine-tuning step, a fully connected layer is first added to BERT for NER classification output; then the BERT model with the fully connected layer attached is fine-tuned using large-scale annotated English patent text data, so that it learns the annotation rules and completes the training of keyword recognition capabilities. After fine-tuning, the fully connected layer can determine the keywords of a single patent text that has not been seen before.

[0072] S4. Use the fine-tuned BERT model to perform NER tasks and extract keywords from a single English patent text:

[0073] When performing NER tasks, the fine-tuned BERT model generates a category judgment value for each word. A judgment value of "I" indicates that the word is a keyword, and a judgment value of "O" indicates that the word is not a keyword. For an English patent text containing n words, the fine-tuned BERT model will generate category judgments for n words and output a sequence containing n judgment results. The fine-tuned BERT model can handle English patent texts of different lengths when performing NER tasks.

[0074] In this embodiment, the method is tested, and the evaluation criteria are precision, recall, and F1 score.

[0075] The experiment is as follows:

[0076] The English texts of 153,812 utility model patents were obtained from the USTPO official website as large-scale patent text input, and the text keywords were extracted using a method that combined TF-IDF and the large language model gpt-4o-mini. Among them, the texts and keywords of 140,613 patents were used as training sets; the texts and keywords of the remaining 13,199 patents were used as test sets.

[0077] The fine-tuned BERT model is used to extract keywords from the test set patent text, and the precision, recall, and F1 scores are calculated respectively. In addition, three other methods are used to extract keywords and compared with the method described in this article. The three comparison methods are KeyBERT, YAKE, and RAKE. The results of the comparison experiment are shown in Figure 2. Figure 4 As shown, it can be seen that the patented method, namely BERTNER, achieves better results than the other three methods.

[0078] The present invention provides a single English patent text keyword extraction method based on a fine-tuned BERT model. The method significantly improves the accuracy and efficiency of keyword extraction by combining deep learning technology with large-scale annotated data. The method first inputs a large-scale English patent text and its keywords, pre-processes each patent text, generates a vocabulary set, and marks the vocabulary set according to the keywords. Then, the pre-trained BERT model is fine-tuned using the marked data so that it can recognize keywords through a NER task. Finally, the fine-tuned BERT model is used to perform a NER task on a single English patent text to generate a category judgment for each word, and the final keyword set is determined through a post-processing step.

[0079] The present invention makes full use of the powerful semantic understanding ability of the BERT model, which can better combine contextual semantics and accurately identify keywords. It is particularly applicable when processing English patent texts of different lengths. In addition, the present invention selects BERT-for-patents released by Google as a pre-training model, which further optimizes the processing of complex semantics and professional terms in patent texts, improves the overall performance, and provides more powerful support for patent information retrieval and analysis.

[0080] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise one" do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0081] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A single English patent text keyword extraction method based on a fine-tuned BERT model, characterized by: The following steps are involved: S1. Input a large-scale English patent text and its keywords, wherein the large-scale English patent text contains more than 100,000 patent texts, and the text is stored in the form of a structured file, including patent titles, abstracts and their keywords; S2. Preprocess each patent text, generate a vocabulary set, and mark the vocabulary set according to keywords; S3. Fine-tune the pre-trained BERT model using labeled data. By adding a fully connected layer to the BERT model, the model is trained using labeled data to complete keyword recognition for the named entity recognition (NER) task. S4. Input a single English patent text, and use the fine-tuned BERT model for NER tasks after preprocessing to generate category judgments for each word. The final keyword set is determined through post-processing steps.

2. A single English patent text keyword extraction method based on a fine-tuned BERT model according to claim 1, characterized in that: The preprocessing process in step S2 includes: segmenting the patent text, removing stop words and punctuation marks, and generating the following vectors: a) Character ordinal vector: convert the word into the corresponding numerical ordinal in the BERT-for-patents dictionary; b) Filling token vector: mark valid word positions and filling positions; c) Semantic vector: mark the semantic features of text; d) Label vector: Label keywords as “I” and non-keywords as “O”.

3. A single English patent text keyword extraction method based on a fine-tuned BERT model according to claim 1, characterized in that: The pre-trained BERT model used in step S3 is the BERT-for-patents model released by Google, whose input text length is limited to 512 words, and outputs the NER classification result through a fully connected layer.

4. A single English patent text keyword extraction method based on a fine-tuned BERT model according to claim 1, characterized in that: The post-processing step of step S4 includes: a) de-duplication the words judged as keywords; b) Merge keywords that appear repeatedly or in different forms; c) Generate the final keyword set.

5. The method for extracting keywords from a single English patent text based on a fine-tuned BERT model according to claim 1, characterized in that: The large-scale English patent text in step S1 is derived from public data of patent offices or patent websites, and the keywords are pre-extracted by combining TF-IDF with a large language model.

6. The method for extracting keywords from a single English patent text based on a fine-tuned BERT model according to claim 1, characterized in that: During the fine-tuning process of step S3, the annotated data includes a training set of 140,613 patent texts and a test set of 13,199 patent texts, and the model performance is evaluated by precision, recall and F1 score.

7. The method for extracting keywords from a single English patent text based on a fine-tuned BERT model according to claim 1, characterized in that: In step S4, the fine-tuned BERT model can process English patent texts of different lengths and output a category judgment sequence for each word.

8. The method for extracting keywords from a single English patent text based on a fine-tuned BERT model according to claim 1, characterized in that: The post-processing step further includes performing a semantic consistency check on the merged keyword set to eliminate redundancy and ambiguity.

9. The method for extracting keywords from a single English patent text based on a fine-tuned BERT model according to claim 1, characterized in that: When the input text of the BERT-for-patents model is longer than 512 words, it is truncated or segmented to fit the model input limitation.

10. The method for extracting keywords from a single English patent text based on a fine-tuned BERT model according to claim 1, characterized in that: The final keyword set is used to support patent information retrieval, technology trend analysis or patent text abstract generation.

Images