Medical paper key information extraction method based on large language model
By applying large language models and rich underlying information in the keyword extraction of medical papers, the shortcomings of existing methods in generalization capabilities and extraction effects are solved, and more accurate and efficient keyword extraction is achieved.
Patent Information
- Application Number
- CN202411727455.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-03-04
AI Technical Summary
The existing medical paper keyword extraction methods have shortcomings in generalization ability and extraction effect, and have failed to effectively utilize other feature information of the statement.
Using a method based on a large language model, the pre-trained language model is used as a feature extraction tool, and information such as part of speech characteristics and dictionary characteristics is added. By constructing a medical paper dictionary library and neural network model, structured key information is generated.
It significantly improves the accuracy and extraction efficiency of keywords in medical papers, and enhances the interpretability and comprehensibility of the model.
Smart Images
Figure CN119558310A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data information processing, and more specifically, to a method for extracting key information from medical papers based on a large language model. Background Art
[0002] With the continuous development of electronic information technology, electronic resources are becoming more and more popular, and the amount of data in electronic resources is becoming larger and larger. Therefore, it is particularly important to find the required information from massive data and to classify and search for data. In order to better classify data and facilitate data search and statistics, labels or index words are generally extracted from sentences. These words that can identify the content of sentences are called keywords. The rapid development of technologies such as machine learning and deep learning has made it possible to establish high-performance prediction and classification models.
[0003] However, the high complexity and nonlinear characteristics of these black box models make the interpretability and comprehensibility of the models a serious challenge. For many key application areas, such as finance, healthcare, and justice, it is not enough to have accurate prediction results. It is also necessary to have a clear understanding of how the model makes decisions so that these results can be explained and verified to be reasonable and reliable. In order to solve this problem, data information label processing methods have emerged. With the development and widespread application of large language models, data information label processing methods have also been widely used in natural language processing, text classification, sentiment analysis and other fields.
[0004] However, the information extraction method for medical paper data is mainly aimed at extracting keywords from medical papers. It mainly obtains semantic information through a pre-trained feature extraction model, directly uses the feature representation of the sentence as the word embedding feature, and obtains the final output through a linear network. It does not effectively utilize other feature information of the sentence, has weak generalization ability, and has poor keyword extraction effect. Summary of the invention
[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method for extracting key information from medical papers based on a large language model. The method uses a pre-trained language model as a feature extraction tool for sentences to be processed, and adds rich underlying information such as part-of-speech features and dictionary features to expand the differentiation of words. The difference between keywords and non-keywords in the sentences to be processed is clearer, so that the keywords in the sentences to be processed can be obtained more accurately. By generating structured key information, the efficiency of medical paper extraction and analysis can be significantly improved.
[0006] To achieve the above object, the present invention provides the following technical solutions: A method for extracting key information from medical papers based on a large language model, comprising the following steps: T1. Construct a medical paper dictionary library, which includes multiple groups of keywords, and extract data from the medical paper dictionary library to construct a neural network model.
[0007] T2. Train and adjust the neural network model based on the keywords in the medical paper dictionary to generate a target language model.
[0008] T3. Construct a target sentence block based on the target language model.
[0009] T4. Generate key information based on the multiple groups of target sentence blocks.
[0010] T5. Receive the sentence to be processed and obtain the relevant keyword dictionary in the medical paper dictionary library to determine the keywords in the sentence to be processed.
[0011] T6. Based on the generated key information and the keywords determined in the sentence to be processed, extract medical papers similar to the two in the medical dictionary library.
[0012] The present invention is further configured as follows: the keywords in step T1 respectively generate semantic vectors of the corresponding keywords; according to the grammatical and semantic features of the medical paper, the medical paper is divided into multiple semantic blocks, and corresponding semantic vectors are respectively generated.
[0013] The present invention is further configured as follows: the construction of the target sentence block in step T3 is based on the target language model, and the weight coefficients of the semantic vectors corresponding to each semantic block and the semantic vectors of each keyword are calculated respectively, and the sum of the weight coefficients of the semantic vector corresponding to any semantic block and the semantic vectors of all keywords is calculated to obtain the comprehensive weight coefficient corresponding to each semantic block. When it is determined that there is a weight coefficient greater than the set weight coefficient in the comprehensive weight coefficient, the semantic block corresponding to the comprehensive weight coefficient is used as the target sentence block.
[0014] The present invention is further configured as follows: the key information construction in step T4 is to identify a key sentence in at least one target sentence block among multiple target sentence blocks to form an initial key sentence set, cluster the key sentences in the initial key sentence set using a syntactic and semantic similarity algorithm to obtain multiple target key sentences, and reorganize and sort the multiple target key sentences according to the structure of the medical paper to generate the key information.
[0015] The present invention is further configured as follows: the determination of the keywords in the sentence to be processed in step T5 is to first receive the sentence to be processed, and obtain a keyword dictionary corresponding to the sentence to be processed, obtain the dictionary features of the keyword dictionary based on a feature extraction model, and obtain the sentence features of the sentence to be processed based on a language model, and perform part-of-speech tagging on the sentence to be processed to obtain part-of-speech features corresponding to the part of speech of the sentence to be processed.
[0016] The present invention is further configured to: determine based on the above-mentioned dictionary features, the part-of-speech features and the sentence features, determine based on the dictionary features, the part-of-speech features and the sentence features, and determine the keywords in the sentence to be processed based on the dictionary features, the part-of-speech features and the sentence features.
[0017] The present invention is further configured as follows: the medical paper extraction process in step T6 includes T6a, data preprocessing; T6b, feature extraction; T6c, feature generation; T6d, feature prediction; T6e, feature correction.
[0018] The present invention is further configured as follows: the data preprocessing in T6a is used to clean and prepare the original data to improve the data quality and consistency.
[0019] The feature extraction described in T6b is used to extract key feature representations from raw data.
[0020] The feature generation described in T6c is used to expand the expressive power of the original features and further extract more critical feature representations.
[0021] The feature prediction described in T6d is used to predict the key feature representation content.
[0022] The feature modification in T6e is to match and adjust the predicted key features with the generated key information and the keywords determined in the sentence to be processed.
[0023] The present invention is further configured as follows: the neural network model includes: a BERT model, a BILSTM model and a CRF model, the output data of the BERT model is input into the BILSTM model, and the data in the BERT model is output into the BILSTM model as the input of the CRF model, wherein the BERT model is used to realize the function of word embedding, and the BILSTM model and the CRF model construct a training neural network and constrain the output results.
[0024] The advantages of the present invention are: 1. The present invention uses a pre-trained language model as a feature extraction tool for sentences to be processed, and adds rich underlying information such as part-of-speech features and dictionary features to expand the differentiation of words, so that the difference between keywords and non-keywords in the sentences to be processed is clearer, thereby more accurately obtaining the keywords in the sentences to be processed.
[0025] 2. The present invention inputs the output data of the BERT model into the BILSTM model, and outputs the data in the BERT model into the BILSTM model as the input of the CRF model, wherein the BERT model is used to realize the function of word embedding, and the BILSTM model and the CRF model construct a training neural network and constrain the output results, thereby further improving the accuracy of the output data.
[0026] 3. In the present invention, since each semantic block and dictionary library are related to medical papers, the selected target sentence block can more accurately reflect the keywords in the medical paper, so that when the key sentence is selected in the target sentence block and the selected key sentence is reconstructed, the efficiency and accuracy of key information extraction can be improved; and by generating structured key information, the efficiency of medical paper extraction and analysis can be significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 This is a flowchart of the steps of a method for extracting key information from medical papers based on a large language model according to the present invention.
[0028] Figure 2 This is a flow chart of the extraction process of step T6 of the present invention. DETAILED DESCRIPTION
[0029] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0030] It should be noted that, unless otherwise specified, all technical and scientific terms used in this application have the same meanings as commonly understood by ordinary technicians in the technical field to which this application belongs.
[0031] In the present invention, unless otherwise specified, the directions used, such as "up" and "down", usually refer to the directions shown in the drawings, or to the vertical, perpendicular or gravity directions; similarly, for ease of understanding and description, "left" and "right" usually refer to the left and right shown in the drawings; "inside" and "outside" refer to the inside and outside relative to the outline of each component itself, but the above-mentioned directions are not used to limit the present invention.
[0032] Embodiment 1 See also Figure 1-2 , the present invention provides the following technical solutions: A method for extracting key information from medical papers based on a large language model, specifically, comprising the following steps: T1. Build a medical paper dictionary library. The medical paper dictionary library includes multiple groups of keywords. Data will be extracted from the medical paper dictionary library to build a neural network model.
[0033] T2. Train and adjust the neural network model based on the keywords in the medical paper dictionary to generate the target language model.
[0034] T3. Construct target sentence blocks based on the target language model.
[0035] T4. Generate key information based on multiple groups of target sentence blocks.
[0036] T5. Receive the sentence to be processed and obtain the relevant keyword dictionary in the medical paper dictionary library to determine the keywords in the sentence to be processed.
[0037] T6. Based on the generated key information and the keywords determined in the sentence to be processed, extract the medical papers similar to the two in the medical dictionary library.
[0038] The working principle of the first embodiment is as follows: by using a pre-trained language model as a feature extraction tool for sentences to be processed, and adding rich underlying information such as part-of-speech features and dictionary features to expand the differentiation of words, the difference between keywords and non-keywords in the sentences to be processed is made clearer, so that the keywords in the sentences to be processed can be obtained more accurately. By generating structured key information, the efficiency of medical paper extraction and analysis can be significantly improved.
[0039] Embodiment 2 See also Figure 1-2 , this embodiment 2 makes the following improvements on the basis of embodiment 1. Specifically, in step T1, the keywords respectively generate semantic vectors of the corresponding keywords; according to the grammatical and semantic features of the medical paper, the medical paper is divided into multiple semantic blocks, and the corresponding semantic vectors are respectively generated.
[0040] Specifically, the semantic vector contains the deep semantic information of the keywords, which is beneficial to the subsequent key information extraction, semantic matching, text classification and other tasks. At the same time, by processing each keyword, the generated semantic vector has better generalization and robustness, which can reduce the problems of word ambiguity and description differences in legal documents.
[0041] The construction of the target sentence block in step T3 is based on the target language model. The weight coefficients of the semantic vectors corresponding to each semantic block and the semantic vectors of each keyword are calculated respectively. The sum of the weight coefficients of the semantic vector corresponding to any semantic block and the semantic vectors of all keywords is calculated to obtain the comprehensive weight coefficient corresponding to each semantic block. When it is determined that there is a comprehensive weight coefficient greater than the set weight coefficient, the semantic block corresponding to the comprehensive weight coefficient is used as the target sentence block.
[0042] Specifically, by simultaneously focusing on the semantic vectors corresponding to each semantic block and the semantic vectors of each keyword, it is possible to automatically focus on the most critical sentence fragments (i.e., the sentence fragments corresponding to the target sentence block) from complex medical papers, thereby effectively alleviating the limitations of keyword matching, capturing the implicit contextual associations in medical papers, and improving the accuracy and coverage of key information positioning.
[0043] The key information construction in step T4 is to identify the key sentences in at least one target sentence block among multiple target sentence blocks to form an initial key sentence set, cluster the key sentences in the initial key sentence set using syntactic and semantic similarity algorithms to obtain multiple target key sentences, and reorganize and sort the multiple target key sentences according to the structure of the medical paper to generate key information.
[0044] The determination of the keywords in the sentence to be processed in step T5 is to first receive the sentence to be processed, and obtain the keyword dictionary corresponding to the sentence to be processed, obtain the dictionary features of the keyword dictionary based on the feature extraction model, and obtain the sentence features of the sentence to be processed based on the language model, and perform part-of-speech tagging on the sentence to be processed to obtain the part-of-speech features corresponding to the part of speech of the sentence to be processed.
[0045] Specifically, some target sentence blocks that may contain key information can be located from medical papers. However, the fragments in the target sentence blocks may have some redundant, irrelevant or incoherent problems. Therefore, the key sentences in the target sentence blocks can be screened and reorganized to form key information that is clear and semantically coherent.
[0046] Based on the above-mentioned dictionary features, part-of-speech features and sentence features, the keywords in the sentence to be processed are determined. Based on the above-mentioned dictionary features, part-of-speech features and sentence features, the keywords in the sentence to be processed are determined.
[0047] Specifically, by generating structured key information and keywords, it facilitates the query, extraction and reading of medical papers.
[0048] Embodiment 3 See also Figure 1-2, this embodiment 3 makes the following improvements on the basis of embodiment 2. Specifically, the extraction process of medical papers in step T6 includes T6a, data preprocessing; T6b, feature extraction; T6c, feature generation; T6d, feature prediction; T6e, feature correction.
[0049] Data preprocessing in T6a is used to clean and prepare raw data to improve data quality and consistency.
[0050] Specifically, it includes operations such as removing noise, processing missing values, and data normalization to improve the quality and consistency of the data.
[0051] Feature extraction in T6b is used to extract key feature representations from raw data.
[0052] Specifically, the feature extraction unit is used to extract meaningful feature representations from the raw data for subsequent learning and inference. This is achieved through traditional feature extraction methods (such as bag-of-words models, TFIDF, ngram, etc.) or deep learning methods (such as convolutional neural networks, recurrent neural networks, etc.), with the goal of converting raw data into numerical features that can be understood and processed by machine learning algorithms.
[0053] Feature generation in T6c is used to expand the expressive power of original features and further extract more critical feature representations.
[0054] Specifically, the expressive power of the original features is expanded by clustering features, reducing their dimensionality, or generating new features.
[0055] Feature prediction in T6d is used to predict the key feature representation content.
[0056] Specifically, key feature information of samples is inferred or generated to assist learning and inference in the absence of accurate key features.
[0057] Feature modification in T6e is to match and adjust the predicted key features with the generated key information and the keywords determined in the sentence to be processed.
[0058] Specifically, it is responsible for integrating key information from different sources and keywords determined in the sentences to be processed to obtain the final key information and keyword data results, matching and adjusting the predicted key features, and determining and extracting medical papers similar to the two in the medical dictionary library.
[0059] The neural network models include: BERT model, BILSTM model and CRF model. The output data of the BERT model is input into the BILSTM model, and the data in the BERT model is output into the BILSTM model as the input of the CRF model. The BERT model is used to realize the word embedding function. The BILSTM model and the CRF model construct a training neural network and constrain the output results.
[0060] Specifically, the BERT model is an architecture that can be used for many downstream tasks, such as answering questions, classification, NER, etc. The working principle of the BERT model is the same as that of most Imagenet deep learning models. First, the BERT model is trained on a large corpus (Masked LM task), and then the model is fine-tuned for our own task by adding some extra layers at the end. The model can be classification, question answering, or NER, etc.
[0061] BiLSTM (Bidirectional Long Short-Term Memory) is a special recurrent neural network (RNN) that can process sequence data and maintain long-term memory. Unlike traditional RNN models, BiLSTM considers both past and future information, allowing the model to better capture contextual relationships in sequence data.
[0062] Conditional Random Field (CRF) is a basic model in natural language processing and is widely used in scenarios such as word segmentation, entity recognition, and part-of-speech tagging. With the popularization of deep learning, models such as BILSTM+CRF, BERT+CRF, and TRANSFORMER+CRF have gradually appeared, and their effects have been significantly improved in these tagging scenarios.
[0063] The processing module includes a table image acquisition unit, a text line position detection unit, a text line content detection unit, an image direction correction unit, a table line analysis prediction unit, a table line analysis fusion unit, a table cell information analysis unit, a table cell information fusion unit and a formatted output unit.
[0064] Obviously, the above-described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0065] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0066] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.
[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
[0068] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
Claims
1. A method for extracting key information from medical papers based on a large language model, characterized in that: The following steps are involved: T1. Construct a medical paper dictionary library, wherein the medical paper dictionary library includes multiple groups of keywords, and extract data from the medical paper dictionary library to construct a neural network model; T2. Training and adjusting the neural network model based on the keywords in the medical paper dictionary to generate a target language model; T3, constructing a target sentence block based on the target language model; T4, generating key information based on the multiple groups of target sentence blocks; T5, receiving the sentence to be processed and obtaining the relevant keyword dictionary in the medical paper dictionary library to determine the keywords in the sentence to be processed; T6. Based on the generated key information and the keywords determined in the sentence to be processed, extract medical papers similar to the two in the medical dictionary library.
2. According to claim 1, a method for extracting key information from medical papers based on a large language model is characterized by: The keywords in step T1 generate semantic vectors of the corresponding keywords respectively; according to the grammatical and semantic features of the medical paper, the medical paper is divided into multiple semantic blocks, and corresponding semantic vectors are generated respectively.
3. The method for extracting key information from medical papers based on a large language model according to claim 2, characterized in that: The construction of the target sentence block in step T3 is based on the target language model. The weight coefficients of the semantic vectors corresponding to each semantic block and the semantic vectors of each keyword are calculated respectively. The sum of the weight coefficients of the semantic vector corresponding to any semantic block and the semantic vectors of all keywords is calculated to obtain the comprehensive weight coefficient corresponding to each semantic block. When it is determined that there is a semantic block whose comprehensive weight coefficient is greater than the set weight coefficient, the semantic block corresponding to the comprehensive weight coefficient is used as the target sentence block.
4. The method for extracting key information from medical papers based on a large language model according to claim 3, characterized in that: The key information construction in step T4 is to identify the key sentences in at least one target sentence block among multiple target sentence blocks to form an initial key sentence set, cluster the key sentences in the initial key sentence set using syntactic and semantic similarity algorithms to obtain multiple target key sentences, and reorganize and sort the multiple target key sentences according to the structure of the medical paper to generate the key information.
5. The method for extracting key information from medical papers based on a large language model according to claim 4, characterized in that: The determination of the keywords in the sentence to be processed in step T5 is to first receive the sentence to be processed, and obtain the keyword dictionary corresponding to the sentence to be processed, obtain the dictionary features of the keyword dictionary based on the feature extraction model, and obtain the sentence features of the sentence to be processed based on the language model, and perform part-of-speech tagging on the sentence to be processed to obtain the part-of-speech features corresponding to the part of speech of the sentence to be processed.
6. The method for extracting key information from medical papers based on a large language model according to claim 5, characterized in that: Based on the above-mentioned dictionary features, the part-of-speech features and the sentence features, the keywords in the sentence to be processed are determined. Based on the dictionary features, the part-of-speech features and the sentence features, the keywords in the sentence to be processed are determined.
7. The method for extracting key information from medical papers based on a large language model according to claim 6, characterized in that: The medical paper extraction process described in step T6 includes T6a, data preprocessing; T6b, feature extraction; T6c, feature generation; T6d, feature prediction; T6e, feature correction.
8. The method for extracting key information from medical papers based on a large language model according to claim 7, characterized in that: The data preprocessing described in T6a is used to clean and prepare the raw data to improve data quality and consistency; The feature extraction described in T6b is used to extract key feature representations from raw data; The feature generation described in T6c is used to expand the expressive power of the original features and further extract more critical feature representations; The feature prediction described in T6d is used to predict the key feature representation content; The feature modification in T6e is to match and adjust the predicted key features with the generated key information and the keywords determined in the sentence to be processed.
9. The method for extracting key information from medical papers based on a large language model according to claim 8, characterized in that: The neural network model includes: a BERT model, a BILSTM model and a CRF model. The output data of the BERT model is input into the BILSTM model, and the data in the BERT model is output into the BILSTM model as the input of the CRF model. The BERT model is used to realize the function of word embedding. The BILSTM model and the CRF model construct a training neural network and constrain the output results.