Named entity recognition method for traditional Chinese medicine case based on natural language processing
By combining the BERT model, BiLSTM model and multi-head attention mechanism, the problem of low accuracy of naming entity recognition in traditional Chinese medicine cases is solved, and the accurate recognition of complex text is achieved, which improves the accuracy and generalization of naming entity recognition in traditional Chinese medicine cases is improved. ability.
Patent Information
- Application Number
- CN202411044797.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-12-03
AI Technical Summary
In the prior art, the accuracy of the naming entity recognition method of traditional Chinese medicine case is low, and it is difficult to effectively identify the entity information in traditional Chinese medicine case, especially when the text is complex and the description varies greatly.
The BERT model is combined with the BiLSTM model, and through the multi-head attention mechanism and CRF model, the global dependence and local context information of the text are integrated to enhance the correlation between adjacent information and improve the accuracy of naming entity recognition.
It improves the accuracy of naming entity recognition in traditional Chinese medicine cases, can better identify related information in complex texts, and improves the generalization ability and recognition effect of the model.
Smart Images

Figure CN119067118A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of natural language processing, and more specifically, to a method for named entity recognition of traditional Chinese medicine medical records based on natural language processing. Background Art
[0002] Named entity recognition is a natural language processing technique used to identify and classify entities from text. The goal is to extract entities from text and label them according to predefined categories.
[0003] TCM medical records record a wealth of clinical experience and knowledge. Named entity recognition can be used to systematize and structure the text, which is convenient for subsequent research and analysis. By identifying and analyzing entities in medical records (such as symptoms, diagnoses, treatment methods, drugs, etc.), it can provide decision support for clinicians and improve the diagnosis and treatment effect, which is conducive to the digitization of TCM. However, in the field of traditional Chinese medicine, TCM medical records truly record the consultation process of patients and doctors in the form of Chinese characters. These records usually contain a mixture of modern Chinese, classical Chinese and spoken language, which makes the text of TCM medical records complex, the descriptions vary greatly, and the related information in the text is farther away.
[0004] The patent application named "A method for automatic identification of named entities in TCM patent documents" and the publication number "CN110826333A" discloses a method of first training the delimitation module and the classification module, and then using the trained delimitation module and the classification module to automatically identify named entities in TCM patent documents. In this method, only the BiLSTM model is used to process sentence information, and the ability to obtain context information is poor. In the scheme published in the patent application named "Named Entity Recognition Method Based on Local Text Information" and the publication number "CN117933259A", Bert+BiLSTM+multi-head attention mechanism+Softmax is used. Softmax performs label classification and prediction independently, ignoring the order and dependency between labels, and is not suitable for named entity recognition of TCM medical records. The existing named entity recognition methods of machine learning and deep learning methods have poor generalization ability, and it is difficult to accurately identify entity information in TCM medical records. The accuracy rate in identifying TCM medical record texts is low. Summary of the invention
[0005] The purpose of the present invention is to provide a named entity recognition method for TCM medical records based on natural language processing to address the deficiencies in the prior art, so as to solve the problem of low recognition accuracy of the named entity recognition method in the prior art.
[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:
[0007] The present application provides a method for named entity recognition of TCM medical records based on natural language processing, the method comprising the following steps:
[0008] S1, obtain medical record data and obtain a standard data set after preprocessing;
[0009] S2, building a named entity recognition model;
[0010] S3, training and testing the named entity recognition model;
[0011] S4, input the text to be recognized into the trained named entity recognition model, and the named entity recognition model outputs the prediction result.
[0012] Furthermore, the preprocessing includes data cleaning, entity tagging, and BIO annotation of the marked entities.
[0013] Furthermore, entity tagging is performed using Doccano to assign labels to the text data, with symptom entities labeled as “symptom”, disease entities labeled as “disease”, and drug entities labeled as “medicine”.
[0014] Furthermore, in the BIO notation, B represents the beginning of the entity, I represents the rest of the entity, and O represents the non-entity part.
[0015] Furthermore, the named entity recognition model includes the BERT model, the BiLSTM model, the multi-head attention mechanism and the CRF model.
[0016] Furthermore, the output of the BERT model is connected to the input of the BiLSTM model, and the output of the BiLSTM model is connected to the input of the multi-head attention mechanism.
[0017] Furthermore, the output of the multi-head attention mechanism is concatenated and fused with the output of the BERT model, and the fused information is input into the CRF model for prediction, and finally the prediction result is output.
[0018] Furthermore, in step S3, the data set is divided into a training set and a test set in a ratio of 7:3-8:2.
[0019] Furthermore, in step S3, the test result is evaluated by the parameter F1 value, and the calculation formula is:
[0020]
[0021] Among them, precision represents the accuracy rate and recall represents the recall rate.
[0022] Compared with the prior art, the present invention has the following beneficial effects:
[0023] In this application, the BERT model and the BiLSTM model pay attention to different scales of contextual information of the text; the BERT model focuses on global dependencies, and the dependencies between texts that are far away are paid attention to, while the BiLSTM model pays attention to the direct order and local context between words. The text sequence information output by the BERT model is spliced and fused with the text sequence information processed by the BiLSTM model and the multi-head attention mechanism. The multi-head attention mechanism enhances the association of sequence information between adjacent information, so that the accuracy of the method of this application for named entity recognition of TCM medical records is relatively high. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 A schematic diagram of a method for named entity recognition of TCM medical records based on natural language processing provided by the present invention;
[0025] Figure 2 A schematic diagram of a named entity recognition model constructed in a named entity recognition method for traditional Chinese medicine medical records based on natural language processing provided by the present invention. DETAILED DESCRIPTION
[0026] In order to make the implementation process of the present invention clearer, it will be described in detail below with reference to the accompanying drawings.
[0027] The present invention provides a method for identifying named entities in traditional Chinese medicine medical records based on natural language processing. Figure 1 As shown, the application method includes the following steps:
[0028] S1, obtain medical record data and obtain a standard data set after preprocessing;
[0029] The data in this application can come from ancient medical books or from self-built TCM data sets of TCM physicians’ clinical medical records. First, the medical case text is cleaned, and the cleaning algorithm is written using code to remove duplicate text records and unnecessary parts, leaving only the required text data. This can improve the quality of the data set, remove irrelevant text information, and thus improve the learning efficiency of the model. Then mark the entities in the processed data. Specifically, extract the “chief complaint”, “medical history”, and “diagnosis results” in the medical case text.
[0030] Doccano is used to label the text with entity tags, and the text data is labeled. Symptom entities are labeled as "symptom", disease entities are labeled as "disease", and drug entities are labeled as "medicine", etc. For example, in the text "2 days ago, the whole body had chills and fever, the body temperature remained at 39.5℃, thirst and red urine. The previous doctor diagnosed it as exogenous wind-heat syndrome. After taking 2 doses of Yinqiao Powder, the body temperature decreased, but it turned into a fever in the afternoon, hoarseness, and difficulty swallowing and eating, so I sought treatment from Master Zou." "Chills and fever", "thirst and red urine", "fever", "hoarseness", "difficulty swallowing and eating" are marked as "symptom", "exogenous wind-heat syndrome" is marked as "disease", and "Yinqiao Powder" is marked as "medicine". In this embodiment, the text content is manually labeled under the guidance of a professional Chinese medicine practitioner, and BIO annotation is performed after the labeling is completed.
[0031] In order to enable the model to learn entities in the text during training, the labeled text is annotated with BIO to obtain a standard dataset. Specifically, B represents the beginning of the entity, I represents the remaining part of the entity, and O represents the non-entity part; for example, the text "Thirst and red urine, the previous doctor diagnosed it as exogenous wind-heat syndrome" is annotated with ["B", "I", "I", "I", "O", "O", "O", "O", "B", "I", "I", "I", "I"]. An example of a dataset with BIO annotation is as follows: the text "Asthma has been attacking for several days." is annotated with ["B-disease", "I-disease", "O", "O", "O", "O", "O", "O"]; the text "Coughing and shortness of breath for the past two weeks." is annotated with ["O", "O", "O", "B-symptom", "I-symptom", "I-symptom", "I-symptom"]. Based on the learned context information, the accuracy of the model is tested in the test set. At the same time, the data in the test set is also annotated with BIO, which makes it convenient for the model to judge the applicability of the ability learned from the training set in the test set. At the same time, consistent with the marking rules in open source data, it is only necessary to mark according to the above rules and divide it into training sets and test sets to facilitate joint feature learning. Because there is a lack of TCM-related data sets and no high-quality data sets to train the model, a high-quality TCM data set is built to improve the accuracy of the model. The open source data sets are from general medical fields and common fields to test the generalization ability of the model. Marking is to separate the entities in the text, and annotation is to annotate the separated entities so that the model can learn the information of the entities.
[0032] The data sources also include the Chinese electronic medical record named entity recognition evaluation dataset released by the open source dataset CCKS2019 and the CLUENER2020 fine-grained named entity recognition dataset to verify the generalization ability of the model method in named entity recognition in other fields.
[0033] S2, building a named entity recognition model;
[0034] The named entity recognition model of this application includes an embedding layer, an encoding layer, and a decoding layer, involving a BERT model, a BiLSTM model, a multi-head attention mechanism, and a CRF. Figure 2 As shown, the output of the BERT model is connected to the input of the BiLSTM model and the output of the multi-head attention mechanism, and the output of the BiLSTM model is connected to the input of the multi-head attention mechanism. The fused information is predicted by the CRF model and finally output. After the text input text obtained in step S1 is encoded and the preliminary text sequence is obtained through the BERT model, it is transmitted to the BiLSTM model for further inter-text dependency information acquisition, and then the multi-head attention mechanism is used to understand the dependency relationship in the text more comprehensively and meticulously. Then, the preliminary output sequence information of the first step BERT model is fused with the output sequence information of the multi-head attention mechanism layer, and finally the fused sequence information is predicted through the CRF model, that is, the entity is judged according to the sequence information. The sequence information after the model processes the text exists in the form of vectors, and fusion is to splice different information vectors to achieve fusion.
[0035] Specifically, the data processed in step S1 is input into the above-mentioned named entity recognition model; that is, the data is first input into the BERT model. The embedding layer adopts the Transformer Encoder structure in the BERT model to obtain five kinds of sequence information of the text: character-level embedding information, word-level embedding information, segment embedding information, relative position encoding information and entity type information. The BERT model takes the position of each text into consideration, so it pays more attention to the information of the global scale. Specifically, in the BERT model, each word will be converted into a 768-dimensional vector representation; in order to distinguish the sentences in the sentence pair to which each word belongs, the segment embedding layer sets two vector representations. The former vector assigns "0" to each word in the first sentence, and the latter vector assigns "1" to each word in the second sentence. If there is only one sentence in the input, the segment embedding information will be all "0". Transformers cannot encode the sequentiality of the input text sequence, so by adding position information to each word, the specific position information P t (i) The calculation is as follows:
[0036] P t (i) =f(t)(i) = sin(W k ×t),i=2k
[0037] P t (i) =f(t) (i) = cos(W k ×t),i=2k+1
[0038]
[0039] Among them, the position encoding vectors are composed of a series of sine and cosine values, which are calculated by the above formula and added to the word embedding. The even positions are represented by the sine function and the odd positions are represented by the cosine function. t is the word position index, k is the dimension index, and d is the dimension of the word embedding vector.
[0040] The BiLSTM model is then used to obtain the long-term dependencies between the above entity sequence information. The BiLSTM model is particularly useful in processing medical records. Since TCM medical records may contain a large amount of descriptive text, BiLSTM can ensure that the model does not lose important context clues, and it works well for long-distance entities; however, BiLSTM is limited by the length of the sequence and cannot maintain long-distance dependencies well for very long sequences. Here, long-term refers to long-distance text information. For example, take the text "Two days ago, the whole body had chills and fever, the body temperature remained at 39.5℃, thirst and red urine. The previous doctor diagnosed it as an exogenous wind-heat syndrome. After taking two doses of Yinqiao Powder, the body temperature decreased, but it turned into a more severe fever in the afternoon, hoarseness, and difficulty swallowing food, so I sought treatment from Master Zou." Forward LSTM It reads the text from left to right in order, that is, from "2 days ago" to "Master Zou". Forward LSTM saves the information in front of each word and combines this information with the current word to generate a hidden state containing the previous word information. The backward LSTM reads backward from "Zou Shi" to "2 days ago". It captures the information behind each word, so that the hidden state of the current word includes the influence of subsequent words. After splicing, a more comprehensive word representation is formed, which is conducive to more comprehensive information acquisition, thereby improving recognition accuracy. The forward LSTM will take into account "2 days ago, the whole body appeared", which means that this symptom appeared in the last few days. The backward LSTM will take into account "body temperature continues to 39.5℃", suggesting that chills and fever may be accompanied by high fever.
[0041] That is, the BERT model and the Bi-LSTM model pay attention to different scales of contextual information of the text. The BERT model focuses on global dependencies. Even if the texts are far apart in the sequence, it can still pay attention to the dependencies between these texts. For example, when processing this text, the BERT model can not only understand the direct connection between "chills and fever" and "body temperature continues to be 39.5℃", but also recognize the indirect connection between "chills and fever" and "seeking treatment from Master Zou", that is, how the patient's initial symptoms eventually led him to seek help from Master Zou; the dependencies between contexts with a longer distance and cross-sentence information are taken into account, which improves the accuracy of named entity recognition.
[0042] The BiLSTM model focuses on the direct order and local context between words. For example, when processing "2 days ago, I had chills and fever all over my body", it will focus on the direct order relationship between "2 days ago" and "chills and fever", as well as the association between "chills" and "fever", but may not easily capture the connection between "chills and fever" and "seeking treatment from Zou Shi" across the entire text. This is because the BiLSTM model mainly relies on its internal state to transmit information, and the efficiency of long-distance information transmission is low. The text sequence information processed by the BERT model and then passed through the BiLSTM model can enhance the correlation of dependencies, but some key information will also be lost. Therefore, in texts with more complex contexts such as TCM medical records, related text information is far apart. The sequence information processed by the BERT model and then obtained by the BiLSTM model through the relevant information before and after the context cannot be accurately understood by the model. Therefore, the sequence output by the BERT model that pays more attention to global information is fused with the sequence processed by the BiLSTM-multi-head attention mechanism to better understand the information in the text. This is of great significance for TCM texts with complex texts and related information that is far apart. It pays attention to information of different scales, so that the relationship between information that is far away can be noticed, thereby improving the recognition accuracy.
[0043] The sequence information processed by the BiLSTM model is input into the multi-head attention mechanism, so that the model can flexibly pay attention to different parts of the input sequence and focus on the most critical part of the text for named entity recognition. For example, in TCM medical records, it can pay attention to the relationship between key information of symptoms and symptoms. One head may focus on the development of symptoms, such as "chills and fever" to "fever in the afternoon", another head may focus on time clues, such as "2 days ago" to "after taking 2 doses of Yinqiao powder", and the third head may focus on changes in patients' medical behavior, such as from "previous doctor" to "Zou Shi". This multi-perspective attention mechanism enables the model to understand various dependencies in the text more comprehensively and meticulously.
[0044] Finally, the direct output of the BERT model is concatenated with the output processed by the BiLSTM model and the multi-head attention mechanism. In this way, the global dependency information of the BERT model on the text is retained, and the sequence information association between adjacent information is enhanced through the BiLSTM model and the multi-head attention mechanism. The prediction of entity information is then completed through the CRF in the decoding layer. Specifically, in the CRF model, given an observation sequence X = (x1, x2, ..., x n ) and a label sequence Y = (y1,y2,...,y n ), the goal of CRF is to find the Y sequence that maximizes the conditional probability P(Y|X). After fusion, the model is more comprehensive in understanding TCM texts and the prediction results are more accurate.
[0045] In TCM medical records, the context is complex, including classical Chinese and spoken texts, the text descriptions vary greatly, the text context information is more complex, and the related information in the text is farther apart. For example, "I feel weak all over, have no appetite for anything, and often feel dizzy." and "The patient is tired and weak, food is not sweet, and he feels dizzy from time to time; he has pain around the waist, stiffness, and heel pain, which is wandering, and his appetite is good, his bowel movements are normal, and his sleep is poor." This application takes into account the different scales of attention paid to the contextual information of the text by the BERT model and the BiLSTM model. By fusing the text sequence information output by the BERT model with the text sequence information processed by the BiLSTM model and the multi-head attention mechanism, such processing can improve the model's ability to process TCM texts.
[0046] S3, training and testing the named entity recognition model;
[0047] The model is trained by establishing standard data sets and open source data sets. The open source data sets use two general medical data sets, CCKS2017 and CCKS2019, and the CLUENER2020 general field data set. These data sets are used to verify the generalization performance of the model. Due to the lack of high-quality data sets in the field of traditional Chinese medicine in open source data sets, the applicant established a traditional Chinese medicine data set by collecting ancient medical classics and clinical records of traditional Chinese medicine practitioners. Therefore, the model is trained using both open source and self-built data sets.
[0048] The dataset is divided into training set and test set in a ratio of 7:3-8:2. The data format of the training set and the test set is the same. The training process is to input the training set data and the corresponding labels into the named entity recognition model so that the model can learn the entity information division ability in the training set. In this way, the model can be verified in the test set based on the information learned in the training set. Although the data format of the test set is consistent with the training set, the marking and annotation results in the test set are for the model to check whether the entity is correctly predicted. Finally, certain evaluation indicators are used to evaluate the results of the model test in the test set. The specific indicators are as follows:
[0049]
[0050] In the formula, correct_entities represents the number of correctly identified entities, while pre_entities represents the total number of entities recognized by the model, and true_entities represents the total number of true entities.
[0051] Compared with the BERT-BiLSTM-CRF model (existing model) with precision of 82.67%, recall of 84.01%, and f1 of 83.34%, this model improved by 1.99%, 1.85%, and 1.91%, reaching 84.66%, 85.86%, and 85.25% respectively on CCKS2019. Compared with the BERT-BiLSTM-CRF model with precision of 79.99%, recall of 81.18%, and f1 of 80.58% on CLUENER, this model improved by 2.42%, 0.97%, and 1.7%, reaching 82.41%, 82.15%, and 82.28% respectively. On the self-built TCM medical records dataset, compared with the BERT-BiLSTM-CRF model with precision of 89.37%, recall of 87.03%, and f1 of 88.19%, this model improved by 0.43%, 1.15%, and 0.79%, reaching 89.8%, 88.17%, and 88.98%, respectively. It can be seen that the method of this application improves the recognition accuracy of named entities in TCM medical records, and the recognition effect is good.
[0052] S4, input the text to be recognized into the trained named entity recognition model, and the named entity recognition model outputs the prediction result.
[0053] The text to be recognized is input into the above model, which can be an unstructured TCM medical record text. During the training process, the model has learned the relationship between text entities and labels. The trained named entity recognition model will automatically identify the three types of entities contained in the medical record text, and the model outputs the entity category label corresponding to the text entity. For example, after the medical record text "The patient is tired, has no appetite, and feels dizzy from time to time" is input into the model, the model will recognize "tired", "no appetite", and "dizziness" as symptoms and automatically mark the text.
[0054] The named entity recognition model in this application performs entity recognition on a large amount of unstructured text and unlabeled data, and is committed to converting the unstructured data in TCM medical records into structured data, constructing relevant knowledge graphs in the field of TCM, and promoting the establishment of standardization in the field of TCM.
[0055] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A named entity recognition method for traditional Chinese medicine medical records based on natural language processing, characterized in that: The method comprises the following steps: S1, obtain medical record data and obtain a standard data set after preprocessing; S2, building a named entity recognition model; S3, training and testing the named entity recognition model; S4, inputting the text to be recognized into the trained named entity recognition model, and the named entity recognition model outputs a prediction result.
2. The method for named entity recognition of TCM medical records based on natural language processing according to claim 1, characterized in that: The preprocessing includes data cleaning, entity tagging, and BIO annotation of the marked entities.
3. The method for named entity recognition of TCM medical records based on natural language processing according to claim 2, characterized in that: The entity tagging is performed using Doccano to assign labels to the text data, with the symptom entity being labeled as "symptom", the disease entity being labeled as "disease", and the drug entity being labeled as "medicine".
4. The method for named entity recognition of TCM medical records based on natural language processing according to claim 3, characterized in that: In the BIO notation, B represents the beginning of the entity, I represents the remaining part of the entity, and O represents the non-entity part.
5. The method for named entity recognition of TCM medical records based on natural language processing according to claim 4, characterized in that: The named entity recognition model includes a BERT model, a BiLSTM model, a multi-head attention mechanism and a CRF model.
6. The method for named entity recognition of TCM medical records based on natural language processing according to claim 5, characterized in that: The output of the BERT model is connected to the input of the BiLSTM model, and the output of the BiLSTM model is connected to the input of the multi-head attention mechanism.
7. The method for named entity recognition of TCM medical records based on natural language processing according to claim 6, characterized in that: The output of the multi-head attention mechanism is concatenated and fused with the output of the BERT model, and the fused information is input into the CRF model for prediction, and finally the prediction result is output.
8. The method for named entity recognition of TCM medical records based on natural language processing according to claim 7, characterized in that: In step S3, the data set is divided into a training set and a test set in a ratio of 7:3-8:
2.
9. The method for named entity recognition of TCM medical records based on natural language processing according to claim 8, characterized in that: In step S3, the test result is evaluated by the parameter F1 value, and the calculation formula is: The precision represents the accuracy, and the recall represents the recall rate.
