A method for named entity recognition in electronic medical records

Abstract

A method for identifying named entities in electronic medical records, firstly, vectorizes the text sequence of the original electronic medical record data set with the word vector and word vector of the pre-trained language model as the input of the neural network, then uses the multi-head self-attention mechanism module and the BiLSTM neural network module of the parallel structure to train the input vector respectively, the output of the neural network layer is obtained by splicing the output vectors of the two modules, characterizing the richer text features and feature association weights at multiple levels, and finally uses CRF to decode and sequence-label the output of the neural network layer, and outputs the entity prediction label. The electronic medical record named entity recognition model constructed by the present invention can comprehensively model and predict the text sequence, the model performance is more excellent, and it can ensure that the entity recognition is carried out more quickly and accurately.

Description

Technical Field

[0001] The invention discloses a method for identifying named entities in electronic medical records, and belongs to the technical field of natural language processing. Background Art

[0002] The application of hospital information systems has accelerated the development of medical informatization, carrying a large amount of electronic medical records including clinical diagnosis and treatment of patients. Facing medical clinical information stored in unstructured text, how to conduct effective data mining and knowledge discovery is the research focus of natural language processing in the medical field.

[0003] Named Entity Recognition (NER) is one of the key basic tasks in natural language processing research. It aims to identify entities with specific meanings from unstructured texts, such as names of people, places, and organizations, and lays the foundation for tasks such as relationship extraction and knowledge graphs. In the field of medical electronic medical records, it mainly mines medical knowledge that is closely related to clinical knowledge.

[0004] In the early days, named entity recognition was mainly based on rules and dictionaries, but it required a lot of effort for manual annotation, and the rule availability was poor, which made the rules unusable. Machine learning models include Maximum Entropy Models (MEM), Hidden Markov Models (HMM), Support Vector Machines (SVM), Conditional Random Fields (CRF), etc. Although they have good flexibility and robustness, they have disadvantages such as requiring a large number of annotation sets and over-reliance on the correctness of feature selection. With the continuous emergence of deep learning neural network technology, manual data preprocessing is gradually abandoned, and automatic feature extraction is realized. Compared with traditional statistical rule-based or machine learning algorithms, it has the advantages of avoiding the high cost and complexity of manual feature extraction, reducing dependence on word segmentation, and enhancing model generalization. As the mainstream architecture of Chinese named entity recognition models, BiLSTM-CRF can effectively establish contextual association information, but it does not take into account the importance of different words and characters in sentences, and the recognition results still have room for further improvement. Summary of the invention

[0005] The purpose of the present invention is to provide a method for identifying named entities in electronic medical records. By comprehensively modeling and predicting text sequences, the model performance is more excellent and entity recognition can be ensured to be performed more quickly and accurately.

[0006] In order to achieve the above object, the present invention provides a method for identifying named entities in electronic medical records, comprising the following steps:

[0007] Step 1: After the electronic medical record dataset is introduced into the medical terminology dictionary, it is preprocessed to obtain the original corpus, and the words are randomly initialized to obtain the electronic medical record word vector lookup table word_emb; combined with large-scale unlabeled medical record text in the same field, low-dimensional dense word vectors and word vectors are trained to form a pre-trained language model. For the electronic medical record text input sequence S = (w1, w2, ..., w n ), where w i is the word in the sentence, w i If it contains m words, then the word vector representation of the electronic medical record word vector lookup table word_emb can be obtained from the word vector lookup table vec of the pre-trained language model char And word vector lookup table vec word Matching and assigning values:

[0008]

[0009] Where: e i The word vector lookup table vec for the pre-trained language model word The word vectors in ;

[0010] p ij is the word vector lookup table vec of the pre-trained language model char The word vectors in ;

[0011] Step 2: Based on the electronic medical record word vector lookup table word_emb obtained in step 1, the vectorized representation vec of the electronic medical record text input sequence S input :

[0012] vec input =w i ∩word_emb,w i ∈S;

[0013] Step 3: The vectorized representation vec of the text input sequence S of the electronic medical record obtained in step 2 input , as the input of the neural network layer composed of parallel structures, and respectively input into the BiLSTM neural network module and the multi-head self-attention module, the BiLSTM neural network module is used to train the input vector to obtain the contextual features of the text sequence, and the output vector H is obtained. t , the input vector is trained with a multi-head self-attention module to obtain the global features and feature association weights of the text sequence, and the output vector Att is obtained t ;

[0014] Step 4: The output vector H obtained in step 3 t and Att tSplice and use it as the overall output text feature vector O of the neural network layer t ;

[0015]

[0016] Step 5: Use the CRF layer to transform the text feature vector O obtained in step 4 t Decode and predict, set the label sequence y of the text input sequence S of the electronic medical record to be {y1, y2, ..., y n}, through the text feature vector O = (O1, O2, ..., O n ), calculate the conditional probability of the output label sequence y:

[0017] P(y|O)=CRF(O,y);

[0018] Step 6: For the text input sequence S of the electronic medical record in step 1, S = (w1, w2, ..., w n ), the text feature vector O of step 4 t , the conditional probability of the label sequence y in step 5, and the predicted label y with the largest conditional probability * The final output label sequence S as the text input sequence of the electronic medical record is:

[0019] y * =argmaxP(y|O).

[0020] The preprocessing in step 1 of the present invention is sentence segmentation and word segmentation.

[0021] The output vector H in step 3 of the present invention is t It is obtained by concatenating the output vectors of the bidirectional long short-term memory network LSTM training:

[0022]

[0023] Compared with the prior art, the model proposed in the present invention mainly includes four layers: input layer, neural network layer, CRF layer and output layer. First, by introducing a medical terminology dictionary, combining large-scale unlabeled medical record text in the same field to train character vectors and word vectors, the text input sequence of the electronic medical record is represented by vectors through the fusion of characters and words, and then input into the BiLSTM neural network module and the multi-head self-attention module respectively, wherein the BiLSTM neural network module is used to learn the temporal characteristics and contextual information of the text sequence, and the self-attention mechanism is used to obtain the global characteristics of the text sequence and the correlation strength between words. The output of the neural network layer is obtained by splicing the output vectors of the two modules, representing the richer text features and feature association weights at multiple levels, and finally using CRF to decode and sequence annotate the output of the neural network layer, and output entity prediction labels. The electronic medical record named entity recognition model constructed by the present invention can comprehensively model and predict text sequences, and the model performance is more excellent, which can ensure that entity recognition is carried out more quickly and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described below in conjunction with the accompanying drawings.

[0026] like Figure 1 As shown, a method for identifying named entities in electronic medical records includes the following steps:

[0027] Step 1: After the electronic medical record dataset is introduced into the medical terminology dictionary, the original corpus is obtained after sentence and word segmentation, and the words are randomly initialized to obtain the electronic medical record word vector lookup table word_emb; combined with large-scale unlabeled medical record text in the same field, low-dimensional dense word vectors and word vectors are trained to form a pre-trained language model. For the electronic medical record text input sequence S = (w1, w2, ..., w n ), where w i is the word in the sentence, w i If it contains m words, then the word vector representation of the electronic medical record word vector lookup table word_emb can be obtained from the word vector lookup table vec of the pre-trained language model char And word vector lookup table vec word Matching and assigning values:

[0028]

[0029] In the formula: word_emb i Look up the i-th word in the word_emb table for the electronic medical record word vector;

[0030] ei The word vector lookup table vec for the pre-trained language model word The word vectors in ;

[0031] p ij is the word vector lookup table vec of the pre-trained language model char The word vectors in ;

[0032] Step 2: Based on the electronic medical record word vector lookup table word_emb obtained in step 1, the vectorized representation vec of the electronic medical record text input sequence S input :

[0033] vec input =w i ∩word_emb,w i ∈S;

[0034] Step 3: The vectorized representation vec of the text input sequence S of the electronic medical record obtained in step 2 input , as the input of the neural network layer composed of parallel structures, and respectively input into the BiLSTM neural network module and the multi-head self-attention module, the BiLSTM neural network module is used to train the input vector to obtain the contextual features of the text sequence, and the output vector H is obtained. t , the input vector is trained with a multi-head self-attention module to obtain the global features and feature association weights of the text sequence, and the output vector Att is obtained t ;

[0035] Step 4: The output vector H obtained in step 3 t and Att t Splice and use it as the overall output text feature vector O of the neural network layer t ;

[0036]

[0037] This step 4 is a sequential process. For time t, the output vector H t and Att t Concatenate to form text feature vector O t , O t It can represent the text feature vector at any time, and the text feature vector O at all times t It forms O, which can be called a text feature vector or a text feature vector matrix;

[0038] Step 5: Use the CRF layer to transform the text feature vector O obtained in step 4 t Decode and predict, set the label sequence y of the text input sequence S of the electronic medical record to be {y1, y2, ..., y n}, through the text feature vector O = (O1, O2, ..., O n ), calculate the conditional probability of the output label sequence y:

[0039] P(y|O)=CRF(O,y);

[0040] Step 6: For the text input sequence S of the electronic medical record in step 1, S = (w1, w2, ..., w n ), the text feature vector O of step 4 t , the conditional probability of the label sequence y in step 5, and the predicted label y with the largest conditional probability * The final output label sequence S as the text input sequence of the electronic medical record is:

[0041] y * =argmaxP(y|O).

[0042] The output vector H in step 3 t It is obtained by concatenating the output vectors of the bidirectional long short-term memory network LSTM training:

[0043]

[0044] The present invention proposes a method for named entity recognition of electronic medical records. It uses a method combining characters and words, and according to the parallel structure of a multi-head self-attention mechanism module and a BiLSTM neural network module, learns the multi-level features of the semantic association information of electronic medical record text, captures the internal structure features and dependency relationships of sentences, so as to improve the effect of the Chinese clinical named entity recognition model.

[0045] In order to verify the effectiveness of the model of the present invention, it was evaluated on the named entity recognition task of the 2019 National Knowledge Graph and Semantic Computing Conference. At the same time, four groups of comparative experiments were set up for model comparison. The comparative model design is as follows:

[0046] (1) Abbreviated as: BiLSTM-CRF, in the form of word vector fusion, the BiLSTM neural network is used to train text feature vectors, and CRF is used for decoding and prediction.

[0047] (2) is abbreviated as: BiLSTM-CRF-SA. Based on (1), Self-attention is used to weight the output vector of BiLSTM, and then CRF is used for decoding and prediction.

[0048] (3) is abbreviated as: BiLSTM-CRF-MHSA. Based on (1), Multi-head Self-attention is used to calculate the feature association weights of the output vector of BiLSTM, and then CRF is used for decoding and prediction.

[0049] (4) The model of the present invention, referred to as MHSA-BiLSTM-CRF, performs named entity recognition of electronic medical records based on (1) according to steps 1-6.

[0050] The entity recognition model uses precision, recall and F-score as evaluation indicators; precision can indicate the proportion of positive samples in the prediction results that are truly positive, and recall can indicate the proportion of positive samples in the standard answer that are correctly predicted; F-score is the harmonic mean of precision and recall, which can balance the two indicators of precision and recall; correct indicates the number of correctly labeled entities, recognized indicates the total number of labeled entities, and entities is the total number of entities contained in the standard answer. The specific calculation methods of precision, recall and F-score are shown in the following formulas:

[0051]

[0052]

[0053]

[0054] The experimental results are shown in Table 1:

[0055]

[0056] The model of the present invention achieved the best results in the results of multiple comparative experiments, and achieved the best recognition results in precision, recall and F-score, with the F-score reaching 82.95%, proving the effectiveness of the model for named entity recognition tasks.

Claims

1. A method for identifying named entities in electronic medical records, characterized in that: The following steps are involved: Step 1: After the electronic medical record dataset is introduced into the medical terminology dictionary, it is preprocessed to obtain the original corpus, and the words are randomly initialized to obtain the electronic medical record word vector lookup table word_emb; combined with large-scale unlabeled medical record text in the same field, low-dimensional dense word vectors and word vectors are trained to form a pre-trained language model. For the electronic medical record text input sequence S = (w1, w2, ..., w n ), where w i is the word in the sentence, w i If it contains m words, then the word vector representation of the electronic medical record word vector lookup table word_emb can be obtained from the word vector lookup table vec of the pre-trained language model char And word vector lookup table vec word Matching and assigning values: Where: e i The word vector lookup table vec for the pre-trained language model word The word vectors in ; p ij is the word vector lookup table vec of the pre-trained language model char The word vectors in ; Step 2: Based on the electronic medical record word vector lookup table word_emb obtained in step 1, the vectorized representation vec of the electronic medical record text input sequence S input : vec input =w i ∩word_emb,w i ∈S; Step 3: The vectorized representation vec of the text input sequence S of the electronic medical record obtained in step 2 input , as the input of the neural network layer composed of parallel structures, and respectively input into the BiLSTM neural network module and the multi-head self-attention module, the BiLSTM neural network module is used to train the input vector to obtain the contextual features of the text sequence, and the output vector H is obtained. t , the input vector is trained with a multi-head self-attention module to obtain the global features and feature association weights of the text sequence, and the output vector Att is obtained t ; Step 4: The output vector H obtained in step 3 t and Att t Splice and use it as the overall output text feature vector O of the neural network layer t ; Step 5: Use the CRF layer to transform the text feature vector O obtained in step 4 t Decode and predict, set the label sequence y of the text input sequence S of the electronic medical record to be {y1, y2, ..., y n }, through the text feature vector O = (O1, O2, ..., O n ), calculate the conditional probability of the output label sequence y: P(y|O)=CRF(O,y); Step 6: For the text input sequence S of the electronic medical record in step 1, S = (w1, w2, ..., w n ), the text feature vector O of step 4 t , the conditional probability of the label sequence y in step 5, and the predicted label y with the largest conditional probability * The final output label sequence S as the text input sequence of the electronic medical record is: and * =arg max P(y|O)。

Images