Traditional Chinese medicine case named entity recognition method

Abstract

The invention aims to provide a Chinese medicine named entity recognition method based on radical and character features, and the method comprises the following steps: firstly, obtaining character and vocabulary features in a Chinese medicine case text through an ALBERT model; according to the method, a Top-k attention score screening mechanism is introduced, character features and vocabulary features are screened, a multi-scale channel attention module (MS-CAM) is used, and semantic information expression of the features is enhanced by performing multi-scale processing on the spliced features. And inputting the fused features into a bidirectional long short-term memory network (BiLSTM), capturing context information in a traditional Chinese medicine case text by utilizing the modeling capability of the BiLSTM on sequence data, and finally correcting an output result of the BiLSTM through a conditional random field (CRF) layer to obtain an optimal tag sequence.

Description

Technical Field

[0001] The present invention proposes a method for identifying named entities in TCM medical records, which improves the accuracy and efficiency of TCM medical record named entity recognition and is suitable for the digitization, standardization and intelligent processing of TCM medical record data, thereby promoting the widespread application and dissemination of TCM knowledge. Background Art

[0002] As an important record of TCM theory and practice, TCM medical records contain a wealth of TCM knowledge and are a valuable resource for TCM academic research and clinical practice. However, since the text content of TCM medical records is often expressed in natural language and contains a large number of proper nouns, ancient Chinese terms and TCM-specific terms, traditional natural language processing technology is difficult to effectively identify and extract key information.

[0003] Existing named entity recognition methods are mainly based on rule matching, statistical learning or deep learning technology. Although these methods perform well in dealing with named entity recognition tasks in general fields, there is still a lot of room for improvement in the field of TCM medical records due to its unique context and complexity. For example, rule-matching-based methods often perform poorly when dealing with TCM medical records because the rules are complex to write and difficult to cover all situations; statistical learning-based methods rely on a large amount of annotated data, but annotated data in the field of TCM medical records is relatively scarce; and although deep learning-based methods have strong modeling capabilities, the ability to capture semantic understanding and contextual relevance still needs to be strengthened.

[0004] Therefore, it is necessary to develop a named entity recognition method that can fully utilize the text features of TCM medical records to improve the accuracy and efficiency of recognition, thereby promoting the digitalization, standardization and intelligent processing of TCM medical records. Summary of the invention

[0005] The present invention aims to provide a method for named entity recognition of traditional Chinese medical records based on attention feature fusion, so as to solve the problems existing in the prior art such as low recognition accuracy and insufficient feature extraction.

[0006] The method of the present invention mainly includes the following steps: first, the medical case data set is processed and analyzed. The data set comes from the "Research on Medical Cases of the Qing Palace", and 600 high-quality medical cases are selected. The data set is manually annotated, and the data is supplemented by combining the Chinese medicine named entity extraction data set and the Chinese medicine instruction entity recognition data set. The data set includes 4 entity types, namely disease, symptom, prescription, and dosage. The data set is divided into a training set and a validation set at a ratio of 90% and 10%, and the ratio of the training set to the validation set is 9:1. For the training samples, a series of data enhancement operations such as translation, flipping, adjusting saturation and contrast are performed to increase the generalization ability of the model. This study uses the BIO annotation method, BX represents the beginning of entity X, IX represents the middle part of entity X, and O represents a character that does not belong to any type.

[0007] The character and vocabulary features in the TCM medical records are obtained through the ALBERT model. In order to make full use of these features, the present invention proposes a Top-k attention score screening mechanism to screen the character features and vocabulary features to retain the feature information most relevant to the named entity. Then, the convolutional neural network (CNN) is used to extract the radical and stroke features of Chinese characters, which can capture the detailed information of Chinese characters and are spliced ​​with the screened character and vocabulary features.

[0008] In order to further improve the expressiveness of features, the present invention introduces a multi-scale channel attention module (MS-CAM), which enhances the semantic information expression of features by performing multi-scale processing on the spliced ​​features. The fused features are input into a bidirectional long short-term memory network (BiLSTM), which uses its modeling ability for sequence data to capture contextual information in TCM medical records. Finally, the output of BiLSTM is corrected through a conditional random field (CRF) layer to obtain the optimal label sequence.

[0009] The method of the present invention can not only effectively improve the accuracy of named entity recognition in TCM medical records, but also can process a variety of different types of named entities, and has strong robustness and versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a structural flow chart of the present invention.

[0011] Figure 2 It is a data collection flow chart of the present invention.

[0012] Figure 3 This is a data annotation structure diagram of the present invention.

[0013] Figure 4 Compute score graph for Top-k dictionary matcher.

[0014] Figure 5 This is the module diagram of MS-CAM (Multi-Scale Channel Attention Module). DETAILED DESCRIPTION

[0015] The present invention will be further described below in conjunction with the accompanying drawings:

[0016] The present invention is a method for identifying named entities in traditional Chinese medicine medical records, comprising the following specific steps:

[0017] S11: Collect and prepare the dataset of TCM medical records as shown in the attached Figure 2 , as shown in Figure 3. The dataset comes from the "Research on Medical Records of the Qing Palace", and 600 high-quality medical records are selected. The dataset is manually annotated.

[0018] S12: Divide the dataset into a training set and a validation set at a ratio of 90% and 10%, and the ratio of the training set to the validation set is 9:1.

[0019] S13: Use BIO notation, where BX represents the beginning of entity X, IX represents the middle part of entity X, and O represents a character that does not belong to any type.

[0020] S21: Top-k dictionary matcher as attached Figure 4 As shown. The attention score of each word and character is calculated. The attention score is obtained through the self-attention mechanism of the ALBERT model to measure the importance of each word and character in the current context.

[0021] S22: At the i-th position, the input of the Top-k dictionary matcher is represented as This is a character feature-word feature pair, where the output of the Transformer layer in ALBERT is, is a set of vocabulary embeddings, The jth word in is represented as follows:

[0022]

[0023] S23: Due to the inconsistency between character vectors and word vectors in terms of dimension, this study first performs nonlinear mapping on word vectors to achieve dimensional matching to ensure that word vectors and character vectors are aligned in the same dimension.

[0024] Dimension Alignment Formula

[0025]

[0026] S24: For each character, the importance of each word in the matching vocabulary is different, so it is necessary to calculate the corresponding attention score for each word vector. This mechanism effectively identifies and highlights the words that are most representative and important to the character by calculating the attention score of each word, thereby improving the overall performance and accuracy of the model. The attention score of each word is calculated as shown below.

[0027]

[0028] S25: Filtering with Top-k Attention Score Figure 4 As shown. Among them, a i ={a i1 ,a i2 ,a i3 ,...,a im}, represents the weight of the jth word vector of the i-th character, and S represents the bilinear attention score matrix. The calculation formula is shown in the following formula.

[0029] a i =softmaxS i

[0030] S26: Specifically, the word vector of each word is weighted and summed according to its corresponding weight, so as to obtain the weighted word vector corresponding to the character. The weighted summation formula is shown as follows.

[0031]

[0032] S27: Next, through layer norm and residual connection operations, the final output of the dictionary matcher is completed. The feature fusion formula is shown in the following formula.

[0033]

[0034] S31: Radical and stroke feature extraction. A convolutional neural network (CNN) is used to perform fine-grained analysis of Chinese characters. CNN extracts low-level to high-level feature information of Chinese characters step by step through multiple convolutional layers, and can accurately identify the radical and stroke features of Chinese characters.

[0035] S32: Decompose Chinese characters into radicals and strokes, decompose Chinese characters into their components based on the convolution operation of CNN, and perform feature dimensionality reduction through the pooling layer to retain important local information.

[0036] S33: Concatenate the radical stroke features with the character and vocabulary features. Through this concatenation operation, the composition information of the Chinese characters is combined with the context information of the text, further enhancing the semantic information expressed by the features and improving the model's ability to recognize complex named entities.

[0037] S4: Multi-scale channel attention module (MS-CAM) performs deep fusion of the combined features as shown in the attached Figure 5 Shown

[0038] S41: Assign weights to each feature channel, and adjust the weights of feature information of different channels through the channel attention mechanism, so that the model can dynamically focus on important feature channels and capture multi-scale information.

[0039] S42: Capturing multi-scale information. The MS-CAM module processes features through convolution kernels of different scales to capture multi-scale semantic information in the text, making the feature expression richer and more accurate.

[0040] S43: Through the multi-scale feature fusion mechanism, feature information of different scales is effectively fused. The feature information of different scales is fused by weighted summation or splicing to generate a more expressive feature representation and improve the effect of named entity recognition.

[0041] S51: Feature input to a bidirectional long short-term memory network (BiLSTM) to generate context feature representation,

[0042] S52: The BiLSTM network adopts a bidirectional information flow mechanism, which can capture contextual dependencies in both forward and backward propagation of text features, thereby improving the recognition of named entities in complex sentence structures.

[0043] S61: Sequence labeling of BiLSTM outputs through the Conditional Random Field (CRF) layer

[0044] S62: The conditional random field (CRF) layer can accurately divide the boundaries of named entities through global serialization modeling, and output the optimal annotation sequence through the decoding algorithm to avoid misclassification and missing labeling problems of entity boundaries.

[0045] S62: For the input sequence X=(x1, x2, ..., x n ) predicts the output sequence Y = (y1, y2, ..., y n ) can be expressed as the sum of the transition probability and the state probability. Where A represents the transition matrix and P represents the output score matrix of BiLSTM.

[0046]

[0047] S63: Using the softmax function, the probability value of the label sequence Y can be calculated

[0048]

[0049] S64: By finding the most likely path on the predicted sequence output by BiLSTM, this method can determine the label of the output given entity, thereby achieving entity identification. Therefore, the training goal is to maximize the probability p(y|X). This can be achieved by maximizing the log-likelihood as shown below.

[0050]

[0051] S65: Evaluation indicators This experiment is evaluated from the precision P (precision), recall R (recall), and F1 indicators.

[0052]

[0053]

[0054]

[0055] S66: The results are compared with the predictions of each entity tag by the model in this paper.

[0056]

[0057]

[0058] S67: To further verify the effect of the model in this paper, we compared it with several other methods. The selected models are: CRF model based on machine learning, BiLSTM model based on deep learning, BiLSTM-CRF model, BERT-BiLSTM-CRF model, ERNIE-BiLSTM-CRF model and ALBERT-BiLSTM-CRF model. We experimented with the Qing Palace Medical Records Dataset on each model and compared it with the model in this paper. The results are shown in the table.

[0059]

[0060] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for identifying named entities in traditional Chinese medicine medical records, characterized by: The following steps are involved: Step 1: Processing and analysis of medical record data sets; Step 2: Use the Top-k dictionary matching strategy to calculate and filter the attention scores of the character features and lexical features generated by the ALBERT model, and then concatenate the filtered lexical features with the character features. Step 3: Use convolutional neural network (CNN) to extract the radical stroke features of Chinese characters and combine them with the concatenated characters and vocabulary features; Step 4: Deeply fuse the combined features through the multi-scale channel attention module (MS-CAM) to enhance the semantic expression of the features; Step 5: Input the fused features into the Bidirectional Long Short-Term Memory Network (BiLSTM) to generate context feature representation; Step 6: Sequence the output results of BiLSTM through the conditional random field (CRF) layer to obtain the final named entity label sequence.

2. The method for named entity recognition of TCM medical records based on attention feature fusion according to claim 1, characterized in that: Step 1: S11: The data set comes from the "Research on Medical Records of the Qing Palace", and 600 high-quality medical records are selected. The data set is manually annotated as shown in Figures 2 and 3; S12: Divide the dataset into a training set and a validation set at a ratio of 90% and 10%, with the ratio of training set to validation set being 9:1; S13: Use BIO notation, where BX represents the beginning of entity X, IX represents the middle part of entity X, and O represents a character that does not belong to any type.

3. The method for named entity recognition of TCM medical records based on attention feature fusion according to claim 1, characterized in that: The second step: S21: The Top-k dictionary matcher, as shown in Figure 4, calculates the attention score of each word and character. The attention score is obtained through the self-attention mechanism of the ALBERT model to measure the importance of each word and character in the current context. S22: At the i-th position, the input of the Top-k dictionary matcher is represented as This is a character feature-word feature pair, where the output of the Transformer layer in ALBERT is, is a set of vocabulary embeddings, The j-th word in is represented as follows: S23: Due to the inconsistency between character vectors and word vectors in terms of dimension, this study first performs nonlinear mapping on word vectors to achieve dimension matching to ensure that word vectors and character vectors are aligned in the same dimension. Dimension alignment formula For each character, the importance of each word in the matching vocabulary is different, so it is necessary to calculate the corresponding attention score for each word vector. This mechanism effectively identifies and highlights the words that are most representative and important to the character by calculating the attention score of each word, thereby improving the overall performance and accuracy of the model. The attention score of each word is calculated as shown below. S24: Filtering based on Top-k attention scores as shown in Figure 4. i ={a i1 ,a i2 ,a i3 ,...,a im }, represents the weight of the jth word vector of the i-th character, and S represents the bilinear attention score matrix. The calculation formula is shown in the following formula. a i =softmaxS i S25: Specifically, the word vector of each word is weighted and summed according to its corresponding weight, so as to obtain the weighted word vector corresponding to the character. The weighted summation formula is shown as follows. Next, through layer norm and residual connection operations, the final output of the dictionary matcher is completed. The feature fusion formula is shown below.

4. The method for named entity recognition of TCM medical records based on attention feature fusion according to claim 1, characterized in that: The step three: S31: A convolutional neural network (CNN) is used to perform fine-grained analysis of Chinese characters. CNN extracts low-level to high-level feature information of Chinese characters step by step through multiple convolutional layers, and can accurately identify the radicals and stroke features of Chinese characters. S32: Decompose Chinese characters into radicals and strokes, decompose Chinese characters into their components based on the convolution operation of CNN, and perform feature dimensionality reduction through the pooling layer to retain important local information. S33: Concatenate the radical stroke features with the character and vocabulary features. Through this concatenation operation, the composition information of the Chinese characters is combined with the context information of the text, further enhancing the semantic information expressed by the features and improving the model's ability to recognize complex named entities.

5. The method for named entity recognition of TCM medical records based on attention feature fusion according to claim 1, characterized in that: Step 4: S41: Assign weights to each feature channel, and adjust the weights of feature information of different channels through the channel attention mechanism, so that the model can dynamically focus on important feature channels and capture multi-scale information. S42: Capturing multi-scale information. The MS-CAM module processes features through convolution kernels of different scales to capture multi-scale semantic information in the text, making the feature expression richer and more accurate. S43: Through the multi-scale feature fusion mechanism, feature information of different scales is effectively fused. The feature information of different scales is fused by weighted summation or splicing to generate a more expressive feature representation and improve the effect of named entity recognition.

6. The method for named entity recognition of TCM medical records based on attention feature fusion according to claim 1, characterized in that: Step 5 S41: The BiLSTM network adopts a bidirectional information flow mechanism, which can capture contextual dependencies in both forward and backward propagation of text features, thereby improving the recognition of named entities in complex sentence structures.

7. The method for named entity recognition of TCM medical records based on attention feature fusion according to claim 1, characterized in that: Step 6 S61, the conditional random field (CRF) layer can accurately divide the boundaries of named entities through global serialization modeling, and output the optimal annotation sequence through the decoding algorithm to avoid the misclassification and missing labeling problems of entity boundaries. S62, for the input sequence X=(x1,x2,...,x n ) predicts the output sequence Y = (y1, y2, ..., y n ) can be expressed as the sum of the transition probability and the state probability. Where A represents the transition matrix and P represents the output score matrix of BiLSTM. S63. Using the softmax function, the probability value of the label sequence Y can be calculated S64. By finding the most likely path on the predicted sequence output by BiLSTM, the method can determine the label of the output given entity, thereby achieving entity identification. Therefore, the training goal is to maximize the probability p(y|X). This can be achieved by maximizing the log-likelihood as shown in the following formula.

Images